Should I go for a PhD at 44?

Wondering what everyone's thoughts/opinions are on this. I'm 42 right now and starting to take classes to meet prereqs and build up my experience when applying for PhD (if I decide to do so). So I'll likely be 44 when I start PhD. Degree is social psychology. A few things on where I'm at in life:

I'm in the USA. Great job that pays about 93k, but I want a career change. If I quit in about a year (my intention), my current job will provide a pension that will pay around $2,200/mo at retirement (62 years old). I have $0 debt. Single, no kids.

I'm saving up a lot in general for any important decisions in the future (buy a house, emergencies, pay expenses as needed in PhD if stipend is not enough, etc.)

I'm free to live/move to anywhere given that I rent an apartment. No hard ties to any specific place.

I'm torn between going for the PhD vs. freedom and travel. I know I probably won't have the luxury of vacation travel for 5 years given the rigorous nature of PhD programs (maybe except in Summer). Yet, for 15 years, my interest in pursuing a degree in this field keeps coming back. Much of it is tied also to personal fulfillment and curiosity, not just a new career. But I also know I have to be serious about this if I'm going to give up 5 years of my life that I would have done something else with, and not being able to save money for 5 years. I don't want to look back and think I should have gone for it.

I can also go for a MA (2 years), then go for PhD if I choose. But going for both will take longer because I can't apply all classes in the MA to the PhD (different schools). Also I can't do as much with an MA careerwise compared to PhD.

In short, want a new career change, which a PhD will fulfill both my desire for a career change AND personal fulfillment. But weighing this against personal freedom, desire for travel, significant drop in income, and all other sacrifices (time with family, friends, etc.)

Thoughts please? Maybe similar personal experiences?