Feeling guilty about quitting makes me postpone that decision. Am I the only one?

I want to share this with you because I really don't know who to share it with. I've been working as a teacher for 3 years. I never really wanted to be a teacher, but I was offered the job by a very close friend of my mother who happens to be principal of a very prestigious school. I only had some experience as an assistant professor at college and I really liked it, but my contract was soon going to end. So I accepted the offer.

First I was given what would be 5th grade, mostly 9 and 10 year olds. Despite me developing very strong bonds with the kids and having little behavior issues from them, I have never enjoyed this. I have a very hard time getting enough motivation to deliver good lessons to these kids, and I find myself barely making it through every day. I simply dread going there, especially at the beginning of the year when they are new faces that I do not know. I only start enjoying it maybe the last month of each school year, while the rest I totally feel like shit.

I also have 10th grade now. While I find it way more tolerable than 5th grade (these kids mostly act like they are grown up, although they have their days), I still have little to no motivation to teach them. I love my subject (Geography) but I feel like I am simply wasting my time there, I have recently started actively looking for other jobs and I have landed a freelance job in insurance. I find it way way more enjoyable, and what's more important, I don't get headaches everytime I think about working. But I don't really want to quit my teaching job yet because I'd feel very guilty about it (and because I'd miss the money, for a while at least) , eventhough I know it'd be the best for my health, as I am afraid that I'd get depressed if I keep working at something that I dislike so much.

I am not sure exactly what I dislike about teaching, but I feel like maybe my personality doesn't match with this job. I am a very calm person, low energy, soft spoken, so being the center of attention for so long mentally exhausts me.