Hypocrisy in how people with low empathy are treated

I find it a bit funny that people seem to have empathy only for those who also have it, and when you don’t, they see you as scummy. Why are people so harsh toward those who predominantly feel apathy, unable to recognize what they’re going through simply because they can’t personally experience it? It is just incredibly hypocritical