Is “ancestors ashamed” a racist remark?

I’m Asian, and someone (non Asian) told me my ancestors should be ashamed.

Is this racist, or just a common insult?