My yarn catches a lot when frogging, which makes it take ages...

When I was frogging my most recent project, it took ages to undo because the yarn kept 'catching' where it was clear that I had inserted the crochet hook through the yarn itself instead of a Stitch's hole, but only by a margin of a few tiny fibres of the yarn. I.e. only tiny fibres of the yarn were responsible for the catching so I didn't notice that I was doing it when crocheting the stitches!

I was wondering whether this is just a normal thing with certain 'hairy' yarns or whether I'm doing something wrong? Any tips to prevent it?