Headset keeps losing controllers

So I'm doing my supernatural boxing workouts, high intensity super speed on, and for some reason the last week and a half or so my headset keeps losing my controllers. My hand will be in position for the strike, the target should explode, but it doesn't register and instead I will often see a phantom glove hovering right in front of my eyes making it harder to see any other targets coming at me. So I just end up flailing like an idiot. It's very frustrating losing my streak and lowering my accuracy score, when I'm trying to build my accuracy and speed. It's not the batteries, because I have just changed those pretty recently. I have no idea but it's very frustrating! Anyone have any ideas of something I can try?

EDIT: My partner told me it was cuz I was doing it in the room when it was too dark, but today I had all the lights turned on, so that shouldn't be it either.