Secure Core vs. Direct

I just signed up for ProtonVPN and have been very pleased with the experience so far. I haven't tried connecting using their Secure Core feature yet. But in reading about how it works, it seems to me that at the end of the day, the effectiveness is all based on the presumption that the Secure Core servers which are located in Sweden, Iceland and Switzerland are all secure.
So assuming these servers are all secure, why wouldn't I just directly connect to a ProtonVPN server in one of these countries with Secure Core disabled, in which case my connection wouldn't have to lose as much speed as with Secure Core enabled since it won't have to go through multiple servers? What am I not understanding?