Ear Tip Sizing Question

Just got my first pair of IEMs (Pilgrims) and the medium and large tips both seem to seal well. Is there a rule of thumb on whether it's better to use a smaller/larger tip when both seal?