Yeah, skill challenge mini games may sound fun in theory, but I think it's really easy for them to end up with the players making no real decisions. Just roll skill again and again with no reason to change what you do, that gets boring fast.

Try have a conversion with the rest of the party to see if they agree that they are supposed to engage with what the GM has prepped or not. It looks like they don't, and that's fine. It just means that it's not worth it to prep for these people and the GM should just expect improvise instead.