Skyrim has no shortage of skilled fighters and frankly most of the Jarls are not particularly impressive, but that doesn't matter because Nords are not Klingons and their nobility clearly does not actually work like this.
Ulfric is an optimal candidate for the Thalmor for a lot of reasons, but just because he kicked off the war by challenging Torygg to a duel doesn't mean that's the only way to kick the war off. If a bunch of rebels raid an imperial manned fort, that's kicking off a war. Or if a bunch of imperials crack down brutally on suspected rebels.