To put it briefly, the Valor Bard basically has a higher damage cantrip, and is better at holding Concentration spells than other Bards.
That's it. Limiting the Bard to relying on weapon attacks like a major feature is like a Wizard relying on Firebolt for most of his damage. Just because it scales doesn't mean it's worthwhile.
Given, Valor does better damage than most other Bard subclasses, it just happens to be on a class that isn't designed to deal damage in the first place.
If you want to focus on dealing weapon damage, be a class that is actually designed around it. You'll get more mileage out of it.