As a swash, you have two approaches: a) Get Booming Blade; you can get Sneak Attack on it but cannot use a dual wield attack (because you're casting a spell, not using the Attack action). b) Dual Wield, which doesn't get you BB damage but does give you a higher chance of landing your Sneak (as well as giving you OA immunity to a second target, if you split your attacks). Either way you run in, deliver your attack and run away again. If you stay in contact, you're doing it wrong.

For DW, the point is primarily about getting a better chance to hit (because you're making two attacks), and therefore to land your Sneak Attack. The damage die size isn't key. There are a number of people who would tell you that DW needs some adjusting; Zman has suggested some tweaks (to DW and other things), linked in his sig.

A level of Hexblade would let you use CHA to attack with, of course, as well as giving you a way to get the BB cantrip, and also, IIRC, shield proficiency so you can actually use a buckler. I think the usual Darkness/Devil's Sight would run counter to the "flashy swashbuckler" feel but YMMV, and of course Hex and Minor Illusion are always nice.