A thousand times yes!!!
My optimization is usually crude but very blatant (a high damage output warblade or a ridiculous stealth droid in SAGA) and my group considers anything I do to be grounds for suspicion (the party wizard who took leadership for a sorcerer cohort called my goliath warblade "broken" because of IHS and Strike of Perfect Clarity).

I mean it's not even that I try that hard to optimize, my knowledge of truly crazy thing is attributed to forum lurking, but everyone considers me the king of cheese in my group (well taking out an enemy base by myself was kind of extreme, but I could stealth and assassinate really well so hey, why not?)

So my plan is in the evil campaign that we start soon, I'm going full wizard possibly core-only just to show how bad(ass) a caster can really be.

So yes, I feel your pain of being the only one who cares about numbers as much as plot.