A couple thoughts...

1.)Why does the rogue feel the need to wantonly steal while in the city? Unless he nails someone who's really rich, you get way more money by going out adventuring and beating monsters, and occasionally collecting quest rewards. I mean, what gets you a better end result: Pick-pocketing a half-dozen people for 5 gold each, or going out and slaying the Young Red Dragon and getting its hoard of treasure?
HINT: Not the first one.
2.)Why is it so terrible the paladin object to the thief taking people's money? In a medieval society, you don't have banks and debit cards. Most likely, what money you have on your person is all the money you have. Like someone pointed up above, stealing without a care could well bring great harm to many people's lives. So, yeah, the paladin should rightfully object to that. If the rogue is only doing a careful "steal from the corrupt rich" schtick, I could see the paladin being willing to "not notice his antics" or some such. But randomly stealing from innocent people walking down the street? What's wrong with the Paladin, who should be a Stand Up Guy, a Good Guy, objecting to that?
3.)Heck, why is the Paladin the only one who would object? Is anyone else in the party Good-aligned? They should probably not be okay with random pick-pocketing either. Even Chaotic Good shouldn't consistently approve of stealing from anyone the Rogue can see, because the return on the harm caused is too small (especially in light of typical adventurer treasure hauls).
4.)Why would it be bad, if this is a reoccurring thing, for the paladin to say that the rogue needs to stop, the rogue needs to leave, or (if no one else cares the Rogue is clearly heading down the path to Evil what with stealing from everyone in sight for no good reason) the paladin leaves (thus the player can make a Chaotic Neutral Fighter who doesn't care what the rogue does)? That seems like a pretty realistic response if someone's doing something you find strongly objectionable.
5.)Nothing about Rogues says they have to be thieves. Heck, even Sleight of Hand can be used for things like "hiding weapons up your sleeves" and "palming a knife when no one can see you". And even with Rogue skill points, there's no way you can get every skill on their list; thus, you can easily make a sort of scout, pathfinder, trap-disarmer LG Rogue. It's completely feasible. Heck, just go for the angle he's like a Medieval Navy SEAL or something.