Mmm. Sounds like you want a LeStrade for your Sherlock. Strictly from a 'role in story' standpoint, I think the best role for the guards would be to reveal clues the PCs missed, or even just to give the PCs clues that the guards can't figure out. The PCs didn't follow up on the letter from library about an overdue book (Evil McNasty's Guide to Necromancy for Fun and Profit), so have a guard contact the PCs "hey, this book actually was returned right before victim died, but it is missing the pages on how to make a flesh golem".

Also, think of the as a PC force multiplier: raiding a thieve's den? Station guards at all exits while PCs do the raid. Boring stakeout seeing who uses the 'Ye Olde Shoppe of EEEEvil Alchemy', let a guard do it. (Of course, this also means you can occasionally use 'guards not as good as PCs to explain "the guild leader sliced through the guards at the sewer entrance and got away" and "the guard dozed and didn't see person X enter")

From a more simulation-ish angle, cops have fundamentally different resources than the PCs. Chain of command means their boss is someone important in the city, so the boss has better contacts with the upper crust. They have a list of 'the usual suspects'. On the flip side, PCs aren't as restrained by laws & police procedure. Do the police have a procedure for "tentacled goop fell from the sky and is taking control of people"?