So, Phantasmal Force, and Illusions in general, do actually have some fairly tight restrictions. These are easy to miss, as I did my self for a few sessions, and it seemed very over powered. Some of these restrictions are in the spell descriptions (PHB/UA), some are in the DMG spell rulings and others acre just scattered around.

First, if you have never seen the creature or spell, you will do a very bad job of making an illusion of it. (DMG-45) This likely gives everyone a chance to save when it's cast, as it looks terrible. Also, while not specifically stated, if you have never seen such a monster fight, you are unlikely to make it believable. This is transferable to spells. If you have never seen a fireball you are unlikely to make a believable one. The general rule I have played with is if you see a spell cast by a 6th level caster, you can make THAT spell at THAT level and be believable.

Second, there are size restrictions. A first level character may be able to make a dragon, but at 1/5 or 1/10 scale. Pretty much everyone will call BS on such a Dragon. (That said, if you are in a small cave, there's no reason you can't make a large creature crouch to get around the size restriction and still be believable)

Third, quality of Illusion. The first level spell does not have sound, smell or thermal components. This means that the illusionary fireball does not burn, the illusionary soldier's sword doesn't ring when it parries and the illusionary monster doesn't howl in pain when struck. While people/monsters may trust their eyes at first, something is going to feel "off" and NPCs are going to start disbelieving real fast. Also, if one disbelieves, he can warn others and they all get a +4 to a roll. (Using Audible Glamer to give the spell an audio component is actually recommended in the PHB!) This means illusionists open strong(scaring the crap out of people with extra monsters), then can go down hill in a hurry.

Fourth, the low level spells require you to concentrate, so you can't move while keeping the illusion alive. If the fight goes around the corner, your illusion cannot react properly and will be found out quickly. At higher levels you can follow it a bit, or even have it fully automated.

Fifth, some things are straight up immune to illusions. This makes a bad day for an illusionist.

As you go up in level you will encounter bigger more dangerous monsters, better spells, and improved versions of the spells. This gives a reasonable power curve as you advance in level.

Illusionists are fun to play, but take a bit of research and creativity to play properly. Illusionists are a real pain in the butt to DM though. The DM knows it's an illusion, but the NPCs don't, so the DM needs to come up with a justification that the NPC might disbelieve what he sees.