I've considered it many times. I also heavily considered 5e Without Ability Scores-- skills, Skills, Skills. It was one sacred cow that I couldn't get my players to go for.
I think it would likely improve the game if the GM had players on board with the idea.
___
Removing armor requirements will be a fine change. The reason the developers put it there was likely to prevent a monk or wild shape druid multiclassing to 1 fighter for +1 AC, but as long as those kind of cases don't occur it should be fine.
A dagger now does 4.5 average damage which is the same as a longsword. That doesn't work for me. :S