Fallout 3 is where the 'modern' iteration of the franchise under Bethesda starts. It established the structure of the games as FPS-RPG hybrids in 3D open-world environments. Fallout: New Vegas operates basically as FO3 with a few minor gameplay and graphical modifications (most notably the color palate due to the shift in environment) and a significantly more vibrant and lived in world with better side quests and NPC interactions. Fallout 4 is similar to FO3 but includes significant upgrades to the FPS elements, a rebuild of the RPG elements, a much larger open world, and a 'settlement management' system that allows for limited base-building. However, the story is generally considered to be inferior to both prior games and certain design choices regarding things like leveling and items make the game feel very sloggy at times and the settlement system is broadly irrelevant to gameplay unless significantly modded. The most recent game in the series Fallout 76, is a massive open-world quasi-survival game that, though it had some gaping holes at launch has been drastically improved over time (and yes, can be played single player), but has a distinct bias to the shooter elements and requires careful research by the player to insure they produce a character with a viable build. However, while it has arguably the worst gameplay and an extremely passive story, has perhaps the best vision of the apocalypse of any of the games.

It should be noted that FO3, FO:NV, and FO4 are all highly moddable games, and playing FO3 and FO:NV especially without at least some mods enabled - just basic quality of life ones to upgrade the UI and such at least - is going to be a painful experience. However, successfully setting up mods for these games is not exactly a simple or swift enterprise.

In terms of where to start, it depends on your interest, patience, and how much you're willing to invest. FO76, for all its problems (though it's much, much better than it was at launch) is the easiest game to just boot up and go and to give you a taste of interest in the general vibe of Fallout in terms of both gameplay and setting. There are also, due to the TV series, various promotions to get your hands on it for free right now. Jump in, bounce around a bit, shoot some things, do some basic quests, and that will provide a feel of whether the games are worth playing, since any deep entry into any of the others will involve dozens of hours of roughly the same gameplay. Otherwise, I'd recommend starting with FO4, which has a pretty good opening act and can be played acceptably well without delving into the mod space.