I personally think the ape and the big demon were a bit harder than good old Isshin, which felt only "reasonably" hard to me, but these things are often different for everyone else anyway.
Still, compared to Dark Souls, Sekiro actually puts their save points really close to the bosses, so that was something to be immensely grateful for.
Yeah, and it's especially exacerbated by the fact that it's a roguelike and you'd have to grind another hour-and-a-half (and that if everything else goes well) just to have another go at it. I always found Stealth and Teleporter to work well at it, though.
***
My contender would be the Nuclear Throne, from Nuclear Throne, honestly. There are times that I cruised up to the final level, no problems, only to die in the first five seconds to it.