PDA

View Full Version : Movies Top movies according to Metacritic



Darth Credence
2016-11-30, 05:37 PM
So, I was reading an article complaining about Rotten Tomatoes, and how it doesn't tell you much. The author went on about how much better Metacritic is, because they give actual scores instead of a yes/no binary, and they have selected critics that they look at. I rolled my eyes a little, as you can also get scores on RT, but whatever. The article contained a link to Metacritics top movies of all time (http://www.metacritic.com/browse/movies/score/metascore/all/filtered), which they used to demonstrate Metacritic is better because they have only given out 3 perfect scores ever.

So I clicked on the link out of curiosity. First thing I see is The Godfather as the number one movie. Sure, that seems reasonable. Number 2 is Boyhood. This was a bit weird to me, as I found it to only be a gimmick and not very good otherwise, but others have liked it, so whatever. Number 3 is a movie I've never seen, although I am aware of the trilogy.

But I wondered - where is The Godfather Part II, or The Shawshank Redemption, or Casablanca, or Citizen Kane, or any other movies that consistently rank high on best movie lists. Well, for Casablanca and Citizen Kane, Metacritic hasn't even put them in the system. But they do have The Godfather Part II and Shawshank in there. At 790 and 860, respectively. I think this needs repeating - according to Metacritic, there are 789 movies, not including older movies that they haven't ranked, that are better than The Godfather Part II. These wonderful movies include Sideways (ranked as the 30th best movie of all time!), Tootsie, Avatar, and a whole lot of other movies that have no business being there. Shawshank sits right next to The Empire Strikes Back, both of them considered to be worse movies than Mad Max Beyond Thunderdome. If anyone can explain to me what kind of a system can cause that to happen, I'd love to hear it.

I can't get over this. I cannot believe that there are people who use Metacritic as a reference for movies, when they consider The Incredibles to be #6 on the list of Pixar movies.

Avilan the Grey
2016-11-30, 05:45 PM
Metacritic and sites like it (Rotten Tomatoes et al) are a huge problem. The "Average Score" thing really takes nothing in consideration except the actual number, and they even "calculate" a number for sites that does not, in fact, use scores for their review.

Lethologica
2016-11-30, 05:50 PM
Rating aggregators have their uses, but ranking movies is emphatically not one of them.

eggynack
2016-12-01, 02:02 PM
Rating aggregators have their uses, but ranking movies is emphatically not one of them.
Yeah, that was my thinking on it. Godfather only had 14 reviews included. Even ignoring the fact that that low number allows for fewer things to "go wrong" in the quest for 100, and the fact that their weighting is apparently strong enough to call an 80 close enough when it's from a "not credited" source, the reality is that that's not that many datapoints, and minor irregularities amongst critics relative to some theoretical hive mind of film quality determination can have decently sized impact on film score. Like, look at the actual ratings for Godfather part II. It's seven really positive reviews, two that are kinda low, and a super aberrantly low New York Times review. If you want to blame the film's poor ranking on something, stop blaming Metacritic and probably start blaming Vincent Canby.

Or maybe don't blame anyone, cause film evaluation is necessarily going to be somewhat wonky. Sure, you've found this often relied upon source that produces these weird results, but find me a critic and I'll find you some relative ratings that go way against what I'd consider reasonable. Critics are imperfect, so no system based upon critics could possibly be perfect. Jeez, even the late, great, Ebert utterly failed to understand The Graduate two separate times, first thinking that the protagonists are treated by the film as in the right, and second, thirty years later, thinking that the protagonists are secretly non-transgressive jerks and that this is a failing of the movie. Of course, proper analysis would find that the characters are indeed empty of the true rebelliousness that they lay claim to, and that this is a major part of the movie, but Ebert had a huge blind spot here, and such things are to be expected. Create a new site which inexplicably aggregates critical analysis of a film into one true interpretation, and Ebert would skew the results, just as critics are generally going to skew simple numbers.

Fri
2016-12-01, 09:21 PM
Metacritic and sites like it (Rotten Tomatoes et al) are a huge problem. The "Average Score" thing really takes nothing in consideration except the actual number, and they even "calculate" a number for sites that does not, in fact, use scores for their review.

It's not a problem, if you consider what they're actually meant for. They're never meant to "rank" a movie from the best to the worst for whatever purpose (for example, making you being able to say "my favourite movie/game is number one movie/game ever." )

It's to see what the general public/reviewer think of an actual piece of movie/game. As long is generally in the right position, it works for the intended purpose (Like, if say a movie score 90 in metacritic/wherever, that means generally public really like that movie. The exact single digit is not that important).

Similar to steam "recommended/not recommended" aggregate. It's to see "oh, 60% of the player recommend this game."

What you use with that information is up to you.

eggynack
2016-12-01, 09:46 PM
It's not a problem, if you consider what they're actually meant for. They're never meant to "rank" a movie from the best to the worst for whatever purpose (for example, making you being able to say "my favourite movie/game is number one movie/game ever." )

It's to see what the general public/reviewer think of an actual piece of movie/game. As long is generally in the right position, it works for the intended purpose (Like, if say a movie score 90 in metacritic/wherever, that means generally public really like that movie. The exact single digit is not that important).

Similar to steam "recommended/not recommended" aggregate. It's to see "oh, 60% of the player recommend this game."

What you use with that information is up to you.
True, though you must admit, 80% for Godfather part II seems rather aberrantly low, on a pure individual ratings basis. The eventual nature of the post I've already made here was based on the fact that I checked that movie's score to reinforce your general premise, that ratings hang out in generally accurate error bars, and that position is highly variable depending on where the rating falls within those bars, but that rating is weird, and others probably are too. Still, as I've noted, the wider than expected error isn't, well, all that unexpected. We can generally tie the error back to clear critical events, and it's not all tied up in something intrinsic to Metacritic. At least not strictly.

Legato Endless
2016-12-01, 10:42 PM
The "Average Score" thing really takes nothing in consideration except the actual number, and they even "calculate" a number for sites that does not, in fact, use scores for their review.

This bears repeating. Metacritic isn't by any means methodologically solid. It's not an aggregate score based on actual subjective opinions, it's an aggregate that stuffs a bunch of competing subjective opinions through a subjective filter to arbitrarily normalize a variety of different rating systems into a singular score. Data requires context, and Metacritic makes a lot of assumptions before it gives you that final 'weighted' average. Imdb suffers from less a systemic error and more the reviews it aggregates suffer from the usual caveats of the internet, from the sexism to polarization.

If aggregate sites were some holy grail, we'd kind of presume they'd match up to each other more than they do, even if one ignores the critic-fan divide.

https://espnfivethirtyeight.files.wordpress.com/2015/10/hickey-datalab-fandango-2.png?quality=90&strip=all&w=575&ssl=1

Knaight
2016-12-01, 10:54 PM
Metacritic and sites like it (Rotten Tomatoes et al) are a huge problem. The "Average Score" thing really takes nothing in consideration except the actual number, and they even "calculate" a number for sites that does not, in fact, use scores for their review.

Rotten Tomatoes doesn't use an average score - they report the proportion of reviews that are positive. If a movie got 100 8/10 reviews that wouldn't get translated to an 80% in rotten tomatoes - that would be 100%. It's a different metric, and it works just fine if it's used for intended uses and not as some sort of greatest movie of all time ranking system. Metacritic uses averages, and it's a bit messier there as the lack of a standard scale causes much more problems when you can't work within existing scales to compile a metric, but it still works fine, it just doesn't work as some sort of idealized movie sorting system.

Razade
2016-12-01, 10:59 PM
So, I was reading an article complaining about Rotten Tomatoes, and how it doesn't tell you much. The author went on about how much better Metacritic is, because they give actual scores instead of a yes/no binary, and they have selected critics that they look at. I rolled my eyes a little, as you can also get scores on RT, but whatever. The article contained a link to Metacritics top movies of all time (http://www.metacritic.com/browse/movies/score/metascore/all/filtered), which they used to demonstrate Metacritic is better because they have only given out 3 perfect scores ever.

So I clicked on the link out of curiosity. First thing I see is The Godfather as the number one movie. Sure, that seems reasonable. Number 2 is Boyhood. This was a bit weird to me, as I found it to only be a gimmick and not very good otherwise, but others have liked it, so whatever. Number 3 is a movie I've never seen, although I am aware of the trilogy.

But I wondered - where is The Godfather Part II, or The Shawshank Redemption, or Casablanca, or Citizen Kane, or any other movies that consistently rank high on best movie lists. Well, for Casablanca and Citizen Kane, Metacritic hasn't even put them in the system. But they do have The Godfather Part II and Shawshank in there. At 790 and 860, respectively. I think this needs repeating - according to Metacritic, there are 789 movies, not including older movies that they haven't ranked, that are better than The Godfather Part II. These wonderful movies include Sideways (ranked as the 30th best movie of all time!), Tootsie, Avatar, and a whole lot of other movies that have no business being there. Shawshank sits right next to The Empire Strikes Back, both of them considered to be worse movies than Mad Max Beyond Thunderdome. If anyone can explain to me what kind of a system can cause that to happen, I'd love to hear it.

I can't get over this. I cannot believe that there are people who use Metacritic as a reference for movies, when they consider The Incredibles to be #6 on the list of Pixar movies.

To be fair. A lot of "BEST MOVIES EVER" lists only ever cover films from the United States. All your examples are films from the U.S (Except Mad Max. That's Australian). I'm surprised Ran outperformed Seven Samurai on Metacritic's list but it certainly deserves to be on there somewhere.

Gastronomie
2016-12-01, 11:24 PM
To be fair. A lot of "BEST MOVIES EVER" lists only ever cover films from the United States. All your examples are films from the U.S (Except Mad Max. That's Australian). I'm surprised Ran outperformed Seven Samurai on Metacritic's list but it certainly deserves to be on there somewhere.Any "best movies ever" list without Godzilla Resurgence is fundamentally wrong. [/Evangelical rant]

tomandtish
2016-12-01, 11:30 PM
To be fair. A lot of "BEST MOVIES EVER" lists only ever cover films that are popular in the United States. All your examples are films from the U.S (Except Mad Max. That's Australian). I'm surprised Ran outperformed Seven Samurai on Metacritic's list but it certainly deserves to be on there somewhere.

Fixed that for you. :smallwink: Even lists that have films from other places are usually only taking a US based perspective on them.

The simple truth is: there's no objective way to make a list of best films ever. Reviews? They are subjective. Box office gross? Well, you'd have to account for inflation (fairly easy to do), variations in ticket price (3D/2d, matinee, etc., not easy to do since theaters don;t report on that), and also repeat viewing (not at all easy to do). Internet pollin... Sorry, couldn't stop laughing.

The best you can hope for from any "Best Of" list (pun intended) is that they give you enough explanation of their methodology to decide how much weight you want to give it. In most cases, that should be fairly little.


I can't get over this. I cannot believe that there are people who use Metacritic as a reference for movies, when they consider The Incredibles to be #6 on the list of Pixar movies.


So would you rate it higher or lower?

I love the Incredibles, and have it as 3rd on my list. But I could see Toy Story (1-3), Finding Nemo, and Cars beating it out in an overall popularity contest. Really, it's just in what formula you want the plot....

Pixar, 1995: What if toys had feelings-Toy Story (1995 movie)

Pixar, 1998: What if bugs had feelings-A Bug's Life (1998 movie)

Pixar, 2001: What if monsters had feelings-Monsters Inc. (2001 movie)

Pixar, 2003: What if fish had feelings- Finding Nemo (2003 movie)

Pixar, 2004: What if superheroes had feelings-The Incredibles (2004 movie)

Pixar,2006: What if cars had feelings- Cars (2006 movie)

Pixar, 2007: What if rats had feelings-Ratatouille (2007 movie)

Pixar, 2008: What if robots had feelings- WALL-E (2008 movie)

Pixar, 2009: What if dogs had feelings-Up (2009 movie)

Pixar, 2012: What if Scotland had feelings-Brave (2012 movie)

Pixar, 2015: What if feelings had feelings-Inside Out (2015 movie).

Razade
2016-12-01, 11:43 PM
Fixed that for you. :smallwink: Even lists that have films from other places are usually only taking a US based perspective on them.

The simple truth is: there's no objective way to make a list of best films ever. Reviews? They are subjective. Box office gross? Well, you'd have to account for inflation (fairly easy to do), variations in ticket price (3D/2d, matinee, etc., not easy to do since theaters don;t report on that), and also repeat viewing (not at all easy to do). Internet pollin... Sorry, couldn't stop laughing.

These are both true. Not just because Objectivity is in itself subjective but ya know....that's a different discussion.



Pixar: What if X had feelings joke

I too go on the internet. (https://www.buzzfeed.com/simoncrerar/what-if-feelings-had-feelings?utm_term=.iaMWD7gJdD#.xskYGE79QG)

Aedilred
2016-12-02, 12:36 AM
I find Rotten Tomatoes preferable precisely because it doesn't attempt to account for how much a critic liked a film, only whether the review was positive or negative. The downside is of course that it doesn't account for the quality of that opinion and it's possible for reviewers who know their stuff to be drowned out by a cloud of worse ones, as well as the occasional critic panning an excellent film apparently just so they can get their review noticed and hence damaging its score, but as a one-stop-shop to tell you whether a film is likely to be enjoyable, you could do a lot worse. Especially since it links the reviews in question if you want to find out more.

I don't really get on with star ratings and the like in the first place and they're not comparable from critic to critic (some will mark out of the full range, others only use the 6-10/3-5 for 95% of reviews) so Metacritic has lost me there before it even starts, really.

The other issue which faces all review aggregators is that they tend only to deal with more recent reviews which are available online. Check out The Godfather on Rotten Tomatoes, for instance: the earliest reviews on there are from about 2000. The score might be a good judge of how well the film has aged and what current critics think of it, but how many films from 1972 are still being regularly reviewed in the 21st century? The system is skewed in favour of those films which are already considered classics, because they're more likely to be picked up and re-reviewed, whether to prove a reviewer's chops or for a DVD release, one-off big screen return or the like. But only a tiny handful of films get that sort of treatment.

I think that is why films from the 21st century dominate the Metacritic list, and lists like it. It's not deliberate, nor necessarily reflective of critical opinion, just that the way the data is selected and aggregated inherently favours newer releases. If you select "top by year" and scroll down you'll see they don't even offer such lists for years before 1995, so the whole of cinema's Golden Age and the glorious seventies are clearly underrepresented.

With regard to Shawshank at least, I think it's possible to get carried away. It's certainly a very good film, but, without wishing to sound overly snobbish, it tends to be a favourite film of people who don't watch a lot of films/ As a result I'm not surprised to see it has scored less well with critics than it does with popular conception (see that the users rate it about 10% higher than the critics do). Somewhere between those figures is probably fair. I suspect it was slightly marked down by critics precisely because of its popularity. I also suspect it suffers partly from the above problem with regard to age: that score is aggregated from a mere twenty reviews, which means a couple of bad ones can really hit its overall rating.

Legato Endless
2016-12-02, 01:35 AM
Pixar, 2012: What if Scotland had feelings-Brave (2012 movie)

Overly reductive comedy notwithstanding, this line doesn't really make any sense.

/nitpick

Razade
2016-12-02, 01:46 AM
Overly reductive comedy notwithstanding, this line doesn't really make any sense.

/nitpick

Most things stolen from Tumblr rarely do.

Fri
2016-12-02, 03:21 AM
Overly reductive comedy notwithstanding, this line doesn't really make any sense.

/nitpick

On the other hand, for me that nonsensical line is what makes the joke. Basically I see it as how the premise get more ridiculous until you arrive at "what if feeling had feelings".

I'll put Up as "what if old people had feeling" to improve the joke from how I see it.

factotum
2016-12-02, 03:38 AM
I prefer the IMDB user ratings for this sort of thing, because it gives an overall score of what regular people who watched the movie thought of it and doesn't include critic opinions--because the problem with movie critics, IMHO, is that they watch lots of movies and tend to form slightly different opinions because of it compared to people who maybe watch one or two a week. Just to give an idea where the movies mentioned in the OP are in their top 250 list:

The Godfather: #2
The Godfather, part 2: #3
Shawshank Redemption: #1
Citizen Kane: #68
Casablanca: #33

GloatingSwine
2016-12-02, 08:49 AM
Rotten Tomatoes doesn't use an average score - they report the proportion of reviews that are positive. If a movie got 100 8/10 reviews that wouldn't get translated to an 80% in rotten tomatoes - that would be 100%. It's a different metric, and it works just fine if it's used for intended uses and not as some sort of greatest movie of all time ranking system. Metacritic uses averages, and it's a bit messier there as the lack of a standard scale causes much more problems when you can't work within existing scales to compile a metric, but it still works fine, it just doesn't work as some sort of idealized movie sorting system.


To be honest, this is why RT is useful and Metacritic is garbage.

The concept that you can accurately score something as subjective as a piece of entertainment on a scale from 1 to 100 is facile and wrongheaded.

RT's approach of "most people liked this" or "most people did not like this" is actually way more informative than "this is a 73, that is better than a 72 but not by any degree any human being could ever confidently explain"

Ruslan
2016-12-02, 12:00 PM
I think GloatingSwine hit the nail on the head. I agree that the difference between "I liked it" and "I didn't like it" is a lot more useful and informative than the difference between 6/10 and 9/10.

I cannot believe that there are people who use Metacritic as a reference for movies, when they consider The Incredibles to be #6 on the list of Pixar movies.Out of curiosity, can you please clarify whether you consider this too high or too low?

Knaight
2016-12-02, 06:40 PM
The concept that you can accurately score something as subjective as a piece of entertainment on a scale from 1 to 100 is facile and wrongheaded.

RT's approach of "most people liked this" or "most people did not like this" is actually way more informative than "this is a 73, that is better than a 72 but not by any degree any human being could ever confidently explain"
I wouldn't necessarily agree. I do prefer the RT approach, largely because something like "73% of people liked this" is a much more solid metric based on people reliably using a two point scale the same way in a way they really don't once more points show up. With that said, metacritic isn't really saying that "this is a 73". It's more along the lines of "when people use a 5 or 10 point scale they average 3.65 or 7.3. The problem is that while there's really only one 2 point scale there's a whole bunch of different 5 and 10 point scales. Some 10 point scales are loosely based on the classical A-F scale and a 7 is considered average, others place the average at 5. If the distribution of ratings systems are reasonably consistent then this eventually creates a new aggregate scale to learn, but with fewer ratings this makes it less reliable.

Again though, this isn't a big deal. The problem here is usage, where people see a 73 and a 72 and assume that the 73 is better than the 72, rather than sufficiently close to be uncertain. Error bars are the obvious analogy here - if you've got a measurement of two different things you're only confident that the higher measured number actually corresponds to a higher true value if the lower error bar of the higher value is above the higher error bar of the lower value (for whatever the decided certainty is). That doesn't make sorting by score useless, it just makes treating that sorting as some sort of list of definitive bests bizarre, particularly given the amount of tie breaking that has to involve either a secondary characteristic as a tie breaker or using decimal scores that aren't even shown. There are at time of writing 114 movies on metacritic with a 90+ score. If an uncertainty of +/- 5 is assumed (this is, for reference a 1/4 star on a 0-4 star system), that goes from a list of 114 movies to 114 movies any one of which could be the actual best. There are 763 movies in the 80-90 range, and again, if even a 1/2 star of uncertainty (1/4 in each direction, so it's slightly less than that in this particular range) is assumed there's nothing reliable said about the position of any of them relative to each other. The 70-80 range has 1698 movies, which again could all be ranked 1-1698 relative to each other with that 1/2 star of uncertainty.

The 1/4 star is arbitrary, but it demonstrates something useful. In terms of evaluating an individual film, getting something like 3-3 1/2 stars is perfectly adequate. That's absolutely fine as information, and then there's links to individual reviews and such. If used as a ranking system, it produces a titanic mess for things reasonably close to each other. As such, seeing weirdness in the ranking doesn't really mean much. Take the pixar example again - the Incredibles got a 90/100, which is 3.5/4 stars on a 0-4 scale. Someone assuming the 1/2 star range of fairly minimal error would look at that and conclude that the Incredibles is a good to great movie. That sounds like Metacritic doing fine to me, and while you could object by the ranking system, it's worth noting that the same 1/4 star spread doesn't reliably give 6th. Incredibles is at 90, and can be assumed to deserve from 85-95 with that range. On top of that something getting 100 could be as low as 95 and thus in the Incredibles's range, and something as low as 80 could be 85 and thus within the Incredibles's range. The ratings place it somewhere in the top 9, and that seems pretty reasonable.

There's information there, and it's useful. Keep in mind that there's a certain uncertainty and that your personal tastes aren't going to line up perfectly with aggregate tastes anyways, and you're fine, much like on Rotten Tomatoes. Both are totally usable sites, and while I personally favor the Rotten tomatoes scale and am less than impressed with the paraphrased arguments about Metacritic being better in the article the OP refers to, I'm also really not convinced by the OPs arguments about how bad Metacritic is. That claim that Metacritic says there are 759 movies better than the Godfather 2 goes down to 79 if only +/-5 uncertainty is allowed. At +/- 6 it's down to 45. At +/-7 it's down to 18. At +/- 8 it's 7, at +/-9 it's 4, and at +/- 10 it's 0. The case of the Incredibles has been covered above. All of these signs that Metacritic is terrible vanish the instant a minimal level of uncertainty is allowed.

Avilan the Grey
2016-12-03, 02:44 AM
I am sure this has been brought up before, but as far as I know there is no way for a site like that to take different (yet seemingly similar) rankings in consideration.

One critic may use the full 10, not giving a truly awful movie more than 2, say. Another may in practice only us a scale of 5-10, and use 1-4 as novelty scores, almost for only the worst of the worst.
And then the first person gives a movie a 6, and the second a 7, and in reality the first one ranks it higher.

Knaight
2016-12-03, 03:59 AM
One critic may use the full 10, not giving a truly awful movie more than 2, say. Another may in practice only us a scale of 5-10, and use 1-4 as novelty scores, almost for only the worst of the worst.
And then the first person gives a movie a 6, and the second a 7, and in reality the first one ranks it higher.

If both critics are reliably represented though that still creates a distinctive scale in the aggregate - and that's for Metacritic. Rotten Tomatoes bypasses this entirely by using a simple positive-negative per reviewer and aggregating a lot of them together.

GloatingSwine
2016-12-03, 12:47 PM
Again though, this isn't a big deal. The problem here is usage, where people see a 73 and a 72 and assume that the 73 is better than the 72, rather than sufficiently close to be uncertain.

And once you agree that 73 and 72 are close enough to be uncertain, you use a less idiotic rating system in the first place.

To be honest, I can't see a strong argument for anything more than a three or maybe four point scale. Something is either really bad and you should avoid it, really good and you should watch it, or somewhere in the middle and you'll have to make your own mind up based on your existing preferences.

dps
2016-12-03, 02:58 PM
Regardless of anything else, if I had a rating system that said that there are 788 movies better than The Godfather, Part II but worse than The Godfather, I'd redesign it. While I would agree that the original is better than the sequel, they're not that far apart in quality.

Yora
2016-12-03, 03:39 PM
Rankings have nothing to do with quality but are all about popularity. A movie that is pretty good to most people will rank higher than one that is highly loved by fans of the specific niche but seen as meh by the masses.

When you have works of art competing for specific spots on a ladder, "best work" is a really meaningless category. "Widest appeal" is really the only thing you can measure and rank.

Knaight
2016-12-05, 01:06 AM
Regardless of anything else, if I had a rating system that said that there are 788 movies better than The Godfather, Part II but worse than The Godfather, I'd redesign it. While I would agree that the original is better than the sequel, they're not that far apart in quality.

I'd argue that a) it doesn't say that, and b) there's enough movies that it's entirely plausible that there are 788 between them anyways.


To be honest, I can't see a strong argument for anything more than a three or maybe four point scale. Something is either really bad and you should avoid it, really good and you should watch it, or somewhere in the middle and you'll have to make your own mind up based on your existing preferences.
In the context of a scale used by one person, I agree*. In the context of aggregate data though, it doesn't hurt to have more. Rotten Tomatoes reporting that 63% of reviewers liked something seems fine to me, you just take that into consideration using your own personal scale that probably doesn't have so many increments. I could see more than four points being useful, just not 100. For instance:
0 - Absolute garbage.
1 - Bad, but possibly worth watching for the explicit purpose of watching a bad movie.
2 - Bad, but worth watching if you enjoy the particular underrepresented sub genre a great deal.
3 - Decent, and worth watching for people who generally like movies and watch a lot of them or it's in genre.
4- Good, and worth watching for people who are pickier.
5- Great, and worth watching even if you dislike the genre its in as a whole.
6- An outright masterpiece

That's a 7 point scale there, and those are all meaningfully distinct categories in a way that 72 vs. 73 really isn't.

*With a caveat that detail emerging from how category averages pan out is a different matter.

Chen
2016-12-05, 07:48 AM
Rotten Tomatoes can give some ridiculous skewing if the film is right on the borderline of like/dislike though. Dawn of Justice for example. 27% critical approve/disapprove rating, average numerical rating 4.9/10. That's a pretty big difference.

GloatingSwine
2016-12-05, 08:47 AM
Rotten Tomatoes can give some ridiculous skewing if the film is right on the borderline of like/dislike though. Dawn of Justice for example. 27% critical approve/disapprove rating, average numerical rating 4.9/10. That's a pretty big difference.

Not really. Both of them say "most people do not like this movie".

And that's literally the only useful information that comes out of a ratings aggregator.




In the context of a scale used by one person, I agree*. In the context of aggregate data though, it doesn't hurt to have more. Rotten Tomatoes reporting that 63% of reviewers liked something seems fine to me, you just take that into consideration using your own personal scale that probably doesn't have so many increments. I could see more than four points being useful, just not 100. For instance:
0 - Absolute garbage.
1 - Bad, but possibly worth watching for the explicit purpose of watching a bad movie.
2 - Bad, but worth watching if you enjoy the particular underrepresented sub genre a great deal.
3 - Decent, and worth watching for people who generally like movies and watch a lot of them or it's in genre.
4- Good, and worth watching for people who are pickier.
5- Great, and worth watching even if you dislike the genre its in as a whole.
6- An outright masterpiece


Yeah, but how usefully different are your 0-2 anyway? If a movie is so bad it's good, or is bad but people who are really invested in the genre will watch it anyway because they have no other options, the movie is still bad and you can happily say it's bad and people shouldn't watch it, and people who fit into the niche categories you identified will make their own mind up.

And your 4 & 5 are pretty similar too. The distinction between "not fan of the genre" and "sort of likes the genre but is very picky within it" isn't actually amazingly massive.

Legato Endless
2016-12-05, 11:12 AM
And once you agree that 73 and 72 are close enough to be uncertain, you use a less idiotic rating system in the first place.

To be honest, I can't see a strong argument for anything more than a three or maybe four point scale. Something is either really bad and you should avoid it, really good and you should watch it, or somewhere in the middle and you'll have to make your own mind up based on your existing preferences.

I think actually while an individual reviewer can get away with a binary (watch/don't watch) or trinary (good/bad/caveats) an aggregate review demands a four point differentiation. Largely because good vs. excellent becomes meaningful at that point.

A 50% split of ten people where half say it's good versus half saying the film is incredible is frequently a different animal. Hidden gems or underappreciated upon release is often enough around that 50-60 approval rating, (because some of the critics were hidebound) and a wild approval from those who do recommend it is the only indicator I've found to reliably imply this.

I wish instead of the averaged value rating we got a bar graph indicating whether the film was polarzing vs. mediocre.

Kato
2016-12-05, 12:02 PM
I love the Incredibles, and have it as 3rd on my list. But I could see Toy Story (1-3), Finding Nemo, and Cars beating it out in an overall popularity contest.
Hahahaha... Really? That old movie with Michael J Fox except with cars? Above Incredibles or Wall-E or even Inside Out? Uhm, okay. (no, I'm kidding, like whatever you want but even as a kid I couldn't quite see that movie's appeal.)



When you have works of art competing for specific spots on a ladder, "best work" is a really meaningless category. "Widest appeal" is really the only thing you can measure and rank.

Lies. Taste it totally objective. I might need twenty hours to convince you that Green Lantern is truly an under appreciated masterpiece but after that time alone with me talking about nothing else you will clearly see my point!



More seriously, I agree with people saying the binary system. Yes, good taste is not binary, a good critic is aware you will not think as he does and you will need a more detailed judgement than yay or nay. But until you find a critic with exactly your taste, you're better of relying on an average over those than an average of numbers that only partially agree with each other.

Darth Credence
2016-12-05, 01:12 PM
I think GloatingSwine hit the nail on the head. I agree that the difference between "I liked it" and "I didn't like it" is a lot more useful and informative than the difference between 6/10 and 9/10.
Out of curiosity, can you please clarify whether you consider this too high or too low?

I was wrong - it is actually 7th. I find it too low. Especially when the ones above it are Ratatouille, Toy Story, WALL-E, Inside Out, Toy Story 3, and Finding Nemo. I would put the first and last near the bottom of Pixar's movies, and WALL-E and Inside Out as in the middle range. I realize that this is subjective, but that is the lowest I have ever seen the Incredibles rank.


I'd argue that a) it doesn't say that, and b) there's enough movies that it's entirely plausible that there are 788 between them anyways.
...

How does it not say that? The movies are ranked, and there are 788 movies in between them, so I don't get how you can argue that it is not, in fact, saying that. Unless you are saying that that is the minimum separation, and it could be more if you add in movies that Metacritic doesn't list.

Lethologica
2016-12-05, 01:46 PM
I was wrong - it is actually 7th. I find it too low. Especially when the ones above it are Ratatouille, Toy Story, WALL-E, Inside Out, Toy Story 3, and Finding Nemo. I would put the first and last near the bottom of Pixar's movies, and WALL-E and Inside Out as in the middle range. I realize that this is subjective, but that is the lowest I have ever seen the Incredibles rank.
Your choice to put Ratatouille and Finding Nemo near the bottom of Pixar's movies is more surprising than Metacritic putting Incredibles in the middle, tbh. The core of Pixar's oeuvre (which, to me, includes all of the above movies plus Monsters, Inc. and Toy Story 2) is very consistent in quality, so it's hard to rank them. But the bottom of Pixar's movies is Cars 2, Monsters U, and The Good Dinosaur--good luck finding people who rank Nemo and Ratatouille around that level.


How does it not say that? The movies are ranked, and there are 788 movies in between them, so I don't get how you can argue that it is not, in fact, saying that. Unless you are saying that that is the minimum separation, and it could be more if you add in movies that Metacritic doesn't list.
Knaight's argument is that the ratings have implied error bars, and that ratings whose distance is less than the error don't clearly count as 'better' or 'worse' than each other.

Knaight
2016-12-05, 01:59 PM
Knaight's argument is that the ratings have implied error bars, and that ratings whose distance is less than the error don't clearly count as 'better' or 'worse' than each other.

Pretty much, although it's more that there is a certain level of error regardless (as in any measurement), and as per any comparison if the measurements are within each others error it's not safe to say that one is higher than the other. My argument is more that the amount of error actually needed to cause that number to plummet is tiny.

137beth
2016-12-05, 05:31 PM
Yeah, that was my thinking on it. Godfather only had 14 reviews included. Even ignoring the fact that that low number allows for fewer things to "go wrong" in the quest for 100, and the fact that their weighting is apparently strong enough to call an 80 close enough when it's from a "not credited" source, the reality is that that's not that many datapoints, and minor irregularities amongst critics relative to some theoretical hive mind of film quality determination can have decently sized impact on film score. Like, look at the actual ratings for Godfather part II. It's seven really positive reviews, two that are kinda low, and a super aberrantly low New York Times review. If you want to blame the film's poor ranking on something, stop blaming Metacritic and probably start blaming Vincent Canby.

Or maybe don't blame anyone, cause film evaluation is necessarily going to be somewhat wonky. Sure, you've found this often relied upon source that produces these weird results, but find me a critic and I'll find you some relative ratings that go way against what I'd consider reasonable. Critics are imperfect, so no system based upon critics could possibly be perfect. Jeez, even the late, great, Ebert utterly failed to understand The Graduate two separate times, first thinking that the protagonists are treated by the film as in the right, and second, thirty years later, thinking that the protagonists are secretly non-transgressive jerks and that this is a failing of the movie. Of course, proper analysis would find that the characters are indeed empty of the true rebelliousness that they lay claim to, and that this is a major part of the movie, but Ebert had a huge blind spot here, and such things are to be expected. Create a new site which inexplicably aggregates critical analysis of a film into one true interpretation, and Ebert would skew the results, just as critics are generally going to skew simple numbers.
Pretty much all of this.


In Stephen Sondheim's memoir Finishing the Hat, he gives a really good analysis of "awards" in the arts, and why he believes they are essentially meaningless. I'm not even going to try paraphrasing it, because I couldn't do it justice: Sondheim is a spectacular writer and I am not (And you know he must be a great writer, since he has a Pulitzer prize and nine Tony awards as a lyricist:smalltongue:)

A lot of Sondheim's critiques of awards also apply to review aggregation, with the added issue that review aggregation isn't actually designed to produce a "best" ranking list in the first place.


Besides, Rotten Tomatoes rankings are obviously all wrong, because <my favorite movie> should be WAY higher. Also, <some movie I didn't like> totally doesn't deserve that many good reviews. It should be way lower. Bah, that just proves that the critics are wrong about everything.

Blackhawk748
2016-12-05, 05:41 PM
They have Gravity as the 12th best movie of all time. Let that tell you something.

Really i never muched care for these rating agregates, and what happened with Suicide Squad just reinforced it for me. In short, i like movies that critics dont like (Suicide Squad) and i like movies that they do like (Spirited Away), so i pretty much just ignore them

Aedilred
2016-12-05, 11:28 PM
Regardless of anything else, if I had a rating system that said that there are 788 movies better than The Godfather, Part II but worse than The Godfather, I'd redesign it. While I would agree that the original is better than the sequel, they're not that far apart in quality.
But then, what's the range? Given the number of films released each year, a ranking in the top thousand films of all time probably puts them within 1% of each other, even excluding foreign-language films, documentaries, porn, etc: given that The Godfather is ranked #1, that would be one in the hundredth percentile and one in the 99th. That's pretty hard to argue, I think. Without knowing how many films are being considered for inclusion in the list, specific placement below the top two or three is basically a crapshoot and impossible to judge even if we were familiar with every film ever made (which I don't believe anyone is or could be - which in turn also means any such ranking system can't really hope to be fully accurate anyway).


Your choice to put Ratatouille and Finding Nemo near the bottom of Pixar's movies is more surprising than Metacritic putting Incredibles in the middle, tbh. The core of Pixar's oeuvre (which, to me, includes all of the above movies plus Monsters, Inc. and Toy Story 2) is very consistent in quality, so it's hard to rank them. But the bottom of Pixar's movies is Cars 2, Monsters U, and The Good Dinosaur--good luck finding people who rank Nemo and Ratatouille around that level.

Knaight's argument is that the ratings have implied error bars, and that ratings whose distance is less than the error don't clearly count as 'better' or 'worse' than each other.
As an aside, when I saw The Incredibles (in 2004), I thought it was the worst Pixar film to date. Which is not to say I thought it was bad, just that its predecessors had been better. That alone would rank it #6 by my reckoning and I'd also say I preferred Up, Toy Story 3 and Inside Out to it among the films that followed; I could further be convinced of the relative merits of Wall-E, and though I could possibly be persuaded to demote Toy Story 2, that still leaves The Incredibles a fair way down the list.

So it's pretty subjective, and I don't think that's an unfair ranking for The Incredibles at all.


They have Gravity as the 12th best movie of all time. Let that tell you something.

Really i never muched care for these rating agregates, and what happened with Suicide Squad just reinforced it for me. In short, i like movies that critics dont like (Suicide Squad) and i like movies that they do like (Spirited Away), so i pretty much just ignore them
I think people have forgotten how good Gravity was. I got the impression that the poeple who came away claiming it wasn't any good were either expecting something else and unhappy about that, or trying to look cool by dissing it. It was the second-biggest original film of that year in sales (behind Frozen), received near-universal critical acclaim and won seven Oscars (indeed, had it not been for the very worthy 12 Years a Slave released the same year, it would likely have taken Best Picture too). In terms of Oscars won it's in the top thirty films of all time, and given that as previously discussed Metacritic is unreliable for the period before about 1995, only five other movies since then have won seven or more Oscars, and while I like all the others* I'd be hard-pressed to say that many of them are, in fact, better films.

Yeah, I know the Oscars aren't a reliable metric, but I still think Gravity is underappreciated, and given the period for which Metacritic actually works something like as intended, I don't think #12 is as far off the mark as one might instinctively think.

But this is an inherent problem with any kind of ranking of anything to do with art, especially given the diversity of films available. How can you compare The Wicker Man with The Godfather with Singin' in the Rain with Once Upon a Time in the West with South Park: Bigger Longer and Uncut? How can you compare any of those to, say, Birth of a Nation? And how do you compare any of them with Gravity? Do you lump Gravity in with the popcorn character thrillers like Jaws or the middlebrow space adventure film like Interstellar or a disaster film like Armageddon? In any of those categories I think it's towards, if not at, the top of its class. To compare it with The Godfather or The Third Man would be to take at least one of those films out of where it belongs; apples and oranges spring to mind.

I honestly don't think film ranking can or should be taken remotely seriously. It can be entertaining as an intellectual exercise for film enthusiasts (explain how Jurassic Park is a better film than Brief Encounter!) or the subject for a pub debate but any kind of ranking system can't take into account the context, conditions and caveats that help to determine how good a film actually is, and so inevitably leave everyone dissatisfied.


*In order, The English Patient, Titanic, Shakespeare in Love, Lord of the Rings: The Return of the King and Slumdog Millionaire.

eggynack
2016-12-06, 06:58 AM
I think people have forgotten how good Gravity was. I got the impression that the poeple who came away claiming it wasn't any good were either expecting something else and unhappy about that, or trying to look cool by dissing it. It was the second-biggest original film of that year in sales (behind Frozen), received near-universal critical acclaim and won seven Oscars (indeed, had it not been for the very worthy 12 Years a Slave released the same year, it would likely have taken Best Picture too). In terms of Oscars won it's in the top thirty films of all time, and given that as previously discussed Metacritic is unreliable for the period before about 1995, only five other movies since then have won seven or more Oscars, and while I like all the others* I'd be hard-pressed to say that many of them are, in fact, better films.

Yeah, I know the Oscars aren't a reliable metric, but I still think Gravity is underappreciated, and given the period for which Metacritic actually works something like as intended, I don't think #12 is as far off the mark as one might instinctively think.
I dunno. I watched it, and I don't remember ever thinking, "Wow, I love this movie, and it's easily one of my favorites." It had some really good effects, and a generally strong aesthetic sense, but I didn't get all that much out of it. I'd call it somewhere above average, but not hugely so. Far from being a movie I'd consider 12th best of all time, I'm not all that convinced it was the 12th best in the year it came out. Checking the Wikipedia list for films from 2013, I can recall seeing Before Midnight, This is the End, Despicable Me 2, Monsters U, The World's End, Blue Jasmine, Don John, Captain Phillips, 12 Years a Slave, Frozen, Inside Llewyn Davis, American Hustle, most of The Wolf of Wall Street, and Her. Out of those, I'd consider only Despicable Me 2 significantly worse than Gravity. The rest either hover a bit below, a bit above, or are what I'd consider significantly better. Jeez, Despicable Me 2 was such a disappointment.

As you note, the Oscars are a pretty awful metric. I mean, I just checked the Oscars for that year, and The Act of Killing was considered for that year. Saw that one too, and it was way better than Gravity by my reckoning. Probably the best movie of the year, and it was neither nominated for Best Picture, nor did it win best doc. I usually have a list of stupid Oscar things, and I thought I'd have to use that, but, y'know, that one's near the top of my personal list. So, yeah, I could probably assemble a solid list of 12 movies that I'd consider superior to Gravity from 2012, and that's just out of stuff I've personally seen. Oh, wait, I've seen Nebraska too. That movie was pretty cool. That title is so forgettable to me.

Anyways, point is, ridiculously off the mark. So so many cinematic classics out there, enough that Gravity isn't close to the top 100 for films strictly from America, or in the top significantly larger number for all films everywhere. It had its positive qualities, there's no arguing that, but I didn't think it was great. This isn't coming from some crazy hype machine either, cause I definitely wasn't super hyped for it. It basically delivered what I thought it would. Also, for genre, I'd consider Gravity's closest cousin to be Castaway. Solo survival adventure with isolation as a heavy theme, and such.

Darth Credence
2016-12-06, 12:32 PM
Your choice to put Ratatouille and Finding Nemo near the bottom of Pixar's movies is more surprising than Metacritic putting Incredibles in the middle, tbh. The core of Pixar's oeuvre (which, to me, includes all of the above movies plus Monsters, Inc. and Toy Story 2) is very consistent in quality, so it's hard to rank them. But the bottom of Pixar's movies is Cars 2, Monsters U, and The Good Dinosaur--good luck finding people who rank Nemo and Ratatouille around that level.


Knaight's argument is that the ratings have implied error bars, and that ratings whose distance is less than the error don't clearly count as 'better' or 'worse' than each other.

Well, I guess a big part of that is that I pretty much forgot about the existence of the three you mention. I've never seen any of them, so I cannot make a call there. Sure, those three can be the worst, with Nemo and Ratatouille above them (I also have no knowledge of Finding Dory).

As to the implied range, that is not how Metacritic sells it. If they sold it as that, sure. But they publish lists of 10 best movies of the year, or of all time, so they believe that the rank order matters.


I think people have forgotten how good Gravity was. I got the impression that the poeple who came away claiming it wasn't any good were either expecting something else and unhappy about that, or trying to look cool by dissing it. It was the second-biggest original film of that year in sales (behind Frozen), received near-universal critical acclaim and won seven Oscars (indeed, had it not been for the very worthy 12 Years a Slave released the same year, it would likely have taken Best Picture too). In terms of Oscars won it's in the top thirty films of all time, and given that as previously discussed Metacritic is unreliable for the period before about 1995, only five other movies since then have won seven or more Oscars, and while I like all the others* I'd be hard-pressed to say that many of them are, in fact, better films.

And I would say a lot of people were tricked into thinking that it was a good movie when it really wasn't. It was sold on how good of a job it did at showing reality of being in space. Then it had to throw all of that away in order to kill Clooney.

Avilan the Grey
2016-12-06, 01:52 PM
Personally I think Pixar's quality has gone down at about the same rate as the other studios quality has gone up. Cars 2 being the obvious first blatant Cashgrab, of course.
(To be fair I have not yet watched Finding Dory).

Legato Endless
2016-12-06, 03:24 PM
I dunno. I watched it, and I don't remember ever thinking, "Wow, I love this movie, and it's easily one of my favorites." It had some really good effects, and a generally strong aesthetic sense, but I didn't get all that much out of it. I'd call it somewhere above average, but not hugely so. Far from being a movie I'd consider 12th best of all time, I'm not all that convinced it was the 12th best in the year it came out. Checking the Wikipedia list for films from 2013, I can recall seeing Before Midnight, This is the End, Despicable Me 2, Monsters U, The World's End, Blue Jasmine, Don John, Captain Phillips, 12 Years a Slave, Frozen, Inside Llewyn Davis, American Hustle, most of The Wolf of Wall Street, and Her. Out of those, I'd consider only Despicable Me 2 significantly worse than Gravity. The rest either hover a bit below, a bit above, or are what I'd consider significantly better. Jeez, Despicable Me 2 was such a disappointment.

As you note, the Oscars are a pretty awful metric. I mean, I just checked the Oscars for that year, and The Act of Killing was considered for that year. Saw that one too, and it was way better than Gravity by my reckoning. Probably the best movie of the year, and it was neither nominated for Best Picture, nor did it win best doc.

The thing with Gravity is it's a perfectly executed film, but it has extremely modest aims. The pacing is relentless, the opening shot is gorgeous, and the rest hits all the right notes for a survival thriller. But...that's all it is. Bullock's character has enough depth to be serviceable, but not enough to paint any extraordinary picture of humanity. The psychological portrait is too fundamentally shallow. The life-death imagery is good enough, but simplified and not particularly thoughtful. The question one comes to is, does a film of such narrowness deserve greatness? If you're not a fan of the genre, the film doesn't have anything else to woo you. When compared to the breadth of The Act of Killing, you start inevitably wandering into judging a film based on what it lacks. And while that's considered a critical faux pa, it's a bit inevitable when you're being presented with an ocean versus the loveliest of fish globes.


Well, I guess a big part of that is that I pretty much forgot about the existence of the three you mention. I've never seen any of them, so I cannot make a call there. Sure, those three can be the worst, with Nemo and Ratatouille above them (I also have no knowledge of Finding Dory).

Finding Nemo would never be ranked as middling for Pixar regardless of the site, it's their most populist movie. A quick google of over a dozen Pixar rankings puts Nemo perennially in the top 3, with the Incredibles near but usually not quite as high. If you polled a randomized group of 100 about their absolute favorite Pixar film, Finding Nemo would probably win out more often than anything else. Not to downplay your opinion, just that it is an outlier. Any aggregate site based on quality would put Nemo near the top currently.

Ruslan
2016-12-06, 03:43 PM
I was wrong - it is actually 7th. I find it too low. Especially when the ones above it are Ratatouille, Toy Story, WALL-E, Inside Out, Toy Story 3, and Finding Nemo. I would put the first and last near the bottom of Pixar's movies, and WALL-E and Inside Out as in the middle range. I realize that this is subjective, but that is the lowest I have ever seen the Incredibles rank.Yes, this is indeed subjective. I love The Incredibles too (our family went as The Incredibles for Halloween!), but I can see a valid case for putting several Pixar films above it. Because they are all great films, and let's face it, neither me or you are sufficiently qualified to put such greatness on a scale. Neither of us is qualified to utter a statement "Incredibles is better than Finding Nemo" (or vice versa). At the most, we can say we like one over the other.

At this point, you can note some hypocrisy, because Metacritic does rank those films. Well, it really doesn't rank them in terms of better. More in terms of collected information as to how people liked them. Which may or may not be useful, but definitely not worth getting worked up over. So what if Metacritic didn't find enough people who like your favorite film to place it above #7? The film doesn't become worse as a result.

Legato Endless
2016-12-06, 04:02 PM
Yes, this is indeed subjective. I love The Incredibles too (our family went as The Incredibles for Halloween!), but I can see a valid case for putting several Pixar films above it. Because they are all great films, and let's face it, neither me or you are sufficiently qualified to put such greatness on a scale. Neither of us is qualified to utter a statement "Incredibles is better than Finding Nemo" (or vice versa). At the most, we can say we like one over the other.

At this point, you can note some hypocrisy, because Metacritic does rank those films. Well, it really doesn't rank them in terms of better. More in terms of collected information as to how people liked them. Which may or may not be useful, but definitely not worth getting worked up over. So what if Metacritic didn't find enough people who like your favorite film to place it above #7? The film doesn't become worse as a result.

No, I don't believe this. The very act of placing films into a generalized explicit ranking order from qualitative reviews is heavily implying something is better to some degree, even with error margins. While that might not be the literal truth, it's definitely what the site presents. Furthermore, I have a fair amount of difficulty remaining sympathetic to a site that uses a hidden balancing system for...pop culture grading. To quote Nate Silver, there's a moral imperative to transparency when you're dealing with large amounts of data and reaching any kind of systemized conclusions, and Metacritic isn't handling anything that would merit secrecy. To hide something like that is pretentious at the most charitable.

eggynack
2016-12-06, 08:12 PM
The thing with Gravity is it's a perfectly executed film, but it has extremely modest aims. The pacing is relentless, the opening shot is gorgeous, and the rest hits all the right notes for a survival thriller. But...that's all it is. Bullock's character has enough depth to be serviceable, but not enough to paint any extraordinary picture of humanity. The psychological portrait is too fundamentally shallow. The life-death imagery is good enough, but simplified and not particularly thoughtful. The question one comes to is, does a film of such narrowness deserve greatness? If you're not a fan of the genre, the film doesn't have anything else to woo you. When compared to the breadth of The Act of Killing, you start inevitably wandering into judging a film based on what it lacks. And while that's considered a critical faux pa, it's a bit inevitable when you're being presented with an ocean versus the loveliest of fish globes.

I think a big problem with gravity is that the big best features intrinsic to it don't have that much depth. It's like, okay, here's a really good shot of space, or a space station, or the Earth, or whatever, and that's really interesting, but what does that amazing shot, in and of itself, tell me? What am I getting out of it, that I'll be tossing around in my head for weeks afterwards? This, to my mind, is the fundamental difference between a movie having these really amazing qualities, like great effects or strong artistic direction, and a movie heading towards greatness premised on visuals. It's the difference between Gravity, and, say, Citizen Kane, It's Such a Beautiful Day, or The Shining, movies which reinforce, maybe even create, their characters, narrative, and tone from those fundamental elements of film making. Or, hell, even The Act of Killing, which so often uses visual language for communication purposes, both in the art produced by the gangsters and in Oppenheimer's overall film.

In summary, if visuals are to be a film's path to greatness, then I really expect more out of them than this. I expect the visuals to tell me a really complicated and depthy story on their own. In a sense, I expect all those things you listed to still be in the movie, except transmitted in whole or part by this one possibly great element. And Gravity, because it's shallow when you have the entire film in front of you, is obviously still going to be shallow when you take away the dialogue. By that metric, it tends to be a lot easier to determine a movie's greatness, because you'll still have all these qualities, even if you have to look harder for them. After all, we do manage to have lots of pretty paintings without calling every one that crosses our path a masterpiece, and we still have masterpieces in spite of the harsher criteria.


No, I don't believe this. The very act of placing films into a generalized explicit ranking order from qualitative reviews is heavily implying something is better to some degree, even with error margins. While that might not be the literal truth, it's definitely what the site presents. Furthermore, I have a fair amount of difficulty remaining sympathetic to a site that uses a hidden balancing system for...pop culture grading. To quote Nate Silver, there's a moral imperative to transparency when you're dealing with large amounts of data and reaching any kind of systemized conclusions, and Metacritic isn't handling anything that would merit secrecy. To hide something like that is pretentious at the most charitable.
Y'know, I was thinking about this kinda thing awhile ago, separate from this thread. And what I was thinking was that it'd be cool if review aggregate sites made use of 538's "house effects", except applied to genre. That'd mean that a reviewer who consistently rates romances lower than the general population would have their number adjusted upwards by whatever distance is generally present. Dunno if such a system would make sense in the universe of film, but it'd certainly be interesting.

Velaryon
2016-12-11, 09:20 PM
Since selecting the movie collection for my library is a huge part of my job, I find this whole discussion pretty fascinating. For what it's worth, when I'm deciding whether to purchase a film, I do look at Rotten Tomatoes, both the critic and audience scores (and I give more weight to the audience scores), but I almost never bother with Metacritic. The information I need is "how much are people likely to check this out, and how likely are they to like it?" and RT serves that need pretty well.


Regardless of anything else, if I had a rating system that said that there are 788 movies better than The Godfather, Part II but worse than The Godfather, I'd redesign it. While I would agree that the original is better than the sequel, they're not that far apart in quality.

Personally, I did think The Godfather, Part II was a pretty big step down from the original. Mostly my problem is that you're essentially watching two mostly unrelated films at the same time. If there was any sort of narrative thread connecting the two stories beyond "these are members of the same family and one of them is the father of the other" then it would work better for me, but as it is I think they should have been fleshed out into two separate films.



Not really. Both of them say "most people do not like this movie".

And that's literally the only useful information that comes out of a ratings aggregator.

A binary like/dislike score makes no distinction between "Meh" and "Oh god, please make it stop," which can actually be a meaningful difference when deciding whether it's worth watching a particular film, especially if you can do so on the cheap (matinee prices, free tickets, Netflix, etc.). Certainly it's better to consult individual reviews if there's a particular critic whose tastes more or less match your own, but if that's not the case then it would be useful for an aggregator like Rotten Tomatoes to make that distinction.

GloatingSwine
2016-12-12, 03:40 AM
A binary like/dislike score makes no distinction between "Meh" and "Oh god, please make it stop," which can actually be a meaningful difference when deciding whether it's worth watching a particular film, especially if you can do so on the cheap (matinee prices, free tickets, Netflix, etc.).

Not really?

I mean in 100% of instances you can find something better to do with your time than go and see a movie that rates a "meh" even if it's only go to the pub for a couple of hours.

eggynack
2016-12-12, 06:59 AM
Not really?

I mean in 100% of instances you can find something better to do with your time than go and see a movie that rates a "meh" even if it's only go to the pub for a couple of hours.
Well, the inverse is more generically useful, obviously, the difference between, "This was a rather enjoyable film," and, "this was the best movie I've seen in years," a trivially seen one. However, I think that the differences on the lower end of the scale could be important, at least in aggregate. Instead of just this one person saying either that the movie is mediocre or that the movie is awful, imagine instead that that person is surrounded by some positive reviews, ranging from good to incredibly positive. Now, that negative opinion's scale becomes important for how much you should let it modify those positive reviews. If the single person is just on the bottom half of the scale, then you'd maybe say, "I guess this person just values elements of a film in a somewhat different way, in such a manner that it accounts for ten or so points of difference." At the very bottom though, there's clearly a massive discrepancy between this person and the rest of the critics, or, in a more balanced field, they're clearly making a more impactful statement.

Also worth note that personal modifiers can move the needle on a rating enough to make even a single reviewer on the lower end of the spectrum have meaning. for example, if you're heavily invested in a series of films, then you may have enough momentum behind you to pursue future entries in the series given that those entries cross a pretty low threshold. A 50, for example, could be acceptable where otherwise you would turn away, while a 3 would keep you away from even a heavily anticipated film.

Knaight
2016-12-12, 02:35 PM
Not really?

I mean in 100% of instances you can find something better to do with your time than go and see a movie that rates a "meh" even if it's only go to the pub for a couple of hours.

It depends. Sometimes you're just really in the mood for a movie of a particular genre, and if there's one that rates a "meh" overall it might still be worth watching. Or maybe you're watching a movie as part of a social event, and while you like movies you're willing to put up with a "meh" rated one for the socialization while not really up for one that is just incessant unmitigated garbage.

Liquor Box
2016-12-12, 03:10 PM
I didn't like The Incredibles that much - it was ok, but not great. My favourite Pixar movie was Inside Out, but I also like the Croods. I know that one of the Toy Story movies is considered to be quite iconic.

I am by no means an expert on Pixar films, but I would have thought The Invredibles might have been lower than number 7 on the list.

GloatingSwine
2016-12-12, 03:47 PM
Not just as a Pixar film, but until Marvel Studios really got their act together The Incredibles was the best superhero movie for at least a decade.

Liquor Box
2016-12-12, 05:59 PM
Not just as a Pixar film, but until Marvel Studios really got their act together The Incredibles was the best superhero movie for at least a decade.

Yeah, it may just be that superhero movies are not my genre. I quite like some of the Marvel movies, but I'm sure there are some I missed and that doesn't worry me.

Lethologica
2016-12-12, 06:30 PM
I didn't like The Incredibles that much - it was ok, but not great. My favourite Pixar movie was Inside Out, but I also like the Croods. I know that one of the Toy Story movies is considered to be quite iconic.

I am by no means an expert on Pixar films, but I would have thought The Invredibles might have been lower than number 7 on the list.
I'm not sure whether you mean this separately from the Pixar discussion or not, but for clarity, The Croods was a DreamWorks film, not a Pixar film.

Liquor Box
2016-12-12, 10:58 PM
I'm not sure whether you mean this separately from the Pixar discussion or not, but for clarity, The Croods was a DreamWorks film, not a Pixar film.

No, I thought it was Pixar.

Velaryon
2016-12-13, 01:17 PM
Not really?

I mean in 100% of instances you can find something better to do with your time than go and see a movie that rates a "meh" even if it's only go to the pub for a couple of hours.

Yes, really. There are definitely films I would pay $5 to see but not $10, and films I would watch for free but not if I have to pay for them.

It may not be a meaningful distinction for you, but there are plenty of situations in which others might find that information useful. I for one would prefer even a truly terrible film over going to the pub. Bars make me horribly uncomfortable.

As someone else above mentioned, sometimes people go to movies with friends as a social thing, and the actual quality of the movie is secondary (so long as it's not horrible) to the experience. Or maybe it's Netflix night with the SO and you just want to watch something you haven't seen before, so something that looks "alright" might be preferable to spending 45 minutes scanning through the various options. Or in my case, sometimes I just want something on in the background while I'm reading or browsing the interwebs that's vaguely interesting but not engrossing enough that it monopolizes my attention.

dps
2016-12-13, 07:26 PM
More seriously, I agree with people saying the binary system. Yes, good taste is not binary, a good critic is aware you will not think as he does and you will need a more detailed judgement than yay or nay. But until you find a critic with exactly your taste, you're better of relying on an average over those than an average of numbers that only partially agree with each other.

To be honest, even with an individual critic, I'm more interested in reading his review to see why he rated it 4 stars out of 5 or 7.9/10 or gave it a thumbs up than I am in the actual rating itself. A 5 out of 10 tells me that he thinks it was mediocre, but I want to know what was good or bad about it. Is it a movie does is just mediocre in all aspects, or one that does some things well but is poor in other regards? And what things does it do well or poorly? For example, if it's a comedy that has great dialogue but a weak plot, I might very well still be interested, but if it's a murder mystery with great dialogue but a weak plot I'm probably not wanting to see it. Did the reviewer mark the film up or down because he disagrees with it's political/cultural/etc. POV? If my position on those things is different than the critic's, my reaction to the film is likely to be different.

Of course, any site or rating system that aggregates ratings/reviews is going to lose that element.

Aedilred
2016-12-13, 09:53 PM
Not just as a Pixar film, but until Marvel Studios really got their act together The Incredibles was the best superhero movie for at least a decade.

For that sentence to be true, it would need to be better than at least one of X-Men 2 and The Avengers*. You also have the Nolan Batman films coughing politely in the background. So... I disagree. :smalltongue:

*That is, Marvel's The Avengers. Not The Avengers (http://www.imdb.com/title/tt0118661/). Anything is better than that.

GloatingSwine
2016-12-14, 04:41 AM
For that sentence to be true, it would need to be better than at least one of X-Men 2 and The Avengers*. You also have the Nolan Batman films coughing politely in the background. So... I disagree. :smalltongue:

*That is, Marvel's The Avengers. Not The Avengers (http://www.imdb.com/title/tt0118661/). Anything is better than that.

The Incredibles was definitely better than all the X-Men movies.

When it was released it was the best superhero movie since at least Batman Returns, and it wouldn't be until Iron Man that another one as good came out.

eggynack
2016-12-14, 09:26 AM
The Incredibles was definitely better than all the X-Men movies.

When it was released it was the best superhero movie since at least Batman Returns, and it wouldn't be until Iron Man that another one as good came out.
It's probably just my favorite superhero movie period. There is what I'd consider a rather dry period, between Mr. Incredible's start of solo heroing and the introduction of Elastigirl to that plot-line, but the parts not in that period are amazing. She just makes the movie to such a great extent.

Legato Endless
2016-12-14, 06:28 PM
Ranking and arguing the entire run of superhero films from Superman to present might as well be it's own thread considering how often it leaks into other topics.

GloatingSwine
2016-12-14, 06:34 PM
Ranking and arguing the entire run of superhero films from Superman to present might as well be it's own thread considering how often it leaks into other topics.

We all know Batman and Robin would win though....