What is review bombing, and how is it recognized?

[TheSummoner]

If substance is what you want, then a numerical rating is useless to begin. The only way you're going to get that is actually reading through or listening to a more in-depth review. The numbers - the thing the supposed "review bombing" would actually effect, is only relevant as an initial impression and as a metric for companies.

[Batcathat]

If substance is what you want, then a numerical rating is useless to begin. The only way you're going to get that is actually reading through or listening to a more in-depth review. The numbers - the thing the supposed "review bombing" would actually effect, is only relevant as an initial impression and as a metric for companies.

While it's certainly not perfect, getting a general idea of how people feel about the product can be useful, if the people rating it have experienced it and is giving their honest opinion (while the overlap between my opinion and the public's is very far from a 100 percent, I'm certainly more likely to enjoy something with five stars than with one).

Of course I can and do read individual reviews, but in most cases I won't read anywhere near all of them and as such I can't really know how representative they are of the whole.

[halfeye]

Amazon have disconnected reviews from scores, now there are often scores with no reviews at all. I'm not sure whether that's good.

[NichG]

I agree that there's not total overlap, but I don't think that there's no overlap. Honestly informing someone of your own impressions can done with the intent to convince them that theirs would be the same. And that's probably standard in this case, because that's largely the purpose of the communication.

Like, someone writing a review tries to convince those with the same taste that they'd have the same impressions, and someone reading reviews hopes to be convinced by/of the impressions of those with the same taste. Those goals are in harmony.

The areas where they don't overlap is where a concept like 'detecting and excluding review bombing' applies though. The point isn't to filter out negative reviews, its to filter out reviews where in pursuit of either pushing up or pushing down a project, things like 'honest communication' have been sacrificed. For instance, wanting to destroy a project so you get 500 of your friends and internet contacts and so on to all leave copy-paste of your own bad review. That sort of behavior is compatible with 'I want to convince people not to buy this product', but its not compatible with 'I want to honestly inform people'. And that's the sort of behavior such that its pointless to talk about any single review as a 'bomb', but we can meaningfully talk about subpopulations.

Because the 500 1-star reviews that all land on the same day when the product averages 20 reviews a day spanning the range is a pretty good indication that something is up.

Edit: As an aside, there are cases where you could detect bombing behavior in a small population of reviews, but not really in a 5-star system. For example, in paper submissions to conferences, whether a paper gets accepted or not is often thresholded by an average score where reviewers are able to score from 1 to 10. The tricky thing is that generally by convention scores lower than 4 are reserved for papers that are severely incorrect, plagiaristic, etc. However, giving someone a 1 lets you push the average further than if you gave someone a 4 or 5. If the other two reviewers are giving scores in the 5-8 range (usual distribution), then someone tossing in a 1, especially if the text of their review isn't like 'this paper is literally copy-paste blocks from these other papers, a falsified figure, and someone writing redrum fifty times in a row in this one spot' is probably trying to prevent the paper from being published rather than trying to evaluate the paper. But in order to do that kind of detection, there needs to be a well-established community norm about what each score value means first, which is absent from most 5-star systems.

[halfeye]

The areas where they don't overlap is where a concept like 'detecting and excluding review bombing' applies though. The point isn't to filter out negative reviews, its to filter out reviews where in pursuit of either pushing up or pushing down a project, things like 'honest communication' have been sacrificed. For instance, wanting to destroy a project so you get 500 of your friends and internet contacts and so on to all leave copy-paste of your own bad review. That sort of behavior is compatible with 'I want to convince people not to buy this product', but its not compatible with 'I want to honestly inform people'. And that's the sort of behavior such that its pointless to talk about any single review as a 'bomb', but we can meaningfully talk about subpopulations.

Because the 500 1-star reviews that all land on the same day when the product averages 20 reviews a day spanning the range is a pretty good indication that something is up.

Edit: As an aside, there are cases where you could detect bombing behavior in a small population of reviews, but not really in a 5-star system. For example, in paper submissions to conferences, whether a paper gets accepted or not is often thresholded by an average score where reviewers are able to score from 1 to 10. The tricky thing is that generally by convention scores lower than 4 are reserved for papers that are severely incorrect, plagiaristic, etc. However, giving someone a 1 lets you push the average further than if you gave someone a 4 or 5. If the other two reviewers are giving scores in the 5-8 range (usual distribution), then someone tossing in a 1, especially if the text of their review isn't like 'this paper is literally copy-paste blocks from these other papers, a falsified figure, and someone writing redrum fifty times in a row in this one spot' is probably trying to prevent the paper from being published rather than trying to evaluate the paper. But in order to do that kind of detection, there needs to be a well-established community norm about what each score value means first, which is absent from most 5-star systems.

This is sort of a different thing, but also sort of isn't: What about a "15" star system, where only scores of zero stars, 6 stars, 9 stars or 15 stars are allowed? it's really a 4 star system, but because the allowed scores don't exactly fall on 0, 5, 10 and 15 it doesn't at first sight appear so, and possibly evades statistical detection.

[Smoutwortel]

I'm not aware of any case where "Review bombing" was factually demonstrated to be the result of an organized effort by a small minority. As far back as I can remember hearing the term (which would be back when it mostly applied to game review sites), it was just something that was thrown around when fan opinion diverged from professional reviewers.

One of the earliest I can remember off-hand was Dragon Age II, a game that was very unpopular with Bioware's then-fanbase for moving away from their classic RPG style. This was probably the most common case pre-culture wars: sequels to games would make changes viewed by the existing fan base as dumbing the game down for more mainstream audiences, which they would harshly criticize, while professional reviewers who were at best casually interested in the previous games would rush through it in a weekend and slap a "8/10" score on it as long as it didn't make their console/PC physically explode.

There has always been, at very least, a strong element of bad-faith dismissal of criticism by developers/publishers/creators associated with claims of "review bombing".

My opinion of fan reviews remains large unchanged from when they first appeared: If a game is being "review bombed", then I want to know why (especially if I'm going to pay $60 for a game), because if something about it is angering the fans then there's a good chance I'll dislike it too. Especially since professional reviewers are, to put it kindly, overly generous toward corporate products with large marketing budgets. I certainly don't trust them to tell me, for example, that an adaptation/spin-off plays fast and loose with the source material's lore or that the latest MCU movie is riddled with plot holes and questionable logic.

If the hardcore fans are upset about something that I don't really care about, I can ignore their reviews and consume the product anyway. But I've been on their side of the fence far too often to dismiss the validity of their opinions.

Meanwhile, I find it a very strange notion that hardcore fans are unrepresentative of general audiences but that professional critics aren't. People who go out of their way to write a review will, by the very nature of the effort required, whether they are being paid or not, be a bit unrepresentative, but hardcore fans are at least, you know, fans, whereas professional critics have a relationship with media that is often utterly alien to your average consumer.

Even with franchises that I'm not a hardcore fan of I generally find myself more likely to agree with a hardcore fan than I am a professional critic. I'm not really interested, for example, in consuming most of the content that the Star Wars machine pumps out- even pre-Disney. I never cared about the EU, the Clone Wars cartoon, or getting an Obi-Wan TV series. But the fans that do are almost always more in line with my opinions about the Star Wars media that I do consume. If a new Disney+ series manages to win over the hardcore fans, I'll be interested in checking it out. Otherwise, stuff like the Obi-Wan series can only evoke enough mild curiosity in me to get me to, at most, get me to watch an RLM view of it while I play Minecraft.

The MCU is the same story: I've lost almost all interest in it at this point. I haven't seen any of the TV Shows or any of the movies past Black Widow. Yet, somehow, it seems to be able to more-or-less consistently generate positive reviews from professional critics. If the people who have been critical of Phase 4 (and parts of Phase 3) start buzzing again, it'll get my attention.

I'm not a hardcore LotR fan. I've only read The Hobbit and Lord of the Rings, seen the LotR movies, and watched first Hobbit movie. But I can still see far more of myself in someone who has read every letter Tolkien ever wrote and can repeat all of the songs from the books from memory than someone who is only writing articles about it because their editor assigned them to next corporate product and gave them a pile of press releases to regurgitate. I, like Elrond, was there 10,000 years ago when the strength of professional critics failed, and rather than cast Dragon Age IIinto the fires of Mount Doom where it belonged, they named it GotY.

Exactly if a group of fans is large and/or motivated enough to pull such a stunt it at least means the franchise just took a heavy turn they probably still need to adjust to.

[NichG]

This is sort of a different thing, but also sort of isn't: What about a "15" star system, where only scores of zero stars, 6 stars, 9 stars or 15 stars are allowed? it's really a 4 star system, but because the allowed scores don't exactly fall on 0, 5, 10 and 15 it doesn't at first sight appear so, and possibly evades statistical detection.

If you want me to be very general, it comes down to the shape of the distribution as given by community norms and standards, how much residual entropy there is in the distribution of conditional parameters for that community distribution after observing the overall score (e.g. if the community does accept a delta function 1 star review for the lowest 5% of works for example), and the log probability of the extreme ratings under that manifold of community distributions. That is to say, if you have a three point system but it is absolutely unacceptable in that community to give 1 point in all but 1% of cases, then seeing that something has a 1 but five 2s and five 3s could be enough signal to identify the 1 as anomalous. But if the community norms say, sure, give 1s as easily as you'd give 2s or 3s, then that 1 is not going to be enough signal to say much about that single review. Fortunately you don't usually need to identify each and every single review this way in order for tools to be useful, nor do you generally have to do it in isolation just using the raw scores.

[veti]

Thinking about it, I doubt very much that review bombing of a high-profile show is going to have much short-term effect anyway. Basically everyone has already heard of it and probably has already formed their first impression, before they even see the review ratings.

There might be more of an impact in a few years' time when someone is trolling through the archives looking for "the best" of previous years, because by then the hype will have worn off and they won't be looking for this show specifically. I have no idea what magnitude of effect that would be.

There may also be an effect on the morale of the makers and sponsors of the show, and I'm guessing they are the primary audience being targeted both by the bombers and the defenders.