Does anyone else DESPISE A/B tested thumbnails and titles?

[gomipile]

Several of the media channels/websites I frequent A/B test headlines, video titles, thumbnails, etc. Often I'll open up the day's feed on a site, and mentally note which articles, videos, whatever seem interesting to come back to over the course of the day. Then, later in the day I come back and don't notice the title, whatever I thought I was looking for because the *other* one that was part of an A/B test won. The double-take that results is always at least a little jarring.

It always makes me feel like I'm being gaslighted, in the original sense from the film Gaslight(1944.)

Does anyone else have a similar reaction to this?

[Sigako]

First time I hear (well, read) the term. What's that?

[Rynjin]

First time I hear (well, read) the term. What's that?

Basically an A/B test is when you mock up two idea for marketing and test them by actually putting them in the wild to see which one does better. If you eg. title your Youtube video "I think Supernatural is the Greatest Show Ever Made" and it gets 10000 views in an hour and then switch the title to "Top Ten Reasons Why Supernatural is Better than YOUR Favorite Show!" and it gets 100000 views in an hour, you know which title works better.

These titles (if you're capable) will hit the market at the same time, so it's not a matter of testing Title A 1 hour and then Title B the next. The titles are put out there simultaneously and then the better performing one becomes the "real" title while the poorly performing one is put away.

[Sigako]

Either I've never encountered that, or I have a major case of change blindness (which I do have).

[Rynjin]

Sort of the point is that you shouldn't notice it, like any good marketing technique. If you're not part of the "test audience" you'll never know anything changed.

Given that the "losing" test will by its very nature be seen by less people, you have to be in the right place at the right time to notice.

[gomipile]

I have noticed it a lot on Ars Technica. They may do this more often than most places. I'm not sure quite how common it is industry-wide.

[Ionathus]

I'm quite familiar with A/B testing, but my only experience with it is for ads, marketing email copy, or other short-lived content. You test two banner ads and then pick the better one for a long-term campaign. You try out new structures in your email marketing. That kind of thing. My only experience with it is in these transient settings, though: you send an email and then it's done, and there's no chance of somebody getting the emails swapped out in their inbox and going crazy or feeling gaslit.

I've never noticed this done for headlines or blog post titles. I don't know if there's a theory driving that choice, but it seems like a bad idea to fiddle with your content's title immediately after posting it, for the exact reason you've stated: it makes it far harder to find! Especially in those first few hours, which are so crucial for a lot of news / social media performance.

This seems like a real weird choice. I'd probably have a similar reaction if I experienced this in the wild.

[IthilanorStPete]

I don't notice it so much on content, but I notice it in UIs, especially on mobile apps (although I'm not sure how much of that is A/B testing vs. just UI change over time). It's really annoying to get used to a certain flow, with buttons in known locations, then have that switch around, then switch back, for no apparent reason.

[gomipile]

Some of the YouTube channels I'm subscribed to do this at times, also. They'll change the thumbnail and/or title a few times until a video starts getting a lot of views per unit time.

[Tevo77777]

Some of the YouTube channels I'm subscribed to do this at times, also. They'll change the thumbnail and/or title a few times until a video starts getting a lot of views per unit time.

Yeah, I've literally had videos change their thumbnail and title WHILE I was watching them. That was kinda weird.

Or I would save them in a playlist and come back later, and they were different.

Typically, I find that the new title and thumbnail is less insulting/clumsy/confusing.

[Ionathus]

I don't notice it so much on content, but I notice it in UIs, especially on mobile apps (although I'm not sure how much of that is A/B testing vs. just UI change over time). It's really annoying to get used to a certain flow, with buttons in known locations, then have that switch around, then switch back, for no apparent reason.

I don't know of any specific examples of a company A/B testing their UI -- I'm sure people do, but it seems like it'd require exponentially more work to create and maintain two separate site-wide designs than it'd be to swap out a thumbnail in a display ad. I think the thing you're noticing is probably just bog-standard UI updates. Whether or not they're actually improvements is a different discussion entirely :D