It seemed interesting to study how does attraction to an idea vary between some adjacent countries. The Fairphone project seemed like something worth promoting, so went with that. A choice of European countries seemed reasonable given that the project has mostly been active in that area, and considering also the ability to either write in the relevant languages or get some help to do so. And indeed, after much harassing of friends knowledgeable in the relevant languages, the messages were the following:
Looking at the results, the vast majority of engagements (98.7%) were clicks, with 2 retweets and 5 follows as well.
Engagement rates are the following:
- Italy: 1.20% (standard deviation 0.15%)
- England: 1.21% (standard deviation 0.17%)
- Germany: 1.86% (standard deviation 0.24%)
- Spain: 2.13% (standard deviation 0.17%)
- French: 1.64% (standard deviation 0.19%)
The formula for the standard deviation is . As a guideline, I’d ignore differences between points that are closer to each other than the corresponding standard deviations, and start taking things more and more seriously with ~1.5-3 standard deviations. It depends also on other factors, such as the number of options available – the more options you have, the more likely it is that noise gives rise to a large difference between two of them, so I’d suggest higher standards.
As for the engagement cost, we have
- Italy: $0.09 (noise level $0.01)
- England: $0.12 (noise level $0.02)
- Germany: $0.10 (noise level $0.01)
- Spain: $0.04 (noise level $0.003)
- French: $0.08 (noise level $0.01)
The method to compute the noise level here is a bit sketchy. It’s based on computing the standard deviation for the number of engagements (according to the observed probability), looking at the extremes of the corresponding interval, dividing the budget by each of these extremes, and then looking at the length of the corresponding interval. Would say it’s way more standard here (and maybe for the engagement rate as well) to build confidence intervals instead.
Some factors to take into account are the following:
- The wording might be a bit off in some of the languages, altering the engagement rate.
- Translating the message literally might not be the best way to obtain comparable results, since style of advertising varies in different countries. The fact that Spain has the highest engagement rate, as well as the original copy, suggests this might be going on.
- Different countries might have a different rate of Twitter users that speak the language used in the ad. Don’t think this is much of a relevant factor, and expecting that number to be > 98% in all countries.
- And more generally, there can be a multitude of reasons why Twitter population cannot necessarily be extrapolated to general population.
- EDIT: As pointed out by @cosenalgo (thanks a lot!), there is a huge factor going on, which is that the Tweet appears as tweeted by “Abel Molina”. It seems quite reasonable to expect engagement rate to increase upon more familiarity of the name (this article presents some possible evidence for this). That would help explain as well the higher engagement rate in Spain.