I’ve been running an A/B test on Preceden, my web-based timeline maker, for the last week to test the impact of gradient bars on the sign up rate.
Half the people saw solid color bars:
And half the people had a slight gradient fade added to them:
I measured the number of unique visitors who saw a timeline and considered it a conversion when the person signed up for an account. Note that the participant figures include anyone who viewed a timeline including existing users, people viewing someone else’s timeline, etc (ie, not just new users visiting the homepage). Because I’m just comparing the relative results, it doesn’t matter that the numbers include existing users, etc.
Users with non-CSS3 compliant browsers were also included in both test groups, but their browsers only rendered the fall-back solid color version. Since these people were evenly distributed between both test groups, it should not have an impact on the relative results.
I expected the gradient timelines to blow the solid colors out of the water, but the opposite was true (no = solids, yes = gradients):
The results per A/Bingo:
The best alternative you have is: [no], which had 102 conversions from 4166 participants (2.45%). The other alternative was [yes], which had 71 conversions from 4229 participants (1.68%). This difference is 99% likely to be statistically significant, which means you can be very confident that it is the result of your alternatives actually mattering, rather than being due to random chance. However, this statistical test can’t measure how likely the currently observed magnitude of the difference is to be accurate or not. It only says “better”, not “better by so much”.
Why did solid colors outperform gradients when (at least to me) gradients look much better?
My guess is that on average, the gradients look gaudy. People want simple timelines. Gradients are not simple.
If that is indeed the reason, it reaffirms what most of my previous A/B tests have taught me. Simple wins.
But I don’t know. Maybe I missed something. I’d love to hear your thoughts in the comments.
