A/B, MVT testing and usability
Written on 30 March 2015, 10:31pm
Tagged with: books, human behavior, usability
Some quick notes after reading A field guide to usability testing and re-reading the Smashing Book #1:
1. A/B testing
– always test both versions simultaneously
– wait for it 🙂 (use a calculator to determine when to end it, and don’t give up earlier)
– keep the A/B tests for new visitors only (don’t surprise the regulars)
– but make sure that a new visitor gets the same version on consecutive visits
– be consistent: make sure that the variation appears on all pages (ex – if you have a promotional price on version A, make sure that the user will always see the promotional price on all the pages)
– the results might be un-intuitive
– naturally, the higher the number of users, the more reliable the result
More
Who would be involved in an A/B test:
– the UI/UX team – to propose the 2 versions and analyze the metric results
– the dev team – to implement the metric, manage sessions and make changes consistent across all the interfaces
– the network team – to handle various types of redirects (ex – run the A/B test only for users in a given geographical area, or only users on mobile)
2. MVT (multi variate testing)
– it needs a lot of traffic and time
– keep the number of combinations to 25 or less and make sure you preview them all
– global vs local optimum (A/B vs MVT)
– if you don’t have the traffic and cannot use full factorial testing, you can still use partial factorial testing.
3. Some usability rules/principles
- Likes (0)
- Comments (0)
-
Share