Just found an option in the settings of Malwarebytes 3.0 that made me think. It shouldn’t have, so I fixed it 🙂

Their version:

My version:

Update 7/Mar/2017: Read the follow-up to this post – Make the questions easy to answer

Some quick notes after reading A field guide to usability testing and re-reading the Smashing Book #1:

1. A/B testing

– always test both versions simultaneously
– wait for it 🙂 (use a calculator to determine when to end it, and don’t give up earlier)
– keep the A/B tests for new visitors only (don’t surprise the regulars)
– but make sure that a new visitor gets the same version on consecutive visits
– be consistent: make sure that the variation appears on all pages (ex – if you have a promotional price on version A, make sure that the user will always see the promotional price on all the pages)
– the results might be un-intuitive
– naturally, the higher the number of users, the more reliable the result
More

Who would be involved in an A/B test:
– the UI/UX team – to propose the 2 versions and analyze the metric results
– the dev team – to implement the metric, manage sessions and make changes consistent across all the interfaces
– the network team – to handle various types of redirects (ex – run the A/B test only for users in a given geographical area, or only users on mobile)

2. MVT (multi variate testing)

– it needs a lot of traffic and time
– keep the number of combinations to 25 or less and make sure you preview them all
– global vs local optimum (A/B vs MVT)
– if you don’t have the traffic and cannot use full factorial testing, you can still use partial factorial testing.

3. Some usability rules/principles

(more…)

An interesting usability issue that affects the people with a color vision deficiency (CVD) – like me :). Here is how Canon and Sony chose to indicate the charging status for 2 battery chargers that I own:

Both chargers have a LED indicator.
Canon LED blinks if it’s charging, and is always on if the charging is done.
Sony LED changes color when the charging is done.
For me, this means that I can tell when the Canon device finished charging, but I have no way to tell when the Sony device finished its job.
GG Canon 🙂