Small screen detected. You are viewing the mobile version of SlideWiki. If you wish to edit slides you will need to use a larger device.
Purpose: Test a single innovation
Prerequisite: You have a large search engine up and running.
Have most users use old system
Divert a small proportion of traffic (e.g., 1%) to the new system that includes the innovation
Evaluate with an “automatic” measure like clickthrough on first result
Now we can directly see if the innovation does improve user happiness.
Probably the evaluation methodology that large search engines trust most
In principle less powerful than doing a multivariate regression analysis, but easier to understand
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License