This question is regarding the “Search Ranking” problem in “Grokking The Machine Learning Interview”
How do you account for situations where the user is searching for multiple topics in one session. For example during one session the user searches for
- Best Hotels in France
- How to make homemade cookies
- Who invented the internet?
Let us assume that each search results in a success meaning that the user finds the information which was being sought. Since all the searches were done in one session, the session success rate would be 100% because there’s a single session which was successful. However including the total number of queries and time to success will negatively affect the session success rate which should not be the case.
A valid example of the total number of queries and time to success negatively affecting session success rate is when during one session the user searches for
- How to make homemade cookies
- How to bake cookies at home
- Simple recipe for making cookies at home
How can you define your metrics such that total number of queries and time to success does not negatively affect the first scenario but does negatively affect the second scenario?