I am confused what is the purpose of the B/A test in the course of backtesting. It looks to me just a repeat of the A/B test. This is because in the test if A is outperforming B by 5%, B should be underperforming A by the same number. Swapping “control” and “variation” does not seem to change anything. - Thanks
No, that is not the case. B/A testing would have been redundant if we used the same data that we collected for A/B testing.
For B/A testing, the experiment is rerun, and new data is collected. If the new data confirm our hypothesis, it will ensure that the changes made in the system improve performance and will work in the long run.