During online evaluation, the model has to predict on non-sampling data. If we do not do anything, what would happen to the precision metric?
Do we need to perform negative downsampling on the non-sampling data e.g. test data set or new incoming data in production?
Hey @Tan_Kah_Hooi!
The negative downsampling will be done only on the training data. This is done because our training data is unbalanced and we want the minority class to be better represented.
Your test data, however, needs to be kept far away from the downsampling process to prevent the performance of our model from being affected.