p-value ≤ 0.05, there is a 5% probability that the variation that we see is by chance
It’s very popular but incorrect interpretation.
It’s not the probability of it being by chance, but the probability of it or more extreme happening if it is by chance.
Basically, what we see or larger difference happens with <5% probability when H0 is true so we agree to reject it.