Would you make a definitive judgment on a batsman whose career was 10 Tests long? Of course not, you'd say that it's far too soon to really tell. Why it's considered fine to make conclusive judgments purely on statistical grounds after 20 innings, which is an objectively miniscule sample size, is beyond me.
Even after 200 innings you can easily have a +/- of about 3-5 runs on a batsman's batting average IIRC. You'd expect that to be an order of magnitude higher after just 20. There's just very little meaningful to be gleaned just by looking at an average of, say, 30, when the +/- on that average could easily be upwards of 50%.
EDIT: This is some of the testing I did some time back on this exact question, and why I have become increasingly sceptical of the precision of batting average as a measure of quality.
This is after
200 innings, and the "underlying" average should be 48.95. Look at how much variation there is in the "actual" batting averages after
200 completed innings, a 10-run spread.
(I gave Cribb this Mathematica notebook a while back hoping he would look at it. Hopefully this post prods him into doing so
)