Every play is reviewed by the booth so if there is not a buzz downstairs it means they already reviewed it and they agree with the call - so why would you waste a challenge and you know they already looked at it?
It's not like the referee or a different person looks at it if you challenge
It's not like the referee or a different person looks at it if you challenge