Most reliable measure of error in a model

[antonyp22]

What is the most reliable measure? Model error v line error, strike rate?

The reason I'm asking is that I seem to be getting some strange results. My model error is quite close to the line error and my model's strike rate increases as the difference between the predicted line and the actual line increases. However, if I analyse strike rates in intervals the strike rate decreases when the interval is changed from a 5-15 pt difference to a 15-25 pt difference, when in theory it should increase. Should I be worried?

[strixee]

Such behaviour isn't that unusual. Large differences can be caused by additional information your model doesn't have like injuries, lineup changes etc.

[antonyp22]

Such behaviour isn't that unusual. Large differences can be caused by additional information your model doesn't have like injuries, lineup changes etc.

So I'm assuming I shouldn't be that worried?

[nash13]

it depends on the number of values for each category. if ithere is a statistical bias in it you can test it by using chi square test. you compare the statistical differences between your model and the one without any effect beside.