The refs have an impact, yes. But to say that the refs were just out to screw the Pats is silly. If that is the case they ignore the blatant tackle on the PI a few plays earlier. They do not give the Pats another second on the clock for the final play. They call holding on the Pats on the final play or the final few plays.
If it's a fix, there has to be a benefactor. One would say the refs or a syndicate the refs were involved in...same problem, you don't give the Pats the opportunity to beat "the fix." You just don't. With 22 players on the field, coaches, injuries, and careers on the line in some instances, you cannot fix an NFL game without having many people in on it.
It gets a little old to hear the boys who cried "Fixed" here every time a game is close. Give Vegas credit once in a while for posting a sharp line with the plethora of information and sharps they have access to.