The idea to paint a first-down line across the field on people's TV screens sounds so simple. As it turns out, implementing this is incredibly complex. It takes a tractor-trailer rig of equipment, including eight computers and at least four people, to accomplish this task!
Here are some of the problems that have to be solved in order for this system to work:
* The system has to know the orientation of the field with respect to the camera so that it can paint the first-down line with the correct perspective from that camera's point of view.
* The system has to know, in that same perspective framework, exactly where every yard line is.
* Given that the cameraperson can move the camera, the system has to be able to sense the camera's movement (tilt, pan, zoom, focus) and understand the perspective change that results from the movement.
* Given that the camera can pan while viewing the field, the system has to be able to recalculate the perspective at a rate of 30 frames per second as the camera moves.
* A football field is not flat -- it crests very gently in the middle to help rainwater run off. So the line calculated by the system has to appropriately follow the curve of the field.
* A football game is filmed by multiple cameras at different places in the stadium, so the system has to do all of this work for several cameras.
* The system has to be able to sense when players, referees or the ball cross over the first-down line so it does not paint the line right on top of them.
* The system also has to be aware of superimposed graphics that the network might overlay on the scene.
This is not an easy process!
A key piece of hardware used in the system is a special camera mount that holds the television cameras. This mount encodes all of the camera's movement (such as tilt, pan, zoom and focus). This data enables the computers to understand exactly what each camera is doing in real-time.
Another key piece is a computerized 3-D model of the field. The computers know exactly where the cameras are located in the 3-D model and can orient the virtual first-down line on the field accordingly. The model also accounts for things like the crest of the field and the location of the yard lines on the field.
The color palettes are also critical to the system. The computers must be able to distinguish between grass, on which the line should be painted, and everything else (players, referees, the ball, etc.), on which it should not. Color palettes solve this problem. You can see the palettes at work in this frame:
Photo courtesy SporTVision
Players seem to run over SporTVision's "1st and Ten" line.
The player does not have the line painted over his jersey because of the color palettes.
All counted, there are eight computers used in the system:
* Four SGI computers
* One PC
* Three special computers used in conjunction with the television cameras
These special computers' sole task is to record aspects of each camera's movement 30 times per second from the camera mount, and then send that data back to the production truck for analysis and use.
Drawing the Line
In order to determine where the line should go, a central computer utilizes several pieces of information:
* The virtual field modeled from measurements of the field (done before the game), and the data from the camera mounts showing what each camera's range of view is
* The raw video feed from the camera that is currently on-air (which is determined by a separate computer in the Sportvision production truck)
* Two distinct color palettes, one representing the on-field colors that should be changed to yellow, and another representing those colors that should not be changed (like colors in the players' and officials' uniforms -- this allows a player to appear to "obscure" the line, making the line appear as if it were really painted on the field)
Once the computer determines exactly which pixels should be colored yellow, this information, along with the raw video feed of the tallied (on-air) camera, is sent to a computer whose job it is to draw the yellow line 60 times per second. The line is then sent to a linear keyer to superimpose the yellow line onto the program video. Since it takes time for all this to occur, the program video is sent through several frame delays so that the generated yellow line and delayed program video can be synchronized and turned into what you see on your TV screen.
On game day, it takes four people to run the system:
* A spotter and an operator work together to manually input the correct yard line into the system. The spotter is in the press box and the operator is in the production truck physically keying in the correct number.
* Two other Sportvision operators are on hand to make any adjustments or corrections necessary during the course of the game. These adjustments might include adding colors to the color palettes due to changing field conditions, such as snow or mud.