Why inference rate, not intelligence, decides which layer keeps the robot alive.
Six blind men are asked to describe an elephant. The one at the trunk reports a snake, the one at the leg a tree, the one at the ear a fan. None of them is wrong. Each has a correct local measurement of a surface he cannot see the whole of, and the argument only starts when they compare notes. The same problem occurs when you try to describe intelligence and even for what should be a very straight forward concept — bandwidth.
A machine learning researcher, a computer vision engineer and a control engineer walk into a bar. Ask them what the bandwidth of a neural network is and you get three clammy hands on the same metaphorical elephant. The first will tell you about spectral bias, the tendency of a network to learn low frequency structure in its input before fine detail. The second will tell you about aliasing in feature maps and why you blur before you downsample. The third will ask what the sample period is, because to a control engineer the bandwidth of anything in a loop is set by the loop frequency. All three are measuring bandwidth. None of the three measurements have anything to do with the other two.
Leave a Reply