Stereo vision systems rely on precise geometric calibration to accurately measure depth. Any misalignment or calibration error, even at a subpixel level, can significantly degrade performance. One often overlooked factor that affects calibration accuracy is camera synchronization—particularly in dynamic environments where the stereo rig is in motion. When cameras are not perfectly synchronized, the relative position and orientation of the stereo pair can shift between frames, introducing additional calibration errors. These errors can make it impossible to match features correctly between images or result in inaccuracies in large-depth estimation. Here, we explore the relationship between stereo calibration and synchronization, demonstrate how even small timing mismatches can lead to substantial errors, and provide a real-world example using the Nodar HDK to illustrate the impact of synchronization on depth accuracy.
$$ \Delta t = t_1 - t_0, \quad \text{ time between the left and the right cameras take images}\\ V \quad \text{ - speed, with which the cameras are moving}\\ D = V \Delta t \quad \text{ - distance travelled by the object}\\ f, B \quad \text{ - cameras' focal length and stereo baseline}\\\vec{T}, \vec{T}' \quad \text{ - real and perceived translation vector due to synchronization error}\\ \alpha \quad \text{ - angle between the real and perceived baselines due to synchronization error} $$
For stereo matching to be accurate, the calibration of the stereo cameras must be known with extreme precision—down to subpixel accuracy. Even minor calibration errors can have significant consequences. If the calibration is inaccurate, two major issues arise:
The required calibration precision is determined by the focal length and is inversely proportional to it, with one pixel corresponding to the following angle in a pinhole camera:
$$ iFOV = \arctan\big( 1 / f\big). $$
For example, with 5.4 MP Sony IMX490 cameras, a 30-degree field of view (FOV) corresponds to approximately 0.01 degrees per pixel. In contrast, a 70-degree FOV increases this to 0.03 degrees per pixel, highlighting the importance of precise alignment to ensure accurate depth estimation.
Imperfect synchronization between the cameras mounted on a moving platform can lead to a calibration that differs from the initial calibration performed in a static scenario. This issue occurs when one camera captures an image, the stereo setup moves forward by a distance of $D = V \Delta t$, and then the second camera takes its image. As a result, an additional angular difference $\alpha$ is introduced between the originally calibrated stereo setup and the configuration used for the actual measurement. The magnitude of change in calibration can be expressed as:
$$ \tan \alpha = \frac{D}{B} $$
and since $D = V\Delta t$, we get
$$ \alpha = \arctan \frac{V\Delta t}{B}. $$
The impact of synchronization errors on camera pose calibration increases with both the speed of motion and the level of synchronization error. Additionally, the calibration error has an inverse relationship with baseline length; this indicates that stereo systems with longer baselines are more resilient to synchronization errors.
Let’s use the baseline $B = 1m$ that’s part of the HDK, mounted on a car traveling at $V=30km/h \approx 8.33 m/s$. Let's use the 5.4MP cameras available in the HDK with $30^{\circ}$ field-of-view lenses (focal length $f=5333 px$).
To demonstrate the importance of synchronization, we would like to look at two options: