Ultra-Wide Baseline Stereo Vision for Ultra-Long-Range Detection

NODAR Inc. / NODAR Sensor GmbH / January 22, 2025

NDR-HDK-2.0-XX

NDR-ORIN-R1

Cables that come with NDR-HDK-2.0-XX

Ultra-wide baseline stereo vision revolutionizes depth perception by expanding baseline distances from 100 mm to 1000 mm — 10 times wider than conventional setups. This extended baseline increases depth accuracy and range by a factor of 10, enabling precise 3D vision over greater distances for applications in autonomous vehicles, robotics, and industrial automation.

Furthermore, the high-resolution 5-MP HDR cameras produce 5-MP depth maps, a factor of 5 times greater than many of our competitors.

The hardware development kit (HDK) comes with an outdoor stereo camera with 1-m baseline, 5 MP cameras, 30° or 65° field of view, and an optional rugged Orin computer pre-loaded with NODAR software. It has everything you need to generate point clouds “out of the box.”

Experience the power of HDR Stereo Vision!

System Block Diagram

graph TD
  A[Left Camera] -- M12 <br/> 1G PoE <br/> RJ-45 --> C[Rugged Orin]
  B[Right Camera] -- M12 <br/> 1G PoE <br/> RJ-45 --> C
  C -- RJ-45 <br/> 10G Ethernet <br/> RJ-45 --> D[User Computer]
  C -. HDMI .-> Monitor
  C -. USB .-> Keyboard/Mouse
  
  style Monitor fill:none,stroke:none
  style Keyboard/Mouse fill:none,stroke:none

The left camera and right camera are both connected to the rugged Orin processing unit via 1G PoE (Power over Ethernet), enabling power and data transmission through the same cable. The rugged Orin is connected to the user system via 10G Ethernet, providing high-speed data transfer to the user’s computing environment. The rugged Orin also connects to a monitor using HDMI, providing a visual interface for output. Additionally, the rugged Orin is connected to a Keyboard/Mouse through USB, enabling user interaction for system control.

The NODAR Viewer can be installed on the Rugged Orin and/or the user computer. The Viewer takes up some GPU resources, so it does reduce the frame rate when running on the Rugged Orin.

HDK Software Block Diagram

NODAR HDK software runs on the rugged Orin and consists of three blocks:

Stereo Camera Driver - captures images from two cameras. Supported cameras are listed here.
Auto-Exposure - automatically sets the camera exposure for both left and right cameras. The feedback loop is limited to approximately 1 second for Lucid cameras.
HDK API - converts raw images into high-quality color point clouds and occupancy maps. We currently support three communication frameworks: ROS2, ZeroMQ and a ZeroMQ/ROS2 bridge.

graph TD
  Cam[Two Cameras] --> A
  A[Stereo Camera Driver] -- Image Pair --> Hammerhead
  Hammerhead -- Point Cloud --> GridDetect
  Hammerhead --> B[Hammerhead Output Topics]
  GridDetect --> C[GridDetect Output Topics]
  
  A -- Image Pair --> D[Auto-Exposure] -- Exposure --> A
  
  style Cam fill:none, stroke:none
  style B fill:none,stroke:none
  style C fill:none,stroke:none

The input and output topics are discussed in the corresponding API documentation with links above.

Output data topics and files

Topic	Description
`disparity`	Contains disparity maps in unsigned 16-bit format. They are saved as uncompressed TIFF files.
`details`	Stores metadata in CSV format. These CSV files provide additional information about the dataset, such as capture conditions or object details.
`left-rect`	Left rectified images; saved in uncompressed TIFF format.
`topbot`	Contains the raw Bayer left and right images stacked vertically; saved in TIFF format.
`timestamp-<DATE>.csv`	A CSV file that contains timestamps associated with the data.
`bounding_boxes_and_velocities`	Contains CSV files with bounding box and velocity data for objects in the scene. Each row in the CSV files has the following format: x1, z1, x2, z2, x3, z3, x4, z4, vx, vz, where x and z represent coordinates and vx and vy are velocities. Currently, the velocities are zeroed out.
`master_config.ini`	The master configuration file (master_config.ini) is a simple text file used for configuring Hammerhead. Its complete description can be found here: master_config.ini.
`rectification_config.enc`	Hammerhead’s internal rectification configuration file.

Camera Specifications

Parameter	Value
Class	Automotive
Make	Lucid
Model	Triton
Format	5.4 MP
Pixels	2880 x 1860
CMOS	Sony IMX 490
Pixel Size	3 µm
Shutter	Rolling Shutter
HDR Type	Two photodiodes and four TIAs
Dynamic Range	120 dB
Companded Bit Depth	16-bits
Exposure	86 µs to 47 ms
Gain	0 to 30 dB
Ingress Protection	IP67
Focus	Infinity
Lens	S-mount
IR Filter	650 nm
Synchronization	PTP (IEEE 1588)
Camera-to-camera timing jitter	10-µs average
<20 µs
Dataflow from each camera	<1 Gbit/s (800 Mbit/s typ.)