Ultra high speed megapixel video photogrammetry and correspondence

The video shown above is a demonstration of a 3 to 6 million match-per-second megapixel-resolution video photogrammetry and correspondence software system.

In simple terms this technology determines position and velocity of 3 to 6 million simultaneous points per second, all in real-time and with minimal latency. This is a 100% software defined and 100% passive sensor, requiring no special hardware and requiring zero emitted radiation or illumination.

Few if any image correspondence algorithms can leverage multi-megapixel resolution and gigapixel-per-second readout imaging sensors more effectively than this algorithm.

In fact this is possibly the fastest photogrammetry and image correspondence algorithm ever developed. By fair estimates it is 10 to 100 times faster than related published methods. This also implies 10 to 100 times lower power consumption or CPU/GPU utilization for similar match-per-second throughput.

This algorithm generates simultaneous spatial and temporal baseline correspondences (feature matches) with minimal additional computation. It does not require baseline calibration (epi-polar constraints). It can process calibrated, non-calibrated, rectified and highly distorted imagery equally well. It can handle real-time variable baselines (camera separation) and unknown spatial and temporal baselines. It is somewhat insensitive to focus, magnification and rotation variances between the image pairs.

This is currently developed and packaged as a software application for static and moving platform sensing applications. The system is composed of a C++ multi-threaded CPU + GPU application core, Qt 5 GUI, dual synchronized Point Grey Research 2048 x 2048 pixel x 90 Hz USB 3.0 cameras, OpenGL data display, Ubuntu Linux host and a mid-range laptop with an Intel Core i7 4700HQ CPU and NVidia GTX 770M GPU.

Specifically the demonstration shows 1024 x 1024 pixel x 15 frame per second stereo video being processed into approximately 200K to 400K correspondences per frame or 3 to 6 million correspondences per second. The correspondence yield is scene dependent. Very minimal outlier and noise filtering are being applied. Processing time varies linearly with the number of input pixels. 2048 x 2048 x 4 frame/second video can be acquired and processed but not displayed effectively due to the large image size. 21 megapixel stereo pairs can be processed at a rate of slightly less than 1 stereo pair per second.

Additional demonstration videos, images and live demonstrations are available upon request.

Please Click here to view this video on Youtube at 1080p resolution

Assistance provided by Alton Reich, Streamline Automation LLC and John Bossard, BSRD LLC.


Jason Heym
Autonomous Vision Research