Photometric Stereo Vision at Breakneck Speed

A JAI Sweep SW-16000M-CXP4A line scan camera unlocks the full performance potential of MSTVision's FPGA-based photometric stereo line scan solution.

The Challenge

The photometric stereo method uses multiple images of the same object or scene captured under different lighting directions. In the individual images, the surface shape creates characteristic brightness variations, often simplified as “shadows.” For this reason, the method is also commonly referred to as shape from shading. The computationally intensive processing of the captured images allows for the generation of image data with additional information content. In particular, topography (surface orientation) and texture properties (albedo) can be analyzed separately. This makes it possible to visualize even subtle defects such as scratches, dents, or embossing errors with high contrast — defects that often remain hidden in conventional images.

Accurate synchronization of the different image captures is easiest to achieve when the inspected objects are not moving during image acquisition. Conventional photometric stereo systems are therefore often either limited in speed or highly complex in design, for example through the use of multiple cameras or external computing units. This makes integration into high-speed production processes considerably more difficult. For that reason, there was a clear need for compact, scalable solutions that enable photometric stereo at high speeds with one or more line-scan cameras and deliver real-time results — without compromising accuracy and stability.

To use this method efficiently with line-scan cameras at high speeds, the different illumination directions must be captured within a single relative motion. For photometric stereo systems, even the smallest timing deviations between the illumination channels are critical and directly affect the quality of the computed surface information. At the same time, the system must be capable of processing very high data rates in real time, remain economically viable, and be flexibly adaptable to different applications. This requires tight integration between camera control, lighting control, and FPGA-based image processing.

Description of the Vision System

German development service provider MSTVision has introduced a photometric stereo line-scan solution that addresses this challenge through time multiplexing using its in-house MultiChannel technology. Multiple illumination directions are synchronized with a single line-scan camera down to the pixel level, enabling precise acquisition within a single motion.
Both image acquisition and the complete photometric stereo computation are implemented directly on a programmable frame grabber using FPGA processing, eliminating CPU load on the connected PC. This approach enables continuous inspection of material flows at previously unattainable process speeds while delivering both raw images and computed photometric stereo results in real time. Using the high-speed JAI Sweep SW-16000M-CXP4A camera takes the whole system to the next level.

What is not sufficiently illuminated cannot be optimally captured — even by the best camera. To meet the high demands for speed and light intensity, the company has developed its own high-performance line light specifically for photometric stereo. This line illumination module called Photometric Stereo Line Light MST-PSL features integrated LED drivers and four directional channels through which light is emitted at defined oblique angles. Compared to other systems, this arrangement offers clear advantages in terms of irradiance and angular precision. The individual LEDs are focused in such a way that, in strobe operation, irradiance levels of up to 6 kW/m² can be achieved, with a depth measurement range of 40 mm to 60 mm below the system. The lighting concept is based on an integrated illumination unit with four lighting directions. This simplifies mechanical integration at a constant working distance and enables a particularly compact design. Depending on the application, MST-PSL can be flexibly combined with other than the standard lighting types, such as backlight or front light illumination.

The Solution

A key factor behind the enormous performance increase of MSTVision’s photometric stereo solution is the use of a JAI Sweep SW-16000M-CXP4A line-scan camera. Its high line scan rate and data transfer speed allow the system to fully realize its potential in high-speed applications. JAI cameras have impressed MSTVision with their high sensor quality, industrial reliability, and excellent support for high-speed line-scan camera systems. For MSTVision’s photometric stereo solution, the stable temporal characteristics and the excellent integration capability into FPGA-based processing pipelines were particularly decisive. The JAI Sweep SW-16000M-CXP4A line-scan camera offers exactly the performance and control required for systems with synchronized multi-channel illumination and the photometric stereo method.

Another decisive factor for choosing this camera in MSTVision’s system was its extremely high line rate of up to 277 kHz at 16k resolution. According to MSTVision, this enables data rates of up to 4.5 GB/s and propels their photometric stereo system into speed ranges that were previously hardly achievable. Additional arguments in favor of the JAI Sweep SW-16000M-CXP4A included its excellent signal-to-noise ratio, stable temporal behavior, and the CoaXPress interface, which allows reliable real-time processing with maximum throughput. MSTVision’s photometric stereo solution delivers both raw images and computed photometric stereo results in real time and integrates seamlessly with high-performance cameras such as the JAI Sweep SW-16000M-CXP4A line-scan camera.

In principle, the system was developed for high-speed inline inspection applications in which detailed surface information of continuously moving objects is required. Typical fields of application include quality inspection of web materials such as metal strips, films, paper, or similar products. The system is particularly well suited for applications where even the finest surface structures, defects, or shape deviations must be reliably detected at very high line speeds. Compared to conventional line-scan camera systems or 2D image processing, users can analyze surface shapes and textures much more precisely with MSTVision’s system.

Oliver Gräff: ”The result of this collaboration between JAI and MSTVision is a compact and highly scalable system for photometric stereo line-scan solutions that can handle extremely high data rates while simultaneously delivering immediately usable results with outstanding level of detail.”

About the JAI camera

The monochrome line scan camera JAI Sweep SW-16000M-CXP4A is the perfect choice for this task because of several reasons. First of all, with an outstanding scan rate of up to 277 kHz at 16K line scan resolution with 1 x 16,384 monochrome pixels, this camera offers sufficient performance for a wide range of applications, ensuring that object features remain clearly visible even at the highest speeds. In addition, the CoaXPress 2.0 standard interface with CXP-12 enables high-speed data transfer, so that image acquisition and data flow enable the development of reliable systems even for high-speed applications. Numerous other features, including the camera’s high sensitivity with 5 μm square pixels, versatile GPIO and trigger options for flexible synchronization with lighting, rotatory encoders and other machinery peripherals, as well as an excellent shock (80G) and vibration (10G) resistance make the Sweep SW-16000M-CXP4A a powerful basis for use in a wide variety of industries.

Learn more about the JAI Sweep SW-16000M-CXP4A line scan camera here!

About MSTVision

MSTVision was founded in 2016 and has established itself as a specialized partner for sophisticated machine vision. In addition to comprehensive development services, ranging from feasibility studies and optical design to the development of CPU-, GPU-, and FPGA-based systems, the company, based in Ginsheim-Gustavsburg, Germany, offers its own highly integrated technologies and image acquisition modules. A particular focus is on innovative solutions in the field of photometric stereo as well as high-performance multi-channel systems. In addition, MSTVision supports its customers with in-depth market research and practical technical training throughout all phases of a machine vision project.

Learn more about MSTVision here!