Introduction
It is often said that the world gets smaller every day. For designers of vision systems, that phrase takes on a special meaning as increasing numbers of today’s vision systems must concern themselves with microscopic details. Tasks which once might have been performed manually by a single individual with an optical microscope, are now being performed faster and with higher accuracy and repeatability than ever before by combining microscopy with digital cameras and sophisticated image capture and analysis software.
These microscope-based machine vision systems can be found in a wide range of life sciences applications, as well as in industrial applications ranging from semiconductor inspection to materials science to failure analysis and QA.
For many of these digital microscopy systems, the capture and analysis of color information is critical to meeting the application’s goals. Thus, the system’s camera or cameras, which serve as the “eyes” of the system, must be able to provide the right levels of both spatial and color information to enable the software “brain” of the system to perform its function.
Read on to learn more about digital microscopy solutions and how various camera capabilities can be utilized to meet your application requirements.
Types of color microscopy systems
Modern color microscopy-based systems fall into several categories. The most common are:
Systems using color dyes or stains with traditional brightfield techniques to illuminate samples and capture color images
For some industrial systems, illumination might also include darkfield techniques, as well as near-infrared wavelengths to simultaneously analyze both visible and non-visible details.
Brightfield microscopy technique Color microscopy systems using color fluorescence
This technique can be used with both organic and inorganic compounds, but is most common in life sciences where one or more fluorescent stains (fluorophores) are added to biological samples in a way that specific structures or objects of interest emit specific light wavelengths (colors) when struck by the excitation light source.
Fluorescence microscopy technique
Color Microscopy Challenges
Designers of color microscopy systems face various challenges depending on the targeted application. Some of the most important issues which must be considered are:
Color accuracy and differentiation
In many applications, particularly life sciences applications involving stains or dyes, detecting specific colors and distinguishing them from similar colors is critical for proper analysis.
Brightfield images before (left) and after adding color stains to sample (source: Olympus) Sensitivity
Microscopy applications often must work with relatively low light levels. This is most noticeable in fluorescence applications where absorption of the initial light source reduces the intensity of the resulting “glow” that is emitted. Even in brightfield applications, light levels might need to be limited to minimize phototoxicity and avoid harming the sample being inspected.Image sharpness/level of detail
While the microscope or other optics provide the necessary magnification, this does not insure the sharpness of fine details. In applications where precise positions or measurements are required, the machine vision detector/camera must provide color images with a minimum of softening, moiré patterns, or other loss of detail.Color enhancement, especially red
Many of the dyes, stains, and fluorophores used in life sciences applications are red in color. Microscopy-based systems must perform exceptionally well in this portion of the spectrum, either through native sensitivity or the ability to add emphasis to specific colors. Extended sensitivity in the red channel may also be useful for some industrial applications where non-visible information from the NIR band is required.
Brightfield microscopy image of skin cells Low noise
Related to both the sensitivity and sharpness issue, microscopy-based systems must seek to minimize noise levels so that exposures captured under low light conditions remain “clean” enough for effective analysis.Dust and/or foreign object debris (FODs)
While electronic noise can add virtual “dirt” to an image, microscopy applications must also concern themselves with actual dirt. Dust particles, lint, and other so-called foreign object debris (FODs) that finds its way into the optical path of the camera can significantly hamper the efficacy of a microscope application. This is especially true for brightfield microscopy, which tends to accentuate any FODs on the sensor or the optical glass elements. It can be a key factor in cell-counting and other similar types of applications where dust/FODs are mistakenly identified and counted.Frame rate
In life sciences applications such as live cell imaging, “real-time” or faster frame rates may be required for time-lapse sequences, while for wafer inspection and other industrial applications high frame rates are required to meet throughput demands. Furthermore, techniques such as extended depth of field (EDOF), which require stacking of multiple images, can also benefit from higher frame rates.Compatibility with robust analysis tools
Though many systems may utilize custom-developed analysis software, many other microscopy-based systems may benefit from leveraging specialized third-party software capture and analysis tools already available in the market. It is important for system designers and integrators to consider the type of functionality their system must possess and whether commercial or open source software available in the market can be easily integrated to provide those capabilities.
Addressing the color camera challenge
As the preceding section suggests, utilizing the proper camera technology is an important starting point for creating an effective color microscopy imaging solution – regardless of whether the imaging solution is within a larger machine vision application, or it is part of a simpler laboratory-style rig.
Bayer color technology supports basic system requirements
For systems where the color requirements are relatively modest, a standard single-sensor camera (detector) featuring an imager equipped with a Bayer color filter may provide adequate results. Of course, one should still keep the key challenges in mind, and search for a camera with low noise, good sensitivity and a reasonable frame rate, as well as specialized features like color enhancement and/or color space conversion. Integration with leading microscopy software packages can also be quite important.
Bayer color cameras have improved considerably in recent years, giving system designers many good options for basic color microscopy systems. In particular, new cameras based on state-of-the-art CMOS imager technology, such as that found in Sony’s PregiusTM sensor line, provide fast frame rates and deliver low-noise color images that can meet the needs of many applications.Multi-sensor prism cameras for advanced applications
If one is looking to build a more advanced color microscopy system where superior color and spatial performance provides added value to the application, a more advanced camera technology should be considered.
Fluorescence image using red, green, and blue fluorophores Three-sensor prism cameras (3-CMOS or 3-CCD) use high-grade optical prisms to split incoming light to three separate sensors based on spectral wavelengths (red, green, blue). Compared to Bayer cameras which must use interpolation to estimate RGB values, this provides higher color accuracy when subtle differentiation is required (see below). It also results in higher effective sensitivity. This is because the Bayer filter matrix essentially blocks 2/3 of the wavelengths falling on each pixel, while prism cameras use their multiple sensors to capture nearly 100% of the light emanating from the sample. With a higher overall signal-to-noise ratio, prism cameras are able to produce better images with lower light levels to reduce the stress on delicate samples.
Bayer color methodology vs. prism technology
Bayer filters (left) cause each pixel to only capture a single color band (R, G, or B). The other two colors in the pixel's RGB value are "interpolated" by using data captured in neighboring pixels. Prism technology (right) separates incoming light to three precisely aligned sensors so that true RGB values can be captured for each pixel with no loss of accuracy due to interpolation.
The JAI Microscopy Solution
JAI utilizes its expertise in color imaging and prism technology, as well as integration with leading microscopy software packages, to offer system builders multiple options for creating microscopy systems with advanced color capabilities.
JAI’s multi-sensor prism technology provides higher levels of color accuracy, better differentiation of subtle shading differences, better low-light sensitivity, and better sharpness of small details than comparable Bayer color cameras (read more about this in our Color Imaging Tech Guide). This enables developers to build systems capable of analyzing microscopic samples – whether biological or industrial – in ways that traditional monochrome or single-chip color cameras can’t.
JAI’s prism color cameras feature three 1.6-megapixel CMOS sensors or three 3.2-megapixel CMOS sensors mounted on a three-way dichroic prism that splits the incoming light into three separate color bands. This provides precise RGB values for each pixel without any of the averaging or estimation of color information that occurs with Bayer interpolation algorithms.
The JAI Microscopy Solution encompasses eight of these Apex Series prism color cameras, including two different resolutions and featuring models configured with or without IR-cut filters (read more about the “no filter” models in the “Extended Red Channel Sensitivity Section” below). These cameras offer a variety of capabilities tailored to the needs of advanced microscopy applications. These capabilities include:
Integration with leading third-party software solutions
Depending on the type of system you are building, you may choose to develop your own proprietary analysis software from scratch. But many systems and individual laboratory setups can save time and effort by using specialized software capture and analysis tools already available in the market. The JAI Microscopy Solution supports either approach. Standard GenICam-compliant firmware integrates with a wide range of software development tools. In addition, the Apex Series cameras have been fully integrated with Image-Pro® and µManager -- two of the most widely-used third-party microscopy software packages.
The Image-Pro image analysis software platform from Media Cybernetics (Rockville, Maryland, USA) enables users and system designers to more easily capture, process, measure, analyze and share microscopy-based images. Custom drivers allow the Apex Series cameras to seamlessly pass images to the Image-Pro software while allowing certain camera functions to be controlled from within the Image-Pro environment.
For those who prefer an open source, non-commercial software solution, device adapters have also been developed for the μManager (Micro-Manager) software originally created by Vale Lab at the University of California, San Francisco and now being developed and maintained by Open Imaging, Inc. These adapters have been developed to allow the Apex Series cameras to fully interact with the μManager software which works with microscopes from all four major manufacturers (Leica, Nikon, Olympus and Zeiss) and supports many types of peripherals (stages, filter wheels, shutters, etc.) used in microscope imaging.
Extended red channel sensitivity options
Many microscopy-based systems require a spectral range that extends beyond the visible. There can be several reasons for this. In fluorescence microscopy systems, for example, some of the common fluorophores used have emission plots that go beyond the typical 670-700 nm cut-off range used by most color cameras to avoid the effects of near-infrared light on color balancing. But that can greatly reduce the capture of these “deep-red” fluorophores, making it hard to make out certain image details. Similarly, industrial inspection systems often need to leverage the special characteristics of near-infrared light to see through plastics or see below the surface of some materials in order to better analyze for defects.The JAI Microscopy Solution includes color models where the normal IR-cutoff point has been eliminated, thereby allowing extended sensitivity into the deep-red and near-infrared wavelengths. These “NF” models provide a great deal of added flexibility when configuring the proper imaging components for a given application.
Extended sensitivity spectral chart with red fluorophore plot Color enhancement function
All of the JAI Microscopy Solution cameras – whether equipped with standard or extended red-channel sensitivity – also give the user the ability to add emphasis to certain colors in order to facilitate better analysis. Users can choose one of six colors to enhance, including the three primary colors of red, green, and blue or the three complementary colors of yellow, cyan, and magenta. Strengthening a color by up to 2X its normal intensity can be used to make certain items “stand out”, such as the red color of blood vs. surrounding tissue in medical applications, or a particular fluorophore in a typical fluorescence application.Low-noise white balancing
The starting point for any microscopy application is to make sure that the camera is properly white-balanced for the type of lighting that is being used. Typical Bayer color cameras can only be balanced by adding gain (amplification) to two of the three color channels in order to match the channel with the highest response. However, adding gain not only multiplies the signal, it also multiplies the noise in the image which can be a critical issue in some low-light microscopy applications.The prism cameras in JAI’s Microscopy solution provide independent control over each sensor, including shutter speed as well as gain. This creates the option to use shutter speed for white balancing – either by lengthening the exposure time for the two channels with lower response, or shortening the exposure time for the two channels with the highest response. While noise may increase slightly if longer exposures are used, the increase is far less than when gain is applied.
Enhanced dust suppression
All of JAI’s microscopy cameras are inspected and qualified to ensure high image quality with a low degree of FODs (foreign object debris). Some applications, however, may require more stringent suppression of dust and FODs than is found in the standard cameras. For these applications, JAI offers several “LSX” models that have been subject to even higher levels of FOD screening.Plug-and-play USB3 Vision interface
The JAI Microscopy Solution utilizes a modern USB3 Vision interface on all of the available camera models. USB is by far the interface of choice for microscopy-based systems providing an excellent blend of plug-and-play convenience with high data bandwidth. The 3.2-megapixel models can output 38 frames-per-second at full resolution, supporting the high throughput needed for industrial inspection systems, live cell imaging, extended depth of field (EDOF) and more. 1.6-megapixel models provide up to 79 frames-per-second at full resolution.
For more detailed product information including datasheets and other documentation, be sure to visit the Apex Microscopy Solutions page on this website.
Where to get more information
To discuss your specific project requirements or to get a price quote on products in the JAI Microscopy Solution, please use the contact link shown below. Let a JAI consultant answer your questions and provide you with personalized advice.
