CCD merging in imaging systems: improving signal-to-noise ratio and frame rate

Infineon / Mitsubishi / Fuji / Semikron / Eupec / IXYS

CCD merging in imaging systems: improving signal-to-noise ratio and frame rate

Posted Date: 2024-01-30

CCDs, and the engineers who integrate them into functional systems, are surprisingly good at preventing thousands or millions of electron beams from getting mixed up on their way from the pixel dot to the readout amplifier. However, if for some reason we want the charge of one pixel to mix with the charge of another, the structure of the CCD makes this easy to achieve.

Deliberately combining the charges produced by light from different pixels is called binning - as if we were throwing electrons into some kind of common bin.

Let's consider why we do this.

Advantages of binning

The advantages of utilizing merging in CCD can be divided into two major points:

Increase frame rate

Improve signal-to-noise ratio

Increase frame rate

One of the benefits of binning is speed. Binning is like downsampling; the resolution of the image data decreases depending on how much binning is performed.

Suppose we have a 1000 × 1000 pixel sensor. If we combine the two rows into one before activating the horizontal shift register, we will only have 500 rows of data. Therefore, the total readout time is reduced by about a factor of two.

Then, if we merge the two pixels into one during horizontal readout, we will create a line width of 500 pixels and reduce the readout time by a factor of two. Remember, this is not a crop - the entire image still appears in the final dataset, but at a lower resolution.

This technology can be used in a variety of applications requiring flexible imaging operations. An example is a digital camera that needs to produce high-quality still images and low-quality video. Binning provides downsampled image data at a high frame rate, and then a non-binning readout is used to obtain a single image at full resolution.

In this diagram, two vertically adjacent pixels are merged, and then two of these merged charged packets are deposited onto the output node, resulting in a merge factor of four.

Improving Signal-to-Noise Ratio (SNR)—and taking additional observations of noise sources in CCDs

Binning is similar to digital downsampling, but different. If we downsample by eliminating for example every other line and every other pixel, then data will be lost. We lowered the resolution but did nothing to improve the remaining data.

When we do binning, the data is not simply discarded because we are combining the charges of adjacent pixels. This means that binning is a way to improve SNR in low light conditions.

It is important to understand the effect of binning on SNR, and to do this we need to understand the nature of CCD noise. I plan to explore image sensor noise more thoroughly in a future article, so this is just the basics.

The main sources of noise in CCDs are photon noise, dark noise, and read noise. Photon noise and dark noise become part of the charge packet generated in each pixel. Read noise includes all noise introduced during the process of converting the CCD's charge packets into usable digital data.

Binning is not a solution to photon noise or dark noise. For example, when you combine charge packets from two adjacent pixels, you simply add the dark noise of one pixel to the dark noise of the other. The signal-to-noise ratio is unchanged.

However, the impact of read noise can be greatly reduced because the operation of the external electronics is not affected by the merger. These circuits don't even know that the merge has occurred. If two pixels receiving equal amounts of incident light are combined, the signal is doubled, and the combined pixel will see the same amount of read noise during off-chip processing. Therefore, the SNR increases by two times. If four pixels are combined, the SNR increases four times.

Binning Limitations: Don’t Exceed Your Full Well Capacity

So we can see that binning is a simple and effective way to trade resolution for noise performance. You do have to exercise some restraint, though - this method of improving SNR has its limitations.

Each charge-holding site in a CCD has a full well capacity (also called well depth), which specifies the number of electrons it can contain. Vertical merging moves additional electrons (i.e., more than one pixel's electrons) into the horizontal shift register, while horizontal merging moves additional electrons into the output node. If the amount of charge introduced by merging exceeds the full well capacity, saturation occurs and image quality is degraded.

Implement CCD merge

Let's take a brief look at some timing diagrams that illustrate the relationship between control signals and staging. We will use the example of the KAI-1003 CCD image sensor from ON Semiconductor.

The image below shows what a standard reading looks like.

Chart taken from KAI-1003 data sheet.

The V1 and V2 pulses represent vertical shift register activity: rows of pixels move toward the horizontal shift register. The V1/V2 pulses are followed by (higher frequency) pulses on the H clock (H1A, H2A, etc.) which control the horizontal shift register. The R (reset) clock pulses every H clock cycle. This signal clears electrons from the floating diffusion so that subsequent pixels' charge packets can move in.

The image below corresponds to 2×2 binning, i.e. combining squares of four adjacent pixels into one output value.

Chart taken from KAI-1003 KAI-1003 data sheet.

V1/V2 are pulsed twice so that the two rows of pixels are combined in the horizontal shift register. This completes the vertical merge.

Horizontal merging occurs because there is only one reset pulse for every two H1/H2 pulses - the horizontal shift register deposits electrons from both pixels into the floating diffusion before the charge is cleared.

in conclusion

Merging is a useful technique that allows CCDs to achieve higher frame rates and higher signal-to-noise ratios. I hope you now understand why imaging systems are graded, how they are implemented, and the limitations imposed by full well capacity.

#CCD #merging #imaging #systems #improving #signaltonoise #ratio #frame #rate