Research on the application of USB 3.0 interface technology in machine vision

“The bandwidth of USB 3.0 reaches 5.0 Gbps, which is an enchancment over at present used interfaces equivalent to USB 2.0, IEEE 1394b, GigE and Digicam Hyperlink. Larger bandwidth might help designers enhance picture decision and body price whereas sustaining picture high quality. With out compression, designers can use smaller FPGAs and scale back the quantity of reminiscence required within the system, which helps scale back BOM value, PCB space, and energy consumption.

“

By Steven Chen, Andrew Tamoney, Karnik Shah

Given the elevated decision and body price of picture sensors and the rising demand for high-definition imaging, the present bottleneck for machine imaginative and prescient is greater bandwidth necessities. Present programs can solely compromise between picture high quality and compression ratio. Larger compression ratios restrict the accuracy and efficiency of machine imaginative and prescient programs.

The bandwidth of USB 3.0 reaches 5.0 Gbps, which is an enchancment over at present used interfaces equivalent to USB 2.0, IEEE 1394b, GigE and Digicam Hyperlink. Larger bandwidth might help designers enhance picture decision and body price whereas sustaining picture high quality. With out compression, designers can use smaller FPGAs and scale back the quantity of reminiscence required within the system, which helps scale back BOM value, PCB space, and energy consumption.

Just lately, VISION, the world’s main machine imaginative and prescient business exhibition, celebrated its twenty fifth anniversary, attracting 372 exhibitors from 32 international locations world wide, and the quantity reached a brand new excessive. The machine imaginative and prescient business advantages from the continual enchancment of picture sensors, that are more and more utilized in industrial, medical, surveillance, scientific and manufacturing functions.

Present interface requirements utilized by machine imaginative and prescient cameras (GigE, IEEE 1394b, and Digicam Hyperlink) every help distinctive capabilities, however these interface requirements should weigh bandwidth, packaging, value, and energy consumption necessities, and it's troublesome to do it as soon as and for all. USB 2.0, the commonest common interface normal within the computing and shopper product markets, has a spot within the machine imaginative and prescient market, but it surely nonetheless can't meet up with different interface requirements when it comes to bandwidth. And the arrival of USB 3.0 modified all that.

This text will talk about the advantages of utilizing USB 3.0 in machine imaginative and prescient functions and analyze the important thing design issues for constructing a USB 3.0 digicam.

Benefits of USB 3.0

* 1. Its bandwidth exceeds USB 2.0, IEEE 1394b and GigE
* 2. Use one cable to transmit energy and knowledge
* 3. The implementation value is decrease than Digicam Hyperlink
* 5. Plug and play, and simpler to arrange than GigE
* 6. Adopted by the Worldwide Automated Imaging Affiliation (AIA) because the USB3 Imaginative and prescient normal

enhance bandwidth

One of many challenges designers at present face in machine imaginative and prescient is maintaining with the ever-increasing knowledge price calls for imposed by high-resolution and high-frame-rate picture sensors. Designers go to nice lengths to fulfill the required body dimension and body price necessities throughout the bandwidth supplied by the interface normal. Think about if a machine imaginative and prescient digicam was designed with a GigE interface, its decision could be restricted to the VGA degree of 120fps resulting from obtainable bandwidth. With a 5-megapixel picture sensor, the body price drops to a pitiful 5-10fps.

Some programs tackle the shortage of bandwidth by means of compression, which permits greater decision and body price knowledge to be transmitted over slower interfaces. Nonetheless, we're not inclined to make use of compression expertise, particularly in machine imaginative and prescient functions, as a result of compression has two main deficiencies: one is picture high quality, and the opposite is design compactness. The newest compression algorithms are designed with the concept of ​​utilizing progressive elimination of picture element to scale back the required bit price.

In contrast to shopper functions: in shopper functions, most picture particulars might be misplaced as a result of the human eye is imperceptible, however the photographs captured by machine imaginative and prescient programs have to be precisely calculated by picture evaluation software program. Subsequently, we should Seize uncooked knowledge preserving all picture particulars. As well as, machine imaginative and prescient merchandise are getting smaller and smaller, and most machine imaginative and prescient cameras are like an ultra-compact ice dice, lower than a cubic foot in dimension.

Nonetheless, to help the compression expertise, extra {hardware} is required, equivalent to FPGA for encoding, reminiscence for body buffering, and many others., which will increase the PCB footprint and in flip will increase the scale of the ultimate product. As well as, utilizing extra compression components additionally will increase the system invoice of supplies value, complicates the design, and in addition considerably will increase the design effort and considerably will increase the design time.

With USB 3.0, designers can have greater bandwidth: USB 3.0 helps a excessive knowledge price of 5Gbps, which is 10 occasions that of USB 2.0 (480Mbps). After 8b/10b encoding, USB 3.0 can present 4Gbps of obtainable bandwidth for knowledge. USB 3.0 continues to help the majority and isochronous switch mechanisms of USB 2.0, guaranteeing knowledge supply and bandwidth, respectively. When it comes to synchronous transfers, USB 3.0 has been considerably enhanced: the switch velocity of USB 3.0 has been elevated from 24MBps of USB 2.0 to 384MBps, which is equal to 16 occasions that of USB 2.0. Purposes that require real-time knowledge can profit vastly from this velocity enhance.

With the upper bandwidth obtainable, USB 3.0 can switch high-resolution and high-frame-rate video content material with out compromising picture high quality. Subsequently, USB 3.0 doesn't have an effect on picture high quality and helps promote additional miniaturization of machine imaginative and prescient cameras. Determine 1 exhibits a comparability of the obtainable bandwidth supplied by USB 3.0 and different interface requirements in a machine imaginative and prescient digicam. At 5Gbps knowledge charges, USB 3.0 helps extra totally different body sizes and body charges, making it a extra versatile expertise that helps many alternative functions.

Determine 1: Bandwidth comparability between USB 3.0 and customary interface requirements in present machine imaginative and prescient.

High quality and Client Prices of Machine Imaginative and prescient

The general system value of implementing a USB 3.0 machine imaginative and prescient system is way decrease than the price of implementing GigE and IEEE 1394b, and even decrease than Digicam Hyperlink. Prices will proceed to lower as USB 3.0 continues to be adopted by customers. Ninety p.c of PCs bought at present have built-in USB 3.0 ports, so customers don’t have to pay further for them. Elements equivalent to USB 3.0 connectors and cables are available. Moreover, the USB 3.0 cable supplies 4.5W of energy, sufficient to energy a machine imaginative and prescient digicam with out the necessity for added energy.

For functions equivalent to 3D imaging that make use of a multi-camera system, the price distinction is much more pronounced. Since a single USB host can help as much as 255 units, a number of USB 3.0 cameras can function in parallel on a single bus by means of low-cost and business USB 3.0 hubs. Not all different requirements provide this flexibility. Within the case of Digicam Hyperlink, an extra body grabber is required for every digicam. Determine 2 exhibits the price and obtainable bandwidth of USB 3.0 in comparison with different machine imaginative and prescient requirements. As might be seen from the determine, the bandwidth of USB 3.0 is considerably greater than that of IEEE 1394b and GigE, that are comparable in value, and virtually similar to the bandwidth of Digicam Hyperlink, which prices 3 to 4 occasions or extra.

Determine 2: Value and bandwidth comparability of a number of competing machine imaginative and prescient requirements.

Implementation of USB 3.0 Digicam

Integrating a digicam in a system requires a software program utility to learn knowledge from the picture sensor and ship management data to the picture sensor controller. This may be finished by utilizing a USB drive. The usual USB drives obtainable off the shelf for video cameras are primarily based on USB Video Kind (UVC). It's a plug-and-play machine appropriate with all PCs and is extensively utilized in video seize functions like PC webcams. Nonetheless, UVC drivers have some limitations and aren't very best for machine imaginative and prescient functions, particularly UVC drivers solely help photographs in uncompressed YUV codecs (equivalent to YUY2 and NV12), which limits the collection of picture sensors. Since picture sensors sometimes seize Bayer, RGB, or monochrome photographs, the uncooked picture knowledge should be transformed to YUV formatted photographs by means of an ISP (Picture Sign Processing Pipeline) perform. We will use FPGA to finish the above work on the picture sensor or on the PC host by means of software program. However that’s not good for some excessive body price or excessive decision cameras that generate uncooked Bayer and RGB knowledge and don’t want further headers or ISPs within the system.

As a result of UVC drivers have been initially designed for shopper video functions, they don't help the excessive diploma of customization required by machine imaginative and prescient functions, nor do they supply the assorted digicam management options that machine imaginative and prescient functions might require.

Given the shortcomings of UVC, the machine imaginative and prescient digicam business wants a totally totally different machine kind or customized driver resolution. With customized drivers, designers can select their most popular picture sensor and design management traits particularly for the goal utility. Nonetheless, to extend flexibility and add management options, longer design cycles are required.

To keep away from this delay, the Worldwide Automated Imaging Affiliation (AIA) has developed a brand new normal, USB3 Imaginative and prescient, primarily based on proposals from main members. In response to the USB3 Imaginative and prescient normal, the essential discovery of digicam units, reporting of capabilities (equivalent to acquire, brightness, gamma, picture decision, body price, and many others.)

What differentiates USB3 Imaginative and prescient is that it might help extra picture sensors that transmit photographs in non-YUV codecs, extra digicam management options, and application-layer compatibility with software program packages equivalent to GenICam. The concept is to reuse as many modules as doable from current requirements like GigE Imaginative and prescient and CoaXPress, making it simpler for designers to develop in a method they're accustomed to. This permits producers and designers to make use of the identical software program entrance finish with the quickest {hardware} again finish utilizing USB 3.0.

USB 3.0 Imaginative and prescient additionally helps customized driver implementations to fulfill the wants of producers who can't help all options and capabilities in {hardware}.

For instance, if the {hardware} doesn't have sufficient code area to find and retailer all digicam management parameters, a customized driver on the USB host discards these components to take care of compatibility with current software program functions.

Present USB 3.0 in Machine Imaginative and prescient

At current, many producers have launched USB 3.0 machine imaginative and prescient cameras. The most typical designs include a CMOS picture sensor and a Cypress EZ-USB for USB 3.0 connectivity? FX3 controller. Relying on the goal utility, producers can design their cameras in a different way, equivalent to utilizing FPGA to implement ISP and picture sensor interface conversion, or utilizing larger-capacity body buffers for picture processing or guaranteeing dependable video streaming. Determine 3 exhibits the essential block diagram of a machine imaginative and prescient system.

Determine 3: Machine imaginative and prescient design with Cypress FX3.

The FX3 is provided with a configurable Normal Function Programmable Interface (GPIF II), which permits the FX3 to not solely join on to any FPGA or picture sensor, but additionally present an information switch price of 400 MBps. Moreover, the FX3 incorporates a 200 MHz ARM9 processor with 512 KB of RAM, guaranteeing quick switch of real-time picture knowledge. The ARM9 core manages the USB 3.0 stack and might be programmed to perform as a USB Video Kind (UVC), USB3 Imaginative and prescient, or vendor-defined digicam as required.

USB 3.0: Paving the best way for the way forward for machine imaginative and prescient

The necessity for high-bandwidth, low-power, and low-cost interfaces for machine imaginative and prescient is extra pronounced than ever. At this yr’s VISION present, almost each machine imaginative and prescient digicam vendor has a USB 3.0 product or is actively designing one. The widespread help of USB 3.0 by machine imaginative and prescient distributors speaks volumes in regards to the high-quality video, low energy consumption, and low value that may be achieved with USB 3.0. With the formulation of the most recent USB3 Imaginative and prescient normal, the arrival of USB 3.0 clearly marks a significant shift in future machine imaginative and prescient interface expertise.