Case study: Weld Visualization using smart 3D-vision Helmet

This technology has been applied in a next-generation, 3D-welding smart helmet that uses cameras to collect the data and utilize it in real time.
For the camera part of the device, a compact USB3 camera from XIMEA was chosen to provide the required output.
 
Thus Kawada Group´s welding experience, SRI International problem-solving ability and XIMEA´s machine vision expertise help to transform the world of welding.
As a result, welders won´t need years to perfect their craft at a time when the Japanese construction industry already struggles with labor shortages.     

Who is taking part in the project?

The XDR project began with the KTI Kawada Group intending to pass on the core of their unique tradecraft techniques in an effective and sustainable manner.
Kawada Industries is one of the leading construction and bridge-building firms in Japan that manufactures many of the thick steel plates joined by arc welding. 
 
However, to find a way to accelerate the learning curve for novice welding trainees KTI Kawada Group needed the assistance of optical integration experts.
Therefore they partnered with SRI International, from Menlo Park, California, who create world-changing solutions making people safer, healthier and more productive.

"Kawada came to SRI with a need to greatly shorten the time required to train new welding technicians while continuing to pass down welding knowledge in an effective manner to new welders,” said Mike Piacentino, Senior Technical Director of Vision Systems for SRI International. “The real-time weld-visualization technology developed by SRI provides a higher dynamic range to allow humans and robotic welders to see more than just the welding tip. The ability to view real-time welds maintains quality and informs the welder of low quality or possibly failing welds, which is an important advantage over conventional welding where precision welds are critical to structural integrity and public safety of infrastructures such as construction and bridge engineering.” 

What is the goal ?

The arc welding procedure produces a combination of ultraviolet, visible and infrared light so bright, that it can inflame the corneas and/or burn the retina. 
The filter in the specialized helmet’s lens shade darkens this light to allow the welder to look at it directly, but it also blacks out the rest of the scene.
 
Thus the welders have to pause their working process to remove protective visors and check the welding quality.
Such a method is dangerous and considerably prolongs the period of skill and craftmanship accumulation. 

For this project, the objective was to enable a larger number of welders and improve their welding quality by greatly enhancing the visibility of the welding scene.
This includes revealing the welding tip, weld beads, welds and the working environment of the welding scene.
 
Besides Beside achieving a full unobstructed view of the workspace, the target was to streamline inspection in real time with a finer degree of control.
Important was to achieve it through the use of ordinary image sensors rather than special high cost imagers. 

Which technologies are used in XDR Weld ? 

SRI applied advanced vision technologies to invent XDR weld visualization, which uses small camera modules and image fusion to display a composite image in 3D.
Specifically, XDR controls the shutter speed synchronization and the time at which images are captured - 1/1000 seconds.
 
It acquires multiple images with different exposure conditions to avoid under or overexposure and perform a real time XDR Fusion.
The fusion is accomplished by a wearable battery-operated GPU which synthesizes the frames into one 3D stereo vision stream with a 150 dB dynamic range.

Images with this dynamic range are displayed, with minimal latency, inside the helmet on a head-mounted display (HMD) as a stereo image.
To reduce the disconnect between action and visuals, minimizing the operator's disorientation, the display refresh rate is set to 30 fps with image delay under 2-3 frames.
 
Among display's functions is the possibility to record various status parameters related to welding like the voltage, temperature, current, etc.  
Simplified collection and examination of useful information ensures a high-quality weld and expedites the learning curve in the long run. 

Why these components ? 

Arranging the quick image fusion is a job for NVIDIA Jetson AGX Xavier 512-core GPU with an 8-core, Carmel ARM v8 64-bit CPU supporting 7.8 double-precision TFlops.
The GPU is typically attached to the welder’s belt and tethered to the helmet, where the rest of the equipment is located.

The helmet employs cameras from XIMEA which feature Sony CMOS Pregius sensors, and a low-resolution, radiometric longwave infrared imager.
Traditional commercial cameras are typically not suitable to fulfill the demanding requirements of such application or might fail during long operation. 
 
XIMEA specializes in compact high-performance cameras, embedded and multi-camera systems making it an ideal candidate for innovative projects.
The model selected from the XIMEA portfolio was MC031 from the xiC camera series with 3.1 Mpix at 122 fps, low power consumption, mini dimensions and weight. 

In outcome, the limited dynamic ranges of 70 dB are expanded by the XDR welding helmet to 150 dB without the need for special image sensors.
In addition, using professional products permits to benefit from high volume price reductions and proven component reliability and availability. 
 
The whole setup runs on hot-swappable batteries removing the power consumption limit in environments that have no external energy source. 
Finally, the welder’s eyes are completely protected from any damage because the whole weld is viewed indirectly through the display screen in the helmet.

Where does development lead next?

Companies such as Kawada and SRI International aim to be at the forefront through constant innovation of the current technology.
The leading-edge 3D-welding helmet will be refined over the next year and is planned for deployment to teach technical welding skills at Kawada Industries.
 
Since XDR can capture images of the welder’s line of sight, this data gathering can be used for study by both the individual welder and novice technicians.
The KTI Kawada group is expected to gain and analyze the insights from this project which could be applied to support new technological and product developments. 

Conclusion

The motive is simple: welding helmets need a fresh idea to make the users safer and faster while the activity would produce better results and would be easier to master.
 
“Welding is the most important and difficult skill to transfer from master welders to novice welders,” said Noriyuki Kanehira, Project Leader and Director, Kawada Technology Research Center. “Visualization is the key. In conventional welding, the operator can see only the tiny bright area of the arc and everything else is darkened by the thick shading filter attached to a welding helmet. This new technology developed by SRI clearly visualizes the entire area of the workspace with appropriate brightness and contrast. Welding visualization directly leads to stabilizing welding quality and providing traceability.”
 
The new Extreme Dynamic Range weld visualization technology (XDR) enables technicians to weld more safely and assemble useful data. 
XIMEA is glad to be chosen for this project based on the camera tiny size and weight, low power consumption and ease of integration with Nvidia’s powerful products.