The Eyes of Smart Factories - What is Machine Vision?

In a broad sense, "machine vision" refers to the utilization of visual information and optical information using digital technologies. However, because it is being introduced to production sites in the manufacturing industry, machine vision generally refers to digital image technologies utilized in automatic inspection and process automation in factories.
Machine vision consists of the following:

・Imaging section (cameras)
・Interface section (board/general-purpose interface)
・Processing engine section (computer)
・Processing control section (software)

etc.
The image data obtained by the imaging section is loaded into the processing engine via the interface section. In the processing engine section, the processing control section performs binarization^*1, edge detection^*2, and other forms of image processing. Next, it outputs the image data that has completed image processing to robots and industrial equipment.

*1: Processing that converts the pixel values (threshold values) that are preset for the image to the two values of black and white.
*2: Processing that detects the bright and dark areas in an image.

Figure 1: Machine vision configuration

Machine vision is used in automatic inspection, automatic recognition, and automatic measurement, etc. For example, automatic inspection and automatic recognition are very fast compared to manual inspection. Moreover, machine vision used in automatic measurement can accurately identify small value differences that humans cannot measure. In addition, linking machine vision to AI functions is expected to produce more intelligent uses going forward.

Differences Between Machine Vision, Computer Vision, and Robot Vision

Computer vision and robot vision are technologies that are similar to machine vision, but they each play different roles. Here, we explain how their roles differ.

Differences Between Machine Vision and Computer Vision

"Computer vision" is a technology that adds equipment control functions to machine vision functions. Computer vision can control various types of equipment based on data output by machine vision. For example, this would make it possible to combine the image data output by machine vision with CAD data generated by the computer to control processors or to adjust the conveyor transport speed based on image data that captured the progress of line work.
In addition, computer vision has a broad scope of utilization that naturally includes smart factories as well as referee assistance systems and virtual replays in sports and projection mapping, etc. in entertainment.

Figure 2: Example of computer vision utilization

Differences Between Machine Vision and Robot Vision

"Robot vision" is a technology that adds machine vision to robot visual functions. Machine vision attaches cameras to various types of equipment to obtain image data, but robot vision attaches cameras to the end of robot arms to obtain image data. This enables the robot to operate based on visual information obtained by itself.
For example, a robot that does not use robot vision can only execute certain operations programmed in advance. However, when a robot equipped with robot vision moves its robot arm closer to an object, it then sequentially identifies the target based on image data from the camera to execute operations.
Because this type of operational control is possible, a robot equipped with robot vision is capable of detailed, precise, and flexible operations that cannot be specified with a program.

Figure 3: Example of robot vision utilization

Example of Machine Vision Utilization in a Smart Factory

Machine vision is used in various types of work in a smart factory. Here, we introduce an example of machine vision utilization.

Reading Characters at Ultrahigh-speed – OCR (Optical Character Recognition/Reader)

OCR is a technology that scans handwritten or printed characters with an image scanner or digital camera. Machine vision scans the 2D code, serial number, and date of manufacture, etc. printed on products. The scanned data is utilized in inspections of the print quality or whether the correct characters are printed as well as the screening of the next process, etc. In many cases, the products being scanned by OCR pass through the line at high speeds. Therefore, machine vision uses cameras that can capture images at high speed and image processing systems that are capable of high-speed processing.

Figure 4: OCR on a manufacturing line

Scanning Shapes in 2D and 3D – Visual Inspections

The utilization of machine vision is considered to be the most effective means of visual inspection on a manufacturing line. Because machine vision-based visual inspection is able to inspect the accuracy of products after cutting and molding and the mounting condition of components, etc. with high speed and accuracy, it is essential for improving production efficiency and maintaining quality. The surfaces of target objects can vary widely in terms of their glossiness and light absorption, etc. The lighting used in machine vision enables products with various surface conditions to be inspected by changing the method of illumination and the color of the light rays. In addition, 3D cameras equipped with a stereophonic^*3, time-of-flight^*4, and other measurement systems as well as structured light^*5 and other imaging systems can also measure the depth.

*3: Method that uses two cameras to perform triangulation and measure the distance to an object.
*4: Method that measures the distance to an object according to the time it takes for light emitted by a light source to reflect off of the object and return to a light detector.
*5: Lighting method that emits light with a specific shading pattern.

Figure 5: Visual inspection with a 3D camera

Grabbing Specific Components – Picking

"Picking" on a manufacturing line refers to the task of collecting (pickup) the necessary components and products. Utilizing machine vision makes it possible to identify and pick components with the specified color and shape even if components and products with various colors and shapes are mixed together in one case.

Figure 6: Picking by a robot

Providing Visual Functions to Robots – Robot Operational Control

In recent years, industrial robots have also been used in precision work due to their high operating precision. For example, they play an active role in installing, soldering, and welding microcomponents. However, controlling such work with programming requires high-precision positioning devices. With robot vision utilizing machine vision, high-precision positioning devices are not required for the robot to perform precise positioning with the image data from the camera mounted on the end of the robot arm.

Figure 7: Installing microcomponents with a robot

Future of Machine Vision – AI Vision

As various types of industrial equipment shift to using AI, one form of machine vision evolution that shows promise is "AI vision."

What is AI Vision?

AI vision is a system that equips lightweight AI chips in smartphones, etc. with cameras and camera functions to make inferences about captured image data. AI vision resembles computer vision in that it uses AI (artificial intelligence), but computer vision mainly makes inferences with a cloud AI service (Figure 8).
On the other hand, AI vision equips an AI module in an edge device to make inferences, and the inference results are stored on the edge server. The edge server communicates with the cloud AI only when necessary. (Figure 9).

Therefore, AI vision has the following advantages over computer vision:
・Able to decrease the volume of communication with the cloud AI service
・Able to prevent external leaks of data collected by the sensors
・Superior responsiveness
Furthermore, the cameras and smartphones used in AI vision are called "edge AI devices" while the method that makes inferences on the edge device is called "edge AI."

What Technologies Are Required to Realize AI Vision?

Although AI vision has the advantages described above, AI modules equipped on edge devices require the following types of functions.
・Superior energy conservation due to restrictions on battery capacity
・Ability to perform high-speed processing due to the responsiveness requirement
・Superior heat dissipation due to installation in a limited space
More than anything, it is important that the AI modules be compact.

Summary

This article explained the topic of machine vision and the similar phrases of computer vision and robot vision as well as AI vision. Going forward, it is expected that machine vision and AI will evolve in a close relationship with the evolution in deep learning for image data. In addition, new AI systems and products are expected to be developed through the proper use of cloud AI and edge AI.
These trends will likely become essential technologies for smart factory robots, automatic inspection, automatic recognition, and automatic measurement equipment that require visual information that is closer to the human eye and decision-making capabilities that exceed that of humans.