Computer vision in manufacturing: What, why, and how?

Industry 4.0 is the digital transformation of manufacturing and related industries. It’s all about a different, smarter approach to every step of the process, including production, supply chain inventory management, and more. One of the goals is to create ‘smart machines’ that are able to see, communicate, and do the same work as humans with greater precision and better results. Enabling machines to perceive information from the physical world and assist humans in complex processes opens the door for many opportunities. Many companies are either implementing or considering an idea of Computer Vision on their manufacturing sites. If you are interested in this topic as well, keep reading.

Computer Vision and Industry 4.0: Market overview

Industry 4.0 is about creating a ‘smart factory’, a manufacturing site that utilizes data from various types of sensors and all the available sources in order to optimize the processes. Computer Vision is a part of a complex approach to smart manufacturing that allows computers and machines to ‘see’ the physical world by enabling them to extract, process, and analyze information from visual inputs. 

The computer vision market is undergoing a relentless transformation, constantly creating new solutions and technological advancements. Forrester’s analysis shows that we are at the peak of the commercialization phase of Computer Vision development [2]. We already use CV for facial recognition, content intelligence and intelligence recommendation engines, and more. 

Industry 4.0 has benefited from this technology even more in time of pandemic, as it is widely used for inventory purposes.

In 2018, the global market for computer vision stood at over $9,2B, and it is expected to surpass $13.0B by 2025. Both North America and Europe are headliners in the adoption of Computer Vision in manufacturing and a number of other industries. 

Along with the growing interest in technology and market growth, companies are paying more attention to the technological advancements that AI offers. According to the research [3], Computer Vision is one of the most widely adopted technologies. It is used by at least 20% of companies worldwide. 

So why do companies around the globe are investing in AI and Computer vision in particular? 

Benefits of Computer Vision in manufacturing

• Time-efficiency. Not only does a  fully automated system operate much faster, but it can also work  24/7 if needed.
• Accuracy. Adoption of CV-based solutions allows manufacturing companies to achieve a significantly higher level of accuracy within the accepted tolerance. Combining specific equipment and advanced CV algorithms allows achieving near-perfect precision levels in production and quality control.
• Repeatability. When it comes to repetitive work, CV-driven solutions are more effective when it comes to monotonous tasks. A fully automated system speeds up production time and reduces the cost of production on many levels (e.g., there is no need to train or retrain personnel).
• Reduced costs. Apart from reduced labor costs (as fewer staff members are needed to control the process), there is less room for mistakes or deviation from the standards, thus the overall quality of the product is better, and there is less waste.
• Post-pandemic value. It seems like social distancing is here to stay, so reducing the number of staff members on the production lines will help employers maintain sanitation norms and help employees stay healthy. 

Computer Vision use cases in manufacturing

While the actual applications of Computer vision in manufacturing are almost limitless, let’s focus on the key examples of how CV and ML can improve the processes, increase productivity, and grow revenue. 

• Vision-guided robots. The most common use of CV systems are used for tool and/or detail positioning on the production lanes. The system identifies the precise location of an object or a  tool, and sends these coordinates to the robot. Typically, CV is used as a part of pick and place applications.
• Anomaly detection is used to analyze new images and compare them to a pre-existing dataset in order to find anomalies and prevent dangerous situations on manufacturing sites, production lines, etc.
• Defect reduction. When a large number of items need to be inspected on a production line, Computer Vision can be a superior technology that can help manufacturers automate this process. A complex CV solution can not only scan the item from several angles and match it to the acceptance criteria but also save the accompanying metadata. Why is it important? When the number of faulty items reaches XX, the system can inform the manager/inspector or even halt the production, pending further inspection. 
• Packaging inspection. Most manufacturers depend on a certain quality of their product, thus having the system that can automatically find any deviations from the standard is essential. For example, a CV-based inspection can track whether an item has desired color, length, and width, whether the edges are intact, or if a package is filled to the necessary level, etc.
• Barcodes and text labels scanning. Virtually every single item that is being sold now has a barcode. And scanning those barcodes is not a task that humans can do quickly and effectively on a large scale. So introducing a computer vision on a manufacturing site can a) improve the detail management process, b) speed up the order processing, and c) enhance the tracking system.
• Labeling, tracking, and tracing. When a product is mislabeled or misplaced, it’s not only about a manufacturer losing money or customers being not happy. In many cases, a mislabeled item can be harmful, especially when it comes to food/beverages or medicine. So Computer Vision helps manufacturing companies identify such items (as well as misaligned or wrinkled labels), match them to the database, and track them. 

Forrester’s analysis also suggests that by blending Computer Vision and other technologies, manufacturing companies can create new business outcomes [1].

Tools and libraries for Computer Vision in manufacturing

As any technological solution these days, Computer vision in manufacturing can have a rather versatile technical stack. In this section, we are going to take a look at some of the most widely used tools and libraries that are powering the CV.

• OpenCV is one the most widely known, used, and loved computer vision tools. It is written in C++ and has many language bindings: Python, Java, Matlab, JavaScript. It has a library of functions and algorithms for computer vision, image processing, and numerical open source general-purpose algorithms that allow users to interpret images, calibrate the camera to a pre-set standard, decrease or even eliminate optical distortions, and analyze the movement of an object. It also comes with a number of other handy functions like 3D reconstruction, object segmentation, gesture recognition, etc. 
• VisionWorks is another great tool from Nvidia that helps users build CV pipelines, powering many applications like intelligent video analytics, localization algorithms, and more. Nvidia VisionWorks also supplies a thread-safe API, making it easier and faster to track and analyze multiple scenes.
• AForge.NET/Accord.NET frameworks and Computer Vision Sandbox are a nice set of tools for .NET solutions. Both come with a wide range of available instruments, allowing their user access to CV, image and video processing libraries, advanced analysis algorithms, and more. The CV Sandbox, an open-source software package, allows solving different CV-related tasks (e.g., vision-based automation, image/video processing, multi-camera view, barcode scanning, etc.)
• SimpleCV. SimpleCV is a framework for building CV applications. With quick prototyping and interoperability with many other tools, SimpleCV  is a great option for computer vision technology in manufacturing. However, its performance might not be as great as the OpenCV, especially in a high-load environment. Another issue is a rather small community that might lack support in terms of documentation and issue-solving. 
• Vision Workbench library is NASA's take on CV tools, and it's primarily written in C++. And while it’s not a state-of-the-art item yet, this library is designed to be a base for applied research on Computer Vision. Apart from image analysis and enhancements, it can also be used for research purposes and robotics.
• PyTorch is an open source ML library that is often used for computer vision and natural language processing. It has both Python and C++ interfaces. 

Computer Visions is an interdisciplinary field. Even though these tools and platforms have a wider area of application, they can also become the powerful instrument for Computer Vision solutions in Industry 4.0.

• CUDA is a platform for parallel computing that helps users run complex processes in GPUs instead of CPU. Its toolkit includes the NVIDIA Performance Primitives library, which is a compilation of various image, video, and signal processing functions. CUDA works well with a variety of libraries and instruments, which makes it an ultimate tool in the acceleration of CV algorithms.
• TensorFlow. has been gaining more and more popularity due to its power and ease of use. Apart from some nice tools for image processing/classification, it also helps you bring Deep Learning to computer vision. With the help of Python API, one can perform face and expression detection. Among the drawbacks of TensorFlow are  a) high power consumption, b) need for a log of resources, and c) higher-qualification of developers is required.
• Keras is an open-source neural-network library that is capable of running on top of TensorFlow, Microsoft Cognitive Toolkit, R, Theano, or PlaidML. It is a user-friendly, modular, and extensible tool that extends your options while working with deep neural networks.
• FastAI for PyTorch. Developed by a non-profit group of researchers, this library simplifies training of fast and accurate neural networks with the focus on deep learning . 
• Nvidia Data Loading Library (DALI) is a portable, open-source library that powers  decoding and augmentation of images,videos and speech for deep learning applications. 
• H2O.ai is an open-source ML platform with linear scalability. It supports the most statistical and ML algorithms, and it also offers  AutoML functionality..
• OpenVINO (Visual Inference and Neural Network Optimization) is a toolkit written in C++ and Python. It consists of many pre-trained models, allowing you to optimize and deploy them, rather than develop them.
• Kubeflow is a free and open-source ML platform that was designed to make running Machine Learning workflows on Kubernetes clusters easier and more coordinated. 
• The Microsoft Cognitive Toolkit ( former CNTK) is an open-source toolkit for deep learning solutions. It is also one of the first toolkits to support the Open Neural Network Exchange ONNX format.
• Matlab is an excellent instrument for creating image processing applications. This tool helps you create clear and concise code (compared to C++), making it easier to read and debug, thus easier to support. It also allows quick prototyping. On the downside, a) Matlab is a paid tool, and b) it can get quite slow during execution time.

Computer Vision as a Service:

• Google AI Platform and Firebase ML Kit use machine learning models in a simple REST API that can be called in an application. It allows users to detect objects and faces, read printed and handwritten text, and build valuable metadata into an image catalog. AutoML also enables you to optimize the model’s accuracy, latency, and size; and export it  to your application in the cloud, or an array of devices.

• Amazon Rekognition helps users add an image and video analytics layer to their application. This service can identify objects, text, people, and activities based on the deep learning solution.

• Microsoft Azure Computer Vision API allows you to process and classify visual data (images and text in them, video in near-real-time). You can also flag adult content, generate thumbnails of images and recognise handwriting. This system has a flexible pricing policy, including free test access.

How can N-iX help you with Computer Vision for the manufacturing industry?

N-iX is a global software solutions and engineering services company with over 2,200 tech specialists that have experience working with business cases of different shapes and sizes. We have delivery offices across Europe and the Americas. Here’s why you should opt for N-iX:

1. We have strong technical expertise in Machine learning and AI, C/C++, Python, Java, .NET, and Kotlin, DevOps, and QA.
2. N-iX has a proven track record in delivering AI-based solutions to clients in various industries across the globe and one of the strongest expertise in machine learning in Ukraine.
3. We work with mid-size companies, enterprises, and tech giants from North America, Europe, and the UK.

A featured use case

Our client (under NDA) is a German-based, Fortune 100 engineering and technology company. To improve the logistics between 400+ warehouses, the client introduced a platform that turned out to be ineffective and unscalable. So, they have chosen N-iX as a partner to upgrade and extend their system.

Expertise delivered: Industrial Computer Vision, NLP, Embedded Computer Vision, Cloud-native architecture, Microservices architecture.

This project consists of three primary components: 

• Changes in architecture. Migration to microservices architecture allowed us to add  new AI-related services: anomaly detection, delivery prediction, route recommendations, object detection in logistics, OCR (optical character recognition) of labels on boxes, Natural Language Processing for document verification, data mining, and sensor data processing.
• Computer Vision solution. Our client had CV algorithms written by another vendor, which were inefficient and unsuitable for production. Therefore, we found a top-notch CV expert with a Ph.D. degree to run the CV workstream. After careful examination of the existing algorithms, we decided to redevelop them completely. We changed the architecture of the solution and introduced Continuous Delivery for Machine Learning, which allows implementing continuously repeatable cycles of training, testing, deploying, monitoring, and operating the ML models. That is especially important given the global scale at which our client is operating.
• Multiplatform CV mobile app. Also, our team designed the architecture of the multiplatform Computer Vision mobile app and is responsible for its end-to-end development. The app covers object detection, package damage detection, OCR, and NLP for document processing

If you have more questions about computer vision in manufacturing or need a CV-based solution, contact our experts today!

References:

1. All Enterprises Need (Computer) Vision, Forrester Research, Inc., June 14, 2019. 
2. Emerging Technology Spotlight: Computer Vision , Forrester Research, Inc., December 10, 2018.
3. Yoav Shoham, Raymond Perrault, Erik Brynjolfsson, Jack Clark, James Manyika, Juan Carlos Niebles, Terah Lyons, John Etchemendy, Barbara Grosz and Zoe Bauer, "The AI Index 2018 Annual Report”, AI Index Steering Committee, Human-Centered AI Initiative, Stanford University, Stanford, CA, December 2018.