This PDF file contains the front matter associated with SPIE Proceedings Volume 11794 including the Title Page, Copyright information, and Table of Contents.

In this paper, the application of monocular Visual Odometry (VO) solutions for underground train stopping operation are explored. In order to analyze if the application of monocular VO solutions in challenging environments as underground railway scenarios is viable, different VO architectures are selected. For that, the state of the art of deep learning based VO approaches is analyzed. Four categories can be defined in the VO approaches defined in the last few years: (1) supervised pure deep learning based solutions; (2) solutions combining geometric features and deep learning; (3) solutions combining inertial sensors and deep learning; and (4) unsupervised deep learning solutions. A dataset composed of underground train stop operations was also created, where the ground truth is labeled according to the onboard unit SIL-4 ERTMS/ETCS odometry data. The dataset was recorded by using a camera installed in front of the train. Preliminary experimental results demonstrate that deep learning based VO solutions are applicable in underground train stop operations.

Deep learning is now the gold standard in computer vision-based quality inspection systems. In order to detect defects, supervised learning is often utilized, but necessitates a large amount of annotated images, which can be costly: collecting, cleaning, and annotating the data is tedious and limits the speed at which a system can be deployed as everything the system must detect needs to be observed first. This can impede the inspection of rare defects, since very few samples can be collected by the manufacturer. In this work, we focus on simulations to solve this issue. We first present a generic simulation pipeline to render images of defective or healthy (non defective) parts. As metallic parts can be highly textured with small defects like holes, we design a texture scanning and generation method. We assess the quality of the generated images by training deep learning networks and by testing them on real data from a manufacturer. We demonstrate that we can achieve encouraging results on real defect detection using purely simulated data. Additionally, we are able to improve global performances by concatenating simulated and real data, showing that simulations can complement real images to boost performances. Lastly, using domain adaptation techniques helps improving slightly our final results.

This paper proposes a method to estimate 3D shape of puppet head for a digital archiving. To reconstruct an inner shape of head, we use CT images. First, we divide four regions (wood, hair, paint and air) by thresholds based on manual directed regions. Subsequently, we divide these regions by a graph cut method. We also extract 3D shapes of parts in puppet head based on a graph cut method. We apply these methods to puppet heads and we confirm effectiveness of our proposed method.

Light-Field (LF) cameras allow the extraction not only of the intensity of light but also of the direction of light rays in the scene, hence it records much more information of the scene than a conventional camera. In this paper, we present a novel method to detect key-points in raw LF images by applying key-points detectors on Pseudo-Focused images (PFIs). The main advantage of this method is that we don’t need to use complex key-points detectors dedicated to light-field images. We illustrate the method in two use cases: the extraction of corners in a checkerboard and the key-points matching in two view raw light-field images. These key-points can be used for different applications e.g. calibration, depth estimation or visual odometry. Our experiments showed that our method preserves the accuracy of detection by re-projecting the pixels in the original raw images.

In this study, we focus on automatic three-dimensional (3D) face reconstruction from two-dimensional (2D) face images using a deep learning model. The conventional methods have been used to develop models that can reconstruct 3D faces from 2D images. However, for Japanese faces, the models cannot accurately reconstruct images, large errors occur in areas such as the nose and mouth, because most of the training data are foreigner’s face images. To solve this problem, we proposed a method that uses not only a frontal 2D face image but also a side-view 2D face image for the 3D face reconstruction, and the resulting 3D model is a combination of two 3D reconstructed models, which are created from the frontal and side-view 2D face images using iterative closest point algorithm. As a result, the accuracy of the proposed method is better than the conventional method.

Our research work is being carried out within the framework of the joint research laboratory ”Inspection 4.0” between IMT Mines Albi/ICA and the company DIOTA specialized in the development of numerical tools for Industry 4.0. In this work, we are focused on conformity control of complex aeronautical mechanical assemblies, typically an aircraft engine at the end or in the middle of the assembly process. A 3D scanner carried by a robot arm provides acquisitions of 3D point clouds which are further processed by deep classification networks. Computer Aided Design (CAD) model of the mechanical assembly to be inspected is available, which is an important asset of our approach. Our deep learning models are trained on synthetic and simulated data, generated from the CAD models. Several networks are trained and evaluated and results on real clouds are presented.

This work presents how deflectometry can be coupled with a light-field camera to better characterize and quantify the depth of anomalies on specular surfaces. In our previous work,1 we proposed a new scanning scheme for the detection and 3D reconstruction of defects on reflective objects. However, the quality of the reconstruction was strongly dependent on the object-camera distance which was required as an external input parameter. In this paper, we propose a new approach that integrates an estimation of this distance into our system by replacing the standard camera with a light-field camera.

Artefacts in industrial Computed Tomography (CT) compromise the image quality of a CT scan and deteriorate evaluations such as inspections for material defects or dimensional measurements. Due to a large variety of scanning objects made of different materials and of various part sizes, artefacts appear in various manifestations in the reconstructed image. Existing analytical approaches allow quantifying the CT scan quality, but still a lack of generalizability exists. Thus, assessing the scan quality is complex and error-prone, as an inappropriate set of analytical quality metrics might be considered for a certain scan setup. In our work, a scan quality estimation based on a Convolutional Neural Network (CNN) is proposed. In order to train the network, projection images of various scans are used. The reconstructed scans are labeled in a pairwise comparison by an experienced user regarding their image quality. A scalar quality value is assigned to every projection image to assess the quality. The network is deployed to perform regression for the quality value. The network is trained on multiple objects that cover the range of objects which can be sufficiently acquired with the used CT scanner. In order to enrich the features from scans of different qualities, each object is captured with various scanning parameters. Our work showed a test accuracy of approximately 80 % on prior unseen data and of up to 95 % on trained objects. In order to comprehend the black box approach incorporated by the trained CNN, visualizations of feature maps are analyzed, as regions in the projection images relevant for the quality estimation are highlighted.

The purpose of this research is to verify that the proposed method, which generates vibro-stimuli according to the distance variation, has a capability of supporting depth edge detection for aperture localization. We have proposed a method to support the visually impaired understanding their surrounding environment via modality conversion from distance variation to haptic vibration information, and have developed the wearable device to provide the user with vibro-stimuli generated from visual variation inspired by optical flow characteristics. The characteristic of this vibration stimulus is that the user feels a strong vibration when the distance difference per unit time is large. It is inferred that this feature is effective in perceiving the boundary of obstacles, i.e., depth edge, which may contribute to the user's aperture localization. To verify the effectiveness of the proposed method, an aperture localization experiment was conducted to compare it with the ETAslike conventional method by changing the number of vibration motors and the method of vibration stimuli. We asked six blindfolded subjects to guess the location of the aperture. The experimental results showed that the proposed method achieved using a single motor a 98.3% correct rate. The contributions of this research are to propose the way of vibrostimuli based on distance variation for aperture localization, which is one of the important environmental elements, and to validate its effectiveness.

PointNet, which enables end-to-end learning for scattered/unordered point data, is a popular neural network architecture. However, in many applications, large amounts of complete point clouds are hardly available for non-rigid objects such as the human body. To generate the training data of PointNet, in this study, we propose to generate human body point clouds of various postures by uniformly sampling point clouds from meshes with respect to multiple human mesh model datasets. Experiments show that the model trained with the point clouds generated from mesh data is effective in the task of human body segmentation.

In this study, a system has been developed to draw three-dimensional coordinates of a catheter in real time using Unity for cerebral aneurysm coil embolization, which is one of the endovascular treatments. The system detects the tip of a catheter by binarization and delineation from X-ray images taken from two directions (frontal and lateral views) and calculates the tip coordinates of the catheter in the 3D model of blood vessels generated using VTK from DICOM images. In this system, surgeons can more effectively visualize the catheter tip in 3D by using an head-mounted display, a virtual reality device. In the evaluation experiment, we compared the catheter tip detection with conventional methods and found that it was faster and more accurate than conventional methods. Future tasks include accurate positioning of 2D images and 3D models and drawing catheters other than the tip.

Specular and mirror-like surfaces are difficult to reconstruct by the fact that light of the surrounding environment is reflected off the surface. Nowadays, it is still a challenge in many application fields to get a fast and accurate reconstruction of specular objects. 3D acquisition by deflectometry involves estimating the 3D profile of a specular surface by analyzing the image of a light source or a pattern captured by a camera after reflection on the surface. These techniques require the knowledge of the location and the orientation of the camera and the orientation of the light source. If camera calibration is a process widely known and controlled and guaranteeing good accuracy, estimating the orientation of the light source is still an issue. We propose to overcome it by using a light source whose the color changes according to the orientation. Indeed, we implement a vision system of light source’s position recognition by the color in simulation with the 3D computer graphics software Blender, we define a calibration technique and we suggest some possible light sources to export our simulated vision system to reality. Plus, we explore the potential benefit of light field in order to model such a system.

This paper aims to evaluate the visual quality of the dynamic relighting of manufactured surfaces from Reflectance Transformation Imaging acquisitions. The first part of the study aimed to define the optimum parameters of acquisition using the RTI system: Exposure time, Gain, Sampling density. The second part is the psychometric experiment using the Design of Experiments approach. The results of this study help us to determine the influence of the parameters associated with the acquisition of Reflectance Transformation Imaging data, the models associated with relighting, and the dynamic perception of the resulting videos

This paper presents very compact range image sensors for short distance measurement, which is suitable for robot hands, etc. Robot manipulation such as grasping is one of the applications that require a range image sensor to obtain threedimensional (3D) information of the target object. For such applications, it is necessary to avoid the occlusion by a robot manipulator or a robot hand while measurement, and it is effective to attach a sensor to the robot hand for the avoidance. For this aim, a range sensor that is small enough and can measure at the short distance is required. Two sensors are constructed in this paper: one uses a multi-slit laser projector and the other uses a multi-spot laser projector. A small laser projector and a small camera is combined and range images are obtained in real time using the principle of active stereo. Appropriate methods to obtain range image are proposed for both sensors, and especially for the one with a multislit laser projector, a method to use both disparity and the intensity of laser light image is presented. The effectiveness of the proposed sensors is verified through short-range object measurement experiments.

Reflectance Transformation Imaging (RTI) is a technique for estimating the surface local angular reflectance and characterizing the visual properties by varying lighting directions and capturing a set of stereo-photometric images. The proposed method, namely HD-RTI, is based on the coupling of RTI and HDR imaging techniques. The HD-RTI automatically optimizes the necessary exposure times for each angle of illumination by using the response of the scene. Our method is applied to industrial surfaces with micro-scratches from which we will estimate saliency information. Results show that coupling HDR and RTI enhance the characterization and therefore the discrimination on the surfaces visual saliency maps. It leads to an increase in robustness for visual quality assessment tasks.

This paper proposes an efficient imaging sonar simulation method based on 3D modeling. In underwater scenarios, a forward-looking sonar, which is also known as an acoustic camera, outperforms other sensors including popular optical cameras, for it is resistant to turbidity and weak illumination, which are typical in underwater environments, and thus able to provide accurate information of the environments. For those underwater tasks highly automated along with artificial intelligence and computer vision, the development of the acoustic image simulator can provide support by reproducing the environment and generating synthetic acoustic images. It can also facilitate researchers to tackle the scarcity of real underwater data in some theoretical studies. In this paper, we make use of the 3D modeling technique to simulate the underwater scenarios and the flexible automated control of the acoustic camera and objects in the scenarios. The simulation results and the comparison to real acoustic images demonstrate that the proposed simulator can generate accurate synthetic acoustic images efficiently and flexibly.

This paper proposes a polynomial-fitting based calibration method for an active 3D sensing using a dynamic light section method. In the dynamic light section method, the relative position of the line laser is dynamically changed at high speed to extend the measurement area with a low computational cost. To conduct 3D sensing, it is necessary to find the equation of the laser plane. In the proposed calibration method, a part of the line laser is irradiated on a reference plane fixed in the 3D sensing system, and correspondences between the normal vectors of the line laser and the image coordinates of the bright point on the reference plane are obtained. Then, the correspondences are regressed to a polynomial function. As a result, the plane equation of the line laser can be obtained at any given moment without considering the complicated system model. Through the measurement accuracy evaluation of the dynamic light section method calibrated by the polynomial fitting, we showed that a measurement target at a distance of 800 mm can be measured with an accuracy of an average of -5.94 mm and a standard deviation of 13.19 mm by rotating the line laser at 210 rpm.

In this paper, we propose to measure the EMGs of the wrist and fingers using dry-type sensors worn near the wrist, and to separate the measured data into wrist and finger EMGs by using independent component analysis (ICA). Then we can confirm the EMGs of the wrist and fingers from the complex motion and realize individual identification in more complex motions. The final goal of this study is to identify individual motions from complex motions. In this paper, as a preliminary step, the ICA is used to isolate compound motions and the validity of the method is evaluated. We measured the EMGs for three days and four motions. The results of the combination of FastICA, Infomax and JADE, respectively, were evaluated by the correlation coefficient with the original signal. The most accurate combination was FastICA + Infomax with an accuracy of 70.5%

The human corneal endothelium, the posterior most layer of the cornea, is a monolayer of flat cells that are essential for maintening its transparency over time. Endothelial cells are easily visualized in patients using a specular microscope, a routine device, but accurate cell counting and cell morphometry determination has remained challenging since decades. The first automatic segmentations used mathematical morphology techniques, or the principles of the Fourier transform. In recent years, convolutional neural networks have further improved the results, but they need a large learning database, which takes a long time to collect. Thus, this work proposes a method for simulating digital twins of the images observed in specular microscopy, in order to enrich medical databases.

Generally, most face detection and recognition tasks are based on the training of intact facial images and their corresponding labels. The training image is supposed to contain as much facial area as possible, and sometimes expanding the training image area to the upper body may also enhance the learning ability. However, we noticed that both the three-dimensional structure and two-dimensional appearance from the frontal view of human faces are bilaterally symmetrical. Few research makes use of this characteristics to simplify the learning process. We have proposed a flipping strategy to apply the facial symmetrical characteristic to transfer learning and proved training with half faces can also achieve equivalent performance in face recognition for a small group of individuals. This paper extend the transfer learning of cropped half face images for face recognition rather than flipping the half face. The facial symmetrical characteristics is utilized to improve face recognition through transfer learning of only a half of the common human face image. We also investigate and explain the reason why the half face area is enough to accurately classify small groups of individuals. A variational autoencoder network is utilized to impose the probability distribution on the facial latent space. Finally, the dimensions of the facial latent space are reduced to visualize the distributed perceptual manifold for face identity.

Since contamination of food products by insects can cause serious damage to manufacturers, it is necessary to determine the timing of contamination as a countermeasure. However, manual surveys are time consuming and expensive. In this paper, we propose a method for automatic identification of insect infiltration routes using image processing.

In this paper, we propose a method to analyze and support the performance status from the images of hands and keys obtained by installing a web camera on a head-mounted display (HMD).

As devices such as depth cameras and projectors become cheaper, various methods have been proposed to make people's work environment smarter with AR technologies. In a kitchen, there would be a wide variety of advantages in an intelligent work environment, because people have to manage so many things such as various ingredients, typology of procedures with many recipes, and complicated work environment by mixing materials and tools. In this research, we aim to support cooking by recommending recipes with an installed projector and camera in the kitchen and projecting an operation interface. Our system recognizes ingredients with a DNN-based method called Mask R-CNN. It also has gesture controls that offer users contactless operations based on a Kinect depth sensor. We present four popular recipes obtained from a famous site named the Rakuten recipe. Displayed information includes dish names, ingredients, cooking procedures, process photographs, and time to cook. A series of user experiments with 10 participants was conducted to evaluate the usability of gesture operation with Kinect of the proposed system. The distance between Kinect and the hands of the participants is 0.8 (m). Each participant is given one trial and uses gestures to select one of four recipes displayed on a iMac screen. We received high evaluations (average 4.2 to 4.5 on a 5-point scale) in the results of the experiment questionnaire. Future work includes integrating more functions into our system, such as estimating ingredient amount based on the areas of recognized materials with Mask R-CNN, and cooking process recognition.

The purpose of this research is to propose the method providing visually impaired users with environment understanding via haptic vibration information. According to studies on ecological perception, optical flow, which represents dynamical visual variation, plays a significant role in environment understanding for humans. We have developed a head-mounted wearable device equipped with a 2-dimensional distance sensor and five vibro-motors arranged around the head. The vibration magnitude of vibro-motors is defined as not only a static distance to an obstacle but also dynamic distance variation caused by the user movement. The vibro-stimuli are generated based on the optical flow characteristics. The feature of the vibro-stimuli is the user feels no vibration when the distance to obstacles keeps constant. Therefore, we considered less discomfort caused by vibro-stimuli generated on the proposed method than the vibro-stimuli based on only static distance. To investigate uncomfortable feelings against the vibro-stimuli of the proposed method, we moved an obstacle closer to eight subjects for five minutes. Then, we evaluated impression for the vibro-stimuli using Semantic Differential scale composed of 15 impression word pairs. As a result, the proposed method gave fewer impressions related to uncomfortable feelings than the vibration based on only static distance. The experiment demonstrated the basic usability of the proposed method.

In this research, we propose a method to support sports instruction by displaying visualized comparison between a model motion and that of the player. Human pose estimation with the OpenPose framework based on deep learning technologies is first applied to both model and user motions. The motion difference between the model and the user is visualized with their trajectories superimposed on those two motions. To help users improve their motions and forms, our system displays where the specific body part, for example, the elbow of the throwing arm, should be located at a specific time, computed from the location and timing in the model motion. Our system also presents changes of velocity and position of body parts over time with line graphs for supposing users better understand that motion. Our current implementation is designed for pitching motions, and several other motions is included in our future work. We conducted user experiments where four participants used our system to improve their pitching forms. After recording pitching motion of these four participants, they were divided into two groups. First two participants were given the visualizations of our system while other two were presented only a pair of motions from a model and their own. The participants then performed their improved pitching motions again in front of our system to evaluate which of these two groups showed more refinements. The experimental results have shown that their forms are improved in both groups, and the usability of the proposed system has not successfully tested. Future work includes conducting larger-scale experiments, automatic selection of model motions for each user, as well as extending our method to other motions in different sports.

In this paper, we propose a support method for PC users to monitor their task engagement. Our approach is based on the number of keyboard strokes and pixel changes on the screen, and images from an ordinary webcam equipped with a PC. With our system, supervisors or teachers would improve their quality of guidance and instructions given at the right moment when workers or students require some support due to reasons such as slow progress and technical difficulties. In conventional methods, a special device such as an acceleration sensor for each individual is often required to acquire information on one’s working status and body movements, which is difficult to deploy in a real environment due to its cost for sensors. A face detection method based on Deep Neural Network, such as SSD, allows as to implement a cheaper system using an ordinary web camera. We calculate the average difference between two grabbed frames from the user’s screen to estimate the amount of screen changes between a given time interval. The number of key strokes typed by the user is another factor to estimate their task engagement. These factors are used to categorize the work mode of users. We use the K-Means method based on the Euclidean distance to cluster the recorded factors to determine thresholds for task categorization. We conducted experiments seven participants to evaluate the accuracy of our categorization method. Every participant is asked to categorize 15 scenes into four work modes. A scene includes a camera image with the PC user’s face, the screenshot path the moment, and the number of key strokes. The results from these participants are then compared with those of our system that categorized the same scenes with the thresholds on three factors. Approximately only 60% of these results matched each other, where we have enough room to improve our approach. Future work includes the selection of features that are far more effective for categorization, a better estimation of pixel changes on the PC screen, and evaluation experiments with more participants.

Social distancing is a suggested solution by many scientists, health care providers and researchers to reduce the spread of COVID-19 in public places. Over a year ago most countries have closed their borders, put people under lockdown, and have been suspending people from work and travel. However, there are still many organizations that need to operate, especially hospitals, services industry, governments, etc. However, people cannot maintain social distancing which includes staying at least 1.5 2 meters from other people because they need to communicate with each other. As a result, this increases the infection of Covid-19. This work proposes a social distancing tracking tool in offices or indoor places. We propose a YOLOv5-based Deep Neural Network (DNN) model to automate the process of monitoring the social distancing via object detection and tracking approaches. We detect office objects of known size and use it to estimate the social distance in real-time with the bounding boxes in indoor environments.

This study proposes a method of data augmentation by background replacement for the species classification of smashed mosquitoes using convolutional neural networks (CNNs). To augment data from a limited number of images of smashed mosquitoes, varieties of foreground mosquito and background are ensured by clipping a foreground mosquito image and pasting it into different backgrounds. For the background images, a white image is prepared as the ideal background, and a hand palm image is assumed as the background for practical use. Images extracted from three publicly available datasets are also prepared, which are considered as the variable backgrounds. A CNN-based deep classification is used with three types of architecture, and the classification accuracy is compared using training images corresponding to different background conditions. The classification accuracy using training images with a variety of backgrounds is better than that with a white or palm background. Moreover, deep classification with a residual network achieves the highest classification accuracy. The results of this work show that the species classification of the smashed mosquitoes can be achieved by using datasets with the proposed data augmentation method.

In this study, we proposed a new non-contact cardiac motion measurement for the purpose of treating cardiac mechanical activity (physical contraction / expansion mode) as clinical information. The proposed method is a hybrid method that combines passive stereo and active stereo, and simultaneously realized three-dimensional reconstruction of the chest shape and measurement of minute changes in the chest by cardiac motion. As a result of the experiment using the prototype system, it was clarified that the respiratory motion and the cardiac motion appearing in the chest can be extracted separately by the proposed method. In addition, based on the extracted signal of cardiac motion, it was clarified that the cardiac mechanical activity can be visualized by expressing the minute displacement on the chest surface due to cardiac motion as a color change.

The previous 3D pose and shape estimation methods often suffer from the problem of depth ambiguity. Hence, we present a novel method to reduce the depth ambiguity by explicitly considering the depth of a person’s body surface. The key idea is minimizing the difference between the depth estimated from an input image and the projected depth of a reconstructed 3D mesh. This allows the proposed method to estimate 3D pose and body shape with plausible 3D joint locations. Evaluations show that the proposed method produces more appropriate 3D meshes and reduces both 3D pose and shape estimation errors.

We propose a refinement module to improve action recognition by considering the semantic relevance between verbs and nouns. Existing methods recognize actions as a combination of verb and noun. However, they occasionally produce the semantically implausible combination, such as “drink a cupboard” or “open a carrot”. To tackle this problem, we propose a method that incorporates a word embedding model into an action recognition network. The word embedding model is trained to obtain co-occurrence between verbs and nouns and used to refine the initial class probabilities estimated by the network. Experimental results show that our method improves the estimation accuracy of verbs and nouns on the EPIC-KITCHENS Dataset.

Gestures complement the utterance contents to help the human understand. In the field of gesture generation, the task of generating gestures from utterances have attracted attention. The main method for generating gestures from utterances is to associate utterances with gestures using deep neural networks. To associate utterances with gestures using deep neural networks, a co-speech gesture dataset is required. However, building such datasets is costly and time-consuming because it requires a reliable pose estimation system (such as motion captures) and manual adjustments. We proposed an automatic method to collect a co-speech gesture dataset from online speech videos. The method extracts various utterance and gesture pairs from online speech videos. In addition, we use the collected dataset to train a deep neural network and confirm that our automatically-collected dataset can be a supervisory signal for speech-driven gesture generation.

In this paper, we propose a “Multi-modal Action Segmentation approach” that uses three modalities: (i) video, (ii) audio, (iii) thermal to classify cooking behavior in the kitchen. These 3 modalities are assumed to be features related to cooking. However, there is no public dataset containing these three modalities. Therefore, we built the original dataset and frame-level annotation. We then examined the usefulness of Action Segmentation using multi-modal features. We analyzed the effects of each modality using three evaluation metrics. As a result, the accuracy, edit distance, and F1 value were improved by up to about 1%, 2%, and 8%, respectively, compared to the case when only images were used.

We present a series of adaptations in low probability distributions scenarios to detect and track multiple moving objects of interest. We investigate the benefits of the linearization of the loss trajectory¹ in training neural networks, mainly addressing the lack of auto-differentiation in MOTA² evaluations, and observe what characteristics can support parallelism³ and differential computation and to what extent these observations contributes to our objectives. Using benchmarks from DeepMOT⁴ and CenterNet^,5 we highlight the use of sparsemax activations by mounting a finite number of independent, asynchronous detectors to augment performance and gain from compounded accuracy.∗ Empirical results show optimistic gains when applying parallelization on low-powered, low-latency embedded systems in cases where automatic differentiation is available.

We use Leap Motion and a deep neural network to perform personal authentication and character recognition of all hiragana characters entered in the air. We use Leap Motion to detect the index finger and store its trajectory as time series data. The input data was pre-processed to unify the data length by linear interpolation. For identification, the accuracy of Long Short Term Memory (LSTM) was compared with Support Vector Machine (SVM). As a result, SVM and LSTM achieved 97.25% and 98.18% F-measure in character recognition, respectively. In personal authentication, SVM has an accuracy of 92.45%, False Acceptance Rate (FAR) was 0.73%, and False Rejection Rate (FRR) was 41.59%. On the other hand, LSTM had an accuracy of 96.13%, FAR of 1.73% and FRR of 14.55%. Overall, the LSTM performed better than the SVM.

In this paper, we attempt to answer to a quality control problem in the context of an industrial serial production of lower plates (wheel suspensions) for the automotive industry. These frame parts are produced by a 2000-ton stamping machine that can reach 1800 parts per hour. The quality of these parts is assessed by a visual quality control operation. This operation is time-consuming. Moreover, many factors can affect its performance, as the attention of the operators in charge, or a too rapid inspection completion time, and non-detection defects lead to high supplementary costs. To answer this issue and automate this process operation, a system based on a vision system coupled to a pre-trained Convolutional Neural Networks (Mask R-CNN)1 has been designed and implemented. In addition, an artificial enlargement of the reference image base is proposed to improve the robustness of the identification, and reduce the sensitivity of the results to potential imaging artefacts due to non-controlled environments factors such as overexposure, blur, shadows or oil fog.

Because manufacturing processes evolve fast and production visual aspect can vary significantly on a daily basis, the ability to rapidly update machine vision based inspection systems is paramount. Unfortunately, supervised learning of convolutional neural networks requires a significant amount of annotated images in order to learn effectively from new data. Acknowledging the abundance of continuously generated images coming from the production line and the cost of their annotation, we demonstrate it is possible to prioritize and accelerate the annotation process. In this work, we develop a methodology for learning actively,¹ from rapidly mined, weakly (i.e. partially) annotated data, enabling a fast, direct feedback from the operators on the production line and tackling a big machine vision weakness: false positives. These may arise with covariate shift, which happens inevitably due to changing conditions of the data acquisition setup. In that regard, we show domain-adversarial training² to be an efficient way to address this issue.

Localized weed control is one of the promising solutions to improve the application of herbicides in crop fields. To target weeds exclusively during the spray, their location in the field must be accurately determined. As weeds have colorimetric properties similar to crops, their detection may be difficult especially under varying illumination conditions. Among available technologies, multispectral cameras provide radiance images with a high spectral resolution allowing for the analysis of vegetation signatures beyond the visible and near infrared domains. In this study, we address the problem of outdoor weed detection and identification using multispectral and RGB-NIR imaging.

Automatic solar panel inspection systems are essential to maintain power generation efficiency and reduce the cost. Thermal images generated by thermographic cameras can be used for solar panel fault diagnosis because defective panels show abnormal temperature. However, it is difficult to identify an anomaly from a single panel image when similar temperature features appear in normal panels and abnormal panels. In this paper, we propose a different feature based method to identify defective solar panels in thermal images. To determine abnormal panel from input panel images, we apply a voting strategy by using the prediction results of subtraction network. In our experiments, we construct two datasets to evaluate our method: the clean panels dataset which is constructed by manually extracted panel images and the noise containing dataset which is consisting of panel images extracted by the automatic panel extraction method. Our method achieves more than 90% classification accuracy on both clean panels dataset and noise containing dataset.

The current work proposes to move towards a quantitative characterization through advanced image processing. Four steps are followed: pin-on-disk experiments (to generate third body), image acquisition of third body particles, image processing, and extraction of quantitative characteristics of third body particles.

There is substantial interest among livestock farmers in estimating the value of live beef cattle from ultrasound images. In this study, we aimed to clarify the knowledge employed for the estimation of beef marbling standard (BMS) by experts in visual inspection. To do this, we conducted a field survey, which revealed that the experts observe ultrasound images containing various body parts of live cattle: in particular, the loin part, iliocostalis part, and shoulder clod part. To automatically estimate the BMS value, we designed a convolutional neural network architecture that incorporates the experts’ knowledge. We demonstrated that our multi-input neural network, with ultrasound images of various body parts, obtained a high accuracy in BMS estimation.

Deep learning resulted in a huge advancement in computer vision. However, deep models require a large amount of manually annotated data, which is not easy to obtain, especially in a context of sensitive industries. Rendering of Computer Aided Design (CAD) models to generate synthetic training data could be an attractive workaround. This paper focuses on using Deep Convolutional Neural Networks (DCNN) for automatic industrial inspection of mechanical assemblies, where training images are limited and hard to collect. The ultimate goal of this work is to obtain a DCNN classification model trained on synthetic renders, and deploy it to verify the presence of target objects in never-seen-before real images collected by RGB cameras. Two approaches are adopted to close the domain gap between synthetic and real images. First, Domain Randomization technique is applied to generate synthetic data for training. Second, a novel approach is proposed to learn better features representations by means of self-supervision: we used an Augmented Auto-Encoder (AAE) and achieved results competitive to our baseline model trained on real images. In addition, this approach outperformed baseline results when the problem was simplified to binary classification for each object individually.

Accurate species identification of parasitic bees is needed for studies of bio-pesticide use and habitat investigation. Therefore, we use differences in the shape of the wing veins between parasitic bee species. In this paper, we build a system that can automatically recognize species based on the extracted features from wing vein images.

Humans and animals have the ability to continually acquire, fine-tune, and transfer knowledge and skills throughout their lifespan. This ability, is referred to as lifelong learning. In machine learning, however, algorithms are trained on available data and re-training them on new data is a major challenge due to catastrophic forgetting. Therefore, lifelong learning remains a long-standing challenge for machine learning and neural network models. In this research we aimed to solve the continual learning catastrophic forgetting issue using image processing methods. We proposed multiple methods including alpha blending, histogram equalization and a pruner. We build a universal network, using a previously trained one as a feature extractor. After that, we implemented our proposed methods. In alpha blending method, we used a technique that separate knowledge from unknown knowledge. This method shows better results than normal concatenation technique. The other method was applying histogram equalization. In this method we used two kind of techniques; one is equalizer as pruner and the other is equalizer as enhancer. Our methods got promising results.

This paper describes an evaluation method of shot peened surface using image processing. Shot peening is a process that applies compressive residual stress to the product surface, and its evaluation is performed visually by an expert. If visual inspection can be replaced with image processing, the inspection of the entire product will be easier. Therefore, first reference samples that experts evaluate are prepared, next these samples are evaluated by image processing. relationship between expert evaluations and image processing evaluations are compared and the estimation function is defined using gausian distribution. Unknown processed surfaces are evaluated as a classification problem. For image processing, after binarization and labeling, the number of labels and the area ratio of binarization are used.

The main reasons behind the traffic jam and accidents are illegal/double parking, over-speeding, violating signal lights, construction, wrong-way driving, reckless driving, unsafe lane changing, etc. To determines the problems and the solutions, this study proposes a two-step approach. One is data collection and the other is Optimization. In the data collection part, traffic information is obtained from various traffic information units through cameras installed in traffic signals and roads. By analyzing the collected data, the systems can take the next step in the optimization part. In the collected data, it is required to detect vehicles, pedestrians, and lanes. Yolov3 method was used for vehicles and pedestrians’ detection. For lane detection, the Hough transform was used. The main goal of the research is not detecting objects but to determine the intelligent systems which can combine all the collected data and give the optimum solutions to control the traffic signals depending on the situations. The study found that sometimes the vehicles are unnecessarily waiting for the signals. If the unnecessary time could be saved through signals then it would reduce time consumption, oil consumption, and mental impatience. The result would give us opportunities to reduce accidents, pollution, money, and time. Moreover, the systems can measure the speed which helps find out rule violating vehicles. This paper also proposed a method for shortest path calculation. In addition, automatic penalty execution can be carried out through the collected data. For that number plate recognition is included in future works.

Toward realizing Automatic High Beam, we are working on a study to classify the beam mode of headlights as "High" and "Low" from night-time in-vehicle camera images using deep learning. Deep learning requires a large amount of training data, and the more effective data are used for training, the better the accuracy of the model is. However, since creating training data is very time consuming, it is often not possible to prepare sufficient data. One of the approaches to perform any task with limited data is to use the learned weights of another task as initial weights of the model and finetune the model with actual limited data. In the image classification, the weights learned on large-scale dataset such as ImageNet are used as initial weights to perform another task. However, in such initialization, if the dataset domain differs from the target domain, there is a report that it is not a factor of the accuracy improvement and it is necessary to learn a model on the pre-training dataset which is suitable to the target domain. Therefore, in this study, we propose a method to create a pre-training dataset easily suitable for the target domain by utilizing public dataset. Experiment results show that proposed method improves the classification accuracy of "High" and "Low" of headlights beam from nighttime in-vehicle camera image.

In order to create a classification model for fungi, it is necessary to have robustness against out-of-distribution data from the viewpoint of practicality. Therefore, in this paper, we perform out-of-distribution detection on a fungi. Unlike the case of conventional out-of-distribution detection, the characteristics of in-distribution data and out-of-distribution data in this paper are very similar. Therefore, the problem in which conventional methods using out-of-distribution data for validation are not effective is mentioned. We also verify whether the accuracy of out-of-distribution detection can be improved using the attention branch network.

Random noise injures both the basic image quality and also the following image processing procedures. The low-pass filter is commonly used as the image denoising. Low-pass filter can reduce noise; however, the edge becomes always blur as the side effect. In order to suppress this side effect, we proposed edge preserving noise reduction filter using Fast Mestimation method. As the Proposed method is applied experimentally to the noisy image, it was clarified that the noise was clearly reduced and the performance of edge preserving was realized at the same time. In this study, a quantitative evaluation of the denoising performance of the proposed method is obtained by varying the amount of noise applied and obtaining the denoising ratio.

In this paper, we propose a method for human detection from low-resolution camera images. The proposed method uses video images as input and uses 3D-CNN for classification, which is an extension of 2D-CNN and that can take into account temporal features such as gait motion. In our experiments, we used Caltech Pedestrian Detection Benchmark to make datasets of low-resolution still and video images and compared the performance between 2D-CNN and 3D-CNN. As a result, 3D-CNN with low-resolution video images achieved 91.8 % accuracy rate, 99.0 % precision rate, and 82.8 % recall rate, and showed higher performance than 2D-CNN with low-resolution images, and comparable performance than 2D-CNN with high-resolution images.