Metal Nuts Use Case

The task is to detect defects on metal nuts and classify them. This is a defect detection followedby a 3-class classification task and can be realized as a pipeline.

Dataset:

The dataset contains 70 positive (defected) and 22 negative (normal) metal nut samples. There are 3 types of defects, all annotated in LabelMe format, as depicted below.

Data Loading & Annotation:

You have 2 options:

Download data annotated with LabelMe format. There are two folders, namely “OK” and “NOK”,in the downloaded dataset corresponding to defect free samples and defected samples. For both folders, load the imageset and the annotations that you downloaded separately using ReliUI. Import the imageset using “Import Data” and the annotation file per image folder using the “Import Annotated Dataset” (with the LabelMe annotation format option). You will have two annotated datasets, one for each folder, available in the gallery.
Load the imageset and annotate the data yourself in any industry standard format you like using the ReliUI’s intuitive annotation functions. Download the OK imageset and NOK imageset separately. Follow the ReliVision Knowledge Hub User Guide (ReliUI: Data Curation) to define the target states/labels as “bent”, “color” and “scratch” and to annotate the NOK imageset using polygon annotation tool. Save them in an appropriate industry standard format (for which LabelMe is a good option). Leave the OK imageset unannotated as they do not have any defects. A sample annotation of a “bent” sample is as follows:

Building A Metal Nut Defect Detection and Classification Pipeline:

We will be building a pipeline that is composed of a series connected defect detector and a defect classifier. The defect detector will be responsible for detecting the defects, if there are any, and outputting a bounding box. The classifier will be fed with defect detection results (the cropped images of the defects detected) and will decide on the type of the defect (bent, color, scratch).

Training a Defect Detector

Prepare your data for defect detector training: In order to train a defect detection model, you need to have training images in which all defects, of all types, are annotated with a common label, such as “defect”. The dataset (NOK imageset) provided has polygonal annotations with 3 labels corresponding to different defect types (bent, color, scratch). Merge your NOK labels in your NOK imageset into a single class called “Defect”. Visit ReliVision Knowledge Hub User Guide to see how to do this (Edit ROI States). Then merge the OK imageset with the single class labeled NOK imageset, that you have generated by merging the 3-class NOK labels, into a single dataset. Visit ReliVision Knowledge Hub User Guide to see how to merge two datasets (ReliUI: Manipulate Datasets/Merge & Download Datasets).
Follow the ReliVision Knowledge Hub User Guide (ReliTrainer: Training an AI Block) to train your AI model for metal nut defect detection. The main steps, detailed in the User Guide, include

Model type selection: Object Detection
Annotated dataset selection
Automated or manual train/test split which essentially spares some data for training validation purposes.
Hyper parameter setting: We have chosen the following in this use case:

Epochs: 1000
X-Y Resolution: 448
Learning Rate: 0.01 (default)
Momentum: 0.937 (default)
Weight Decay: 0.005 (default)

Sample training and validation loss and mAP curves as a function of epochs are as follows:

Training a Defect Classifier

Prepare your data for defect classifier training: In order to train a defect classification model, you need to have training images of individual (cropped) defects labeled with the defect type (bent,color, scratch). The dataset (NOK imageset) provided has polygonal annotations with 3 labels corresponding to different defect types. Extract the polygonal annotations (defects) using the ReliUI data curation functions. Visit ReliVision Knowledge Hub User Guide to see how to extract annotations. Use the dataset thus created with extracted defects for training your classifier.
Follow the ReliVision Knowledge Hub User Guide (ReliTrainer: Training an AI Block) to train your AI model for whole image classification (bent, color, scratch). The main steps, detailed in the User Guide, include

Model type selection: Classification
Annotated dataset selection
Automated or manual train/test split which essentially spares some data for training validation purposes.
Hyper parameter setting: We have chosen the following in this use case:

Epochs: 1000 (default)
X-Y Resolution: 244 (default)
Learning Rate: 0.0001 (default)
Momentum: 0.99 default)
Weight Decay: 0.0005 (default)

Sample training and validation loss and accuracy curves as a function of epochs are as follows:

Training a Defect Classifier Building and Running the Pipeline:

Use the pipeline editor to build your pipeline by dragging and dropping the AI/Basic blocks. For this specific case, we do not need any Basic Block (Digital signal/image processing - DSP/DIP - functions). You will need to select an input data source (your raw image set), a detection AI Block (the defect detector you trained) and a defect classifier (the classifier you trained) connected in series as depicted below. Follow the ReliVision Knowledge Hub User Guide (ReliTrainer: Pipeline Editor).

You can review your results using the ReliUI data annotation interface. Your pipeline’s outputs will be saved as a separate set of annotations for each image it is run on. Visit the ReliVision Knowledge Hub User Guide (ReliUI: Data Curation) for more details on reviewing annotations / inference results.

Detected Scratches:

Detected bents:

Try it by yourself.
Download ReliVision now!

AI Model Train / Test

AI powered solution design-train/test-build functions of the ReliVision are performed in the ReliVision's Training Server [ReliTrainer] which is accessible through the user-friendly ReliUI application. You can train detection, segmentation and classification models, test your models, build complex visual analytics pipelines using your trained AI blocks (AI models) together with Basic blocks (Digital signal/image processing - DSP/DIP - functions).

Refer to ReliTrainer user guide for further details. AI Model Train/Test functions are available with a licensed ReliTrainer.

Deployment

AI powered visual analytics solutions that are built, trained and tested, can be deployed to ReliAudit through the user friendly ReliUI interface. The deployed solutions are run, managed and monitored at the edge by the ReliAudit. The user feedback received from the ReliAudit can also be pulled back to ReliTrainer via ReliUI for retraining.

Refer to ReliAudit user guide for further details. Deployment functions are available with a licensed ReliAudit.