Automate detection of broken utility poles using the Amazon Rekognition Custom Labels SDK

The following image is an example of a damaged pole.

If your images are already identified, Amazon Rekognition Custom Labels can start training in simply a few click the Amazon Rekognition console. If not, you can identify them directly within the Amazon Rekognition Custom Labels identifying interface, or use another service such as Amazon SageMaker Ground Truth to identify them. After you train your image set with Amazon Rekognition Custom Labels, it can produce a customized image computer system vision model for you in simply a few hours. After this custom model is trained, you can utilize it as an endpoint to make inferences on new images.
In this post, we use the Amazon Rekognition Custom Labels API and the AWS SDK to demonstrate how quickly you can integrate this technology into your applications.
Prepare dataset pail with images.
Just like all ML designs, we start with some information– for this post, images of broken and not broken energy poles. This dataset is fully labeled and kept in Amazon Simple Storage Service (Amazon S3). The area of this dataset is fed into the SDK to train the design
Train the design.
Now that our identified data is in our S3 pail, lets produce the design for our information.

Electric utility poles, for example, are an example of a cumbersome facilities issue to deal with. The market standard right now is to utilize drone or street cameras to create images of these damaged poles. The poles in the images are then by hand examined to ensure theyre in good condition and dont need repair work or replacement.
To address this, we propose a service utilizing Amazon Rekognition Custom Labels. You can feed images of energy poles, drawn from street video cameras or from drones, into a machine computer system vision model trained on Amazon Rekognition Custom Labels to immediately detect whether an energy pole is in good condition or damaged.
Amazon Rekognition is a computer system vision service within the AWS AI/ML stack. It enables for the automation of image and videos analysis. With Amazon Rekognition, you can recognize objects, people, text, scenes, inappropriate material, and activities in videos and images.
We utilize Amazon Rekognition Custom Labels for our solution, which allows us to create customized artificial intelligence (ML) designs to analyze images With Amazon Rekognition Custom Labels, you can train a robust, deployable design with a few images as opposed to thousands of images.
For our usage case, we utilize images of electrical poles. The following is an example of a regular pole.

Import the needed libraries:

Because Amazon Rekognition Custom Labels needs you to create a task for a provided usage case, we use the following function to create a task:

#Create a task.
print( Creating job: + project_name).
response= client.create _ project( ProjectName= project_name).
arn = reaction [ ProjectArn] print( project ARN: + response [ ProjectArn].
return arn.

shot:.
reaction= client.create _ project_version( ProjectArn= project_arn,.
VersionName= version_name,.
OutputConfig= output_config,.
TrainingData= training_dataset,.
TestingData= testing_dataset).

The function returns the job ARN, which you should write down or store in a variable due to the fact that you utilize it later on to train the model.
We next define an approach to train a design using Amazon Rekognition Custom Labels. This approach needs the project_arn as an input (the job ARN variable that you conserved earlier), a distinct version_name for this variation of the design, out_config to inform where to save training outcomes, along with places of the training and test data manifest files.

customer= boto3.client( rekognition).

Next, call the train_model() technique, as revealed in the following code:.

# Train model helper method.
import json.

def train_model( project_arn, version_name, output_config, training_dataset, testing_dataset):.

# Wait for the job version training to finish.
project_version_training_completed_waiter = client.get _ waiter( project_version_training_completed).
project_version_training_completed_waiter. wait( ProjectArn= project_arn,.
VersionNames= [version_name].

Run the following train_model() technique, and take down the job variation ARN to use later:.

print( Starting training of: + version_name).

#Get the completion status.
describe_response= client.describe _ project_versions( ProjectArn= project_arn,.
VersionNames= [version_name].
for model in describe_response [ ProjectVersionDescriptions]:.
print( Project Version ARN: + model [ ProjectVersionArn].
print(” Status:” + design [ Status].
print(” Message:” + model [ StatusMessage].
other than Exception as e:.
print( e).

customer= boto3.client( rekognition).

def create_project( project_name):.

print( Done …).

project_arn= arn: aws: rekognition: us-east-1: xxxxxxxxxxxx: project/project _ utility_pole_blog/ yyyyyyyyyyy

version_name= v1

Random rotation of that image produces the following rotated image.

#Call DetectCustomLabels.
response = rek_client. detect_custom_labels( Image= ,. MinConfidence= min_confidence,.
ProjectVersionArn= design).

# Helper method that makes an inference on a Custom Labels trained model for a binary classification problem.
import boto3.
import io.

testing_dataset= json.loads( ).

We set “AutoCreate”: true for testing_dataset because were utilizing Amazon Rekognition Custom Labels to divide the training information arbitrarily into an 80/20 split. You can define the testing_dataset as a manifest file, just as is done for the training_dataset in the preceding code, if you have a different test dataset.
Training can take a couple of hours to complete.

Original model AUC = 0.92.
Enhanced model AUC = 0.96.

from sklearn.metrics import accuracy_score.
print(” Original design accuracy=”, round( accuracy_score( y_true, y_pred_original), 2)).
print(” Enhanced design precision=”, round( accuracy_score( y_true, y_pred_enhanced), 2)).

except ClientError as err:.
print(” Exception in get_custom_labels()”).
raise.

client = boto3.client( rekognition).

def delete_project( project_arn):.

After you train your image set with Amazon Rekognition Custom Labels, it can produce a customized image computer system vision model for you in just a couple of hours. For the 2nd design, instead of feeding the raw images as they are, we do some information enhancement on these images, which is typical in computer system vision problems. ==” broken”:
y_pred. ==” good”:
y_pred. ==””:
raise Exception(” Invalid label”).

“Bucket”: “my-bucket”, “Name”: “datasets/cables-ds/manifests/ output/output.

output_config = json.loads( “S3Bucket”:” my-bucket”, “S3KeyPrefix”:” blog ).

reaction= client.describe _ project_versions( ProjectArn= project_arn,.
VersionNames= [version_name].

def horizontal_flip( image_array: ndarray):.
# horizontal flip does not require skimage, its simple as flipping the image variety of pixels!
return image_array [:,:: -1]

return reaction [ CustomLabels] [0] [” Name”], response [ CustomLabels] [0] [” Confidence”]
else:.
“Confidence”]

client= boto3.client( rekognition).

try:.
#Get image from S3 container.
s3_object = s3_connection. Object( pail, image).
s3_response = s3_object. get().

You can get the present status by calling DescribeProjectVersions and, when its total, calling DescribeProjectVersions to get the training results and assess the model:.

Our original information of 80 images is arbitrarily split into 60 pictures of training information and 20 pictures of test information. The initial model is constructed using Amazon Rekognition Custom Labels on 60 images of the training information.
For developing our boosted model, we enhanced the 60 training information images by running them through the preceding code, which includes rotation and horizontal flip. That increases the training data size from 60 to 180. We build a new model utilizing this dataset.
After we construct these 2 designs, we run test data these models to see how the outcomes compare. We utilize the following code to acquire the outcomes:.

import random.
import numpy as np.
from scipy import ndarray.
import skimage as sk.
from skimage import change.
from skimage import util.

from botocore.exceptions import ClientError.

def describe_model( project_arn, version_name):.

def getLabelAndConfidenceBinary( bucket, image, design):.
“””.
: param bucket: The name of the S3 pail which contains the image that you want to evaluate.
: param image: The name of the image that you want to analyze.
: param design: The ARN of the Amazon Rekognition Custom Labels model that you wish to utilize.
“””.

# set minimum self-confidence to 0 as we are interested in results for all condidence levels.
min_confidence = 0.

import numpy as np.
from sklearn import metrics.
def calculateAUC( y_true, y_prob, pos_label):.
fpr, tpr, thresholds = metrics.roc _ curve( y_true, np.array( y_prob), pos_label= pos_label).
return metrics.auc( fpr, tpr).

train_model( project_arn, version_name, output_config, training_dataset, testing_dataset).

The following is our original image.

bucket=”my-bucket”.
# Assign Project Version ARN returned by train_model() listed below for both the designs.
original_model=”arn: aws: rekognition: us-east-1: xxxxxxxxxxxx: project/upole-new-aug 14/version/upole-new-aug 14.2021-08-14T17.20.06/ 1628976006624″.
enhanced_model=”arn: aws: rekognition: us-east-1: xxxxxxxxxxxx: project/upole-new-aug 14/version/upole-new-aug 14.2021-08-14T18.36.45/ 1628980605880″.
y_pred_original, y_prob_original = getPredictions( original_model, container, test_images).
y_pred_enhanced, y_prob_enhanced = getPredictions( enhanced_model, bucket, test_images).

About the Authors:.
Winston Nwanne is an AWS Solutions Architect working with public sector partners. He specializes in AI/ML and has actually helped partners and customers broaden their capabilities within the AWS Cloud. Apart from supporting clients, he likes to check out, make YouTube videos about monetary literacy, play basketball, and hang out with his family.
Raju Penmatcha is a Senior AI/ML Specialist Solutions Architect at AWS. He deals with education, federal government, and nonprofit customers on artificial intelligence and expert system associated tasks, assisting them construct solutions using AWS. He likes traveling to brand-new places when not helping consumers.

Conclusion.
In this post, we effectively trained, assessed, and inferred on a customized ML design for finding broken and broken energy poles. This when time-consuming and manual job was normally very prone to human error or overlook, but by using Amazon Rekognition Custom Labels, we were able to make this process quicker while maintaining precision.
For more details about using customized labels, see What Is Amazon Rekognition Custom Labels?

print(” Original model AUC=”, round( calculateAUC( y_true, y_prob_original, 1),2)).
print(” Enhanced model AUC=”, round( calculateAUC( y_true, y_prob_enhanced, 1),2)).

Initial design precision = 0.79.
Enhanced design precision = 0.89.

for design in reaction [ ProjectVersionDescriptions]:.
print( json.dumps( design, indent= 4, default= str)).

print( Deleting task: + project_arn).
reaction = client.delete _ project( ProjectArn= project_arn).
print( Status: + reaction [ Status].
print( Done …).

# This code loops through all images and produces a y_true list objects based upon image labels.
image_folder=”blog/REK-CL/utilitypoles/ new_test_images/”.
y_true = [] names = [] test_images = getImageNames( my-bucket, image_folder).
for image in test_images:.
iclass = image [1] name = image [0] names.append( name).
if iclass==” bad”:
y_true. append( 1 ).
else:.
y_true. append( 0 ).

# Helper method that iterates through an S3 container and obtains image labels.
def getImageNames( container, prefix):.
import boto3.
client = boto3.client(s 3).
paginator = client.get _ paginator( list_objects_v2).
outcome = paginator.paginate( Bucket= bucket, Prefix= prefix).
images = [] for page in outcome:.
if “Contents” in page:.
for secret in page [” Contents”]:.
keyString = crucial [” Key”] name = keyString.split(“/”) [-1] iclass = name.split(“-“) [0] if len( iclass)>> 0:.
images.append( [keyString, iclass].
return images.

def random_rotation( image_array: ndarray):.
# select a random degree of rotation between 25% left wing and 25% on the right.
random_degree = random.uniform( -25, 25).
return sk.transform.rotate( image_array, random_degree, preserve_range= True).

Performance review.
As we can observe from the model results, doing image augmentation has actually assisted improve model precision significantly, from 0.79 to 0.89, and design AUC from 0.92 to 0.96.
The initial design gave great results as is, doing image augmentation has actually further enhanced the outcomes. You dont always need to augment your images all the time, however you can utilize this method to see if it can even more enhance your model.
Clean up.
After you end up utilizing this option, its important that you stop your model to stop accumulating charges. To erase your project, simply run the following function:.

rek_client= boto3.client( rekognition).
s3_connection = boto3.resource(s 3).

Design results and enhancement.
We train two models in this post. The first design takes 80 pictures of damaged and excellent energy poles, which are similarly split between the good and broken. These images are fed into Amazon Rekognition Custom Labels and the model metrics are evaluated.
For the second design, instead of feeding the raw images as they are, we do some information enhancement on these images, which prevails in computer system vision issues. Amazon Rekognition Custom Labels does not do information augmentation by itself because it doesnt know your images too well. We recommend explicitly doing image augmentation for scenarios where you want to more improve your design metrics.
We then compare how the design metrics such as accuracy and AUC score compare for the original design and the improved design.
Image enhancement is accomplished utilizing the following code:.

# The assistant approach below makes forecasts on a deploy Custom Labels model and returns predictions.
# Custom Labels associates self-confidence level to the label it forecasts.
# In the code below, we are turning that to a [0 to 1] likelihood scale, where 1 shows a broken pole.
def getPredictions( design, pail, images):.
y_pred = [] y_prob = [] for image in images:.
labelconf = getLabelAndConfidenceBinary( bucket, image [0], design).
, if labelconf [ 0] ==” broken”:
y_pred. append( 1 ).
prob = labelconf [1]/ 100.0.
y_prob. append( prob).
if labelconf [ 0] ==” excellent”:
y_pred. append( 0 ).
prob = 1.0 – labelconf [1]/ 100.0.
y_prob. append( prob).
if labelconf [ 0] ==””:
raise Exception(” Invalid label”).
return y_pred, y_prob.

Leave a Reply

Your email address will not be published.