AWS Panorama

What is AWS Panorama?

AWS Panorama allows you to apply machine learning algorithms of computer visions to the on-premises camera or image/video input devices. It uses machine learning models for AWS to build its computer vision application. It improves operations with computer vision at the edge. You can create your applications using AWS support for machine learning and computer vision without sharing the image data with the AWS cloud.

AWS Panorama provides us to execute some of the everyday computer vision tasks using the on-premises setup without uploading the data to AWS Cloud like:

1. Object Detection
2. Object Tracking
3. Easy Object Character Recognition

AWS Panorama has in-built libraries and templates for carrying out each of these operations. You can also configure and optimize your machine learning models as per your need.

How Does It Work?

AWS Panorama integrates machine learning devices and software, making it possible for on-premises IP cameras to conduct computer vision processes without sending the data directly to the AWS cloud.

Let's understand the detailed working of AWS Panorama with the help of the given diagram.

• AWS Panorama contains some pre-defined machine learning models and templates, which can be used, or some custom models can also be created to run CV applications.
• The AWS Panorama Appliance is a hardware device installed in the network, and the on-premises IP cameras are connected to it so that they can be further connected to AWS Panorama.
• Multiple cameras can be added to the AWS Panorama Appliance, and multiple CV applications can be run parallelly.
• The results generated by the application hosted on-premises using AWS Panorama can be redirected to other AWS services for storage or actions like Amazon S3, Cloudwatch, etc. It can also be redirected to other on-premises devices to trigger other actions as needed.

AWS Panorama Client

AWS Panorama client allows us to connect programmatically to AWS Panorama. It contains methods that connect with AWS Panorama API to get things done.

We can install a panorama client using the age manager of our choice. Some of the common ones are:

• npm install @aws-sdk/client-panorama
• yarn add @aws-sdk/client-panorama
• pnpm add @aws-sdk/client-panorama

OR

You can use it by importing Boto3 packages.

You can check the API Documentation for AWS Panorama to see the list of methods and their uses through AWS Panorama Client.

Features of AWS Panorama

There are multiple features of AWS Panorama. Some of the common features are listed below.

Quick Setup

AWS Panorama can be set up within minutes. The process of setting up AWS Panorama is quite easy, and we need to execute the following few steps:

• Install AWS Panorama on your device and connect it to the network.
• Register your AWS Panorama device using the AWS Management Console.
• After registering, the video feeds from cameras set up on-premises can be added, on which machine learning operations can be performed.

Connects to IP Cameras

You can connect your IP cameras to AWS Panorama. It provides support for RTSP-enabled IP cameras. In addition, it also allows connection for IP cameras with ONVIF standards.

Supports Multi-Model Multi-Stream Parallelly

• We can connect multiple cameras to AWS Panorama to add video/image feeds.
• It also allows running multiple machine learning models on a given feed to generate desired outcomes.
• Not only this, AWS Panorama can integrate the above two features, i.e., It can run multiple machine learning models on top of multiple connected devices in parallel.

Inference at the Edge

• AWS Panorama reduces the need for manual intervention or human monitoring.
• It serves the purpose of automating the task of running high-level machine learning models on the on-premises image input devices.
• Ensuring low latency, enhanced data privacy, and filling the gap of limited internet bandwidth are the major concerns taken care of by AWS Panorama.

GPU Compute

AWS Panorama provides high computational speed of machine learning models using the built-in NVIDIA Xavier GPU.

Edge to Cloud Managed Service

• AWS Panorama devices can deploy applications based on computer vision with live input.
• It takes care of the versioning and lifecycle of these computer vision-based applications.
• Multiple computer vision-based applications are taken care of by a single interface in the AWS Console for management.

Flexible Options for Deploying CV

• AWS contains some inbuilt templates for standard machine learning algorithms like counting the number of persons in an image, classifying the vehicles, recognizing the license plate, and many more.
• You can get started with applications using the inbuilt machine learning templates without prior knowledge of Machine Learning.
• You can also train your machine learning models and optimize them to increase speed and efficiency.

AWS Panorama Use Cases

There are various use cases of AWS Panorama. Listed below are some common use cases. Let's discuss them.

Improve Supply Chain Logistics

Under these, Computer vision models can be used on live input to monitor the following:

• Recognize objects which are being transported via cargo
• Ensure the cargo is moving safely and efficiently
• keep an eye on the activities of transportation yards

Optimize Traffic Management

Traffic is one of the major issues to address in metropolitan cities. The cv based machine learning models can be used to manage traffic. It helps in:

• Ensuring a safe environment at airports and other important places.
• In emergencies, alerts can be sent to ensure a smooth traffic flow.
• Customer delays can be reduced by monitoring the vehicles.

Evaluate Manufacturing Quality

When companies manufacture products, there can be some errors that get ignored if monitored manually, and this can be automated using machine learning models

• Real-time monitoring can be done with the help of AWS Panorama.
• Any defects or anomalies can be easily and accurately detected, saving on extra costs.

Alarms Specifying Unsafe Zones

In a company spread across hectares of land comprising various manufacturing units, some areas might be prohibited.

Prohibition can be for the security of the products or the safety of the workers. In such cases, alerts can be sent if a person is detected to have entered a prohibited area using CV-based applications with real-time inputs.

Analysis of Retail Visitors

Retail shops can be monitored to make some analysis of the visitors. Some of the information retrieved includes:

• Interest areas of visitors.
• The time when maximum visitors visit the retail shop.
• Monitoring of queues is also covered.
• The product which appeals to a particular age group etc.

Drive-Thru Pickup Optimisation

Pickups can be monitored to ensure customer satisfaction. It helps in keeping track of the time taken to deliver products. Other ML models, like models to detect a person's mood, can be used to check the satisfaction level of a particular customer. All these optimizations can be made by connecting the IP cameras with an input feed connected to the AWS Panorama appliance.

AWS Panorama Pricing

Device and Usage Pricing

• For the device referred to as AWS Panorama Appliance, you need to pay a one-time upfront cost of $4,000 per device.
• Instead of AWS Panorama Appliance, you can also use Lenovo ThinkEdge SE70, for which the one-time upfront cost is $2,339 MSRP* per device.
• You can add multiple camera devices. For each device streaming and processing video using the appliance, $8.33 is charged per month.

Additional Charges

• The application assets, which include the machine learning models and business logic, are stored in a versioned form by AWS Panorama. Storing this asset requires some space, for which you are charged $0.10 per GB monthly.
• If you want to store the business logic on AWS S3 for carrying out analysis offline, then extra charges are incurred based on the pricing structure of AWS S3.

AWS Panorama Appliance

AWS Panorama Appliance is a hardware that helps to provide computer vision to the applications. We can create our computer vision model using machine learning and use the model with the help of AWS Panorama Appliance.

To develop the computer vision model, we use the AWS SageMaker and then deploy that model to the AWS Panorama Appliance. This Appliance can add computer vision to the existing IP cameras and automate tasks to inspect and monitor.

AWS Panorama Appliance has some features and functions. They are:

• It can connect to the IP cameras and process video to run simultaneous machine learning models per stream.
• The IP cameras do not need any built-in Machine Learning capabilities. AWS Panorama Appliance provides computer vision to these cameras.
• This Appliance is dust-proof and water-resistant.

AWS Panorama Endpoints and Quotas

Endpoints

We need endpoints to connect programmatically with the AWS Panorama. These endpoints differ from one region to another. The table below lists the endpoints which follow the HTTPS protocols for different regions.

Quotas

Quotas are also referred to as limits. These are used to define the maximum number of resources that can be used for your AWS account. We will discuss all the limits for the AWS resources. It is important to note that these quotas are non-adjustable and apply only to regions supported by AWS Panorama.

• Applications per device: You can run ten application instances on a device for any supported region.
• Assets per package: You can have 50 assets per node package.
• Camera nodes per application: A maximum of 8 camera nodes can be attached.
• Devices: You can register for a maximum of 50 devices for a particular account.
• Node packages: A maximum of 50 node packages can be created for an AWS account.
• Package artifact size: Maximum size of Node package artifact can be 8 gigabytes.
• Versions per package: A maximum of 20 versions can be created per node package.

Getting Started with AWS Panorama

We know the concepts of AWS Panorama. Let's start installing Panorama in AWS and deploy the Panorama in AWS Infrastructure. It involves three sets of procedures:

• Creating and Configuring AWS Infrastructure
• Deploying a Panorama Virtual Appliance in the Management VPC
• Updating the Virtual Appliance and Activating Services

In this demo, we are covering the two procedures because up to the deployment of Panorama in VPC, for updating and activating the services, we need the serial number of the Panorama devices to which we want to connect. So, let's get started with the hands-on activity.

Creating and Configuring AWS Infrastructure

In this procedure, we must create a ManageManagement VPC, IP subnets, VPC Internet Gateway, VPC Route Tables, Security Groups, and an IAM Role with Read-Only Access.

Create a Management VPC

Follow these steps:

• Login to the AWS console and choose the US West(Oregon) region.
• Search VPC in the search bar and click on Your VPCs.
• Click on Create VPC.
• Choose VPC only, fill in the name of the VPC, and provide the IPv4 CIDR block. Scroll and click on Create VPC.
• VPC is successfully created. Go to the Actions and click on Edit DNS hostnames.
• Mark the Enable checkbox and click on Save changes.

Create IP Subnets

Follow these steps:

• On the next screen, click on subnets and click on Create subnet.
• Choose the VPC and fill in the name for the subnet.
• Choose the Availability zone and provide the IPv4 CIDR block. Click on the Create subnet.
• Subnet is created successfully.

Create a VPC Internet Gateway

Follow these steps:

• Go to the Internet Gateway in the VPC menu and click on Create internet gateway.
• Provide a name to the internet gateway and click on Create internet gateway.
• Now, we have to attach this internet gateway to the VPC. Click on Attach to a VPC.
• Choose the VPC and click on Attach internet gateway.
• The internet gateway is successfully attached to the VPC.

Create the VPC Route Tables

Follow these steps:

• Go to the Route table in the VPC menu and click on Create route table.
• Fill in the name of the routing table and choose the VPC. Scroll and click on Create route table.
• The route table is created. Click on Edit routes.
• Click on Add route. Enter 0.0.0.0/0 in the destination box and choose the created internet gateway in the target box. Click on Save changes.
• Go to the Subnet Associations tab and click on Edit subnet associations.
• Select the subnet and click on Save associations.
• The VPC route table is configured successfully.

Create Security Groups

Follow these steps:

• Search Security groups in the search bar and open it.
• Click on Create security group.
• Provide a name to the security group and a description, and choose the VPC.
• Scroll down and set up the inbound rules.
• Click on Add new tag and provide a name tag. Click on Create security group.
• We created the security groups successfully.

Create IAM Role with Read-Only Access

Follow these steps:

• Search IAM in the search bar and click on Roles.

• Click on Create role.

• Choose the AWS services as a trusted entity and choose EC2.

• In the permissions, choose the AmazonEC2readonly access and click on Next.

• Provide a name and description of the role. Scroll down and click on Create role.

• The IAM role is created successfully.

We are done with creating and configuring AWS infrastructure. Now let's deploy Panorama Virtual Appliance in the Management VPC.

Deploying a Panorama Virtual Appliance in the Management VPC

In this procedure, we have to create Panorama Virtual Appliances, label the primary network interfaces, create the EIP address for the Virtual appliances, and configure the admin login for browser access.

Create Panorama Virtual Appliances Follow these steps:

• Click on the AWS CloudShell icon.

• aws ec2 describe-images --filters "Name=product-code,Values=eclz7j04vu9lf8ont8ta3n17o" Name=name,Values=Panorama*10.1.6* --region us-west-2 --output json Run this command to get the image ID. Copy this image ID.
• Search the EC2 in the search bar and click on it.
• Go to the AMI menu in the EC2 console and choose the Public images.
• Search the copied image ID. Select the image ID and click on Launch instance from AMI.
• Provide a name to the instance and the number of instances.
• Scroll down and click on Create a key pair.
• Fill name for the key pair and choose the file format as ppk. Click on Create key pair. It will create a key pair and save it to your machine.
• Scroll down to the Network setting. Choose the VPC subnet and select the existing security groups. Choose the security group created earlier.
• In the advanced network configurations, provide the primary IP.
• In the Advance details, choose the IAM profile we created earlier. Click on Launch instance.
• It may take some time to launch the instance. Click on View all instances.
• The instance is successfully launched.

Label the Primary Network Interfaces

Follow these steps:

• Go to the Network interface in the EC2 console menu.

• Edit the name of the Network interface.

Create EIP Addresses for the Virtual Appliances

Follow these steps:

• Go to the Elastic IPs in the VPC console. Click on Allocate Elastic IP address.
• Click on Allocate.
• Select this Elastic IP address and click on Actions. Click on View Details.
• Click on Manage tags.
• Click on Add tag and provide a Name. Click on Save.
• Now, we have to Associate the Elastic IP address. Click on Associate Elastic IP address.
• Choose the network interface and select the network interface and IP address. Click on Associate.
• We set up the Elastic Ip and successfully associated it with the network interface.

Configure the Admin Login for Browser Access

Follow these steps:

• Go to the instance and copy the Public IPv4 DNS.
• Open putty to connect to this instance.
• Fill the DNS in the Hostname and click on Credentials.
• Select the ppk file from the local machine and click on Open.
• login as admin.
• Run the commands configure, set mgt-config users admin password, and commit to set a strong admin password, and then commit the changes.
• Open the IPv4 DNS and log in with the admin user and the password we set in the last step.
• Click ok on a few prompts and we will get on to the Panorama dashboard page.

We have successfully deployed Panorama Virtual Appliance on AWS Infrastructure.

Benefits of AWS Panorama

Some of the common benefits of AWS Panorama are:

• Add machine learning to the existing IP camera systems
• Make decisions in real-time.
• Enable computer vision in a connected network environment.
• Meet data privacy and governance requirements.
• Integrating AWS Panorama into consulting solutions and SafetyVisor products.
• Fast development in a familiar programming environment.