At Port of Seattle, machine learning streamlines air cargo operations

The Port of Seattle may be more than 100 years old, but it’s learning some very new tricks. The government agency, which oversees Seattle’s seaport and the Seattle-Tacoma International Airport, is using artificial intelligence (AI) and computer vision (CV) for object detection and cataloging in the air cargo area of the airport to improve operational efficiency.

Skip Tavakkolian, senior systems architect at the Port of Seattle, says the agency believes machine learning (ML) will fundamentally change the way the port operates. This project, dubbed PlaneInsight, is just the beginning. Tavakkolian says PlaneInsight was primarily developed to give the Information and Communications Technology (ICT) team first-hand experience with modern ML techniques and CV, though he stresses it wasn’t just an academic exercise. The air cargo team at the port expects PlaneInsight to help them increase efficiency, reduce delays, and increase accountability with cargo carriers.

“Machine learning is a new discipline for ICT and the port and indeed for most of the tech sector,” Tavakkolian says. “As we understood what is possible with ML, we began to identify problems where non-ML solutions would be impractical or expensive.”

Manual solutions don’t scale, Tavakkolian explains, while the existing automatic solutions tend to rely on expensive and specialized hardware, like proximity sensors.

“Using computer vision allowed us to take advantage of our existing security camera infrastructure for video streams and our datacenter infrastructure for collection of snapshots, creation of training datasets, training of neural networks, and performing inference (i.e., analysis) on images,” he says.

From pilot to production

PlaneInsight leverages the port’s security cameras to determine the type, location, and approximate outline of aircraft and ground equipment of interest — ladders, ground power units, belt loaders, and so on. It can analyze an image of an aircraft gate and determine the type of aircraft (if any) docked there, a description of the surroundings, and any visible text (such as the airline name on an aircraft). The project has earned the Port of Seattle a FutureEdge 50 Award for applications of emerging technologies.

 

Port of Seattle

Tavakkolian and Port of Seattle CIO Matt Breed started exploring applications of ML and CV at the port in 2016. In 2017, working closely with Chris Evans, general foreman aviation and electrical systems at the port, Tavakkolian created a simple proof-of-concept CV prototype to the port’s innovations “Shark Tank.” Based on that prototype, the director of aviation operations and manager of air cargo ops and development sponsored the pilot project that would become PlaneInsight.

“The initial decision to dedicate a moderate amount of time and computing resource to the classifier prototype was entirely internal to the ICT department,” Tavakkolian says, but it was the “Shark Tank” process that helped it win sponsorship in the business. “This process allows ideas and prototypes to be presented to directors across the port, where it is possible to receive some capital for pilot projects to take those ideas and prototypes to the next stage.”

With executive sponsorship supporting the project, ICT worked closely with the air cargo operations team to identify the types of information that would be useful for both near real-time use, investigative analysis, and historical trends analysis. Tavakkolian notes that the two teams continue to identify additional features and uses for the object detection data to this day.

Training up for PlaneInsight

The biggest challenge still lay ahead: finding the expertise to make PlaneInsight go.

“Learning the concepts of ML and CV well enough to put them to use, learning the tools and frameworks (e.g., Tensorflow), and then helping educate others in ML concepts have been the most challenging,” Tavakkolian says. “ML expertise is in high demand and organizations who can’t compete on compensation alone need to plan to train existing personnel if they want to take advantage of ML.”

 

Port of Seattle

Training staff wasn’t the only issue, though. The convolutional neural network (CNN) at the core of PlaneInsight needed training datasets for transfer learning. Creating such a dataset is a tedious task: It requires annotating tens of thousands of images. Objects of interest in each image must be labeled (e.g., “Boeing 737”), their locations defined (e.g., a bounding box around it), and their shapes outlined (e.g., a polygon around the aircraft outline). It’s extremely time-intensive and there are no standard training datasets for aircraft and ground equipment. To solve that problem, the port started a new high school summer internship for machine learning.

“Nearly half of the training dataset was created by our high school interns,” Tavakkolian says.

The port deployed the PlaneInsight pilot in 2019, and it has been in operation since. Not only has it helped the air cargo team improve efficiency, Tavakkolian says it has helped build greater awareness of ML within the organization and the port has identified many more operational uses for ML, including wayfinding (using natural language processing and machine translation), surface operations, safety, inventory, and more. The air cargo team wants to use CV for automatic inventory of equipment, verification of actual operations at any location versus the scheduled events, and calculation of variance between planned and actual schedules. Even some existing project RFPs have been augmented to require the use of ML.

“Professionals in IT are familiar with the certainty and pace of change,” Tavakkolian says. “Machine learning techniques in general and deep neural networks in particular are readily available technologies and will perhaps have the most transformative effect on IT since the general availability and adoption of the internet in the mid-1990s. It’s best to start now to prepare for that eventuality.”

Original Article: https://www.cio.com/article/3530279/at-port-of-seattle-machine-learning-streamlines-air-cargo-operations.html#tk.rss_digitaltransformation