How ultra high-resolution geospatial data powers deep learning models to build scalable solutions
At the start of March 2022, Pompeu Fabra University student, Maria Alba, began to develop her thesis on how to build a road detection model using geospatial data. The model, when fully built, could be used to improve the accuracy of georeferenced mosaics, enhance route planning, and help analyze the impact of infrastructural growth on the world.
This blog outlines an overview of Maria’s project, difficulties she encountered, and key findings from the model. To apply to use our high-quality geospatial data for your research and education based projects, visit our Community Resilience & Innovation Earth Imagery Grant program.
“A Novel Approach for Road Detection on Aerial Imagery”
The goal of the thesis was to use geospatial data to build an algorithm that can detect the presence of roads, accurately and reliably. Road detection is a type of image segmentation and classification task that machine learning models derive insights from and can be used for various applications such as improving autonomous vehicles, optimizing traffic routes, and creating more accurate maps.
“When I began my thesis, I was excited to use Near Space Labs’ data because of the quality and frequency of data they provide,” said Maria. “Having frequently updated data of cities to keep track and study how roads and the world is changing through time is a very demanding approach to road detection. Using imagery of roads in different settings with a variety of details down to the type of road material, helped me build a robust model, something very powerful in the world of machine learning.”
Maria began building the road detection system using a neural network called U-Net, a deep learning method that has shown promising results for image segmentation and classification. Essentially, what this method does in the case of binary segmentation is create an images’ binary mask (meaning two colors) of whatever you are teaching it to identify. In the example below, you can see how the binary road mask darkens out the “non-roads” and identifies the true roads in a white outline.
Inputting Near Space Labs’ imagery through the U-net neural network to create a binary road mask. By Maria Alba from “A Novel Approach for Road Detection on Aerial Imagery.”
This neural network can be used to create a model that can classify the difference between nearly anything you teach it to; whether it be cats or dogs, cars or trucks, roads or “non-roads”. Maria’s thesis surrounds the idea of teaching this neural network to automatically output binary road masks for any given image in order to reliably identify the presence of roads. In the future, this would save countless hours of manual work and produce quicker and more accurate results.
In order for the model to learn what a road is in a given image, it needs a large dataset of imagery that already has its respective binary masks created for them. Maria used a dataset consisting of 742 satellite images from an area in Massachusetts Roads Dataset to begin teaching the model. The pros of this data set was that it was large and the binary masks were already created, allowing the model to learn how to classify between roads and non-roads. The cons of this dataset was that the images were all from the same location in Massachusetts consisting of the same types of roads and similar colors, which is very limiting when teaching a model. Another limitation of this dataset was the resolution of the imagery was approximately 1 m which doesn’t give great details into smaller roads or paths.
It’s ideal when teaching a road detection model, or any model for that matter, to feed it a robust dataset that has lots of variety in locations, types of roads, and colors so it can learn how to identify roads in any image and reduce the amount of miss detections and errors.
Maria then used 100 images from a Near Space Labs sample set to fine tune the model. The pros of using our data was that it consisted of 50% 30 cm and 50% 10 cm resolution imagery, and it included a variety of roads, consisting of roads of different sizes, materials, and geographic areas. By using a robust dataset, it enabled the model to learn how to classify a wider variety of roads than if it had been built using images with the same characteristics. Since Maria had to manually create the binary road masks for each NSL image — taking two weeks to build 100 masks — the con of this dataset was that it was much smaller. Ideally, there would be thousands of binary road masks to teach the road detection model how to create its own, but for the purpose of her thesis, the data she used was enough to draw meaningful conclusions.
Example of the model improving accuracy from the first fine-tuning to the second fine-tuning. The roads are more fully connected and clear in the second fine-tuning compared to the first. By Maria Alba from “A Novel Approach for Road Detection on Aerial Imagery.”
Overcoming AI Modeling Difficulties
Maria predicted the model would face some challenges in learning how to identify roads in the presence of occlusions and shadows. Occlusions are anything that is blocking the road from being visible, such as a tree or building.
The goal was to teach the model, through the original binary masks, to see a pattern in road behavior and understand that although part of the road was blocked, that there was still a road underneath the occlusion.
Shadows in an image can give deep learning model difficulties because it could make a single road appear with multiple shades of light and colors. In other words, it darkens part of the road which makes it harder for a detection model to understand what it’s looking at.
Example of (1) an occlusion; and (2) shadows. By Near Space Labs, 10 cm.
Part of Maria’s goal in teaching the model using the variety of images from the NSL database, in conjunction with the large satellite database, was for it to overcome these challenges and reliably identify roads in any given image.
Key Findings / Results
Using additional imagery from NSL, Maria tested the accuracy of the model by comparing the image to the binary road mask output it learned to automatically create. She found that her model was able to overcome, although not perfectly, a number of occlusions and shadows found in the images, meaning the model could still predict the presence of a road although it may have partially been blocked from view.
The binary road mask shows the model was able to detect difficult roads that are partially covered by occlusions with Near Space Labs’ 30 cm data. By Maria Alba from “A Novel Approach for Road Detection on Aerial Imagery.”
The model was also able to accurately define roads of different types, including highways, streets, and paths. It had more success defining larger roads, and had more failed detections on smaller roads and paths.
The model accurately identified the presence of a larger path in Texas, with Near Space Labs’ 10 cm imagery. By Maria Alba from “A Novel Approach for Road Detection on Aerial Imagery.”
To further compare how well the model did, Maria compared the original binary road masks she created with the predicted binary road mask the model created. Below you can see the model did a good job of predicting the presence of roads, although there are still some gaps and missing connections. Maria predicts with a larger sample set of high-resolution images with their manually created binary mask attached, the model would be able to identify these smaller roads and paths and produce more accurate and holistic binary masks.
Original binary road masks compared to the predicted binary road masks. By Maria Alba from “A Novel Approach for Road Detection on Aerial Imagery.”
After months of training and fine-tuning the model and adjusting parameters, Maria completed her thesis in June of 2022 and has since been hired as a Data Operations Specialist at Near Space Labs.
Additional Examples of NSL Imagery and their respected binary mask outputs:
Maria’s road detection model accurately identified the presence of highways and residential roads in Near Space Labs’ 30 cm archived imagery. By Maria Alba from “A Novel Approach for Road Detection on Aerial Imagery.”
The model clearly defined the presence of cul-de-sac roads in Near Space Labs’ 10 cm imagery. By Maria Alba from “A Novel Approach for Road Detection on Aerial Imagery.”
