CitySurfaces: City-scale semantic segmentation of sidewalk materials

April 1st, 2022

Categories: Applications, Software, User Groups, Visualization, Visual Analytics, Visual Informatics, Deep Learning, Data Science

Predictions of the model on the held-out test set. Fine details and boundaries of objects like poles, plants, wooden sticks, and fire hydrants are very precisely predicted.
Predictions of the model on the held-out test set. Fine details and boundaries of objects like poles, plants, wooden sticks, and fire hydrants are very precisely predicted.

Authors

Hosseini, M., Miranda, F., Lin, J., Silva, C.T.

About

While designing sustainable and resilient urban built environment is increasingly promoted around the world, significant data gaps have made research on pressing sustainability issues challenging to carry out. Pavements are known to have strong economic and environmental impacts; however, most cities lack a spatial catalog of their surfaces due to the cost-prohibitive and time-consuming nature of data collection. Recent advancements in computer vision, together with the availability of street-level images, provide new opportunities for cities to extract large-scale built environment data with lower implementation costs and higher accuracy. In this paper, we propose CitySurfaces, an active learning-based framework that leverages computer vision techniques for classifying sidewalk materials using widely available street-level images. We trained the framework on images from New York City and Boston and the evaluation results show a 90.5% mIoU score. Furthermore, we evaluated the framework using images from six different cities, demonstrating that it can be applied to regions with distinct urban fabrics, even outside the domain of the training data. CitySurfaces can provide researchers and city agencies with a low-cost, accurate, and extensible method to collect sidewalk material data which plays a critical role in addressing major sustainability issues, including climate change and surface water management.

Keywords: Sustainable built environment; Surface materials; Urban heat island; Semantic segmentation; Sidewalk assessment; Urban analytics; Computer vision

ISSN 2210-6707

https://www.sciencedirect.com/science/article/pii/S2210670721008933

Resources

PDF

URL

Citation

Hosseini, M., Miranda, F., Lin, J., Silva, C.T., CitySurfaces: City-scale semantic segmentation of sidewalk materials, Science Direct, Sustainable Cities and Society, vol 79, Elsevier, April 1st, 2022. https://doi.org/10.1016/j.scs.2021.103630