A Development of Road Crack Detection System Using Deep Learning-based Segmentation and Object Detection

Jongwoo Ha, Kyongwon Park, Minsoo Kim


Many recent studies on deep learning-based road crack detection have shown significantly more improved performances than previous works using algorithm-based conventional approaches. However, many deep learning-based studies are still focused on classifying the types of cracks. The classification of crack types is highly anticipated in that it can improve the crack detection process, which is currently relying on manual intervention. However, it is essential to calculate the severity of the cracks as well as identifying the type of cracks in actual pavement maintenance planning, but studies related to road crack detection have not progressed enough to automated calculation of the severity of cracks. In order to calculate the severity of the crack, the type of crack and the area of the crack in the image must be identified together. This study deals with a method of using Mobilenet- SSD that is deep learning-based object detection techniques to effectively automate the simultaneous detection of crack types and crack areas. To improve the accuracy of object-detection for road cracks, several experiments were conducted to combine the U-Net for automatic segmentation of input image and object-detection model, and the results were summarized. As a result, image masking with U-Net is able to maximize object-detection performance with 0.9315 mAP value. While referring the results of this study, it is expected that the automation of the crack detection functionality on pave management system can be further enhanced.

Full Text:



Ashraf, S., Hegazy, I., and Elarif, T. L., “Algorithm for Automatic Crack Analysis and Severity Identification,” 2019 IEEE Ninth International Conference on Intelligent Computing and Information Systems, pp. 74-79, 2019.

Cubero-Fernandez, A., Rodriguez-Lozano, F. J., Villatoro, R., Olivares, J., and Palomares, J. M., “Efficient pavement crack detection and classification,” EURASIP Journal on Image and Video Processing, Vol. 39, pp. 1-11, 2017.

Feng, C., Liu, M. Y., Kao, C. C., and Lee, T. Y., “Deep active learning for civil infrastructure defect detection and classification,” Computing in Civil Engineering, pp. 298-306, 2017.

Gopalakrishnan, K., “Deep Learning in Data-Driven Pavement Image Analysis and Automated Distress Detection: A Review,” Data, Vol. 3, No. 3, pp. 1-19, 2018.

Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., and Adam, H., “Mobilenets: Efficient convolutional neural networks for mobile vision applications,” arXiv preprint arXiv:1704. 04861, 2017.

https://github.com/tensorflow/models/tree/ master/research/object_detection.

Jo, H., Kim, D., Pak, K. W., and Kim, M., “Road damage detection over road scanner images using deep convolutional neural network,” ICIC Express Letters, Vol. 14, No. 10, pp. 1001-1008, 2020.

Kim, M., Ryu, J., Cha, D., and Sim, M. K., “Stock Price Prediction Using Sentiment Analysis: from “Stock Discussion Room in Naver,” The Journal of Society for e-Business Studies, Vol. 25, No. 4, pp. 61-75, 2020.

Kirillov, A., Girshick, R., He, K., and Dollar, P., “Panoptic Feature Pyramid Networks,” Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6399-6408, 2019.

Koch, C. and Brilakis, I., “Pothole detection in asphalt pavement images,” Advanced Engineering Informatics, Vol. 25, No. 3, pp. 507-515, 2011.

Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C. Y., and Berg, A. C., “Ssd: Single shot multibox detector,” European Conference on Computer Vision, Springer, Cham, pp. 21-37, 2016.

Naddaf-Sh, M., Hosseini, S., Zhang, J., Brake, N. A., and Zargarzadeh, H., “Real- time road crack mapping using an optimized convolutional neural network,” Complexity, pp. 1-17, 2019.

Pauly, L., Hogg, D., Fuentes, R., and Peel, H., “Deeper networks for pavement crack detection,” Proceedings of the 34th International Symposium on Automation and Robotics in Construction(ISARC), IAARC, pp. 479-485, 2017.

Rababaah, H., Vrajitoru, D., and Wolfer, J., “Asphalt pavement crack classification: a comparison of GA, MLP, and SOM,” Proceedings of Genetic and Evolutionary Computation Conference, Late-Breaking Paper, 2005.

Ragnoli, A., De Blasiis, M. R., and Benedetto, A. D., “Pavement Distress Detection Methods: A Review,” MDPI Infrastructure, Vol. 3, No. 58, pp. 1-19, 2018.

Ronneberger, O., Fischer, P., and Brox, T., “U-net: Convolutional networks for biomedical image segmentation,” International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, Cham, pp. 234-241, 2015.

Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., and Chen, L. C., “Mobilenetv2: Inverted residuals and linear bottlenecks,” Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4510-4520, 2018.

Sorncharean, S. and Phiphobmongkol, S., “Crack detection on asphalt surface image using enhanced grid cell analysis,” 4th IEEE International Symposium on Electronic Design, Test and Applications, pp. 49-54, 2008.

Yang, F., Zhang, L., Yu, S., Prokhorov, D., Mei, X., and Ling, H., “Feature Pyramid and Hierarchical Boosting Network for Pavement Crack Detection,” IEEE Transactions on Intelligent Transportation Systems, Vol. 21, No. 4, pp. 1525-1535, 2020.

Yoo, W., Seo, J., Kim, D., and Kim, K., “Machine Scheduling Models Based on Reinforcement Learning for Minimizing Due Date Violation and Setup Chang,” The Journal of Society for e-Business Studies, Vol. 24, No. 3, pp. 19-33, 2019.

Yoo, W., Seo, J., Lee, D., Kim, D., and Kim, K., “Scheduling Generation Model on Parallel Machines with Due Date and Setup Cost Based on Deep Learning,” The Journal of Society for e-Business Studies, Vol. 24, No. 3, pp. 99-110, 2019.

Zhang, A., Wang, K. C., Li, B., Yang, E., Dai, X., Peng, Y., Fei, Y., Liu, Y., Li, J. Q., and Chen, C., “Automated pixel-level pavement crack detection on 3D asphalt surfaces using a deep‐learning network,” Computer-Aided Civil and Infrastructure Engineering, Vol. 32, No. 10, pp. 805-819, 2017.

Zhang, L., Yang, F., Zhang, Y. D., and Zhu, Y. J., “Road crack detection using deep convolutional neural network,” Proceedings of the 2016 IEEE International Conference on Image Processing(ICIP), Phoenix, AZ, USA, 25-28 September 2016, pp. 3708-3712, 2016.


  • There are currently no refbacks.