Calculation of Spatial and Temporal Characteristics of Objects in a Video Stream Using Photogrammetry and Computer Vision Methods
DOI:
https://doi.org/10.52575/2687-0932-2025-52-3-710-725Keywords:
computer vision, photogrammetric resection, basketball, YOLOv8, object tracking, sports analyticsAbstract
This article presents the results of a study aimed at extracting the spatiotemporal characteristics of objects in a video stream using photogrammetry and computer vision techniques. A special feature of the approach is the use of a limited number of non-professional video cameras. The task of automating sports analytics was chosen as an application area, in particular, the analysis of video recordings of basketball matches. Nevertheless, the proposed solutions are versatile and may be adapted for other applications, such as video surveillance systems, public safety, traffic flow monitoring, consumer behavior analysis in retail, as well as improving the efficiency of interaction between sellers and buyers. The adaptation of the proposed methodology to various fields is achieved by changing the initial training data and configuring the corresponding machine learning algorithms, which are discussed in detail in the framework of this study. The developed system includes two consecutive stages. At the first stage, objects are detected in the video stream using computer vision and deep learning methods, which makes it possible to identify objects of interest in each frame. At the second stage, the spatial characteristics of the detected objects are calculated using photogrammetric reconstruction methods, which allow restoring their three-dimensional coordinates and motion trajectories. To implement the task of detecting objects (players, ball, referees, basketball hoop and shield), the YOLOv8 model was used, trained on a specially generated dataset compiled from FIBA match videos. To restore the spatial coordinates, we applied the direct photogrammetric serif method, modified to operate using available equipment. During the experiment conducted on a 23x11–meter basketball court using two smartphone cameras, an average positioning accuracy of 4 cm was achieved with a standard deviation of up to 8 cm. The YOLOv8 metrics demonstrated accuracy and completeness values in the range of 0.85-0.95, and the mAP indicator ranged from 0.70 to 0.85. Improvements have also been proposed as part of the work, including the use of the reverse photogrammetric serif method to automate calibration based on site markings. The developed system is a promising solution for supporting refereeing and analytics at local sports competitions, while providing a cost-effective alternative to traditional sports video surveillance and analysis systems.
Downloads
References
Список источников
Гордиенко, А.С. Дистанционное зондирование и фотограмметрия. Теория стереопары снимков. Основы пространственной фототриангуляции: учеб.-метод. пособие / А.С. Гордиенко. – Новосибирск: СГУГиТ, 2015. – 88 с.
Дэвис, Р. Компьютерное зрение: современные методы и перспективы развития / Р. Дэвис, М. Терк; пер. с англ. В. Яценков. – Москва: ДМК Пресс, 2022. – 690 с.
CVAT // CVAT: официальный сайт. – URL: https://www.cvat.ai/ (дата обращения: 15.05.2025).
Official Basketball Rules 2022: Basketball Rules & Basketball Equipment: as approved by FIBA Central Board on 28.04.2023: valid as of 01.07.2023. – Mies: FIBA, 2023. – 104 p.
Roboflow // Roboflow: официальный сайт. – URL: https://roboflow.com/ (дата обращения: 15.05.2025).
Second Spectrum // Second Spectrum: официальный сайт. – URL: https://www.secondspectrum.com/ (дата обращения: 15.05.2025).
SportsMOT // Github: [источник набора данных]. – URL: https://github.com/MCG- NJU/SportsMOT (дата обращения: 30.04.2025).
STATS SportVU // STATS Perform: официальный сайт. – URL: https://www.statsperform.com/ (дата обращения: 15.05.2025).
+ high-resolution professional basketball images with multiple annotations // Kaggle: [источник набора данных]. – URL: https://www.kaggle.com/datasets/gabrielvanzandycke/deepsport-dataset (дата обращения: 06.05.2025).
Список литературы
Безменов В.М., Сафин К.И. 2020. Оценка точности прямой фотограмметрической засечки для произвольного случая съёмки разными камерами. Известия вузов. Геодезия и аэрофотосъемка. 64(4): 415–422. DOI: 10.30533/0536-101X-2020-64-4-415-422
Боровик И.Г., Шорох Д.К. 2016. Обнаружение и ведение движущихся объектов на примере спортивного мероприятия. Молодой ученый, 6: 75–83. URL: https://moluch.ru/archive/110/26905/ (дата обращения: 22.04.2025).
Волкова М.А., Романов М.П., Бычков А.М. 2022. Трекер объектов на спортивных мероприятиях. Российский технологический журнал, 10(5): 38–48. DOI: 10.32362/2500-316X-2022-10-5-38-48
Симинеев А.А., Тарасова Е.И. 2012. Обратная фотограмметрическая засечка: надежность решения задачи. Вестник СПбГУ. Серия 7. Геология. География, Вып. 4: 129–134.
Ульев А.Д., Донская А.Р., Зубков А.В. 2023. Автоматизированное распознавание и контроль взаимодействия людей по видеоизображению. Известия Юго-Западного государственного университета. Серия: Управление, вычислительная техника, информатика. Медицинское приборостроение, 13(2): 45–64. DOI: 10.21869/2223-1536-2023-13-2-45-64
Buldu J.M., Busquets J., Echegoyen I. 2019. Defining a historic football team: Using Network Science to analyze Guardiola’s FC Barcelona. Scientific Reports, 9(1). Article ID 13602. DOI: 10.1038/s41598- 019-49969-2
Chen L., et al. 2021. A real-time photogrammetric system for acquisition and monitoring of three-dimensional human body kinematics. Photogrammetric Engineering & Remote Sensing, 87(5): 363–373. DOI: 10.14358/PERS.87.5.363
Chibunichev A.G., et al. 2016. Investigation of phototriangulation accuracy with using of various techniques laboratory and field calibration. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences. Vol. 41: 1227–1233. DOI: 10.5194/isprs-archives-XLI-B5-1227-2016.
Egi Y. 2022. Basketball self training shooting posture recognition and trajectory estimation using computer vision and Kalman filter. Journal of Electrical Engineering. 73(1): 19–27. DOI: 10.2478/jee-2022-0003. Ferraz A., Duarte-Mendes P., Sarmento H., et al. 2023. Tracking devices and physical performance analysis in team sports. Frontiers in Sports and Active Living. Vol. 5: 123–145. DOI: 10.3389/fspor.2023.1067969.
Gruen A. 2021. Everything moves: The rapid changes in photogrammetry and remote sensing. Geo-spatial Information Science. 24(1): 33–49. DOI: 10.1080/10095020.2020.1859359.
Kundegorski M.E., Breckon T.P. 2014. A photogrammetric approach for real-time 3D localization and tracking of pedestrians in monocular infrared imagery. Optics and Photonics for Counterterrorism, Crime Fighting, and Defence X; and Optical Materials and Biomaterials in Security and Defence Systems Technology XI. SPIE. Vol. 9253: 139–154. – DOI: 10.1117/12.2067217.
Magre Colorado L.A., Martínez Santos J.C. 2015. Kinematic parameter estimation using close range photogrammetry for sport applications. 11th International Symposium on Medical Information Processing and Analysis. Proceedings of Society of Photo-Optical Instrumentation Engineers. Vol. 9681. DOI: 10.1117/12.2208354.
Monier E., Wilhelm P., Rückert U. 2009. A computer vision based tracking system for indoor team sports. The Fourth International Conference on Intelligent Computing and Information Systems. 5 p.
Rahimian P., Toka L. 2022. Optical tracking in team sports: A survey on player and ball tracking methods in soccer and other team sports. Journal of Quantitative Analysis in Sports. 18(1): 35–57. DOI:10.1515/jqas-2020-0088.
Redmon J., Divvala S., Girshick R., Farhadi A. 2016. You Only Look Once: Unified, Real-Time Object Detection. 2016 IEEE Conference on Computer Vision and Pattern Recognition (Las Vegas, 27–30 June 2016). New York: IEEE, pp. 779–788. DOI: 10.1109/CVPR.2016.91.
Ross G., Donahue J., Darrel T., Malik J. 2014. Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation. Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition (Washington, 23–28 June 2014). Washington: IEEE Computer Society, pp. 580–587. DOI: 10.1109/CVPR.2014.81.
Shankara V. et al. 2024. Object detection and tracking for football data analytics. Proceedings of the 2024 International Conference on Sports Analytics. 10 p. DOI: 10.4108/eai.23-11-2023.2343216.
Thomas G., Gade R., Moeslund T.B., Carr P., Hilton A. 2017. Computer vision for sports: Current applications and research topics. Computer Vision and Image Understanding. Vol. 159: 3–18. DOI: 10.1016/j.cviu.2017.04.011.
Vidal-Codina F. et al. 2022. Automatic event detection in football using tracking data. Sports Engineering. 25(1): 1–15. DOI: 10.1007/s12283-022-00381-6.
Yacouby R., Axman D. 2020. Probabilistic Extension of Precision, Recall, and F1 Score for More Thorough Evaluation of Classification Models. Proceedings of the First Workshop on Evaluation and Comparison of NLP Systems. Online: Association for Computational Linguistics. 79–91. DOI: 10.18653/v1/2020.eval4nlp-1.9.
Ye M. 2024. Application of 3D recognition algorithm based on spatio-temporal graph convolutional network in basketball pose estimation. International Journal for Simulation and Multidisciplinary Design Optimization. Vol. 15: 9. DOI: 10.1051/smdo/2024004.
Zheng Y., Zhang H. 2022. Video Analysis in Sports by Lightweight Object Detection Network under the Background of Sports Industry Development. Computational Intelligence and Neuroscience. 2022(1). Article ID 3844770. DOI: 10.1155/2022/3844770.
References
Bezmenov V.M., Safin K.I. 2020. The accuracy estimation of photogrammetric spatial intersection with different cameras for an arbitrary case of aerial survey. Izvestiya vuzov. Geodeziya i aerofotos’yemka [News of universities. Geodesy and aerial photography], 64(4): 415–422. DOI: 10.30533/0536-101X- 2020-64-4-415-422.
Borovik I.G., Shorokh D.K. 2016. Obnaruzhenie i vedenie dvizhushchikhsya ob"ektov na primere sportivnogo meropriyatiya [Detection and tracking of moving objects using the example of a sports event]. Molodoy uchenyy [Young scientist], 6: 75–83. URL: https://moluch.ru/archive/110/26905/ (accessed: 22.04.2025).
Volkova M/A., Romanov M.P., Bychkov A.M. 2022. 3D object tracker for sports events. Rossiyskiy tekhnologicheskiy zhurnal [Russian Technological Journal], 10(5): 38–48. DOI: 10.32362/2500-316X- 2022-10-5-38-48.
Simineev A.A., Tarasova E.I. 2012. Obratnaya fotogrammetricheskaya zasechka: nadezhnost' resheniya zadachi [Inverse photogrammetric resection: reliability of the solution]. Vestnik SPbGU. Seriya 7. Geologiya. Geografiya [Bulletin of St. Petersburg State University. Series 7. Geology. Geography], Issue 4: 129–134.
Ulyev A.D., Donskaia A.R., Zubkov A.V. 2023. Automated Recognition and Control of Human Interaction by Video Image. Izvestiya Yugo-Zapadnogo gosudarstvennogo universiteta. Seriya: Upravlenie, vychislitel'naya tekhnika, informatika. Meditsinskoe priborostroenie [News of the South-West State University. Series: Management, computing, informatics. Medical instrumentation], 13(2): 45–64. DOI: 10.21869/2223-1536-2023-13-2-45-64.
Buldu J.M., Busquets J., Echegoyen I. 2019. Defining a historic football team: Using Network Science to analyze Guardiola’s FC Barcelona. Scientific Reports, 9(1). Article ID 13602. DOI: 10.1038/s41598- 019-49969-2.
Chen L., et al. 2021. A real-time photogrammetric system for acquisition and monitoring of three-dimensional human body kinematics. Photogrammetric Engineering & Remote Sensing, 87(5): 363–373. DOI: 10.14358/PERS.87.5.363.
Chibunichev A.G., et al. 2016. Investigation of phototriangulation accuracy with using of various techniques laboratory and field calibration. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Vol. 41: 1227–1233. DOI: 10.5194/isprs-archives-XLI-B5-1227-2016.
Egi Y. 2022. Basketball self training shooting posture recognition and trajectory estimation using computer vision and Kalman filter. Journal of Electrical Engineering, 73(1): 19–27. DOI: 10.2478/jee-2022-0003. Ferraz A., Duarte-Mendes P., Sarmento H., et al. 2023. Tracking devices and physical performance analysis in team sports. Frontiers in Sports and Active Living, Vol. 5: 123–145. DOI: 10.3389/fspor.2023.1067969.
Gruen A. 2021. Everything moves: The rapid changes in photogrammetry and remote sensing. Geo-spatial Information Science, 24(1): 33–49. DOI: 10.1080/10095020.2020.1859359.
Kundegorski M.E., Breckon T.P. 2014. A photogrammetric approach for real-time 3D localization and tracking of pedestrians in monocular infrared imagery. Optics and Photonics for Counterterrorism, Crime Fighting, and Defence X; and Optical Materials and Biomaterials in Security and Defence Systems Technology XI. SPIE. Vol. 9253: 139–154. – DOI: 10.1117/12.2067217.
Magre Colorado L.A., Martínez Santos J.C. 2015. Kinematic parameter estimation using close range photogrammetry for sport applications. 11th International Symposium on Medical Information Processing and Analysis. Proceedings of Society of Photo-Optical Instrumentation Engineers. Vol. 9681. DOI: 10.1117/12.2208354.
Monier E., Wilhelm P., Rückert U. 2009. A computer vision based tracking system for indoor team sports. The Fourth International Conference on Intelligent Computing and Information Systems. 5 p.
Rahimian P., Toka L. 2022. Optical tracking in team sports: A survey on player and ball tracking methods in soccer and other team sports. Journal of Quantitative Analysis in Sports, 18(1): 35–57. DOI:10.1515/jqas-2020-0088.
Redmon J., Divvala S., Girshick R., Farhadi A. 2016. You Only Look Once: Unified, Real-Time Object Detection. 2016 IEEE Conference on Computer Vision and Pattern Recognition (Las Vegas, 27–30 June 2016). New York: IEEE, pp. 779–788. DOI: 10.1109/CVPR.2016.91.
Ross G., Donahue J., Darrel T., Malik J. 2014. Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation. Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition (Washington, 23–28 June 2014). Washington: IEEE Computer Society, pp. 580–587. DOI: 10.1109/CVPR.2014.81.
Shankara V. et al. 2024. Object detection and tracking for football data analytics. Proceedings of the 2024 International Conference on Sports Analytics. 10 p. DOI: 10.4108/eai.23-11-2023.2343216.
Thomas G., Gade R., Moeslund T.B., Carr P., Hilton A. 2017. Computer vision for sports: Current applications and research topics. Computer Vision and Image Understanding, Vol. 159: 3–18. DOI: 10.1016/j.cviu.2017.04.011.
Vidal-Codina F. et al. 2022. Automatic event detection in football using tracking data. Sports Engineering, 25(1): 1–15. DOI: 10.1007/s12283-022-00381-6.
Yacouby R., Axman D. 2020. Probabilistic Extension of Precision, Recall, and F1 Score for More Thorough Evaluation of Classification Models. Proceedings of the First Workshop on Evaluation and Comparison of NLP Systems. Online: Association for Computational Linguistics. 79–91. DOI: 10.18653/v1/2020.eval4nlp-1.9.
Ye M. 2024. Application of 3D recognition algorithm based on spatio-temporal graph convolutional network in basketball pose estimation. International Journal for Simulation and Multidisciplinary Design Optimization, Vol. 15: 9. DOI: 10.1051/smdo/2024004.
Zheng Y., Zhang H. 2022. Video Analysis in Sports by Lightweight Object Detection Network under the Background of Sports Industry Development. Computational Intelligence and Neuroscience, 2022(1). Article ID 3844770. DOI: 10.1155/2022/3844770.
Abstract views: 0
Share
Published
How to Cite
Issue
Section
Copyright (c) 2025 Economics. Information Technologies
This work is licensed under a Creative Commons Attribution 4.0 International License.
