Construction of a Geometric Model of a Product Based on Multimodal Processing of Paper Design Documentation

Abstract

The paper proposes a solution to the problem of constructing a geometric (3D) model of a product based on its drawing, presented in the form of design documentation on paper. A previously developed neural network approach was used to highlight the projections of the product (part). In this study, the geometric model of the part is formed from a point cloud constructed within the projection masks generated using a neural network. The YOLOv8 model was used to determine the masks, and the accuracy it achieved was more than 90 % according to the mAP metric. The features of the applied approach to constructing the geometric model of the part include the use of information about the elements of rotation (in particular, the radii of the rounded parts of the part), as well as the application of an algorithm for adjusting the center of rotation. An end-to-end example is considered, demonstrating the prospects of the chosen approach to constructing geometric models based on scalar images of drawings of parts and products. The authors suggest the following areas of further research that ensure increased accuracy of detailing of geometric models: (1) additional training of the model for constructing part masks with the addition of new classes; (2) accumulation and use of information on frequently used elements of drawings; (3) combination of an approach to geometric modeling of a part/product based on the application of generative neural networks to the obtained parameters of simple and complex primitives.