Researchers from IT4Innovations at VSB – Technical University of Ostrava (VSB-TUO) have presented a new approach to processing photographs of identity documents that combines artificial intelligence with a method based on line detection. The solution enables more accurate correction of perspective distortion in images, facilitating their further automated processing. The research findings have been published in the scientific journal Results in Engineering.
A photograph of an identity card taken with a mobile phone often does not look like a perfectly flat scan. The document may be rotated, skewed or partially blended into its surroundings. Before automated systems can reliably read the information on it, its edges must first be precisely identified and the image transformed into the correct perspective.
This is the step that researchers at IT4Innovations have improved. They developed a new algorithm for the geometric rectification of photographed documents that uses information from lines identified in the document rather than seeking corners.
“Conventional methods often look for the four corners of a document. This can be problematic, for example, with documents that have rounded corners, lower-quality images, or unclear edges. We therefore chose a different approach – we look for the lines that form the individual sides of the document and then use their intersections to calculate its exact position,” explains Petr Strakoš from IT4Innovations.
The new method first uses a neural network to separate the document from the photograph's background. It then identifies several short line segments in this mask that may correspond to the document's outline lines. The algorithm groups them by direction and relative position, then uses them to create four main lines representing the top, bottom, and side edges of the document. Their intersections are then used to calculate the transformation that straightens the photograph. A document photographed at an angle can thus be converted into a view as if it had been captured directly from above.
The research also focused on finding a way to train artificial intelligence models without the need to collect large quantities of real photographs of identity documents. The researchers therefore created their own dataset of synthetic identity card images that closely resemble real documents, including security features such as holograms.
Using computer modelling, they generated images under different conditions, with varying lighting intensities, viewing angles, and backgrounds. They then trained several types of neural networks on this data and tested whether they could also handle real photographs. The U-Net model achieved the best results, accurately identifying the document area even in real images taken with a mobile phone.
The experiments showed that the newly proposed approach corrected perspective distortion more accurately than the compared methods that relied on detecting document edges or corners. For real images, the average deviation in corner detection was 4.66 pixels, compared with 11.73 and 20.19 pixels for the other methods. The computations required for the development, training of the models and evaluation of the results were carried out using the supercomputers at IT4Innovations.
The research was carried out by Petra Svobodová, Petr Strakoš and Lubomír Říha from IT4Innovations, and Kamil Postava, Tibor Fördös and Robin Silber from the Faculty of Materials Science and Technology at VSB-TUO.
In further research, the team will focus on making the synthetic training data even more similar to real photographs. The researchers will train the models to handle a wider range of conditions, such as significant blur, shadows, varying lighting, and partially obscured documents. Another goal is to optimise selected models to operate in real time on mobile devices. In the future, the resulting technology could find practical applications, for example, in smartphone-based identity verification systems.
The research was supported by the international InnovAIte Slovakia project and the project Integration of Holographic and Data Security Protection, supported by the Technology Agency of the Czech Republic.
The results were published in the scientific journal Results in Engineering in the article titled "Synthetic-to-real deep learning for identity card segmentation and line-based geometric rectification".
General concept of identity document perspective correction. (1) First, the document is captured, e.g. by a mobile device, then (2) the segmentation process is performed to isolate only the identity document in the captured image. After that, (3) information for the perspective correction is computed, and (4) the correction is applied.
Source: Fig. 1 in 10.1016/j.rineng.2026.110566