Fringe Projection Method’s Evolving Role in Cultural Asset Cataloging and Conservation: Potentiality in Testing Conservative Restoration Techniques Volume 9 - Issue 3

M C Casas Pérez¹*, F Castillo Rivera², G Moreno Chavez³, D Sarocchi⁴

• ¹Department of Habitat, Universidad Autónoma de San Luis Potosí, Mexico
• ²CONACYT-Instituto de Geologia, Universidad Autonoma de San Luis Potosi, San Luis Potosi, Mexico
• ³Unidad Académica de Ingeniería Eléctrica, Universidad Autónoma de Zacatecas, México
• ⁴Instituto de Geología, Universidad Autónoma de San Luis Potosí, México

Received:April 22, 2024;   Published: April 30, 2024

Corresponding author:Maria del Carmen Casas Pérez, Department of Habitat, Universidad Autónoma de San Luis Potosí, Mexico

DOI: 10.32474/JAAS.2024.09.000313

 

Abstract PDF

Abstract

The pursuit of accurately replicating cultural artifacts has been significant throughout history, evolving from mechanical pantographs to modern digital modeling techniques. Today, precise replication enables practical applications in art and conservation, including museography, cataloging, preservation, and virtual exhibitions. Digital models serve as vital records of an artifact’s condition, aiding in its conservation and restoration efforts.

Recent advancements, particularly in the last two decades, have revolutionized the ability to obtain highly detailed digital models of cultural heritage objects. Studies have shown that achieving precision in the range of tens to hundreds of microns is easily attainable, allowing for the documentation and measurement of intricate details in artworks. Fringe Projection Method (FPM) emerges as a valuable tool for cutator of cultural heritage and restorers due to its high resolution and non-invasive nature.

Future applications of FPM in conservation include comprehensive digitization of archaeological sites, inspection of architectural structures, integration into augmented reality and virtual reality, artifact reproduction and restoration. In laboratories, ongoing research involves applying FPM methodology to comparative studies of different relining techniques, providing quantitative data on their invasiveness. This quantitative approach aims to guide operators in selecting the most suitable techniques for conservation interventions.

The growing prominence of FPM suggests its indispensable role in the future of art conservation and restoration. It represents a valuable tool for preserving cultural heritage by enabling precise documentation, analysis, and intervention. As research continues and methodologies evolve, FPM is poised to contribute significantly to the conservation field, ensuring the protection and longevity of our cultural legacy.

Keywords:Fringe Projection Method; Art conservation; Surface reconstruction; Artifact reproduction; Restoration methods test.

Introduction

The most accurate possible reconstruction of the shape of an artwork has been a matter of great relevance since antiquity when such reproductions could only be achieved using mechanical pantographs. Initially, the goal was to obtain the most precise copy of the artwork. Nowadays, the ability to replicate the external aspects and the surface of an object of cultural heritage with high resolution allows for various practical applications [1]. One of the most common applications is the creation of digital models of objects for museography and cataloging purposes. For this use, very high precision is not essential, as a resolution of one millimeter provides acceptable quality [2,3]. Digital models allow for the exact preservation of the artifact’s memory in case of deterioration or loss (due to natural or accidental causes) as well as the temporal recording of its conservation status. Furthermore, it enables the generation of databases to recognize replicas of original works and to create digital museums that make art accessible to people worldwide. With the advent of 3D printers, it is now feasible to transform three-dimensional models into faithful copies of scanned works [4,5] which also provides the possibility of reconstructing or reintegrating missing parts in pieces that require it [6].

Currently, in the conservation and restoration of cultural assets, a scientific approach is indispensable. For this reason, interventions are carried out with the support of quantitative data leading to an objective diagnosis based on a scientific approach to the problem. To achieve this result, it is necessary to quantitatively understand the pictorial surface as accurately as possible. This is obtained through a deep knowledge of the adopted analytical techniques, their precision, and limits.

3D Scanning Methods in Conservation

The registration and documentation of cultural assets have been fundamental aspects of conservation throughout history. The need to create accurate copies of these objects for cataloging, preservation, and, more recently, their exhibition in virtual museums, has driven the development of methodologies for the three-dimensional reconstruction of cultural heritage objects.

The mathematical foundations for the quantitative threedimensional registration of objects date back thousands of years, with ancient Egyptians and Babylonians developing triangulation techniques in astronomy and geodesy. In ancient Greece, figures like Euclid and Archimedes laid the groundwork for trigonometry, while in the 17th century, Willebrord Snell van Rojen established the foundations of optical triangulation, crucial for modern photogrammetry and 3D reconstruction [7].

Although the foundations of modern techniques for the threedimensional reconstruction of objects have been known since ancient times, it was only with the development of digital cameras, personal computers, and increased graphic and computational capabilities that concrete attempts at 3D scanning were made. During the second half of the 20th century, the first applications of 3D scanning were carried out in the industrial field, mainly in surface inspection [7].

Laser holography, in 1972, was the first 3D reconstruction technique used to document statues and other art objects [8,9]. In the 1980s, at the National Research Council of Canada (NRCC), the first attempts at 3D scanning using structured light were made, projecting a point cloud onto museum artifacts to perform precision calculations on replicas and generate digital models [10,11]. These techniques were later applied to various pieces of cultural heritage, such as relics from the Mausoleum of Qin Shi Huang, “La Minerva di Arezzo,” and sculptures by Donatello and San Giovanni Pisano [11].

Prominent projects such as the Michelangelo Project (1988) and the Pietà Rondanini project (1998) allowed for the rigorous and precise documentation of Michelangelo’s works in Italy [12,13]. In 2003, a joint project between the French Center of Museum Research and Restoration and the National Research Council of Canada scanned the pictorial surface and the reverse side of Leonardo da Vinci’s Mona Lisa [14].

Three-dimensional digitization enables the creation of accurate virtual models of real objects, reproducing their volume, texture, and color. Specialized equipment measures thousands of coordinates on the object’s surface, generating a dense point cloud that is subsequently processed to create a faithful 3D reproduction using trigonometric triangulation [15].

Main Three-Dimensional Scanning Methods

For any 3D scanning method, the starting point is the precise registration of points on the object’s surface. Various methods have been developed and applied over time [4,15,16]

The techniques are divided into “contact techniques,” which require touching the surface, and “contactless techniques,” which acquire points without physical contact and are generally less invasive. The former include 3D positioning systems, where a mechanical arm records movement manually or through a robotic arm. On the other hand, contactless techniques are more varied and are based on principles such as emitting or receiving light, sound, ultrasound, microwaves, and X-rays, which are transformed into surface point coordinates [17] Figure 1.

Among the passive techniques are stereophotogrammetry and digital photogrammetry [18], which allow for the reconstruction of 3D models from 2D images. Techniques such as obtaining models from silhouettes or object shadows are also used. Among these techniques, the most commonly used today are those called “structure from motion,” which do not require calibrated cameras. These methods involve taking numerous photographs of the object from different perspectives and distances to reconstruct a point cloud of the object and generate an interpolated surface Figure 2.

On the contrary, active techniques emit some form of radiation towards the object, either through transmission or reflection. Examples of active techniques are computed tomography and structured light, which will be detailed later.

Optical 3D scanning techniques are active methodologies widely used in architecture, archaeology, and conservation. In addition to structured light, they include active triangulation and time-of-flight techniques, which involve projecting a laser beam onto the object’s surface to obtain its 3D coordinates.

The fringe projection method (FPM)

The fringe projection technique is an optical method used in the three-dimensional reconstruction of objects. Essentially, the technique consists of the projection of a structured light pattern (black and white fringes) onto a reference surface and onto the object of interest of which a 3D digital representation is desired. Figure 1 shows the typical configuration of the experimental setup of the technique. This arrangement consists of a projector, which is used to project the fringe pattern, a CCD camera to capture the images and a computer in which the analysis of the images is performed, and the 3D digital representation of the object is obtained by means of specialized algorithms.

The aim of the technique is the acquisition of “n” (usually n=4) images of the fringe pattern with and without the object. In the case of the reference plane images (without the object), the fringes are completely parallel, and their phase remains constant. When the object is placed, the fringes undergo a displacement (deformation) because of the topography of the object, see figure 2.

The distortion fringe pattern is modeled as

where a(x,y) is the background ligth, b(x,y) is the modulation of the fringes, is the carrier phase term and the shift of phase produced by the height of the object. For gray-scale fringes, the background and modulation are adjusted to generate sinusoidal fringes from black to white theoretically. In practice is so difficult achieved this range however, the optical set-up must be adjusted to obtain the result closest to the ideal. The key work is to recover the phase of the fringes image captured because the phase contains the height of the object. There are some techniques to phase recovery as X, Y, Z (references), one of them is phase stepping (reference). This technique is popular for the few images requires and the precision good. Phase stepping involves the capture of N-images (minimum three) with a known N-step of fringe shifting. The N-step images are models by

where n represents the phase-shift index n=0,1,2,...,N-1. The steps of fringe enable to solve a system of equations from which the phase is obtained. The solution of system of equation is given for following expression.

Without indentation ϕ is the corresponding wrapped phase. The unwrapped phase is essential to obtain the information of the object and then get the 3D digital model. This step is critical to getting accurate models, due to this, in literature exist a great number of algorithms to unwrap the phase [19,20]

Results of our study on the precision of the FPM method and its potentiality

The accuracy of FPM is determined by multiple factors such as the characteristics of the projector and camera, configuration of the projected pattern, phase detection and unwrapping algorithm, calibration technique, and properties of the object to be captured, among others [21-24]. In all of the aforementioned aspects, there are proposals to improve resolution, for example, lens distortion elimination [25], gamma correction techniques [26], sampling techniques [27], calibration methods [28], local blur analysis by subsurface scattering [29].

In the study conducted by [20], we experimentally determined the precision and depth variance of a specific system. The experiment involved using a camera and projector with slightly above-standard quality, and more technical details can be found in the manuscript Figure 3. The experiment consisted of capturing a reference piece with steps of varying heights.

These steps were produced using computer numerical control (CNC) machinery and measured with significantly higher resolution than the precision expected from the machinery, employing an optical profilometer. Eight steps were fabricated, with the tallest being 2.0993 mm and the lowest being 0.0141 mm. The smallest fringe period achieved was 0.5 mm. The maximum resolution with acceptable variance (with an unequivocally recognizable step) was 0.1 mm, with a standard deviation of 0.0125 mm.

In addition to the quantitative study, our work included a qualitative analysis of an oil painting on a linen canvas titled “The Rooster.” This artwork was chosen because its creation involved a technique that posed a challenge to our method, making it an ideal test case. It exhibits a variety of reliefs, textures, and tones, which make it difficult for the scanning technique used. The captured surface reveals global aspects such as relief angles and textures. The achieved resolution allows the observation of very fine details, such as aligned ridges that tend towards verticality and small peaks at the edges of impasto, likely due to irregularities at the edges of the ridges generated by the spatula and/or brushstrokes from the base. Moreover, the resolution enables the observation of slight undulations on an almost flat background. This resolution allows us to observe and measure details related to conservation and technical aspects, such as brushstroke orientation, micro-fractures, and crack patterns, among others [20].

The results obtained in our 2023 study have demonstrated the potential of these scanning techniques in the field of artwork restoration, particularly in restoring canvas paintings. However, the achieved resolution opens up the possibility of utilizing this technique in numerous conservation fields with countless applications.

Possible future application of FPM to cultural heritage

The FPM continues to be a subject of ongoing research and development by numerous groups worldwide. In the specific field of conservation, efforts are being made to enhance the quality of captures and to develop increasingly user-friendly software. This suggests that shortly, these technologies will become more widespread, accessible, user-friendly and potentially leading to their adoption in a variety of applications.

Among the potential applications in archaeology and conservation, one notable area is the digitization of entire archaeological sites in high resolution. Currently, most FPM applications focus on individual objects. However, in the next future, this method could be adapted to digitize complete archaeological sites, enabling the creation of precise 3D models of complex structures such as temples, ruins, and archaeological remains.

Another potential application is the inspection and conservation of architectural structures. Fringe projection could be employed to inspect and monitor the structural integrity of historical buildings and architectural monuments. Due to its potential precision, this technology could detect changes in the shape and geometry of these structures over time, aiding in their long-term conservation and maintenance.

Equally significant are applications in augmented reality and virtual reality. The information obtained from these methodologies could be integrated into augmented reality and virtual reality applications to provide immersive and educational experiences related to cultural heritage. Users could digitally explore artifacts and historical sites with an unprecedented level of detail from remote locations. Overall, fringe projection methodologies, along with other structured light techniques, could find applications in the entertainment industry. They could be used, together with Artificial Intelligence applications, to create special effects for movies, video games, and theatrical productions requiring accurate representations of historical objects and ancient architecture.

More specifically, in the field of conservation, the method could be utilized for artifact reproduction and restoration. Structured light could be used to create exact digital replicas of historical artifacts and artworks. These replicas could be 3D printed or used to guide or test restoration processes, allowing conservators to work with a precise digital copy of the original object for intervention simulations.

In summary, the future applications of fringe projection methods are poised to evolve, ranging from the comprehensive digitization of archaeological sites to integration into augmented reality and entertainment platforms. Additionally, these methods hold promise for the conservation and documentation of individual objects, as well as the restoration of artifacts, further extending their potential in augmented reality and entertainment domains.

Potential use of the FPM technique in our laboratory

As a specific example in the field of conservation, this paragraph highlights the technique currently employed in our laboratory, which will be the subject of forthcoming publications. Our laboratory specializes in canvas painting conservation, focusing on assessing the condition of these artifacts, exploring potential intervention techniques, and refining specific conservation methodologies Figure 4.

The methodology developed and previously tested by [20] will be applied to assess the quality and efficiency of conservation interventions. These interventions involve actions taken to preserve and protect cultural, historical, or artistic objects. Typically, these interventions aim to stabilize, repair, and restore artistic objects without compromising their original integrity or authenticity. They may include surface cleaning, consolidation of fragile materials, repair of structural damage, and implementation of preventive measures to avoid future deterioration. Conservation interventions are conducted with extreme care and respect to preserve the authenticity and historical or artistic significance of the object. However, each technique has its own characteristics and technological peculiarities

One highly important conservation technique is conservative lining, which is used to strengthen and preserve the structure of canvas paintings. Relining is carried out using a variety of techniques and materials like animal gelatin glue paste, wax and resin adhesive, synthetic water-soluble adhesive, acrylic resin adhesive among others. Each technique has its advantages and specific considerations in terms of reversibility, strength, flexibility, and compatibility with the artwork. The choice of the appropriate technique depends on the conservation status of the object, the preferences of the conservator, and the specific needs of the artwork. It is important for relining to be carried out carefully and follow the ethical principles of conservation to ensure the integrity and authenticity of the artwork.

To date, there has been no quantitative study of the invasiveness of these techniques. Often, the decision to use one technique over another depends on the preference of the operator or the restoration school. Naturally, changing the type of material for this operation can result in more or less significant changes to the final appearance of the pictorial surface. It is extremely important to evaluate the invasiveness of a specific technique so that the choice of one relining method over another is based on the type of surface that needs to be consolidated rather than the personal preferences of the restorer.

Given the proven high resolution of the three-dimensional models provided by the FPM technique, we are conducting a comparative study of relining techniques using FPM as a highresolution scanning method. The study involves comparing identical samples (same artist, same subject, size, and execution technique) relined with different methods. Each sample will be scanned before and after the restoration intervention. The obtained 3D models will be registered with great precision, and the differences between the models will provide quantitative data on how much the surface has been modified by the treatment. This study is considered highly important for future decision-making, aiming to move away from subjectivity and focus on objective and quantitative data. Each sample will be scanned before and after the restoration intervention. The obtained 3D models will be registered with great precision, and the difference in height between the models will provide quantitative data on how much the surface has been modified by the treatment.

Conclusions

Throughout history, there has been a continuous effort to accurately replicate cultural artifacts. However, the techniques developed in the last twenty years have made it possible to obtain highly detailed digital models of objects, which hold significant potential in the fields of art and conservation. This enables the creation of precise digital models of cultural heritage objects, with practical applications in museography, cataloging, preservation, and virtual exhibitions. These digital models serve as records of the artifact’s condition and can aid in its conservation and restoration efforts.

Previous studies conducted by our group, focusing on assessing the precision of the method in-plane and out-of-plane using a system accessible to any laboratory, have shown that achieving a precision on the order of one hundred microns (and in many cases even tens of microns) is easily attainable. This allows for the documentation and measurement of very minute details of the artwork. The high resolution and non-invasive nature of FPM make it a valuable tool for curator of cultural heritage and restorers. Possible future applications of the technique in conservation include comprehensive digitization of archaeological sites, inspection of architectural structures, integration into augmented reality and virtual reality applications, and use in artifact reproduction and restoration.

Currently, in our laboratories, we are applying the methodology developed and previously validated in the field of conservative restoration. A comparative study of different relining techniques using FPM is underway to provide quantitative data on the invasiveness of each technique. This study is expected to add significant quantitative information that can guide operators in selecting the most appropriate techniques.

The FPM methodology, which is already gaining prominence, will become indispensable in the field of art conservation and restoration in the near future. Overall, it will represent a highly valuable tool in the preservation of cultural heritage.

Acknowledgments

We acknowledge the Centro en Investigaciones en Óptica in León, Guanajuato, the Laboratorio de Análisis de Imágenes y Modelado Analógico de la UASLP, as well as the Maestría en Ciencias del Procesamiento de la Información de la UAZ, for allowing us to use their facilities. We would also like to acknowledge Bernardino Barrientos and Carlos Mares for their invaluable guidance and support during the laboratory research at CIO León Guanajuato.

References

1. Remondino F (2011) Heritage recording and 3D modeling with photogrammetry and 3D scanning. Remote sensing 3(6): 1104-1138.
2. Mara H, Kampel M, Niccolucci F, Sablatnig R (2007) Ancient coins & ceramics–3D and 2D documentation for preservation and retrieval of lost heritage. In Proc. 2nd ISPRS Intl. Workshop 3D-ARCH: 3D Virtual Reconstruction and Visualization of Complex. Architectures pp. 12-13.
3. Lerma J L and Muir C (2014) Evaluating the 3D documentation of an early Christian upright stone with carvings from Scotland with multiples images. Journal of archaeological science 46(1): 311-318.
4. Sansoni G, Trebeschi M, Docchio F (2009) State-of-the-art and applications of 3D imaging sensors in industry, cultural heritage, medicine, and criminal investigation. Sensors 9(1): 568-601.
5. West J, Kuk G (2016) The complementarity of openness: How MakerBot leveraged Thingiverse in 3D printing. Technological Forecasting and Social Change 102(1): 169-181.
6. Inzerillo L, and Di Paola F (2017) From SFM To 3d Print: Automated Workflow Addressed To Practitioner Aimed At The Conservation And Restauration. International Archives of the Photogrammetry, Remote Sensing & Spatial Information Sciences 42(2): 375-382.
7. Breuckmann B (2014) 25 Years of High-Definition 3D Scanning: History, State of the Art, Outlook. Electronic Visualization and the Arts pp. 262-266.
8. Asmus J F, Guattari G, Lazzarini L, Musumeci G, Wuerker R F (1973) Holography in the conservation of statuary. Studies in conservation 18(2): 49-63.
9. Amadesi S, Gori F, Grella R, Guattari G (1974) Holographic methods for painting diagnostics. Applied Optics 13(9): 2009-2013.
10. Boulanger P, Rioux M, Taylor J, Livingstone F (1988) Automatic replication and recording of museum artifacts. In Analysis and examination of an art object by imaging technique 131-147.
11. Múnera E M, Salazar J F R, Bedoya J W B (2010) Construcción de un modelo digital 3D de piezas precolombinas utilizando escaneo lá Revista Avances en Sistemas e Informática 7(1): 197-206.
12. Guidi G, Pieraccini M, Ciofi S, Atzeni C, Beraldin J A, et al. (2001) Immagini digitali tridimensionali di beni artistici. Alta Frequenza 13(2): 30-36.
13. Bernardini F, Rushmeier H, Martin I M, Mittleman J, Taubin G (2002) Building a digital model of Michelangelo's Florentine Pieta. IEEE Computer Graphics and Applications 22(1): 59-67.
14. Mohen J P, Menu M, Mottin B Leonardo D V (2006) Mona Lisa: inside the painting. (No Title).
15. Reyes Solís J A y Jiménez Badillo D (2011) Técnicas para el modelado Tridimensional de Artefactos Arqueoló Primer Congreso Virtual de Arqueología “El Arqueólogo frente a la Sociedad” 15.
16. Carrillo Garza V (2019) La Digitalización Tridimensional del Patrimonio Cultural Mueble como herramienta para la Conservació Fotogrametría Digital y Escáner de Luz Estructurada. Tesis de licenciatura en restauración, Escuela Nacional de Conservación, Restauración y Museografía. p. 350.
17. Wagner H J, Alvarez M, Groenewolt A, Menges A (2020) Towards digital automation flexibility in large-scale timber construction: integrative robotic prefabrication and co-design of the BUGA Wood Pavilion. Construction Robotics 4(4): 1-18.
18. Magnani M, Douglass M, López Varela S L (2018) Photogrammetry and stereophotogrammetry. Encyc l Archaeol Sci 1-4.
19. Barrientos B, Mares C, Sarocchi D, Cerca M, Valdivia R (2019) Dynamic threedimensional displacement analysis of small-scale granular flows by fringe projection and digital image correlation. Landslides 17(5): 825-837.
20. Casas Pérez M D C, Moreno Chávez G, Castillo Rivera F, Sarocchi D, Mares C, et al. (2023) Fringe Projection Method for 3D High-Resolution Reconstruction of Oil Painting Surfaces. Heritage 6(4): 3461-3474.
21. Bothe T, Li W, Von Kopylow C, Juptner W P (2004) High-resolution 3D shape measurement on specular surfaces by fringe reflection. In Optical metrology in production engineering 5457(1): 411-422.
22. Huang J, Wang Z, Xue Q, Gao J (2012) Calibration of a camera–projector measurement system and error impact analysis. Measurement Science and Technology 23(12): 125402-125402.
23. Zheng D, Da F, Kemao Q, Seah H S (2016) Phase error analysis and compensation for phase shifting profilometry with projector defocusing. Applied optics 55(21): 5721-5728.
24. Kulkarni R, Rastogi P (2020) Fringe denoising algorithms: a review. Optics and Lasers in Engineering, 135(11): 106190-106190.
25. Kai Li, Jingjie Bu, Dongsheng Zhang (2016) Lens distortion elimination for improving measurement accuracy of fringe projection profilometry, Optics and Lasers in Engineering 85(1): 53-64.
26. Hoang T, Pan B, Nguyen D, Wang Z (2010) Generic gamma correction for accuracy enhancement in fringe-projection profilometry. Optics letters 35(12): 1992-1994.
27. Villa J, Rodríguez-Reveles G A, Moreno G, De la Rosa I (2023) Temporal phase-unwrapping in fringe projection profilometry: Increasing the accuracy with equidistant long time-steps sampling. Optics and Lasers in Engineering 167(1): 107591-107591.
28. Feng S, Zuo C, Zhang L, Tao T, Hu Y, et al. (2021) Calibration of fringe projection profilometry: A comparative review. Optics and lasers in engineering, 143(2): 106622-106622.
29. Rao L and Da F (2018) Local blur analysis and phase error correction method for fringe projection profilometry systems. Applied optics 57(15): 4267-4276.