FWF Project CrossSAVE-CH: Crossmodal Search and Visual Exploration of 3D Cultural Heritage Objects (2019-2022)

3D objects are used in many application areas to represent form, appearance and function of natural and man-made objects. Besides applications in industry and technology, they are an important documentation type for museums and digital libraries, where 3D objects represent object collections, archaeological artifacts, or architectures for presentation, research and long-term preservation. Novel methods for 3D digitization, interactive modeling and shape synthesis support the creation of increasingly large 3D object repositories. While for smaller repositories the individual inspection of objects by users is possible, this is not efficient anymore for large repositories consisting of many objects. Rather, appropriate techniques for similarity search based on example objects or sketches are an essential functionality for exploring and leveraging large repositories.

In this FWF project, we research new methods for searching, comparing, and visually exploring 3D cultural heritage objects based on appropriately specified user searches. Specifically, we want to develop and evaluate new methods for cross-modal search. We want to enable searching with query modalities including 3D object, image and sketch against target repositories comprising 3D objects, images, documents, drawings, and metadata. We want to evaluate how such search techniques can aid in object comparison and classification tasks as often occurring in archaeological research. One of our foci is to search with incomplete 3D query objects, as objects in Cultural Heritage data may be incomplete, as artifacts over time have become broken or eroded. One of our goals is to enable users to interactively complete an incomplete 3D query shape by sketching editor, allowing to visually express a hypothesis for a possible shape. Another goal is to develop new methods for interactive visual exploration of search results and repositories, by means of shape- and document visualization, supporting the comparison of search results and classification of objects. Our new methods will be applied and evaluated on relevant archaeological 3D object and document data, together with research partners.

Project Partners

• University of Graz, Institute of Archaeology
• University of Chile, Millenium Institute for Foundational Research on Data

Project Publications

		S. Lengauer, R. Preiner, I. Sipiran, S. Karl, E. Trinkl, B. Bustos and T. Schreck. Context-based Surface Pattern Completion of Ancient Pottery. Proc. Eurographics Workshop on Graphics and Cultural Heritage 2022. [PDF|https://doi.org/10.2312/gch.20221234].
		S. Karl, P. Houska, S. Lengauer, R. Preiner, E. Trinkl and T. Schreck. Advances in Digital Pottery Analysis. it - Information Technology. 2022. [PDF|https://doi.org/10.1515/itit-2022-0006].
		S. Lengauer, P. Houska, R. Preiner, I. Sipiran, E. Trinkl, S. Karl, B. Bustos and T. Schreck. Interactive annotation of geometric ornamentation on painted pottery assisted by deep learning. it - Information Technology. 2022. [https://doi.org/doi:10.1515/itit-2022-0007].
		P. Houska, S. Lengauer and R. Preiner. Direct Elastic Unrollings of Painted Pottery Surfaces from Sparse Image Sets. Proc. Eurographics Workshop on Graphics and Cultural Heritage 2021. https://doi.org/10.2312/gch.20211417].
		I. Sipiran, P. Lazo, C. Lopez, M. Jimenez, N. Bagewadi, B. Bustos, H. Dao, S. Gangisetty, M. Hanik, N.-P. Ho Thi, M. Holenderski, D. Jarnikov, A. Labrada, S. Lengauer, R. Licandro, D.-H. Nguyen, T.-L. Nguyen-Ho, L.A. Perez Rey, B.-D. Pham, M.-K. Pham, R. Preiner, T. Schreck, Q.-H. Trinh, L. Tonnaer, C. Tycowicz, T.-A. Vu-Le: SHREC 2021: Retrieval of cultural heritage objects. Computers & Graphics. Vol. 100. November 2021. 1-20. PDF
		S. Lengauer, I. Sipiran, R. Preiner, T. Schreck, B. Bustos: A Benchmark Dataset for Repetitive Pattern Recognition on Textured 3D Surfaces. Computer Graphics Forum. Vol. 40. No. 5. 2021. PDF / Presentation / Dataset
		S. Lengauer, A. Komar, S. Karl, E. Trinkl, I. Sipiran, T. Schreck and R. Preiner: Semi-automated Annotation of Repetitive Ornaments on 3D Painted Pottery Surfaces. Proc. Eurographics Workshop on Graphics and Cultural Heritage 2020. PDF / Presentation
		E. Trinkl, S. Karl, S. Lengauer, R. Preiner, and T. Schreck. "Cross-Modal Search and Exploration of Greek Painted Pottery". The 3 Dimensions of Digitalised Archaeology. State-of-the-art, Data Management and Current Challenges in Archaeological 3D Documentation. (forthcoming).
		S. Lengauer, A. Komar, S. Karl, E. Trinkl, R. Preiner, and T. Schreck: Visual Exploration of Cultural Heritage Collections with Linked Spatiotemporal, Shape and Metadata Views. Proc. VMV. 2020, 137-144. PDF / Spotlight / Presentation
		S. Lengauer, A. Komar, A. Labrada, S. Karl, E. Trinkl, R. Preiner, B. Bustos, and T. Schreck: A sketch-aided retrieval approach for incomplete 3D objects. Computers & Graphics 87 (2020): 111-122. PDF
		T. Schreck, S. Lengauer, and R. Preiner. "Forschungsprojekt cross-modale Suche und visuelle Exploration von 3D Kulturgutobjekten". Forum Archaeologiae. Vol. 91. VI. 2019. PDF
		S. Lengauer, A. Komar, A. Labrada, S. Karl, E. Trinkl, R. Preiner, B. Bustos, and T. Schreck: Motif-driven Retrieval of Greek Painted Pottery. Proc. Eurographics Workshop on Graphics and Cultural Heritage 2019. PDF
		S. Lengauer, A. Komar, A. Labrada, S. Karl, E. Trinkl, R. Preiner, B. Bustos, and T. Schreck: Sketch-Aided Retrieval of Incomplete 3D Cultural Heritage Objects. Proc. Eurographics Workshop on 3D Object Retrieval 2019.  PDF

Related Work
EU FP 7 Project PRESIOUS - PREdictive digitization, reStoration and degradatIOn assessment of cultUral heritage objectS.

Acknowledgment
This work is co-funded by the Austrian Science Fund FWF and the State of Styria, Austria within project Crossmodal Search and Visual Exploration of 3D Cultural Heritage Objects (P31317-NBL).