Digital resources in the Social Sciences and Humanities OpenEdition Our platforms OpenEdition Books OpenEdition Journals Hypotheses Calenda Libraries OpenEdition Freemium Follow us

​Innovative Tools and Current Projects: Artificial Intelligence in Archives and Collections

In his article, Erdal Ayan presents a selection of AI tools for archives and collections as well as current projects that were presented at the Marburg AI Conference.

Key topics of the conference Artificial Intelligence in Archives and Collections: Practices, Potentials and Evidence Production in Dealing with Images and Multimodal Cultural Heritage included computer vision techniques, multi-modal large language models (MLLMs), semi-automated annotation, and innovative tools for search and discovery. Several new and ongoing projects showcased advancements in deep learning, text-image embedding, and graph-based learning.

This blog post is part of a series on the conference held on December 12-13, 2024, at the Herder Institute in Marburg and will present a number of selected tools and topics that have been discussed.

Opportunities and Challenges

The conference emphasized both the opportunities and challenges presented by AI-driven tools, including ethical concerns, explainability, and data bias, while promoting collaboration and knowledge exchange across disciplines. It focused on visual sources such as photographs, graphic collections, and mixed image-text archives, addressing both theoretical and practical challenges in automated indexing, cataloguing, and image processing. The growing role of AI and ML in multi-modal cultural heritage research was highlighted, particularly in semantic segmentation, object classification, and annotation. Key datasets discussed included coins, archaeological record cards, Buddhist murals, Soviet newsreels, and historical photographs, etc. This article will present a number of selected tools and topics that have been discussed.

​Image Archives and Image Processing

Image archiving and processing topics had an important place in the conference, and current studies on these topics attracted a lot of attention from the participants. The presentations at the conference demonstrated how AI and ML are revolutionizing the management and analysis of image archives.

Ralph Ewerth introduced iART, a computer vision-based search engine that enhances the accessibility of art and historical archives. It addresses cultural heritage images that integrate computer vision techniques for object recognition and cross-modal search. The dataset includes coin images, and the tool supports both scholarly research and public engagement. iART enables users to explore large image collections through features like object recognition, pose estimation, and natural language queries. Ewerth demonstrated how state-of-the-art computer vision techniques, combined with large language models, facilitate improved retrieval results and explanations for search outcomes.

Frank Puppe addressed the digital indexing of archaeological record cards, presenting a pipeline that processes scanned documents through Optical Character Recognition (OCR), semantic mapping, and multi-modal large language models like ChatGPT-4o. This pipeline enables the transformation of analogue archaeological records with images into searchable, structured formats such as JSON.

Home page of HikarIA, screenshot taken on 21 May 2025.

Christopher Kermorvant highlighted challenges in applying contemporary deep learning models to early Japanese photographs. He introduced HikarIA, which is a search engine designed for Japanese historical photographs, addressing issues of limited metadata and enabling discovery through advanced AI techniques. While convolutional neural networks and transformer models excel with modern datasets, historical images often present difficulties due to cultural and temporal differences. Kermorvant emphasized the need for tailored approaches to address these limitations.

Erik Radisch mentioned the Segment Anything, integrated with Annotorious for semi-automated image annotation. Segment Anything is a model developed by Meta AI for image segmentation, capable of prompt-based annotation without extensive retraining. And Annotorious is an open-source JavaScript solution for image annotation. This method significantly reduces the time required for detailed polygon-based annotations, making it ideal for large-scale archival projects. Segment Anything’s ability to adapt to new tasks without extensive retraining provides a powerful tool for researchers.

​New and Ongoing Projects in Image Archiving

At the conference, researchers talked in detail about their own work on image archiving and processing and shared their knowledge and experience on new and ongoing projects. The conference, therefore, showcased several innovative projects and tools that push the boundaries of AI-driven cultural heritage research. Among these were:

Celtic coin type “Divinka” from Slovakia. Foto: Marek Sobola, Wikimedia Commons.

ClaReNet (Karsten Tolle): A project utilizing object detection and classification to analyze coins. Tolle demonstrated the use of Orange Data Mining, a visual programming tool, to enable non-programmers to perform clustering and classification tasks. 

Wiedergutmachung (Harald Sack and Mahsa Vafaie): A collaboration with the Landesarchiv Baden-Württemberg that focuses on the classification and extraction of handwritten and machine-printed text in historical documents. Vafaie presented a pipeline for separating handwritten annotations from machine-printed text, resulting in a 26% improvement in OCR accuracy.

Kinokroonika (Mila Oiva): A project that explores clustering of images as graph data, leveraging graph-based methods for visual analysis.

Dight-Net (Mila Oiva): A collaborative research initiative utilizing tools like ResNet-50 to advance digital cultural heritage projects.

Orange: An open-source visual programming tool for clustering, classification, and image analytics, designed for non-programmers.

ResNet-50: A deep learning model architecture for image classification, compatible with Python libraries like Keras.

Collection Space Navigator (CSN): A tool for visualizing and exploring collections through vector encoding of images, built using Python (Flask) and React.

The sharing of these studies and experiences by experts at the conference provided an important impetus for knowledge transfer and the development of future collaborations.

Conclusion

The Artificial Intelligence in Archives and Collections conference underscored the transformative potential of AI and machine learning in managing, analyzing, and understanding cultural heritage data. Projects like iART and Wiedergutmachung demonstrated practical applications of computer vision and text recognition, while tools like Segment Anything and Orange highlighted the growing accessibility of AI technologies for non-experts.

Despite these advancements, the conference also addressed challenges such as data bias, explainability, and the limitations of current models when applied to historical datasets. Speakers emphasized the importance of interdisciplinary collaboration and the need for ethical considerations in the deployment of AI technologies.

Overall, the event successfully fostered dialogue between researchers, developers, and practitioners, paving the way for future innovations in AI-driven cultural heritage research.


Selected Reference List for the Projects Presented

Arnold, T. (2024). Explainable and Auditable Search and Discovery of Visual Cultural Heritage Collections.

Aviles-Rivero, A. I. (2024). Dusting Off the Unlabeled Data: Graph Semi-Supervised Learning for Large-Scale Datasets.

Evans, J. (2024). The Geometry of Culture: Analyzing Meaning through Embeddings of Text and Images.

Ewerth, R. (2024). Unlocking Cultural Heritage: Computer Vision for Art and History Archives.

Markus Huff, Nine Abele, Dominik Kimmel, Helen Fischer, Gerrit Anders, Tolgahan Aydin, Jürgen Bude. (2024). ArchiveGPT: Psychological and Technological Perspectives on the AI-Supported Archiving of Image Material.

Kermorvant, C. (2024). How Contemporary Deep Learning Models Describe Early Japanese Photographs?

Puppe, F., N. Fischer, D. Kimmel (2024). Pipeline for Digital Indexing of Archaeological Record Cards.

Radisch, E. (2024). A New Approach to Semi-Automated Annotations with Segment Anything.

Sack, H., & Vafaie, M., Waitelonis J. (2024). Separation of Machine-Printed and Handwritten Text in Archival Documents.

Tolle, K. (2024). Potpourri of Computer Vision in Cultural Heritage.

All of these projects were presented at the conference. The full programme and all abstracts can be found on the website of the Herder Insitute.


Erdal Ayan is a Software developer working at FIZ-Karlsruhe, Germany. As a young researcher and developer, he has been active in Digital Humanities for many years now. He is currently interested in software development in image processing and Computer Vision and has participated in the conference.


Title picture: An example of image processing. Original image, and resulting image after Laplacian and Sobel filters. Alaens, Wikimedia Commons.


OpenEdition schlägt Ihnen vor, diesen Beitrag wie folgt zu zitieren:
Lab 1.3 Digitale Heuristik und Historik (22. Mai 2025). ​Innovative Tools and Current Projects: Artificial Intelligence in Archives and Collections. Value of the Past. Abgerufen am 17. April 2026 von https://doi.org/10.58079/14018


Schreibe einen Kommentar

Deine E-Mail-Adresse wird nicht veröffentlicht. Erforderliche Felder sind mit * markiert

This site uses Akismet to reduce spam. Learn how your comment data is processed.