The International Image Interoperability Framework (IIIF)

Community-driven standards to deliver, view and annotate digital objects

Julien A. Raemy, PhD Candidate in Digital Humanities
DHLab, University of Basel
ORCID Google Scholar GitHub Mastodon

Digital Humanities Workshops & Vorträge FS 2023
Universität Bern | 04.04.2023

Linked Open Usable Data for Cultural Heritage

Perspectives on Community Practices and Semantic Interoperability

PhD Thesis - https://phd.julsraemy.ch - supervised by:

  • Peter Fornaro (University of Basel)
  • Walter Leimgruber (University of Basel)
  • Robert Sanderson (Yale University)

Agenda

  • What is IIIF?
  • Why do we need IIIF?
  • What does IIIF do?
  • How does IIIF work?
  • Try it yourself
  • Web Annotation Data Model
  • PIA IIIF Annotation Workflow
Julien A. Raemy | IIIF

What is IIIF?

Julien A. Raemy | IIIF

International Image Interoperability Framework

(triple-eye-eff)

  • A model for presenting and annotating content
  • A global community that develops shared application programming interfaces (APIs), implements them in software, and exposes interoperable content

https://iiif.io

Julien A. Raemy | IIIF
Julien A. Raemy | IIIF

IIIF Community

International leaders from:

  • Museums & Galleries
  • Aggregators/Facilitators
  • Universities & Research Institutions
  • State and National Libraries
Julien A. Raemy | IIIF

IIIF Community

Community and Technical Specification Groups

  • CGs: 3D, A/V, Archives, Design, Manuscripts, Maps, Museums, Newspapers, Outreach
  • TSGs: 3D, Authentication, Content Search, Discovery, Maps

https://iiif.io/community/

Julien A. Raemy | IIIF

Why do we need IIIF?

Julien A. Raemy | IIIF

Images are fundamental carriers of information

All images come from the Art Institute Chicago's platform and are IIIF-compliant, check the source code

The problem

A world of silos and duplication

Image delivery on the Web has historically been hard, slow, expensive, disjointed, and locked-up in silos.

Julien A. Raemy | IIIF
Julien A. Raemy | IIIF

What does IIIF do?

Julien A. Raemy | IIIF
Franks, Kendal; Royal College of Surgeons of England. The Germ Theory. via Wellcome Library.
HarvardX: MCB64.1x - Cell Biology: Mitochondria

How does IIIF work?

Julien A. Raemy | IIIF
Julien A. Raemy | IIIF

IIIF Specifications

  • Image API
  • Presentation API
  • Authorization Flow API
  • Change Discovery API
  • Content Search API
  • Content State API

The Image and Presentation APIs are referred to as the core IIIF APIs

Julien A. Raemy | IIIF

IIIF Image API

It specifies a RESTful web service that returns an image in response to a standard HTTP(S) request.

  • Image Request
  • Image Information (JSON-LD)

https://iiif.io/api/image

Julien A. Raemy | IIIF
Julien A. Raemy | IIIF

IIIF Presentation API

It is a JSON-LD based web service which provides the necessary information about the object or collection structure and layout.

https://iiif.io/api/presentation

Julien A. Raemy | IIIF
Julien A. Raemy | IIIF
Julien A. Raemy | IIIF

Try it yourself

Julien A. Raemy | IIIF

Exercise

  • Find IIIF-compliant resources (Manifest or Collection)
  • Display them in a IIIF viewer
  • Compare severals IIIF Manifests on Mirador

Guides to find IIIF resources and how to use viewers: https://guides.iiif.io/

Julien A. Raemy | IIIF

Web Annotation Data Model

Julien A. Raemy | IIIF

Web Annotation Data Model

The Web Annotation Data Model (WADM) derives from the Open Annotation Specification.

https://www.w3.org/TR/annotation-model/

Julien A. Raemy | IIIF

WADM Motivations

To uderstand the reasons why the Annotation was created, or why the Textual Body was included in the Annotation.

Some of the Motivations: commenting, highlighting, identifying, tagging

Julien A. Raemy | IIIF

PIA IIIF Annotation Workflow

Julien A. Raemy | IIIF

PIA

Participatory Knowledge Practices in Analogue and Digital Image Archives

  • Institute for Cultural Anthropology and European Ethnology, University of Basel
  • Digital Humanities Lab, University of Basel
  • Bern Academy of the Arts, Bern University of Applied Sciences
Julien A. Raemy | IIIF

Collections of the Swiss Society for Fokflore Studies (SSFS) within PIA

Developing a Citizen Science platform on the basis of:

  • SGV_05 Atlas der Schweizerischen Volkskunde - Cartography
  • SGV_10 Familie Kreis - Vernacular Photography
  • SGV_12 Ernst Brunner - Photojournalism
Julien A. Raemy | IIIF

PIA Infrastructure / APIs

At the moment:

  • IIIF Image API 3.0
  • IIIF Presentation API 3.0
  • Omeka S API

In the future:

  • Linked Art API 1.0
  • IIIF Change Discovery API 1.0
Julien A. Raemy | IIIF
Julien A. Raemy | IIIF

Annotations

  • Machine Learning: Leveraging vitrivr to detect objects (Faster R-CNN Inception Resnet V2)
  • From humans: a proof-of-concept still needs to be undertaken

The goal is that all these annotation outputs are compatible with the Web Annotation Data Model.

Julien A. Raemy | IIIF
Julien A. Raemy | IIIF
Julien A. Raemy | IIIF

Presentation API 3.0 Resources

  • Manifest: https://iiif.participatory-archives.ch/<id>/manifest.json
  • AnnotationPage: https://iiif.participatory-archives.ch/annotations/<id>-<listid>.json

Example of a Manifest displayed in our Mirador instance: https://mirador.participatory-archives.ch/?manifest=https://iiif.participatory-archives.ch/SGV_12N_19783/manifest.json

Julien A. Raemy | IIIF
{
  "@context": "http://iiif.io/api/presentation/3/context.json",
  "id": "https://iiif.participatory-archives.ch/annotations/SGV_12N_00001-p1-list.json",
  "type": "AnnotationPage",
  "items": [
    {
      "@context": "http://www.w3.org/ns/anno.jsonld",
      "id": "https://iiif.participatory-archives.ch/annotations/SGV_12N_00001-p1-list/annotation-2800001.json",
      "motivation": "commenting",
      "type": "Annotation",
      "body": [
        {
          "type": "TextualBody",
          "value": "person",
          "purpose": "commenting"
        },
        {
          "type": "TextualBody",
          "value": "Object Detection (vitrivr)",
          "purpose": "tagging"
        },
        {
          "type": "TextualBody",
          "value": "<br><small>Detection score: 0.999616265296936</small>",
          "purpose": "commenting"
        }
      ],
      "target": {
        "source": "https://iiif.participatory-archives.ch/SGV_12N_00001/canvas/p1",
        "selector": {
          "type": "FragmentSelector",
          "conformsTo": "http://www.w3.org/TR/media-frags/",
          "value": "xywh=3225,201,943,4051"
        },
        "dcterms:isPartOf": {
          "type": "Manifest",
          "id": "https://iiif.participatory-archives.ch/SGV_12N_00001/manifest.json"
        }
      }
    },
Julien A. Raemy | IIIF

Image Credits

  • [Blick auf das Spalentor]. Basel, 1938. Ernst Brunner. SGV_12N_00115
  • [Katze auf einer Mauer]. Ort und Datum unbekannt. Ernst Brunner. SGV_12N_19553
  • [Ringtanz während der Masüras auf der Alp Sura]. Guarda, 1939. SGV_12N_08589
  • ["Steffenbach-Brücke" der Furka-Bahn: Bau und Erneuern der Brücke]. Kanton Wallis, 1950. Ernst Brunner. SGV_12N_36937

These images are part of the photographic archives of the Swiss Society for Folklore Studies. Licence: CC BY-NC 4.0

I am doing my PhD in Digital Humanities on Linked Open Usable Data, with a focus on its (potential) use in the Humanities and the perspectives it could bring in terms of semantics and interoperability. My research is grounded as part of the Participatory Knowledge Practices in Analogue and Digital Image Archives (PIA) research project, which aims to develop a Citizen Science platform around three photographic collections of the Swiss Society for Folklore Studies (SSFS).

IIIF is a community-driven initiative, which brings together key players in the academic and CH fields, and has defined open and shared APIs to standardise the way in which image-based resources are delivered on the Web. Implementing the IIIF APIs enables institutions to make better use of their digitised or born-digital material by providing, for instance, deep zooming, comparison, full-text search of OCR objects or annotation capabilities.

So why do we need IIIF? Digital images are fundamental carriers of information across the fields of cultural heritage, STEM, and others. They help us understand complex processes through visualization. They grab our attention and help us quickly understand abstract concepts. They help document many the past--and the present--and preserve it for the future. They are also ubiquitous: we interact with thousands of them every day both in real life and on the web. In short, images are important and we interact with large volumes of them online. Image 1: Female Figurine, Chupicuaro, 500/300 B.C Image 2: Vision of Saint Gregory, unknown artist, n.d. Image 3: Iyo Province: Saijo, Utagawa Hiroshige, 1855

Deep zoom with large images

compare images

Reunify

Search within

Annotate

Crowdsourcing - National Library of Wales

The two core specifications are the Image API and the Presentation API. The former is a web service for manipulating an image through a URL and the latter "specifies the information needed to drive a remote viewing experience".

The purpose of the API is to display descriptive information that is intended for humans and does not aim to provide semantic metadata for search engines

The Web Annotation Data Model was created alongside a vocabulary and a protocol by a dedicated W3C group that reworked the Open Annotation specification, developed in 2013, "(...) [specifying] an interoperable framework for creating associations between related resources, called annotations, using a methodology that conforms to the Architecture of the World Wide Web". Its operating principle is based on the division of an annotation into two distinct parts: the body, which corresponds to the resource on which one seeks to annotate something, and the target, which represents the object being annotated. We will see an example later on.

PIA is a Sinergia project funded by the Swiss National Science Foundation (SNSF) led by the University of Basel, the Uni, the Bern Academy of the Arts, and the Swiss Society for Folklore Studies. PIA wants to connect the world of data and things in an interdisciplinary manner. We explore the phases of the analogue and digital archive from the perspectives of cultural anthropology, technology and design. The common goal of this project is to design a visual interface with machine learning-based tools to make it easy to annotate, contextualize, organize, and link both images and their meta-information, to deliberately encourage the participatory use of archives.

IIIF Workflow within PIA