top of page
Instrument of Record

An interactive installation on machine authority and the reading of feeling

Dec 2025 - Aug 2026
CONSENT AND PERMISSION

  • This work requires access to your camera and microphone. Your face and voice are analysed in real time, in your browser. The video and audio themselves are never recorded, uploaded, or stored — each frame is processed and discarded.

  • During and after the session you are asked three questions: what you were actually feeling, whether you would want a clinician to use the report, and — in your own words — what the report told you.

  • At the end you may download two files: a PDF of the report, and a JSON file containing the session's measurements and your answers. Downloading writes them to your own device and sends nothing anywhere.

  • The work requires both camera and microphone access to function, since its subject is the relationship between the two. If either is declined, or only one is granted, the system will not perform the required analysis. In this case, you want to see a recorded walkthrough of the piece instead.

​

    Research use:

  • This installation is also the instrument of an academic study on how people respond when a measurement system shows them its own internal disagreement. The study is conducted by Diaa Ahmedien and is intended for publication in the International Journal of Human-Computer Studies.

  • Participants are randomly assigned to one of two versions of this session when they begin. The versions differ only in the order in which parts of the session are presented; the analysis and the report are identical in both.

  • Beneath the report you will find a form for submitting your session to the study. Doing so takes four steps: download the JSON file, compress it on your own computer, upload the compressed file, and submit it. Nothing is submitted unless you complete all four. If you do not, your session ends and no record of it exists anywhere.

  • What is submitted is the system's measurements, its verdict, and your three answers. No video, no audio, no name, no email address, and no account details are included. Submissions carry no identifier that would allow your session to be traced back to you — which also means that once submitted, a session cannot be located again in order to withdraw it.

  • Submitting is voluntary, and the work is complete without it.

  • This study is conducted in accordance with the Declaration of Helsinki.

​​

BRIEF INFO
A browser-based interactive installation using real-time facial and vocal affect analysis
​
  • Programming languages: JavaScript.
  • Face analysis: MediaPipe Face Landmarker (478 landmarks, 52 blendshapes).
  • Voice analysis: Web Audio API (real-time prosody; pitch, energy, jitter, spectral measures).
  • Interface: 15-cell grid — 8 live analysis panels, 7 dynamic text panels.
  • Four-act sequence: Seduction → Drift → Contradiction → Refusal.
  • Three participant self-report questions.
  • Randomised two-arm design: presentation order assigned per session.
  • Session outputs: printable PDF report; structured JSON session data.
  • Single-channel, single-participant

Documentation:
>> EMPO report. 
>> ACM SIGGRAPH Digital Arts Community Exhibition
SDGs
E-WEB-Goal-03.png
INSTRUCTIONS

  • This work runs on desktop only. The interface is a 15-cell grid; on a phone or small tablet the cells are too small to read, and you will not be able to see your own face in them. Please open this page on a laptop or desktop computer.

  • Grant camera and microphone access when prompted, and remain within frame.

  • Sit and speak naturally — the piece does not require any particular expression or response.

  • The work runs through four acts, then asks you three short questions and produces a report.

  • Two of the three questions are optional; the report is produced either way.

  • You may then download the report as a PDF, and the session data as a JSON file.

  • To contribute to the study: download the JSON file, compress it (right-click → Compress on Mac, → Send to → Compressed folder on Windows), upload the compressed, and submit it through the form beneath the report.

  • Nothing is submitted unless you complete that upload.

  • A full session runs approximately five to seven minutes, depending on how long you sit with it.

  • Please ensure your browser allows camera and microphone access for this webpage

ABOUT THE PROJECT

Instrument of Record is a browser-based interactive installation that stages a real-time comparison between a participant's facial expression and vocal tone. Where the two align, the system reports coherence. Where they diverge, it reports discordance — and it is in this reporting that the work locates its subject.

​

The installation does not accuse the participant. Face-voice incongruence is common, involuntary, and not evidence of concealment; the piece treats it as such throughout. Instead, the installation indicts the instrument itself. Across four acts — Seduction, Drift, Contradiction, and Refusal — the system performs the register of clinical, institutional authority: it claims to standardise, record, and reliably verify affect. But its own outputs contradict it. Three independent measures — the position of the face on a valence-arousal plane, the configuration of facial action units, and the prosody of the voice — are allowed to disagree with each other, sometimes at high confidence, and the system is built to surface that disagreement rather than paper over it.

​

A referee process adjudicates between the three witnesses and can, genuinely, acquit — the work's credibility depends on this not being staged. A known bias in the system is also left uncorrected: eyewear feeds false signals into the facial action-unit measure, skewing it toward false positives. This is not a flaw hidden from the participant; it is left in as documentation of how real affect-recognition systems behave in practice, including their blind spots.

​

At the end of the session the instrument asks the participant a question it cannot answer itself: what were you actually feeling? The answer is recorded beside the machine's verdict, in the same report, with no attempt to reconcile the two. The instrument does not revise its reading in light of what it is told.

​

Instrument of Record ends without resolving into a verdict. It states its position, refuses further certainty, and closes on the line that stands in for every automated system's actual epistemic position: the reading stands as filed.

​

Send your participation report
Add just one file
The Carbon Farm QR

To open this page on your mobile, Scan this QR code via your mobile. The mobile site will ask you to enter the country's code from which you would like the system to derive its data.  You can see the complete list of the countries' Letter codes here:

​

List of country codes by alpha-2, alpha-3 code (ISO 3166) (iban.com)

​

* Please, be sure that your browser allows sound for this webpage 

DOCUMENTARY
The Carbon Farm

The Carbon Farm

An involved piece in the permanent online exhibition entitled: “The Earth, Our  Home: Art, Technology and Critical Action” Organized by: ACM SIGGRAPH Digital Arts Community (DAC)

© 2026 Diaa Ahmed Mohamed Ahmedien. All rights reserved.
Unless otherwise indicated, all materials on this website are copyrighted. No part of these pages — text, audio, video, or images — may be reproduced, modified, stored in a retrieval system, or retransmitted in any form or by any means without prior written permission.

© 2024 | Diaa Ahmed Mohamed Ahmedien | All rights received 

​

Unless otherwise indicated, all materials on this website are copyrighted. No part of these pages, either text, audio, video, or images, may be used for any purpose unless explicit authorization. Therefore, reproduction, modification, storage in a retrieval system, or re-transmission in any form or by any means, electronic, mechanical, or otherwise, is strictly prohibited without prior written permission.

bottom of page