Project
This project is an interactive installation delving into the latent space of large language models and exploring the autonomy of language. Current neural machine translation models prioritize accuracy over the creative or poetic potential of language. Embracing a postmodernist perspective on the task of translator, this work seeks other possibilities of expression by opening the BlackBox of neural networks. It visualizes the hidden word lists from the beam search algorithm which is widely used in a large language model.
The installation integrates computer vision technologies with physical and virtual media to create an immersive experience. Viewers are invited to touch poems printed on paper to trigger word-based interactions. The system responds in real time by revealing all of its potential translations of the words in the poems. Projected words continuously transform in response to ongoing user interaction, foregrounding the mutual influence between humans and language. This multimedia artwork spans poetry, linguistics, and machine learning, transfiguring the latent data behind neural machine translation into a poetic experience.
Installation
/ Machine Learning,
Projection, Paper, Camera,
Led, Electronics,
Acrylic, PVC
Without touching the printed words, all possibilities will show in horizontal movement.
When touching the printed chinese characters, the program will find words in all four poems which have high semantic similarity scores. The drifting effect will then be triggered.
Through the interactions by audience, the effect will be continuous. The color of drifting words will change from black and white to colorful.
Currently the detection is using a webcam and blob detection with an internal reflection acrylic touchscreen. However the printed paper overlays on the acrylic affects the light a lot. I'm thinking about other materials to replace white paper.
The support plinth is 80*100*45 cm.
Special Thanks:
Dr. Hector Rodriguez