Installation
/ Machine Learning
Project
This project is an interactive projection installation exploring the latent words space under neural machine translation and the autonomy of language. It finds other possibilities of expression inter-spaces inside a fixed Model from Transformer. The multimedia artwork spans poetry, language functions, and hidden states under neural machine translation, unfolding the data and poetics behind machine translation.
It visualizes the hidden word lists from the beam search decoding algorithm. Three experimental poems by Chinese poet Duo Duo are chosen for the representative poems to explore. When the audience touches the physical printed lyrics, the projected words will escape from the former grid and start drifting. Words will change continuously with the interaction, representing the mutual influence between humans and language.
Without touching the printed words, all possibilities will show in horizontal movement.
When touching the printed chinese characters, the program will find words in all four poems which have high semantic similarity scores. The drifting effect will then be triggered.
Through the interactions by audience, the effect will be continuous. The color of drifting words will change from black and white to colorful.
Currently the detection is using a webcam and blob detection with an internal reflection acrylic touchscreen. However the printed paper overlays on the acrylic affects the light a lot. I'm thinking about other materials to replace white paper.
The support plinth is 80*100*45 cm.
Sincere Thanks to
Dr. Hector Rodriguez