Give Me a Sign is a tech art installation that uses gestural recognition to present interactive narratives through film. Created collaboratively by India based artist, Upasana Nattoji Roy and UK based artist, Diane Edwards, this art project is an inquiry whether Ai tools can augment our artistic practices and production processes.
The larger goal of the ‘Give me a Sign’ project is to create a dictionary of hand gestures to highlight the significance and power of physical gestures in expressing meaning and to build a bridge between human and machine, as well as between cultures.
Give Me A Sign premiered a prototype at the Future Fantastic Festival at HOME in Manchester, United Kingdom which was held on 25th and 26th February 2023.
It also featured at Rangashankara in Bangalore on the 11th and 12th of March 2023 and at Bangalore International Centre on from the 24th to 26th of March 2023.
2023 Give Me a Sign Project was supported by :
Featured
THE EXPERIENCE :
This audiovisual experience combines aspects of Indian philosophy and practices with narratives around Artificial Intelligence and the impact of human actions on our planet, through an illusionary AI entity called ‘Shunya’ who responds to certain Indian dance mudras with visions and dreams, stories, concerns and hopes for our changing planet.
The experience starts at an animating interface. When someone holds up a mudra / hand gesture in front of the webcam, it triggers an audio visual response on the screen. Shunya manifests as hands and a voice in an animated speculative world and informs the user about the theme in the form of a statement, a philosophical observation or a leading question.
For this prototype of Give me a Sign, the focus is on six topics related to human impact on climate change. These topics were chosen by the two artists, based on the urgency for action.
“The use of gestures in communication is a defining characteristic of the human species” - Upasana
THE PROCESS :
Mudras are hand gestures that are used to communicate and express emotions or ideas without using words typically used in classical Indian dance.
The process of studying and curating from a vast repository of mudras from the Bharata Natya Shastra was done in tandem with researching the themes related to human impact on our planet.
The research included traditional forms as well as ChatGPT3. The scripts needed to be written by the artists. Each gesture was mapped to a theme like Earth & Water, Life & Growth, Human Ego, Tech, Power & Caveat. 3 scripts were written for each - a statement, a poetic thought or a leading question.
18 scripts have been written based on which 18 different short videos are triggered through gesture.
Shunya was created through dance choreography by Upasana, motion captured through DeepMotion (ai tool) and the animation was retargeted onto Metahuman hands in Unreal Engine.
Shunya is designed to be able to respond with gestures, yet abstracted from a full human.
The process has been an exploration of possibilities in creative and production workflows. Upasana sees AI as ‘Augmented Intelligence’ and currently sees machine learning tools as an augmentation to her design and artistic practice.
Switch Studio is already developing collaborative workflows using traditional, digital as well as AI Tools.
Read more about the process of developing the narrative at this - link
Give me a Sign has been designed with the possibility of being experienced in two formats
Immersive : An interactive immersive experience typically projected or a high resolution led wall with immersive sound.
Popup: Using a TV setup as a display and a simple laptop along with a connected camera setup for image capture.
Credits
Artists : Upasana Nattoji Roy & Diane Edwards
Upasana
www.theswitchstudio.com
www.instagram.com/oonattoji
Diane
www.dianerkedwards.com
www.instagram.com/diane._.edwards
Interface Design, 2D animation, deep motion & choreography : Upasana
Unreal and 3D : Diane
Programming : Diane & Creative Technologist: Hasan S
Sound design : Tim Garratt & Ugendran Padiachey
Spoken Word: Diane & Upasana
Handpose guidance : Ambika (www.instagram.com/computational_mama/)
References and Tech Stack
Deepmotion, 3D Hand Pose with MediaPipe and TensorFlow.js, ML5 NeuralNetwork
Stable diffusion -AUTOMATIC1111 Github Hugging face Google Collab, Researching for Mac -
Python, Invoke, invoke GitHub, Xcode Mac install, Mac install guide, Issues Github, Midjourney, Gooey.ai, RunwayML.
Our Main Tools
Blender, Unreal Engine UE5, Adobe Suite : After Effects, Illustrator, Premier, Audition
First presentation for the initial concept for Give me a Sign in 2020
Give Me a Sign started as a collaborative project as part of the Tech Art fellowship Collab 2020 by Befantastic. Supported by Max Mueller Bhavan Bangalore & British Council India. The first draft was developed collaboratively by Upasana , Diane Edwards and Shayek Mohammad Arif