Creating Digital Interactive Experiences & Augmented Reality
Interactivity & Design
Inventive Visions
Digital Interactivity and Augmented Reality represent a dynamic fusion of technology and creativity. Cameras, machine learning algorithms, Python development environments, and audio-visual components converge seamlessly within popular interactive design platforms like TouchDesigner and Unity in the following projects.
CREATION WITH
TouchDesigner
Interactivity, VR and Games
Joiners VR PanoramA
David Hockney's Joiners photo collages inspire this VR panorama. This version shows a series of photographs on a grid in a 360º viewpoint. Afterward, the photos are “joined” like a collage to create a fully immersive VR experience. The non-perfect overlap is meant to introduce a bit of “charm” to the art project.
The panorama is of Laurel Hill Cemetery, located in Philadelphia, PA.
Interactive VR Elements in the Cemetery
Audio and visual interactive elements were added to the Cemetery VR. There's background audio of a crow and a slight breeze, children playing, and a weed hacker operated by a groundskeeper. You can also see a looping video of a bird flying in one area of the sky. All of these elements were present during the two-day photo shoot in the location to create the original Joiners VR assignment.
Facial Recognition and Tracking using Python
A 3D-modeled cube is added to the project to demonstrate the use of secondary visual elements to convey artistic development within the VR. A Python script is attached to an external camera node to show how facial recognition and tracking can move objects within the VR experience.
Augmentation using facial tracking
The final addition to this multi-segmented project added facial tracking to navigate the cemetery and manipulate a 3D object. Audio is fed through to create an additional particle effect on the object.
This is a homage to Future Sound of London's music video for My Kingdom. The audio is not present because it's copyrighted. However, you can view the original here.
Scary Eye Spy Interactive VR Game
This project focused on creating a version of the popular children's game "Eye Spy.” It was made during the popular US holiday Halloween, so revisiting the tranquil cemetery was serendipitous.
In this interactive experience, the eye spy circle/locator changes color from green (safe) to red (danger) when spooky elements are spotted. Audio of the component plays as long as the locator is hovering over it.
This project merged foundational interactive technologies, including NDI virtual cameras, Google's Teachable Machine, Python, TensorFlow, and TouchDesigner.
AR WITH VUFORIA
AND UNITY
Interactive Augmented Reality
Pokemon Augmented Reality
This project utilized machine learning recognition using Vuforia's integration within Unity. Pokemon Playing/Trading cards were used to train the ML, and free Pokemon 3D models were used for the 3D augmentation.
Using a webcam, Vuforia detected the cards and displayed the corresponding 3D models Unity used to display in augmentation. When the Pokemon card rotates, the 3D model rotates with it. Music was added post-production in Adobe Premier, but in-engine sound elements within Unity could undoubtedly be used for the same effect.
Musical Instrument Augmented Reality
This presentation showcases how AR can be utilized in educational landscapes. Inspired by the Musical Instrument Museum in Phoenix, AZ, the project demonstrates how an educational organization could create an application that is downloadable by users to give a fully immersive and interactive experience. In the in-presentation demonstration, a user would point their own device at a poster or other print medium, and a 3D model would be displayed that the user could move around to observe. Additionally, the instrument’s sound is played as the model is shown.