Greetings!
Welcome to the inaugural issue of our newsletter! We’re thrilled to have you on board
as we shine a spotlight on a cutting-edge concept that’s reshaping the digital landscape:
Spatial Computing.
What’s Inside This Issue?
- Basic Concepts and Terminology: Explore basic concepts and key terminology essential for understanding this innovative field.
- Featured Article: Join an app developer exploring exciting spatial computing applications and resources.
- Hands-On Activities: Engage in fun activities tailored to introduce you to the realm of spatial computing.
- Equity and Sustainability in Spatial Computing: Examine how spatial computing can contribute to advancing equity and sustainability.
Join us as we embark on a journey of discovery and innovation. Empower yourselves to navigate the ever-evolving landscape of technology. Let’s unlock the potential of spatial computing together and shape a brighter, more equitable future for all.
Basic Concepts and Terminology
Ever imagine painting a dragon on your wall, then watching it come alive and soar around the room? That’s the kind of experience spatial computing promises. This revolutionary approach merges the physical world with the digital, blurring the lines between reality and imagination.
To achieve this magic, spatial computing leverages a toolbox of cool technologies that fall under the umbrella term Extended Reality (XR). These include virtual reality (VR), where you enter completely virtual worlds; augmented reality (AR), which overlays digital objects onto your real world; and mixed reality (MR), a blend of both.
But spatial computing takes it a step further. It utilizes these XR tools to create a unique experience where you interact with digital elements in 3D space, using natural movements instead of clicks and keyboards. Imagine designing furniture in your living room, then instantly seeing it come to life. That’s the power of spatial computing – it’s like bringing the digital world to life right in front of you! Apple’s recent launch of the Vision Pro, a wearable “spatial computer,” is a sign of things to come in this exciting field.
Ngo Kiu, who is a physician, a member of our Advisory Board, and one of Paul Jr.’s cousins, will take it from here!
Featured Article
Spatial Computing: The Coolest Tech Trend You Need to Know About
By Ngo Kiu
Welcome to 2024: The Year Immersive Tech Got Super Cool
Alright, let’s take the plunge into the adventures of 2024 – a year poised for an immersive tech revolution. Apple’s Vision Pro, generating a buzz since its February release, is just the tip of the iceberg. Tech titans are swarming the scene! Leading the charge are Meta’s pioneering Quest 3 MR headset, the Samsung-Google-Qualcomm XR partnership, Sony’s mind-bending VR spatial headset, and Microsoft’s powerful HoloLens line. All of the players and their wares are pushing the boundaries of spatial computing, seamlessly integrating these technologies into our everyday lives.
Entertainment Just Got an Upgrade
Forget flat screens and IMAX theaters! 2024 brings immersive experiences straight to your living room. Imagine a colossal dinosaur from Jurassic Park towering in front of you, its massive head swaying in response to your cautious movements. See a delicate butterfly land on your fingertips as it flutters around. Gaze upwards and witness constellations that engulf you in the vastness of the night sky. This isn’t science fiction – Apple Vision Pro lets you see these fantastical scenes come to life right before your eyes.
Credit: Apple Vision Pro Dino Demo @8:57
Now, the immersive tech evolution extends beyond these extraordinary moments. Exir.live is transforming sports viewing with real-time spatial computing. Imagine a live 3D basketball game projected right onto your coffee table, allowing you to witness the action unfold from any angle as if you were courtside. Feel the energy of the crowd and the excitement of the game from the comfort of your home. The future of entertainment is here, and it’s nothing short of awe-inspiring.
Credit: Realtime mixed reality simulcast: https://exar.live/
Shopping just got a tech makeover!
Wanna Kicks, the cool AR app, lets you virtually try on those must-have sneakers from the comfort of your couch. See how different brands and styles look on your feet from all angles, making pre-shopping a breeze. Wondering if those trendy glasses will suit your face? Warby Parker‘s got you covered with their virtual try-on feature. No more wondering “will this fit?” – just fire up IKEA’s new AR Place app to see how that sleek Billy bookcase would look in your place. Ditch the tape measure and embrace the shopping spree of the future: seamless, convenient, and all at your fingertips. It’s finally here!
Credit: IKEA mobile AR app.
Biology and Medicine Like Never Before
Here’s where it gets really wild: the medical world. Medical students can now interact with a virtual beating heart, manipulating it in their hands. They can pull it apart, see what’s inside, and even practice surgeries dry first, before getting — you know — messy. Need to master CPR? You can practice on a virtual patient first, gaining valuable confidence before a real-life situation.
Credit: Exploring the heart on HeartX.
Futuristic rendition of practicing CPR on a virtual person
This isn’t a futuristic fantasy—Stanford Medicine is already leveraging augmented reality through the Apple Vision Pro to visualize data during surgery, enhancing procedural efficiency and potentially improving safety. Likewise, a medical team at a hospital in London utilized the same device to help perform spinal surgeries.
Don’t have a headset but want to develop something like the mobile IKEA app? Dive into Apple’s ARKit if you’re an iOS fan, or Google’s ARCore for Android enthusiasts. These tools empower you to bring AR experiences straight to smartphones.
For those seeking more structured learning, platforms like Coursera and Udemy offer comprehensive courses in both Unity development and mobile AR technologies. Check out this ARKit course on Udemy to learn to build eleven AR apps, or this free AR tutorial made by Google on Coursera to learn the basic elements of AR and the ARCore platform. These resources furnish a guided trajectory from novice to expert.
Ready to dive in? Download Unity, explore ARKit or ARCore, and start experimenting. With these tools and educational resources at your fingertips, you’re well on your way to becoming a VR/AR innovator! Moreover, in the rapidly evolving landscape of spatial computing, career pathways are expanding. Opportunities include roles such as spatial computing developer, MR developer, VR content creator, or AR designer.
While these careers often involve advanced coding and math, building a solid base through courses like introduction to coding and basic algebra equips you with the essential building blocks to one day excel in this exciting field. The resources available on this webpage provide a varity of learning opportunities. Tailored for high school students, they serve as an excellent launching pad for exploring coding, engineering, mathematics, and the sciences, ensuring that your journey towards becoming a professional in spatial computing is both enriching and fulfilling. Get ready to shape the future!
About the Author
A beloved cousin of Paul Jr., Ngo Kiu is an Emergency Medicine physician with a deep passion for technology. Ngo Kiu embarked on his artificial intelligence journey in 2018, inspired by Google’s compelling image recognition demo. This pivotal moment led him to develop AI algorithms for medical image classification and language models aimed at enhancing medical diagnoses. Currently, Ngo Kiu is channeling his expertise into creating an virtual reality application for medical education, aiming to transform the way medical knowledge is imparted and applied, showcasing his commitment to innovating healthcare through technology.
Hands-on Activities
Learning about VR and AR can be exciting, and there are many ways to jump in! Your phone can be a powerful tool for getting started. Have you explored the AR apps mentioned in Ngo Kiu’s article – Ikea, Wanna Kicks, and Warby Parker? Did you know that with a simple 3D scanning app installed on your phone, you can create 3D images similar to the furniture in the Ikea app, the sneakers in Wanna Kicks, or the glasses in Warby Parker?
Check out this list of 3D scanning apps to get started. It’s a fun process! Once you have an app on your phone, simply use the phone’s camera to photograph a real object, process the images online using the app, and then download and view the processed 3D virtual version. The following snapshots showcase the Kiri Engine in action.
For example, the apple image was scanned using the Kiri Engine on an Android phone (Motorola Moto G Power 2022). This process involves capturing a minimum of 20 photos from different angles.
If you have access to an iPhone 12 or a newer model, you can seamlessly integrate a 3D object created with the Kiri Engine into your physical environment. Simply tap the AR button located at the top right corner of your phone. Below are two snapshots: the first one showcases a virtual tea canister with the AR button positioned in the upper right corner of the phone screen, while the second one displays the virtual canister positioned on an actual table.
A virtual tea canister created by using Kiri Engine.
The virtual tea canister positioned on a real table.
From Phone Screen to Immersive Experience:
While your phone can give you a taste of AR with the Kiri Engine, an AR headset takes things to a whole new level. Imagine holding a virtual apple on your table. Unlike a phone screen, with an AR headset, you can walk around and the apple stays put! It’s like having a real object you can examine from all angles. Some headsets even add sounds to make it even more realistic (although they can’t quite make it smell like a real apple yet!).
viewing a virtual apple on a table
Making Spatial Computing Accessible:
This evolving technology faces a hurdle in accessibility, but you can help change that! Advocate for AR headsets or other spatial tools at your school or local libraries. Education thrives on innovation. Some public libraries have already implemented VR programs, such as this library in San Jose, CA. If needed, we provide a template letter for contacting officials, like your school district’s superintendent and city mayor, to secure funding.
Your Voice Matters!
By advocating for yourselves, you’re making sure everyone has a shot at being part of the future of spatial computing. We’re also gathering information on how to make this technology more accessible. Together, we can break down these barriers and open doors to amazing new possibilities!
with Spatial Computing
Spatial computing, with its capacity to overlay digital information onto the physical realm, holds considerable potential for fostering both equity and sustainability. Regarding equity, it presents innovative solutions to overcome geographical disparities in education and training. Picture students residing in remote regions of Australia engaging in immersive virtual expeditions to the Great Barrier Reef, acquiring knowledge about its fragile ecosystem from any corner of the globe. Conversely, students attending inner-city schools can embark on virtual journeys to historical monuments or lush rainforests, deepening their comprehension of the world beyond their immediate environment.
Aussie students explore the Great Barrier Reef through VR!
the rainforest alive!
Moreover, spatial computing can offer interactive 3D simulations for vocational training, nurturing practical skills irrespective of geographical constraints. For example, prospective surgeons in areas with limited access to medical facilities can hone their expertise through VR simulations, practicing intricate procedures and enhancing their readiness, potentially leading to life-saving outcomes.
Currently, robotic surgery empowered by 5G cellular communications represents a notable advancement in remote medicine. However, spatial computing holds the potential to push the boundaries even further. Ngo Kiu’s’s article highlights two cases wherein doctors utilize the Apple Vision Pro during surgical procedures, signaling just the dawn of spatial computing in medicine.
VR surgery simulation: Equipping future surgeons, anywhere in the world.
Robotic precision meets spatial guidance: The future of surgery.
Spatial computing applications can be designed with inclusivity at their core, empowering people in underrepresented communities. Take Michael Running Wolf’s work for example. He utilizes virtual/augmented reality and artificial intelligence, as showcased in a Unity platform video, to strengthen endangered indigenous languages.
Imagine using AR tools in everyday life. Project visual or auditory overlays onto your surroundings, aiding navigation in unfamiliar spaces. Real-time sign language translation through AR can bridge communication gaps, fostering a more inclusive society.
Credit: Unity
Unity for Underrepresented Communities
AR bridges the gap between spoken and signed communication
On the sustainability front, spatial computing shines as a powerful tool for environmental monitoring. It facilitates real-time tracking and visualization of resource consumption, enabling the identification of potential inefficiencies and waste. The technologies described here may be used to create what are called “digital twins,” meaning a duplicate of a real object is created in a computer making a virtual copy, so that the virtual copy can be manipulated and studied without disturbing the real original.
For example, Singapore leverages digital twins of its power grid to monitor its health in real-time, allowing for swift intervention when issues arise. South Korea further exemplifies this potential by creating digital twins of its rivers for water management. These digital replicas enable real-time monitoring of water flow and potential flood risks, empowering authorities to optimize water usage, implement preventative measures, and ensure a more sustainable water management system.
Credit: SP Group
Singapore’s First Digital Twin for National Power Grid
“Digital Twin” technology used for smarter water management in S. Korea
Equitable Access to Spatial Computing
Spatial computing offers a wealth of opportunities, but equitable access remains a challenge. The high cost of AR/VR hardware could exacerbate the digital divide, limiting who can benefit from this technology. To address this issue, a number of measures can be taken. Here are three examples:
Implement subsidized programs to provide AR/VR hardware at reduced costs or for free to underserved communities, schools, or individuals.
Equip libraries and community centers with VR headsets and AR glasses for public use. Offer workshops and demonstrations to help people get started.
Engage in advocacy efforts to push for the development of more affordable AR/VR hardware and software solutions, working with manufacturers and policymakers to promote affordability and accessibility.
We look forward to preparing our next issue on a different topic.
Subscribe to the Newsletter!