Apple’s keynote of their new Head Mounted Display (HMD) just finished and hurray!… New hardware incoming! Apple Vision Pro is the name, $3499 is the price! (ouch..) The keynote was full with some interesting buzzwords that set the stage for the ultimate price reveal. Ok, I know it’s Apple, it’s premium.. but that price…. didn’t see that one coming. I guess “The Zuck” (Meta) was laughing out loud!…… Spatial computing! That will be one of the most searched words in June 2023 I guess 🙂 Let me give you some explanation of certain techniques so you understand what’s coming. This blog also includes my two cents about the device, because hey.. a new major player entered theXMR (Extended Reality) arena!
1. First things first: Spatial computing?
“Spatial computing refers to the process of using digital technology to make computers interact seamlessly in a three-dimensional world using augmented reality (AR), virtual reality (VR), and mixed reality (MR). Spatial computing uses physical space to send input and receive output from a computer.”
Source: Techslang
That seems to me the most clear definition of what this is. So you’re in a physical space somewhere on earth. With your HMD you can “digitally” interact with said space. Albeit in complete virtual surroundings (VR), an “extra / augmented” layer on top of some elements in your surroundings (AR) or something of a mix like 3d-models in your environment, that stays on a fixed position where you can walk around and interact with (MR, Mixed Reality).
So with the Vision Pro, you can use all three techniques. VR is interesting for gaming, immersive yourself in and interact with completely virtual surroundings, do simulations, games! (You NEED to play Half Life: Alyx) AR is interesting for stuff like work-related tasks like having set-up multiple screens to work on. Think of a videoconference on the 1st screen, Excel on a 2nd screen and your trusty Netflix series on a 3rd screen 🙂 Or transform a part of your room to a cinema screen. MR is arguably a blend of VR and AR, where you place certain objects in your real-life environment with which you can interact. Think of a digital rendered machine in your own livingroom, a 3d model of a limb / product / design to interact with…. you can see the possibilities. The combination of all three techniques is enticing and Apple has got you covered!
2. Controls + Cameras
Maybe you’ve noticed already, but.. where are the controllers? If you’ve ever seen a VR headset like the Valve Index, The Playstation VR headset or the Meta Quest (2), you’ve surely seen the two controllers which functions as a joystick / mouse. Meta is already experimenting with hand-based controls so you don’t need the dedicated controllers. Apple is all in on voice recognition, hand-tracking and eye-tracking, for controlling the device and the augmented layers in your environment. On the inside of the device, there are a couple of cameras and LED lights, which combined, make eye-tracking possible. The interesting part of eye-tracking is that you don’t have to move your whole head to a certain segment, but you just look at it. Cameras register your eye movement and that way, you can interact with your digitally enhanced environment, bring stuff into focus.
On the outside of the device, the glass-side of things, there are different cameras to register your real-life surroundings. A depth sensor and pass-through camera, so the device can scan and map your surroundings and pass the image through to the screen. There are also cameras that register your hand and head movement, for hand tracking abilities like controlling the device and interacting with your environment and digital objects…. for example pinching, selecting and scrolling with your fingers.
3. What’s up with the transparant design?
Maybe you’ve already seen some promo-images where you can see the eyes of the user through the device! What sort of magic is there? Well, pretty easy: transparancy mode is auto-enabled when the user is not in a simulated / digitally enhanced experience! That way you can “communicate” if you’re “in” a simulation.
4. Battery / power
The Vision pro is powered by two chips: The M2 chip that’s the heart of the computer side of things. Think of running the operating system (Vision OS), graphics and spatial audio. The R1 chip is the heart of all the sensors and cameras. The Vision Pro uses an external batterypack that’s connected to the headset via a detachable cable and lasts around 2 hours….. yeah… it’s always the battery that is one of the weakest elements of portable devices. You can stow the battery pretty easily in your pocket or connect it to a power outlet to last all day.
4. Building worlds / environments with Unity
Well, those are the biggest takeaways from a hardware perspective. What about the virtual and augmented worlds and app experiences that will be used with the Vision Pro? Drumroll… another big player joins the stage, namely “Unity“. Unity is a (virtual)world-building engine used primarily in gaming. Lots of mobile games and some Playstation / Xbox / PC games are build on the foundations of the Unity engine. Epic Games (Fortnite) are prepping the Unreal engine as the defacto world-building engine for virtual worlds and Metaverses, so Unity made a big splash with the Apple partnership. It’s interesting to see where this goes, because the battle of the engines rages on. Based on this news alone (the partnership with Apple), the stock value of Unity increased by 17% in one day!
5. My two cents…..and what Apple didn’t say!
After all those technical details to understand the Vision pro, here are my two cents about this device:
-What about the Field of View? Field of view is the range of the world that you can “see” while looking through a lens, wheter it’s your eyes, a cameralens, a HMD. Especially for AR / MR, this is a technical problem where the FOV is very limited, based on products as the Microsoft HoloLens (54 degrees) and the Magic Leap 2 (70 degrees). Those two products are considered the best AR tech products and that FOV number is not high, especially considering the human eye FOV is around 135 degrees. Apple didn’t give a figure, so we’re still speculating. This is a potential deal-breaker that could really hurt the experience you’re going to get. Some Hololens and Magic Leap developers already expressing their doubts on Twitter. This could be one of the key factors for this device.
-Didn’t see that price coming, but the build-up in the keynote hinted heavily on a big splash of money. The Keynote stated something like: “The Vision Pro experience surpasses the quality of that of the combination of a dedicated home theatre setup, a big a television and your desktop pc”. After that part, I started calculating 🙂
-Will this be the breakthrough device for the mainstream? Haha… no! A day after the keynote, I already discovered some blogs touting questions like “Will this finally be the device that makes MR mainstream”? Uhmm.. no.. based on two things. One is the price: it’s just premium-de-luxe and not affordable for lots and lots of people. Two: the device itself. As long as consumers need this big form factor to experience this technique, it’s a no-go for lots of people. It will certainly be an amazing device, but blocking your face with HMD’s like this? Not going mainstream! BUT: with each iteration, each investment… HMD’s WILL become smaller. Look at the Meta Quest, edition 1, to 2, to 3…. eventually… we’re getting there. Where’s Google Glasses when you need one? 🙂
-When Apple introduces new hardware, there’s always a connection to the masses. When Apple tinkers… then the mainstream IS interested. So this is a big win for the XR (Extended reality) market, where lots of people finally discover these techniques, they will read about it, maybe experience some stuff…. there WILL be an influencer contentwave when the Vision Pro is ready to sell… so yeah, Apple will help the market as a whole. I guess the $500 priced and just recently announced Meta Quest 3, will gain a siginificant boost by way of 1st time VR / MR adopters.
I hope this blog helps you with understanding the Vision Pro a bit better and makes you excited to give all these techniques a try!