Virtual and augmented reality are becoming so integrated into modern life that we often take them for granted. When watching football games on TV, fans intuitively look for the yellow “first-down line,” which marks how far the offensive team needs to go to gain additional opportunities to score. First introduced in 1998, this computer-generated line requires a huge amount of effort to ensure that it always appears in the correct place as the TV broadcast zooms in and out and switches from camera to camera, as explained by HowStuffWorks. In Olympic swimming broadcasts, lanes may be marked similarly, with each swimmer’s name, flag and speed in meters per second shown.
More recently, programmers have developed phone apps that can guide you to a specific address, camera filters that allow you to attend online meetings as a cat, and websites that will put a specific couch in your living room so that you can see how it fits. These are all examples of augmented reality, which is firmly anchored in reality but enhanced with digital input. In contrast, virtual reality (VR) aims to immerse the user in a completely different world — perhaps a famous city, a faraway planet, the insides of a living animal, or a fantasy world limited only by the creators’ imagination. Virtual reality starts with a head-mounted display that blocks out all surrounding light and replaces it with a digital world that the viewer can explore.
How Do Virtual Reality Headsets Work?
When you look at something in real life, your two eyes are viewing it at slightly different angles. Your brain processes these slightly different images to generate a 3D image. You can enhance your understanding of the 3D world by moving your head and viewing objects from different angles. So how do virtual reality headsets work? A virtual reality headset creates the illusion of 3D by presenting each eye with a slightly different view of the same scene. This is a good starting point, but what really sets VR apart from a good movie theater experience or a typical video game is that the user controls the point of view. When the user turns their head, the image they see changes accordingly. In an ideal virtual reality situation, the user can move 360 degrees around an object and interact with the environment in real time: touch things, feel things and move things around. The technology isn’t there yet, but it is steadily progressing.
Currently, a mid-level virtual reality system allows you to explore VR environments with three degrees of freedom. This means that the system can detect when you turn your head left or right, look up or down, or tilt your head from one side to another. The system cannot detect when you change the position of your entire body by walking around or sitting down, for example.
The best virtual reality systems currently available enable you to explore VR environments with 6 degrees of freedom. In addition to tracking head movements, these systems also check your position within the physical room — forward and backward, left and right, and up and down. This allows for complete freedom within the VR environment, while also allowing for safeguards to prevent the user from crashing into a physical object like a wall. This high degree of accuracy typically requires installing cameras and sensors in the room, making it more expensive.
Adding stereo sound provides an even more immersive experience, since humans hear in three dimensions. For example, sounds that originate from a person’s right side will reach the right ear slightly before reaching the left ear, which helps with locating the source of the sound. VR developers use spatial audio to provide the illusion of 360-degree sound. This works by controlling volume, using left/right delay to convey direction, using head tracking to map auditory space, and manipulating reverberation and echo to simulate environmental factors. This amplifies the VR experience because humans normally respond more quickly to sound cues than to visual cues. For example, one might turn quickly to investigate the sound of a creaking door.
Upcoming Advances
While VR has made steady progress, it doesn’t yet provide a fully immersive experience because the technology is still slower than the speed at which humans process visual information. Frames need to move at a fast pace within a VR headset to mimic what we see in real life. Experts believe that the human eye can handle up to 1000 frames per second, but not all of that information goes to the brain, according to XZ Today. The available evidence suggests that humans can discern frame rates up to 150 frames per second (FPS). Anything less than 60 FPS for VR tends to cause disorientation and nausea, so many developers are currently aiming for 90 FPS, with a longer-term goal of 120 FPS.
In addition to fast frame rates, VR requires fast processing speeds. If it takes more than 20 milliseconds for a user’s actions to produce an appropriate visual and audio response, the user’s brain is unlikely to believe that the VR experience could be real. Increases in computing power will therefore aid in producing true, immersive VR experiences. Machine learning and AI advancements will also contribute to a more realistic experience. A graphics technique known as foveated rendering delivers ultra-high definition images only where the eye is focused and can significantly lower the computing power required to create an appropriate image.
Finally, humans have more senses than just vision and hearing. Efforts are well underway to more fully incorporate the sense of touch through wired gloves for tactile stimulation. Other options include wearing bodysuits and adding smells and changes in heat intensity.
Current and Future Applications
The appeal of virtual and augmented reality has become clear, with obvious applications in gaming, tourism and any situation where people want to “try before buying.” These technologies are also being used to safely but realistically train pilots, firefighters, astronauts, police officers, medical students and surgeons. For pilots, virtual reality is paired with a hydraulically operated cockpit that allows them to operate realistic controls and feel every tilt and turn.
Virtual reality is also being used to treat chronic pain and help patients overcome anxiety disorders, including phobias and post-traumatic stress disorder, as reported in Neurotherapeutics. Virtual and augmented reality may also help individuals recovering from stroke or traumatic brain injury, as Brain Science notes.
As the technology continues to improve, virtual and augmented reality will undoubtedly become even more integrated into modern life, assisting with tasks as simple as watching sports and as complex as learning about faraway planets.
Are you interested in all things related to VR and AR? We are too. Check out Northrop Grumman career opportunities to see how you can participate in this fascinating time of discovery.
