Apple has been granted a patent (number 11,217,021) for a “display system having sensors.” It involves the rumored “Apple Glasses,” an augmented reality/virtual reality head-mounted display.

About the patent

Virtual reality systems may display stereoscopic scenes to users in order to create an illusion of depth, and a computer may adjust the scene content in real-time to provide the illusion of the user moving within the scene. When the user views images through a virtual reality system, the user may feel as if she’s moving within the scenes from a first-person point of view. 

Similarly, mixed reality (MR) combines computer generated information (referred to as virtual content) with real world images or a real world view to augment, or add content to, a user’s view of the world. The simulated environments of virtual reality and/or the mixed environments of augmented reality may thus be utilized to provide an interactive user experience for multiple applications, such as applications that add virtual content to a real-time view of the viewer’s environment, interacting with virtual training environments, gaming, remotely controlling drones or other mechanical systems, viewing digital media content, interacting with the Internet, or the like. 

Apple’s patent includes an HMD that includes a built-in projector mechanism for projecting or displaying frames including left and right images to a user’s eyes to thus provide 3D virtual views to the user. The 3D virtual views may include views of the user’s environment augmented with virtual content (e.g., virtual objects, virtual tags, etc.). The mixed reality system may include world-facing sensors that collect information about the user’s environment (e.g., video, depth information, lighting information, etc.), and user-facing sensors that collect information about the user (e.g., the user’s expressions, eye movement, hand gestures, etc.). 

The sensors provide the information as inputs to a controller of the mixed reality system. The controller may render frames including virtual content based at least in part on the inputs from the world and user sensors. The controller may be integrated in the HMD, or alternatively may be implemented at least in part by a device external to the HMD. The HMD may display the frames generated by the controller to provide a 3D virtual view including the virtual content and a view of the user’s environment for viewing by the user. 

Summary of the patent

Here’s Apple’s abstract of the patent: “A mixed reality system that includes a head-mounted display (HMD) that provides 3D virtual views of a user’s environment augmented with virtual content. The HMD may include sensors that collect information about the user’s environment (e.g., video, depth information, lighting information, etc.), and sensors that collect information about the user (e.g., the user’s expressions, eye movement, hand gestures, etc.). 

“The sensors provide the information as inputs to a controller that renders frames including virtual content based at least in part on the inputs from the sensors. The HMD may display the frames generated by the controller to provide a 3D virtual view including the virtual content and a view of the user’s environment for viewing by the user.”

About Apple Glasses

When it comes to Apple Glasses, such a device will arrive in 2022 or 2023, depending on which rumor you believe. It will be a head-mounted display. Or may have a design like “normal” glasses. Or it may be eventually be available in both. The Apple Glasses may or may not have to be tethered to an iPhone to work. Other rumors say that Apple Glasses could have a custom-build Apple chip and a dedicated operating system dubbed “rOS” for “reality operating system.”



Article provided with permission from AppleWorld.Today