Invented by Thomas Marshall Miller, IV, Victor Ng-Thow-Hing, Josh Anon, Frank Alexander Hamilton, IV, Cole Parker Heiner, Rodrigo Cano, Karen Stolzenberg, Lorena Pazmino, Gregory Minh Tran, Stephane Antoine Joseph Imbert, Anthony Marinello, Katsu Entertainment LLC, Magic Leap Inc
The Market for Contextual-Based Rendering Virtual Avatars
Avatars, who may resemble humans in appearance and behavior, inhabit online worlds where people play games or socialize. Here they create, purchase or receive virtual goods – often referred to as metaverses – for these environments.
Marketers can track avatars’ activities in these virtual worlds and target them with tailored products and promotions. Some avatars even accumulate virtual currency to use at real-world locations.
Market Size
The market for contextual-based rendering virtual avatars is expected to experience strong growth during the forecast period. Growing adoption of these solutions across gaming, media & entertainment, fashion and healthcare is expected to fuel growth.
The gaming industry is expected to hold a major share in the global market over the forecast period. This is mainly due to high acceptance of technologically advanced games by consumers and rising demand for VR & AR products worldwide. Furthermore, gaming businesses are using these technologies to enhance their product offerings and provide players with an engaging experience.
Additionally, avatars are becoming increasingly popular in the education sector. These digital human avatars look and behave like real people, providing students with an engaging educational experience. Furthermore, the integration of conversational AI technology into these avatars is expected to further fuel this market’s expansion.
Additionally, the market for avatars is expected to experience significant expansion in retail and healthcare sectors. These avatars can mimic salesperson interactions with customers to better comprehend their preferences and assist in making purchasing decisions.
Avatars can answer customer inquiries much quicker than live human agents and resolve any discrepancies visitors might have. In doing so, avatars save a large number of contact center staff that would otherwise need to handle multiple inquiries from customers.
The growing popularity of e-commerce platforms is further fueling demand for these solutions. E-commerce allows users to purchase items without leaving their homes, and with increased spending per capita on e-commerce in countries such as China, South Korea, and Australia comes an increased need for 3D avatars in these industries.
Moreover, the gaming industry is taking advantage of these solutions to create realistic games that appeal to a broad audience. The gaming industry strives to deliver an authentic experience for customers, and new technologies like augmented reality (AR) and virtual reality (VR) have allowed for a more immersive experience.
Market Segmentation
Market segmentation can help you more precisely target your marketing efforts. This will enable you to craft services and messages that resonate with your customers’ needs and preferences, leading to increased brand loyalty as well as higher sales volumes.
Market segmentation divides a customer base into smaller groups based on factors like age, gender, income and education level. It may also be employed to classify customers according to their behavior, lifestyle and attitudes.
Market segmentation can be beneficial for a range of products and services, such as anti-aging beauty items, local sales and OTT streaming. It also allows you to craft targeted promotions and discounts tailored for specific audiences and create advertising campaigns tailored to meet the demands of your target customers.
The initial step in developing a market segmentation strategy is to identify the various groups of potential customers you wish to reach. This can be accomplished through market research methods such as focus groups, surveys and interviews.
Once you’ve identified potential customer groups, analysis can be done to understand their purchasing behaviors, patterns and reactions. This will enable you to customize your product offerings and pricing accordingly; offering them items they are likely interested in and willing to pay for.
Furthermore, you can conduct market research to understand which type of products your customers prefer and whether they are willing to pay more for them. This will enable you to design products and services tailored specifically for them – helping you hit sales targets while increasing profit margins.
An experiment was conducted to better understand users’ perceptions of avatars. This involved two use cases that required remotely located users to complete a collaborative task using AR and VR gear. The experiments sought to evaluate the impact different avatar designs had on users’ individual and other users’ person perception in telexistence systems.
Results of the study revealed that participants preferred full body avatars over torso with arms & half thigh or head & hands avatars. Furthermore, they desired to view remote user’s face and gaze direction in the telexistence environment. Furthermore, they recommended customizing avatar design; enabling them to choose colors for clothes or shoes as well as height and shape.
Market Trends
The Contextual-based rendering virtual avatars market is expected to experience tremendous growth over the forecast period, driven by an increasing demand for realistic visuals in movies, videos, TV shows, and online content. Furthermore, the rising number of simulation technology applications across various sectors are propelling demand for Contextual-based rendering virtual avatars.
Tele-immersive AR applications require photorealistic avatars in order to create an immersive and co-presence experience. This encourages empathy towards others and increases sense of embodiment within space – leading to stronger identification with one’s avatar in telexistence systems. To achieve this effect, avatars should have similar facial expressions and body motions as their corresponding person in real life.
Implementing a rich embodiment system necessitates accurate tracking of the user’s body, which can be achieved using sensors that identify different body parts (e.g., head, hands, fingers, eyes and face). Furthermore, taking into account the user’s gaze direction should be included in the system to guarantee that avatars respond appropriately to user input.
For this purpose, a virtual mirror is employed to capture an avatar’s movements within an environment and translate them into bodily and facial actions in either augmented reality (AR) or virtual environment (VE) of the AR user. The avatar reacts in real time to these movements of its corresponding AR or VE user, so both parties can observe what transpires.
However, avatars in telexistence systems still face difficulties due to their context-awareness. Furthermore, physical and emotional aspects of avatars should be carefully considered so as not to create an ‘uncanny valley’ effect where users become uncomfortable with realistic representations.
As a result, several researchers have proposed alternative ways to present avatars in systems to boost user satisfaction. Some suggested using an avatar with a cartoon-like or more straightforward appearance for easier interaction, while others recommended more complex representations such as humanoids or mythical characters which might appear more realistic and less susceptible to the uncanny valley effect.
Market Opportunities
The market for contextually-based rendering virtual avatars is expected to expand significantly over the coming years due to an uptick in AR and VR applications, driven by advances in computer graphics and high-speed networking technology. These systems could include photorealistic avatars suitable for both social and professional uses cases.
Recent improvements to virtual avatar graphics [42], [62] have resulted in more realistic-looking designs. Unfortunately, some of these designs can cause people to feel uneasy or disoriented due to their unsettling appearance. To prevent this from occurring, visual representations of avatars in telexistence systems should be designed so as not to cause users any visual shock or distraction.
Studies of self-presentation in virtual environments have revealed that many users prefer more realistic avatars to idealized ones. This is likely because realistic avatars allow for greater self-presence, leading to enhanced sense of identity and trust between remote participants. On the other hand, some may prefer less realistic avatars in order to reduce feeling overwhelmed or distracted by their virtual environment.
A study looking into user preferences for avatar representations in tele-immersive sports and collaborative tasks revealed that both types of user interaction require a recognizable avatar, but the preferred design differed depending on the context. Games generally favored photorealistic full-body human avatars more than professional settings.
The study also reveals that hologram representations of avatars are preferred in both VR and AR contexts, as they provide a see-through avatar that can be utilized for various purposes. Professionally speaking, a hologram presentation may create a stronger sense of presence and boost trust that another participant has in their virtual avatar.
To determine user preference for virtual avatar design, 36 comparable designs were created and presented to a group of participants during a ranking task. They were asked to rank their preferences for each design and provide rationales. Results revealed that overall preference among participants was higher for photorealistic full-body avatars than hologram representations; however both designs were highly acceptable.
The Katsu Entertainment LLC, Magic Leap Inc invention works as follows
Examples are given of systems and methods that render avatars in mixed reality environments. These systems and methods can be used to scale or render an avatar based upon a user’s intention, an interesting impulse, stimuli or user saccade points. When rendering an avatar, the disclosed systems and methods can apply discomfort curves. These systems and methods could allow for a more natural interaction between an avatar and a human user.
Background for Contextual-based rendering virtual avatars
Modern computing and display technology have made it possible to create systems that can be called “virtual reality”, “augmented reality?”, or “mixed reality?” Digitally reproduced images and portions of them are presented to users in a way that makes them appear real or can be perceived as such. A virtual reality (or?VR?) scenario usually presents digital or virtual information without transparency to any other real-world visual input. An augmented reality (or?AR?) scenario typically presents digital or virtual information as an enhancement to the visualization of the real world around the user. A mixed reality (or?MR?) scenario involves merging real and digital worlds to create new environments in which physical and virtual objects coexist and interact in real-time. It turns out that the human visual system is complex. Therefore, it is difficult to create a VR, AR or MR technology that allows for a natural-feeling, rich presentation and interaction of virtual images elements with real-world or virtual imagery elements. The systems and methods described herein address various issues related to VR, AR, and MR technology.
Various examples of mixed reality systems for rendering and adjusting virtual avatars are shown based on context information.
Examples are given of systems and methods that render avatars in mixed reality environments. These systems and methods can be used to scale or render an avatar based upon a user’s intention, an interesting impulse, stimuli or user saccade points. When rendering an avatar, the disclosed systems and methods can apply discomfort curves. These systems and methods could allow for a more natural interaction between an avatar and a human user.
Details about one or more implementations are provided in the accompanying drawings as well as the description. The claims, drawings, and description will reveal other features, aspects, or advantages. This summary and the detailed description below do not attempt to limit or define the scope of the inventive subject matter.
Overview
A virtual avatar can be described as a virtual representation of an actual or fictional person, creature, or personified object in an AR/VR/MR setting. A viewer may perceive an avatar of another AR/VR/MR user within their environment during a telepresence session. This creates a tangible sense that the other user is present in the viewer?s environment. A virtual avatar allows users to interact and collaborate in a virtual environment. A student can see avatars of other students and teachers in an online classroom. They can also interact with the teacher’s avatars.
A user must determine the size of an avatar when placing it in their AR/VR/MR environment. In practice, the avatar can be any size, including tiny, large, and human-sized. Although the avatar can be kept at a 1:1 ratio to its human counterpart, this might not work in all environments due to privacy or space limitations. A poorly sized avatar can cause awkward social interactions and fatigue in interaction with the avatar. If an avatar is too large or small in relation to the viewer, the viewer might need to adjust his or her body position to be able to have an eye-to?eye conversation with the avatar. A poorly sized avatar may send the wrong message, such as an implied superiority or inferiority between the avatar and the viewer. FIG. 11A further explains the problems that can be caused by an incorrectly sized avatar. 11A, and solutions are provided with FIGS. 12A-18B.
Advantageously in some embodiments, a wearable system described herein is able to automatically determine the appropriate size of an avatar at spawning. It can also re-scale the avatar during interaction with other users (or avatars) using contextual information about the interaction. Examples of contextual information include the user’s position, the rendering location of avatar in the environment, a relative height between avatar and user, the presence of objects in rendering environment (e.g. whether avatar can sit on chairs or if they would be able to move through solid objects like tables). Based on contextual information, the wearable system can scale the avatar automatically to increase or minimize direct eye contact and facilitate avatar-human communication. FIGS. provide further details on scaling an avatar using contextual information. 12A-18B.
An avatar can be animated using its human counterpart. A human’s interaction with an avatar is then mapped to the avatar. An AR/VR/MR environment can allow for one-to-one mapping. This means that an avatar’s actions are a direct mapping to a user’s. If a user looks to the left, then its avatar will also look left. If the user stands, his avatar will also stand. A user can walk in one direction and its avatar will follow it. One-to-one mapping can work in VR environments because all participants (including the avatar) see the same virtual content. AR/MR environments can be very different. Each user’s environment and how their avatar appears in it may differ. This is because they might be in different environments. Bob may be in a living area in a home, while Alice might be in an office room. Bob might see Alice?s avatar in Bob?s living room, while Alice may see Bob’s avatar in Alice’s office room. Another example is that avatars can be resized. Bob can shrink Alice’s avatar to fit on a table in his living area, while Alice may be working in an office space and want Bob’s avatar to stand in the corner.
This may lead to Bob looking at Alice’s avatar while Bob is speaking to her avatar. A one-to-one mapping of Bob’s actions to Alice’s avatar may result in Bob’s avatar rendering in Alice’s office looking unusually to Alice. This is because Bob’s real avatar is looking at Alice’s avatar while Alice is talking to him. It may be beneficial to preserve certain aspects of the user and map them to an avatar in another environment. A user can communicate to another user by simply nodding or shaking their heads in agreement. Additional examples of problems in one-to-1 mapping can be found at FIGS. 11B-11D. These problems can be solved by referring to FIGS. 19A-30B.
In some embodiments, the wearable device can analyse an interaction of a user to break it down into a world and local component. The interaction with the environment can be included in a world component. One example of a world motion is walking from A to B, climbing up ladders, standing or sitting, facing a particular direction and interfacing with an object (virtual, or physical) within the environment. The world component can be described in terms of a world reference frame that is associated with an environment. Local components can also include actions relative to users (which can be described relative a body-fixed refer frame). If Alice shakes her head or nods her head, this motion is defined by the angle of her head relative to her body (or torso). Alice can also turn her head 180 degrees and nod her ear. These movements can be considered local as they are specific to Alice’s torso. They may not need interactions with the surrounding environment. Another example is waving a hand. It can be considered a local movement because it can be identified with respect to the user. Some movements can have both a local and a global portion. A gesture such as pointing a finger at an object within the environment may have a local portion and a world portion. Alice might point to Bob’s avatar by pointing at it. This is an example of Alice’s intent. Alice may appear in Bob’s environment in a different orientation or relative to Bob. Alice’s hand gesture might be rendered in one-to-1 correspondence so Alice’s avatar might not point towards Bob. Bob can use his wearable device to map Alice’s intentions to Bob’s environment, so Alice’s avatar will be rendered as pointing towards Bob.
The wearable system is able to extract the intent of a user?s interaction using contextual information such as the user?s environment, user?s movements, user intentions, etc. The wearable system can map the user’s world motion to an avatar’s action using the avatar’s environment, and then map the local action to the avatar. “Mapping the world motion may include the adjustment of one or more characteristics such as the user’s movement, position and orientation, size, facial expression, pose and eye gaze to make it compatible with the environment in which the avatar is rendered.
For instance, Alice may walk to a chair and then sit down on it. The wearable system will access information from Bob’s world map and render Alice sitting on the chair. Another example is that the wearable system might determine that Alice wants to interact with an object (e.g., a tree, or a virtual novel) within her environment. The wearable system can automatically reorient Alice?s avatar so that it interacts with the object in Bob’s environment. However, the location of the object may be different in Bob’s. If Bob’s environment has a one-to-one mapping that would render the virtual books inside or under a table, Bob’s wearable device may show the virtual books as lying on top of the table. This will allow Bob to interact with the virtual book more naturally.
The wearable system might remap world motions but it can also preserve local motions such as nodding or hand tilting. This can be used to indicate confusion, agreement, or refusal. Alice, for example, can move toward a chair by shaking her head. This interaction can include a world motion such as walking towards a chair or shaking her head. The wearable system can adjust Alice?s walking direction based on Bob’s chair location. In the meantime, Alice’s avatar can be rendered shaking her head. FIGS. provides further information on intent-based renderings of avatars. 19-31.
The wearable system can map a local component to an avatar depending on intent. If Alice makes a thumbs-up gesture, the wearable system will interpret it as an emblem gesture. This is a gesture that is consciously used, understood, and used to communicate the same intent. It can map a more expressive thumbs-up animation to Alice’s avatar in Bob?s environment. You can also use this gesture to communicate your intention with other emblem or symbolic gestures such as waving your hands with an open palm gesture or the okay sign.
The wearable system can, in certain embodiments, also be used to animated a virtual avatar based upon the environment in which it is placed. The wearable system can render a virtual avatar so that it can make decisions based on interactions with its environment. For example, the wearable device may show the avatar standing if there is no chair in the environment.
The wearable system of the viewer can detect interesting impulses in their environment. This could include an object, an area, sound or component of an object. These impulses may cause the avatar to respond automatically (e.g. turning around to see the interesting impulses). One example is that the wearable viewer system may detect loud noises and can automatically reorient avatar to face the source of the noise.
The wearable system can animated the avatar based upon the impulse, regardless of whether the viewer is in an telepresence session or not with the avatar’s human counterpart. The avatar can respond to an interesting impulse in the environment of the viewer even if it is not present in their environment. Alice and Bob might be in telepresence sessions. Bob may be speaking and Alice’s avatar might face Alice at first. Bob’s wearable device may detect a loud noise and Alice’s avatar might switch its attention to that direction (and therefore may turn away from Bob). Additional descriptions of animating an avatar based upon the environment of a viewer are provided in FIGS. 31A-37.
In some implementations, the wearable can learn the behavior of one or more users (e.g. the avatar’s human counterpart), and then drive the avatar’s animation using that learning, even though the human counterpart may not be there (either remotely, or in the same environment). The wearable system can, for example, learn from data collected from sensors how the user interacts. This includes the direction of the user’s eyes and the frequency with which they make eye contact. It can also determine the user’s ability to speak or hear objects. Based on user’s learned behavior, the wearable system can drive avatar animations. If a user doesn’t respond to to country music (e.g. does not look at the source of the country music), then the avatar associated to that user might be made so that it does not respond.
Although these examples are about animating a human-shaped avatar of Alice, similar techniques can be used to animals, fictitious beings, and objects (e.g. the virtual book mentioned above). Alice’s system might detect movement in her environment from a dog, for example. Bob’s wearable device can show the movement of a virtual pet in Bob’s environment. Bob’s wearable device may show the movement of the virtual canine in Bob?s environment, based on obstacles (e.g. by having the virtual pet follow a path that doesn’t cause it to move through any physical objects).
Accordingly, embodiments and methods disclosed may allow for a more natural interaction between the wearable system user and avatars in user’s environment.
