MR.Dali—A new approach on the use of mixed reality.
[/vc_column_text][vc_empty_space height=”30px”][/vc_column][/vc_row][vc_row css_animation=”” row_type=”row” use_row_as_full_screen_section=”no” type=”full_width” angled_section=”no” text_align=”left” background_image_as_pattern=”without_pattern” z_index=””][vc_column width=”2/3″][vc_column_text]Mixed Reality (MR) describes the blending of the real and virtual world. Applied to science, this merger has enormous potential. “MR.Dali” supports nature scientists in collecting, editing and sharing data locally.
Through object recognition, information can be bound to objects which have been created by the tool named “stolus“. Eye tracking & muscle sensors help the scientist navigate through the interface quick and intuitively, while walking. The collected data can be shared among researchers and used for analysis, education and planning.[/vc_column_text][/vc_column][vc_column width=”1/3″][vc_column_text css=”.vc_custom_1560091524217{margin-left: 30px !important;}”]DATE —March – May 2018
CATEGORY — Mixed Reality
PLACE —ZHdK, ETHZ & Technopark Zurich
TEAM — Stella Mühlhaus, Manuel Kallen , Micha Weber, Marco Ketzel[/vc_column_text][/vc_column][/vc_row][vc_row css_animation=”” row_type=”row” use_row_as_full_screen_section=”no” type=”full_width” angled_section=”no” text_align=”left” background_image_as_pattern=”without_pattern”][vc_column][vc_separator type=”normal” thickness=”1″ up=”37″ down=”17″ color=”#d8d8d8″][vc_column_text]
Project details
[/vc_column_text][vc_empty_space height=”30″][/vc_column][/vc_row][vc_row css_animation=”” row_type=”row” use_row_as_full_screen_section=”no” type=”full_width” angled_section=”no” text_align=”left” background_image_as_pattern=”without_pattern” z_index=””][vc_column width=”1/2″][vc_accordion active_tab=”” collapsible=”yes” style=”boxed_accordion”][vc_accordion_tab title=”Foreword” title_color=”#0a0a0a” background_color=”#f2f2f2″][vc_row_inner row_type=”row” type=”full_width” text_align=”left” css_animation=””][vc_column_inner][vc_column_text]OUR GOAL[/vc_column_text][vc_empty_space height=”10px”][vc_column_text]The aim was to find an intuitive control system for Mixed Reality. Based on the natural human behavior, our project submits a new control combination: the eye-gaze used as the cursor and the muscle as the trigger. The advantages of our work were then validated and visualizedthrough a user case.[/vc_column_text][vc_empty_space][vc_column_text]
THE NAME
[/vc_column_text][vc_column_text]Early on, our team and project needed a name. We chose: MR for Mixed Reality + the surrealism artist Salvador Dalì. = “MR. Dali”[/vc_column_text][vc_empty_space height=”10px”][/vc_column_inner][/vc_row_inner][/vc_accordion_tab][vc_accordion_tab title=”Use Case” title_color=”#0a0a0a” background_color=”#f2f2f2″][vc_row_inner row_type=”row” type=”full_width” text_align=”left” css_animation=””][vc_column_inner width=”1/2″][vc_column_text]NATURAL SCIENCES[/vc_column_text][vc_empty_space height=”10px”][vc_column_text]The topic of inventory in connection with the user case of natural sciences fascinated us throughout the entire project.[/vc_column_text][vc_empty_space height=”10px”][vc_column_text]At first it may seem rather contradictory to use this modern piece of technology (mixed reality) outside in the nature. But since the natural scientists often have tocarry technical equipmentwith them, it makes especially sense for them to wear a head-mounted display (HMD) instead to carry a laptop.[/vc_column_text][vc_empty_space height=”10px”][vc_column_text]Furthermore in the course of the work it turned out that perhaps mixed reality may be also an initial asset that brings technology and nature together:“seeing the Data of Nature — Making the nature readable in human language … “[/vc_column_text][vc_empty_space height=”25px”][/vc_column_inner][vc_column_inner width=”1/2″][vc_column_text]For example, with our idea the following points would be possible:[/vc_column_text][vc_column_text]
watching changes and symbioses
carrying less equipment with you
local based / attached data
easy recovery of marked objects
clearer comparisons
learn/read directly from the environment
easier planning by judging visible effects locally
[/vc_column_text][vc_empty_space height=”25px”][/vc_column_inner][/vc_row_inner][vc_single_image image=”500″ img_size=”full” onclick=”link_image” qode_css_animation=””][vc_empty_space height=”10px”][/vc_accordion_tab][vc_accordion_tab title=”Making of” title_color=”#0a0a0a” background_color=”#f2f2f2″][vc_single_image image=”926″ img_size=”full” onclick=”link_image” qode_css_animation=””][vc_empty_space height=”10px”][/vc_accordion_tab][vc_accordion_tab title=”Exhibition” title_color=”#0a0a0a” background_color=”#f2f2f2″][vc_empty_space height=”10px”][vc_row_inner row_type=”row” type=”full_width” text_align=”left” css_animation=””][vc_column_inner width=”1/2″][vc_single_image image=”496″ img_size=”full” onclick=”link_image” qode_css_animation=””][vc_empty_space height=”10px”][/vc_column_inner][vc_column_inner width=”1/2″][vc_single_image image=”495″ img_size=”full” onclick=”link_image” qode_css_animation=””][vc_empty_space height=”10px”][/vc_column_inner][/vc_row_inner][vc_column_text]MR.Dali supports researchers in collecting, processing and sharing measurement data on site. Using image recognition, information can be linked to objects that are captured by a pen. Eye-tracking control helps the user navigate the interface swiftly and intuitively. MR.Dali enables researchers to exchange the collected geographical and climatic data and thus to use it for analysis and planning.[/vc_column_text][vc_empty_space height=”10px”][/vc_accordion_tab][/vc_accordion][/vc_column][vc_column width=”1/2″][vc_accordion active_tab=”” collapsible=”yes” style=”boxed_accordion”][vc_accordion_tab title=”the cursor” background_color=”#f2f2f2″ title_color=”#0a0a0a”][vc_row_inner row_type=”row” type=”full_width” text_align=”left” css_animation=””][vc_column_inner width=”1/3″][vc_column_text][/vc_column_text][/vc_column_inner][vc_column_inner width=”2/3″][vc_empty_space][vc_column_text]
Eye Cursor
[/vc_column_text][vc_empty_space height=”10px”][vc_column_text]“Humans are eye-animals“. The first gesture humans normally do is to gaze at the object of interest. So why dragging a cursor after it? Especially if we have a computer in front of our eyes, the eyes are then actually predestined to function for control purposes.[/vc_column_text][vc_empty_space height=”10px”][vc_column_text]We have combined the Microsoft HoloLens with the EyeTrackers from PupilLabs. Through aself-written codethe captured eye gaze can now be used as cursor. This makes the navigation in mixed reality faster and more intuitive to use. One of the advantages, which was not possible before, is for example, that it can now also be used while walking.[/vc_column_text][vc_empty_space height=”10px”][vc_column_text]
and above all…
The feeling that a computer object recognizes that it is being watched was an incredible joyful moment.A new level of proximity has been created.[/vc_column_text][vc_empty_space height=”10px”][/vc_column_inner][/vc_row_inner][vc_empty_space][vc_row_inner row_type=”row” type=”full_width” text_align=”left” css_animation=””][vc_column_inner width=”1/3″][vc_column_text]
[/vc_column_text][vc_empty_space height=”10px”][vc_column_text]To take action we need our muscles.Even if we only think about the action, we minimally tense muscles. So this is our trigger … also for mixed reality purposes.[/vc_column_text][vc_empty_space height=”10px”][vc_column_text]And don’t we forget that it’s also natural for usto use tools instead our bare handsto perform a task. Especially in the case of MR, a tangible tool can be a beneficial extension for the interaction.[/vc_column_text][vc_empty_space height=”10px”][vc_column_text]Therefore we designed “stolus”.
The wristband captures the muscle contractions and comes with a device in the form of a biface.[/vc_column_text][vc_empty_space height=”10px”][/vc_column_inner][/vc_row_inner][vc_empty_space][vc_row_inner row_type=”row” type=”full_width” text_align=”left” css_animation=””][vc_column_inner width=”1/3″][vc_column_text]
[/vc_column_text][vc_empty_space height=”10px”][vc_column_text]In MR, holograms are integrated into the physical world. So it can be under/on/behind/… real objects.But if you want to move a physical object with its hologram, the hologram stays at its spatial place.Reason, MR does not distinguish objects from the room surfaces yet. So the hologram remains unaffected.[/vc_column_text][vc_empty_space height=”10px”][vc_column_text]In our work we defined a parenting system that categorizes the connection between holograms and the physical world in 3 stages:
Space-Child, My-Child, Object-Child.
Through “Object-Child” the holograms gain a bit more reality and a whole new interaction level.[/vc_column_text][vc_empty_space height=”10px”][vc_column_text]In our project we work on all 3 stages. With vuforia we were able to develop our prototype of an object-child. The object was now recognizable and holograms could stick to it.[/vc_column_text][vc_empty_space height=”10px”][/vc_column_inner][/vc_row_inner][vc_empty_space][vc_row_inner row_type=”row” type=”full_width” text_align=”left” css_animation=””][vc_column_inner width=”1/3″][vc_column_text]
[/vc_column_text][vc_column_text]
[/vc_column_text][vc_empty_space height=”10px”][/vc_column_inner][/vc_row_inner][/vc_accordion_tab][vc_accordion_tab title=”interaction” title_color=”#0a0a0a” background_color=”#f2f2f2″][vc_empty_space height=”10px”][vc_column_text]Depending on the purpose of the menu/feature, it is essential to create an appropriate appearance which enables a proper interaction. Here I introduce you to 3 interaction levels:[/vc_column_text][vc_empty_space][vc_column_text]
calling > Satellite
[/vc_column_text][vc_empty_space height=”5px”][vc_row_inner row_type=”row” type=”full_width” text_align=”left” css_animation=””][vc_column_inner width=”1/3″][vc_single_image image=”903″ img_size=”full” onclick=”link_image” qode_css_animation=””][/vc_column_inner][vc_column_inner width=”2/3″][vc_column_text]The satellite is a kind of “start menu” that always follows its user, hence its name. The satellite is generally invisible so the field of view remains unobstructed. Only when the user looks up left*, to the satellite’s position, it becomes visible and unfolds to the main menu.[/vc_column_text][vc_empty_space height=”10px”][vc_column_text]
* This position was chosen, among other factors, as an analogy to psychology: When a person unconsciously looks up to the left, it means that she or he visually remembers something.
[/vc_column_text][vc_empty_space height=”5px”][vc_row_inner row_type=”row” type=”full_width” text_align=”left” css_animation=””][vc_column_inner width=”1/3″][vc_single_image image=”907″ img_size=”full” onclick=”link_image” qode_css_animation=””][/vc_column_inner][vc_column_inner width=”2/3″][vc_column_text]When the satellite is enabled, the main menu unfolds vertically downwards, so that the gaze can be lowered again. The clear line arrangement takes up less space in the field of view and ensures a overview which saves time.[/vc_column_text][/vc_column_inner][/vc_row_inner][vc_empty_space][vc_column_text]
handling – C-tool table
[/vc_column_text][vc_row_inner row_type=”row” type=”full_width” text_align=”left” css_animation=””][vc_column_inner width=”1/3″][vc_single_image image=”905″ img_size=”full” onclick=”link_image” qode_css_animation=””][/vc_column_inner][vc_column_inner width=”2/3″][vc_column_text]Example: Drawing tool If the drawing tool is active, the tool-palette can be opened by a double click. The tool-palette appears around the hand that called it. The C shape allows a quick and practical handling by simply tilting the hand in a certain direction. The open side is not needed as it would be covered by the hand and from ergonomic test is perceived as a rather uncomfortable movement.[/vc_column_text][/vc_column_inner][/vc_row_inner][vc_empty_space height=”10px”][/vc_accordion_tab][/vc_accordion][/vc_column][/vc_row][vc_row css_animation=”” row_type=”row” use_row_as_full_screen_section=”no” type=”full_width” angled_section=”no” text_align=”left” background_image_as_pattern=”without_pattern”][vc_column][vc_separator type=”normal” thickness=”1″ up=”37″ down=”17″ color=”#d8d8d8″][vc_column_text]
[/vc_column_text][/vc_column_inner][vc_column_inner width=”2/3″][vc_empty_space][vc_column_text]
[/vc_column_text][/vc_column_inner][vc_column_inner width=”2/3″][vc_empty_space][vc_column_text]
[/vc_column_text][/vc_column_inner][vc_column_inner width=”2/3″][vc_empty_space][vc_column_text]
[/vc_column_text][/vc_column][/vc_row][vc_row css_animation=”” row_type=”row” use_row_as_full_screen_section=”no” type=”full_width” angled_section=”no” text_align=”left” background_image_as_pattern=”without_pattern”][vc_column][vc_separator type=”normal” thickness=”1″ up=”37″ down=”17″ color=”#d8d8d8″][vc_column_text]
[/vc_column][/vc_row]
