Feel Physics

Develop physics education HoloLens apps and Deliver lessons

Mixed Reality (MR) description template in academic papers

Notice: I haven't confirm whether rights of using pictures below are available or not. Notice: It's still under construction. Any idea is welcome.

Mixed Reality (MR)

MR in recent years

The concept of Mixed Reality (MR) has been in the 1990's. However, the mixed reality headset in recent years (after 2016) has the following appearance (the picture is HoloLens).

f:id:weed_7777:20180906050516p:plain

Fig. 1 Appearance of MR headset

VR headset is closed, immersive type. You can see only virtual images in three dimensions. On the other hand, the MR head set is transparent type (see-through). The wearer can see both the virtual stereoscopic image and the surrounding scenery in the same time.

The MR head set can grasp the wearer's gaze direction, posture and movement in real time. Even the wearer changes his/her gaze direction, posture or he/she moves, it seems to him/her that the virtual stereoscopic image is "placed" in the real space.

f:id:weed_7777:20180815181839p:plain

Fig. 2 A man operating in MR

f:id:weed_7777:20180906051042j:plain

Fig. 3 People working together in MR

In brief, when we wear a MR headset, we can mix real scenes and virtual images and see them. Therefore, it is called Mixed Reality.

History of MR

Historically, the first concept of MR was proposed by Milgram et al. (1994):

An MR experience is one where the user is placed in an interactive setting that is either

  • real with virtual asset augmentation (augmented reality, AR) or
  • virtual with real-world augmentation (augmented virtuality, AV) that overlays real information on virtual world.

f:id:weed_7777:20180826205613p:plain

Fig. 4 First definition of MR

Huges (2005) embodied the concept of MR:

MR is the mixed visual and audio content for both

  • AR (such as image, sound, smell, heat. The left side of the figure below) and
  • AV (Virtual world overlaid with information of real world obtained with such as camera and sensors. The right side of the figure below)

which is captured, rendered and mixed by graphics and audio engines.

f:id:weed_7777:20180809044838p:plain

Fig. 5 MR as mixed of AR and AV

In this case, actual images are supposed to be acquired from the camera. This is a bit different from the current one.

Adriana et al. (2009) sophisticated the definition of MR:

MR is the merging of real and virtual worlds to produce new environments and visualizations where physical and digital objects co-exist and interact in real time.

They removed the somewhat confusing concept of AV from MR and added real space instead (meaning has not changed). They also emphasized spatiality, interactivity and real-time nature.

Citations and references

  • Adriana de Souza e Silva, Daniel M. Sutko: Digital Cityscapes: merging digital and urban playspaces. New York: Peter Lang Publishing, Inc., 2009

  • Dieter Schmalstieg and Daniel Wagner, First steps towards handheld augmented reality, Seventh IEEE International Symposium on Wearable Computers, 1530-0811(2003), 127-135

  • D.E.Hughes, Defining an Audio Pipeline for Mixed Reality, Proceedings of Human Computer Interfaces International, Lawrence Erlbaum Assoc., Las Vegas (2005)

    1. Milgram and A.F. Kishino, “Taxonomy of Mixed Reality Visual Displays,” IEICE Trans. Information and Systems, vol. E77-D, no. 12, 1994, pp. 1321-1329.

Reference: "Mixed Reality in Education, Entertainment, and Training"

https://www.researchgate.net/publication/7454465_Mixed_Reality_in_Education_Entertainment_and_Training

Outline of HoloLens application teaching magnetic field

The application is divided into three scenes.

f:id:weed_7777:20180617132145p:plain

First, students can freely move the bar magnets on a compass indicating a magnetic field aligned in a plane.

f:id:weed_7777:20180617132201p:plain

The second is that students can move the bar magnets in a compass azimuth needle arranged in three dimensions.

f:id:weed_7777:20180617132218p:plain

Third, students can move the magnet freely, and students can see magnetic lines of force between the magnets.

    • -

f:id:weed_7777:20180629170518p:plain

We were introduced on the official homepage of Microsoft:

This MVP Travels Japan To Bring Science To Life With HoloLens – Microsoft MVP Award Program Blog

    • -

We are going to deliver lesson worldwide after December 2018.