Practices and Improvements of Lesson using Magnetic field Visualization
by Mixed Reality (MR) technology
UEDA Tatsuro
Mie University part time lecturer
ueda.tatsuro@gmail.com
Abstract
It is difficult to study the phenomena which cannot be seen. But Mixed Reality (MR) has the potential to support such study. In this case, I take magnetic fields as the phenomenon cannot be seen. I created and improved a visualization teaching application and lesson program to resolve several problems which become apparent through lesson practices. At last the learning effect was confirmed and the satisfaction was very good. In this research, I confirmed that with devises MR is able enough to support the study of phenomena cannot be seen, and I confirmed the requirements to do lesson using MR.
Keywords
Mixed Reality, MR, Augmented Reality, AR, Magnetic Field, Visualization, ICT, Information technology
1. Background
１） Study of phenomena cannot be seen
It is difficult for students learning physics to study and understand the phenomena which cannot be seen.
Thus, how to image the phenomena cannot be seen is important in the study of physics.
In this research, I confirmed that MR information technology is a efficient method of visualization and imagination, using the teachings of magnetic fields as an example.
２） Study of magnetic field
Fig. １ Iron sand around a bar magnet
Magnetic field cannot be seen.
In this research, I created the teaching material with MR technology which includes AR to study magnetic fields.
３） Augmented Reality (AR)
Fig. ２ Famous game using AR
(Pokemon Go)
Augmented Reality (AR) is a variation of Virtual Reality (VR). VR technologies completely immerse a user inside a synthetic environment. While immersed, the user cannot see the real world around him. In contrast, AR allows the user to see the real world, with virtual objects superimposed upon or composited with the real world. Therefore, AR supplements reality, rather than completely replacing it. Ideally, it would appear to the user that the virtual and real objects coexisted in the same space [Ronald, 1997].
Various sensors such as camera, GPS or acceleration sensor make tracking the position and orientation of a mobile phone possible. And this system can display an image in the correct position and orientation, and is important technology to realize AR.
We can use image recognition libraries such as "ARToolKit" released in 1999 for AR tracking.
Fig. ３ Example of using
AR library
Dieter Schmalstieg and Daniel Wagner developed the first real time marker tracking system for mobile phone and personal digital assistant (PDA) in 2003.
Fig. ４ The first real time marker
tracking system
for mobile phone and PDA
４） Mixed Reality (MR)
MR in recent years
The original concept of Mixed Reality (MR) was put forward in the 1990's. However, the Mixed Reality headset in recent years (after 2016) has the following appearance (the picture is HoloLens).
Fig. ５ Appearance of MR headset
In the case of VR, a headset is a closed, immersive type. You can see only virtual images in three dimensions. On the other hand, the MR head set is transparent type (see-through). The wearer can see both the virtual stereoscopic image and the surrounding scenery at the same time.
In the case of MR, a headset can grasp the wearer's gaze direction, posture and movement in real time. Even the wearer changes his/her gaze direction, posture or he/she moves, it seems to him/her that the virtual stereoscopic image is "placed" in real space.
Fig. ６ A man operating in MR
Fig. ７ People working together in MR
In brief, when we wear a MR headset, we can mix real scenes and virtual images and see them. Therefore, it is called Mixed Reality.
History of MR
Historically, the first concept of MR was proposed by Milgram et al. (1994):
An MR experience is one where the user is placed in an interactive setting that is either
real with virtual asset augmentation (augmented reality, AR) or
virtual with real-world augmentation (augmented virtuality, AV) that overlays real information on a virtual world.
Fig. ８ First definition of MR
Huges (2005) embodied the concept of MR:
MR is the mixed visual and audio content for both
AR (such as image, sound, smell, heat. The left side of the figure below) and
AV (Virtual world overlaid with information from real world obtained with such as camera and sensors. The right side of the figure below)
which is captured, rendered and mixed by graphics and audio engines.
Fig. ９ MR as mixed of AR and AV
In this case, actual images are supposed to be acquired from the camera. This is a bit different from the current one.
Adriana et al. (2009) sophisticated the definition of MR:
MR is the merging of real and virtual worlds to produce new environments and visualizations where physical and digital objects co-exist and interact in real time.
They removed the somewhat confusing concept of AV from MR and added real space instead (the meaning has not changed). They also emphasized spatiality, interactivity and real-time nature.
５） MR and physics education
Fig. １０ Physical phenomena
with additional image
In many cases of physics study, we learn the mechanics of phenomena around our everyday life.
Therefore, if we can add information such as images of vector with color to the real world around us (e.g. running car, current carrying electric wire, air of a room), it is efficient for physics study.
References of AR visualization ***
Comparison with AR and MR ***
So, MR can do this, that it is suitable for physics study.
Unfortunately, I couldn’t add information directly to real phenomena in this research, but I created MR teaching materials able to operate and experience virtual phenomena.
2. Purpose of research
The purpose of this research is, to confirm the requirements that the teaching application and lesson program using MR should satisfy by performing lesson practices, for learning phenomena cannot be seen and are difficult to understand but popular around everyday life especially learning magnetic field.
3. Methods
１） Improvement of teaching application and lesson program
I confirmed problems in every lesson to analyze responses of students and the results of the post questionnaire. Afterwards, I improved the teaching application and lesson program to overcome confirmed problems.
２） Evaluation of efficiency
Learning effect
I carried out learning tests to write directions of magnetic field before and after lesson, and compared scores.
Degree of satisfaction
I measured students' satisfaction of MR lesson by NPS (Net Promoter Score).
Fig. １１ How to calculate NPS
NPS is the index to measure attachment, reliance and satisfaction degree of corporation, brand and service. It has been widely adopted with more than two thirds of Fortune 1000 companies using the metric (Jennifer, 2016).
It is calculated based on responses to a single question:
"How likely is it that you would recommend our company/product/service to a friend or colleague?"
It can be as low as −100 (everybody is a detractor) or as high as +100 (everybody is a promoter). In many case NPS is negative (i.e., lower than zero), and positive NPS (i.e., higher than zero) is felt to be good, and an NPS of +50 is top level in the industry.
4. Results
１） Improvement of teaching application and lesson program
Version 1
Fig. １２ Appearance of version 1
Description
Only 3-dimensional magnetic force lines are drawn around a virtual bar magnet. Because it is a stereoscopic image of MR, it can be watched from any angle by walking around freely. But the magnet cannot be moved.
Practice
I provided a demonstration in HoloLens meetup Osaka the first time.
Problems
After watching 3-dimensional magnetic force lines, it is still intuitively difficult to understand the circumstances of a magnetic field.
There is little reality because the magnet is unable to move and the magnetic field does not change.
Version 2
Fig. １３ Appearance of version 2
Description
A user can move a virtual bar magnet limitedly by moving the paper on which special pattern is printed. The application doesn't draw magnetic force lines but arranges many virtual azimuth needles 3-dimensional grid pattern. Each azimuth needle points in each direction according to the position of the bar magnet. Azimuth needles are bright where a magnetic force is strong, and dark where a magnetic force is weak.
Therefore, this teaching material can express both the direction and the strength of a magnetic field.
In addition, when an azimuth needle rotates, it makes sounds.
Improvements
Since magnetic force lines are difficult to understand, I introduced azimuth needles representing the state of the magnetic field (At first FPS - Frames Per Second - fell to 5, but I improved performance). It is easier to understand for users who are able to move the bar magnet, so I made it possible. I adopted the AR marker for this reason.
Response
I tweeted a new app's video, and the tweet was retweeted 400 times. Also, I was interviewed by net news dealing with VR.
Practices
Fig. １４ Exhibition of DIY mindset (Osaka Maker's Bazaar)
I exhibited the MR teaching material at the exhibition of DIY mindset (Osaka Maker's Bazaar) and 60 visitors experienced it.
Problems
It was confirmed from experienced people's responses and free description on a questionnaire that there were problems as follows:
Operability and visibility are not good
The operation is not stable (The recognition of AR marker is often lost).
Direction such as sound is not good (Noisy).
Version 3
Fig. １５ Appearance of version 3
Description
A user can move a virtual bar magnet by a gesture to move his/her hand in front of him/her. Azimuth needles are arranged in 2-dimensional and 3-dimensional grid pattern which rotate with the movement of the bar magnet and change their brightness with the strength of the magnetic force. In addition, when a user operates the bar magnet, it makes sounds according to the operation state.
Improvements
Because the operability of the bar magnet was bad, I changed the operation method from AR to hand tracking. And also, I added the operation sound to improve operability on.
Practices
Fig. １６ Exhibition of DIY mindset (Maker Faire Tokyo 2017)
I exhibited the MR teaching material at the exhibition of DIY mindset (Maker Faire Tokyo 2017) and 54 visitors experienced it.
Fig. １７ Suzuka High School
Because responses were very good, I next practiced lessons in the high school (Suzuka High school, Mie, Japan). 73 students took my lessons.
Responses
The teaching material was experienced individually in the exhibition, and the response was very good. So, I finished the individual phase and entered the lesson practice phase. I practiced lessons using MR teaching material in a high school, and found a few problems.
Problems
There were problems as follows:
The live video on a projector is small and difficult to watch (Objects displayed must be bigger).
Because there is a long time-lag between the experience and the live video displayed by projector, it is difficult to catch up with the lecture (The time lag must be shorter).
The behavior is slow (The program must be optimized more).
Version 3.5
Fig. １８ Appearance of version 3.5
Description
I added the features below to version 3.
There is a gradation of color and it moves from the south pole of the azimuth needle to the north pole. A user can display magnetic force lines.
In the scene of 3-dimension, at first, the bar magnet automatically does reciprocating motion slowly. A user can observe how azimuth needles rotate by the reciprocating motion of the bar magnet. This makes the understanding of a 3-dimension magnetic field easy.
In addition, I let students confirm the behavior of real magnets before and after the experience so that they could connect real and virtual.
Improvements
I made the Azimuth needle bigger to make it easy to watch the live video on the projector. I changed the way of displaying the live video on the projector and make the length of time lag shorter. I optimized the program (I wrote the shader directly) to make it work faster.
I also improved as follows:
Display of magnetic force lines
Addition of Azimuth needle animation
Automation of the reciprocating motion of the bar magnet in 3-dimensional scene
Practices
Fig. １９ Mie High School
I practiced lessons in the high school (Mie High school, Mie, Japan). 52 students took my lessons.
Figure X Prefectural Aichi High School of Technology and Engineering
I practiced lessons in the high school (Prefectural Aichi High School of Technology and Engineering, Aichi, Japan). 29 students took my lessons.
I practiced lessons in the junior high school (Inagi Civic 6th Junior High School, Tokyo, Japan) (No picture). About 180 students took my lessons.
Fig. ２０ Kyoshin School One
Yokkaichi Tokiwa
I practiced lessons in the private-tutoring school (Kyoshin School One Yokkaichi Tokiwa, Mie, Japan). 5 students took my lessons.
Fig. ２１ IT College for Handicapped
I practiced lessons in the IT skill development school for handicapped people (IT College for Handicapped, Mie, Japan). 19 students took my lessons.
Responses
Lessons were popular in all schools. In addition, because the satisfaction of experienced students was much higher than other students, I let all students experience, so that the global satisfaction became higher.
On the other side, many students got interested in magnetic field. For example, many junior high school students said "I am looking forward to lessons on magnetism".
Problems
When I let all students experience MR, they see similar live video many times, so that they get tired in about 20 minutes (A devise to avoid students from getting tired is necessary).
Version 4
Fig. ２２ Appearance of version 4
Description
I added the features below to version 3.5.
More than one student can see and operate the same 3-dimensional image together. When a student moves his/her bar magnet, other students can see the movement. Because students each operate one bar magnet, the number of bar magnets in the space is same as the number of students. In brief, students share one mixed reality space.
Azimuth needles arranged in a grid pattern are affected by more than one bar magnet, so rotate and change brightness in real-time. Also, magnetic force lines are affected by more than one bar magnet, so the form changes in real-time.
Improvements
Because students share the same mixed reality space, the live video on the projector is not similar but changes every time, so that I can avoid students getting tired.
Practices
I provided a demonstration in Asahi Shimbun Publishing Co. (dealing with newspaper). Two members of staff experienced the lesson.
I provided a demonstration in Benesse Co. (dealing with education). Six members of staff experienced the lesson.
Fig. ２３ Tsu Higashi High school
I practiced lessons in the high school (Tsu Higashi High School, Mie, Japan). 31 students took my lessons.
Responses
Each demonstration and the Lessons were popular.
As my objective impression, the dimension of experience would change higher when people share same mixed reality space.
２） Learning effect
At Suzuka High School where I took the first lesson, the score of the learning test rose 16% after the lesson. At Mie High School, the score rose 40%. At other schools where I carried out learning tests, scores rose too.
Satisfaction score of experience
The NPS answer distribution at the exhibition on June 2017 (Osaka Maker's Bazaar) is as below:
Fig. ２４ NPS answer distribution at Osaka maker's bazaar
NPS was -41. It can be said that the satisfaction level was low.
Then, NPS answer distribution at the high school on February 2018 (Mie high school) is as below:
Fig. ２５ NPS answer distribution at Mie high school
NPS was 0. It can be said that the satisfaction level was normal or a little high.
Then, NPS answer distribution at the high school on March 2018 (Prefectural Aichi High School of Technology and Engineering) is as below:
Fig. ２６ NPS answer distribution at Prefectural Aichi High School of Technology and Engineering
NPS was +50. It can be said that the satisfaction level was very high.
Thus, by improving the teaching material application and the lesson program, it turned out that the students were satisfied in very high level.
5. Conclusion
１） The requirements of teaching application and lesson program about magnetic fields with MR
Based on the above results, the requirements to be satisfied by the teaching materials and the lesson program on the magnetic field using MR in actual class can be summarized as follows:
Because It is difficult to understand with only magnetic force lines, another devise to express a magnetic field (i.g. azimuth needles arranged in a grid pattern) is required.
Because a static magnetic field is difficult to understand, it must be possible to experience a dynamic magnetic field.
２） The requirements of teaching application and lesson program with MR
In addition, more generally (even in unit learning other than magnetic field), in order to do lesson using MR in an actual class (i.e. 50 minutes 40 people), the following devises are required:
If the operation is not stable (i.e. the marker tracking is often lost), the experience becomes annoying.
To display to many people by projector, objects displayed must be big
Because the time lag between experience and live video displayed by projector arrests understanding, it must be short.
If the behavior is slow, the experience becomes annoying.
To let the satisfaction be at high level, it is necessary that everyone experience MR.
If students see similar live video many times they get tired, so a devise to avoid them getting tired (i.e. more than one student can share one mixed reality space) is required.
３） Learning effect and degree of satisfaction
The scores of the learning test rose. It can be said that the teaching of magnetic fields using MR has learning benefits.
Many students got interested in magnetic field. It can be said that the lesson of magnetic field using MR is effective to attract students' interests. Therefore, for example, to perform MR lesson at the beginning of the unit considered to have a good effect.
About the degree of satisfaction, it was very high when students took lessons which satisfy the aforementioned requirements. The timing when we can carry out the lesson of magnetic field using MR is limited, but the lesson can be said to be better to carry out at the timing.
４） Another problem and solution
MR headsets are still in the development phase and very expensive. The price is predicted to decrease, but it is considered difficult for schools to purchase them in a few years. Therefore, the way that organizations able to carry out lessons using MR independently deliver lessons to schools, is considered to be realistic.
6. Future work
The true value of MR in learning physics is that, by the addition of stereoscopic image to physical phenomena which we cannot watch, we can visualize those phenomena. It is considered that if real world and stereoscopic image are connected with each other, we can feel high reality.
Presently, I visualize the magnetic field, but the bar magnet does not exist in the real world. In other words, I don't visualize the magnetic field around a real bar magnet.
The future work is to provide experiences with higher reality by a directly visualizing the magnetic field around a real bar magnet.
Citations and references
Adriana de Souza e Silva, Daniel M. Sutko： Digital Cityscapes: merging digital and urban playspaces. New York: Peter Lang Publishing, Inc., 2009
Dieter Schmalstieg and Daniel Wagner, First steps towards handheld augmented reality, Seventh IEEE International Symposium on Wearable Computers, 1530-0811(2003), 127-135
D.E.Hughes, Defining an Audio Pipeline for Mixed Reality, Proceedings of Human Computer Interfaces International, Lawrence Erlbaum Assoc., Las Vegas (2005)
Jennifer Kaplan：The Inventor of Customer Satisfaction Surveys Is Sick of Them, Too, 1, Bloomberg.com, 2016
P. Milgram and A.F. Kishino, “Taxonomy of Mixed Reality Visual Displays,” IEICE Trans. Information and Systems, vol. E77-D, no. 12, 1994, pp. 1321-1329.
Ronald T. Azuma, A Survey of Augmented Reality, Teleoperators and Virtual Environments 6, 4 (1997), 355-385.
