Technical issues
As with the initial study, the children in the main studies exhibited
diversity in their use of the wand, the interaction device.
The instructions given were not exactly the same for everyone
but depended on the situation and the environmental or personal
distractions. Generally,
these instructions started with
showing the representation of the virtual hand to the leader, then the use
of the joystick for navigation, and finally, once the child was able to
move into the garden, the function of the buttons.
Learning the functions of the wand lasted from 2 minutes, for the children
that learned quickly, to 7 minutes.
After learning how to use the wand, the children's effort was focused on orientation, as noted in the following section. Limitations of the physical design of the wand caused discomfort to young users, as both hands were needed to reach the buttons and press the joystick at the same time. The joystick of the current wand is difficult even for adult users and requires applying considerable force when navigating. The children obviously had a problem doing this. It was expected that the boys would generally be better at using the wand, partly because of their strength and partly because of their familiarity with similar input devices from playing video and computer games. According to both parents' and kids' reports, 92\% of the boys play electronic games weekly, as opposed to 42\% of the girls (see \ref{tablevideogame}). The majority of these games have joystick-based interface devices. We did not notice, however, any gender differences in learning to use the wand.
A larger problem was the size of the stereo glasses. Despite the glass-ties used to tighten the glasses on the children's heads, the glasses would still fall off. Most children had to hold the glasses with their free hand and, when tired holding them, would just take them off. Not only did this contribute to the subjects' fatigue, but also to their level of motivation and excitement. Since the stereo glasses and the wand are an integral part of the virtual experience, these limitations are a current hindrance not only to usability but also to learning.
Some of the best ``drivers'' consulted the menu that was attached to the virtual hand. These instances, however, were very few and sometimes inconsistent: some of the drivers who consulted the menu at the beginning forgot about it later, while others remembered to use it only part of the time.
The children's susceptibility to simulator sickness was not as large as expected. Less than 5\% of the subjects complained about getting a headache or being dizzy during or after the experience, and for most it was so slight that they had not noticed until asked. Only a few girls, mainly ImmersaDesk users, felt dizzy during the experience and for longer afterwards. One girl felt slight nausea commencing about 15 minutes into the experience, and lasting for about a half hour.
Evaluation of the system with respect to its robustness and cost effectiveness for broader use must be put off until the system is in a public locale. Nevertheless, it is necessary to mention the large number of human staffing required for these studies. Approximately eleven people helped each day of the classroom evaluations. Part of this number relates to the evaluation procedure itself (videocamera person, guides, interviewer, etc.), but a core number of at least three people, including a technical person, an instructor to handle the glasses and teach the use of the interface devices, and a teacher-avatar are needed for even the simplest case study. The NICE software is flexible enough to eventually expand into a user-authoring system. To be effective, however, it needs to be used by a small number of learners for an extended period of time.
\begin{table}
\begin{center}
\caption{ VIDEO \& COMPUTER GAME EXPOSURE OF SUBJECTS}
\begin{tabular}{|l|c|c|}
\hline
Activity & Boys & Girls\\
\hline
Video-game playing & 3.44 & 1.34 \\
Computer use for school & 4.14 & 2.4 \\
Parent computer usage & 5.85 & 6.16\\
\hline
in average hours per week
\end{tabular}
\label{tablevideogame}
\end{center}
\end{table}
Orientation
After learning how to use the wand, the children focused on
trying to navigate and orient themselves in the virtual environment.
With respect to the classroom groups, this proved to be the effort
of the leader and not of the other children in the group, although their
mission was to help the leader. The drivers were the only ones focused on
the orientation task at hand, as they were the ones navigating, while
the other children were distracted by the movement and the three-dimensional graphics.
The girls seemed slightly better at orienting themselves in the environment, possibly because they were generally
more focused and reserved compared to the boys.
Even with the case studies, although not nearly to the same extent,
there were times when the other child (the one not using the wand)
would wander around, instead of observing or directing the driver's actions.
While it was not expected that all children's full attention would be given
at orientation, the result in these studies was that
each child came up with their own version of the right
direction, voiced them at the same time as the other children and
confused the leader, who then
individually decided which was the right path to take.
As a result, apart from the difficulty in using the joystick for navigation,
the leaders exhibited noticeable individual differences in their
abilities to interact with the 3-D environment.
These differences seemed to relate to their level of ``independence'':
the ones pursuing their own goals did well, while the ones that attempted to
listen to the others in their group ended up confused and disoriented.
A test for spatial orientation was the ability to find areas in the space, such as the hole that leads to the area under the garden. This was a relatively hard task, although there were spatial clues: the passage was located near the only set of trees behind one of the garden fences. These were the only instances where verbal interaction seemed to work well, largely because the goal was very specific and required the kids' complete attention.
Another test for orientation was the concept map -the plan of the garden on paper. In the planning stage, students developed different strategies for planting (\ref{plannedgardens}, Appendix A). We wanted to see how they were able to implement this plan in VR. The case studies were more focused and, therefore, the children attempted to stick to their plan. With the exception of the boy in the initial study, the children were not successful at completing the task. Most children began planting as planned, but then changed their plans when running into difficulty. A younger girl who tried following the plan, commented that it was very hard to be precise in separating the vegetables. The teacher-avatar helped her with directions, but that ``wasn't enough''. The classrooms, on the other hand, hardly even tried to implement the plan, although constantly reminded by the teacher-avatar. Their entire experience was consumed by dealing with the group's behavior. None of the children admitted that they did not try; rather they stated that implementing the plan was a difficult task. One boy, after seeing the look of the group's final version of the garden asked his group: ``how come we didn't get it right?'' to receive the overwhelming response ``because it was very hard!''.
As perceived through observation, most kids felt immersed. This was indicated by their motion and excitement. Almost all children attempted to ``touch'' the virtual objects by moving and clasping their hands in the air. This was particularly noticable in the case of the virtual beam that extended from the user's hand to help point to and select objects. As the beam was always attached to the hand and close to the user, it felt very ``three-dimensional'' to the children. Many children, however, would take their stereo glasses off and put them back on constantly during the experience. One of the boys in the case studies would take the stereoglasses off every time he needed to accomplish a more demanding task, such as finding his way underground. When asked why, he answered that it was easier to see without the glasses because ``they were heavy and I couldn't do things right''. Technical obstacles, such as the size of the stereo glasses, possibly hindered the perception of presence.
The present feedback seems effective, as most children understood the function of the yellow balloon for picking, and the virtual hand. Many leaders waved at the other avatars with the hand that was holding the wand, indicating that they understood the relationship between the wand, their real hand, and the virtual hand.
Affective
Measuring motivation is difficult, as it is indirect. Anderson et al. (1975) note that we do not see motivation, but behavior. Moreover, in the case of virtual reality, motivation is highly driven by other factors, such as the novelty effect, media hype, and social issues. It is significant to look through these factors and try to identify whether the content taught within this medium is motivating for children, what it is that motivates them, and most importantly, for how long. This was difficult, as all of the children were excited before starting, just by the fact that they would experience virtual reality. Therefore, we had to look at their level of engagement during the actual experience.
The amount of time the children spent in VR ranged from 30 minutes to 1 hour and 30 minutes. Each classroom group, due to time constraints, stayed in the experience for about 30 minutes. The case-study subjects, on the other hand, were allowed to stay until they displayed noticeable fatigue, at which point they were asked if they wished to continue. Most cases wished to remain in NICE for at least 45 minutes and started getting tired after one to one and a half hours.
Interactive activities ranked high amongst the preferences of the children, as shown by their responses in the post study questions. Planting was a favorite. An equal number of responses were in favor of the area under the garden. The fantasy was another fundamental driving force for many of the children. Many liked the water (or ``swimming''), the rain, sun, umbrellas and sunglasses, and the vegetables. The three things that were most disliked by the children included ``the stuff that we had to move with'', the ``glasses falling off'', and the fact that some did not get to drive. Most (73\%) of the children answered ``nothing'' to the question ``what did you dislike the most?''.
The most important issue related to motivation is control. As mentioned in the discussion of orientation, the children that were leading were more on-task and engaged, while all others were distracted and unfocused. This was also perceived, to a lesser degree, with the pairs of children in the case studies: the driver was focused on the task even if that meant only navigation, and was consequently more engaged, while the second child seemed less engaged. The post-experience questions verify these observations: Children that were leaders listed that what they enjoyed the most was being the leader, while most others that did not get that chance were very disappointed.