Week 11


Collaborative Virtual Reality

People have been sharing virtual worlds of different kinds for quite a while now

In this class we are going to focus on shared / collaborative Virtual Reality worlds (3D displays, head and hand tracking) which tend to be less massive in terms of the number of participants, but richer in terms of the interaction with the virtual world and the other participants. Many of the same issues apply to collaborative Augmented Reality worlds.

One way to do this is to have a large shared display like a CAVE or CAVE2 showing a single virtual world where multiple people in the same location share that virtual world where one person controls the viewpoint but one or more people might have interaction control. The participants see each other's real bodies, and talk to each other directly. This is co-located collaboration.

If the participants are not co-located then more infrastructure is needed. In this case each participant is in their own virtual world (whether via HMD or a large display) and is connected via a server to other participants sharing the same space. Participants have avatar bodies - representations of their tracked selves in the virtual world for others to see - and voice (and / or video) is streamed between the participants.

Back in week 3 we showed a video on Dactyl Nightmare from 1991 where players wearing HMDs could see each other and shoot each other  https://www.youtube.com/watch_popup?v=dji9YiPZ4AM

Local Work from the mid to late 1990s exploring a range of options:

Caterpillar - audio and video on moving screens that showed the position of the various collaborators - the screens were located where the users were located in the space, and the videos always oriented themselves (billboard) towards the user on the local site. The lack of articulated avatars made it difficult to point in the virtual space, but the trade-off was that you had a richer view of the expressions of the remote users. At the time available bandwidth only allowed low resolution greyscale video, but today you could stream much higher resolution as in skype or google hangout. Video of this kind works very well with fish tank VR setups, less well with room scale setups, and not very well at all with HMDs. shared VR with NCSA and Caterpillar
Virtual Temporal Bone - full body avatars would get in the way of this educational environment so only different coloured pointers are used showing where the user's tracked wand is in the space. Full body avatars also tend to give a sense of up and down which may not make sense in worlds such as this. Audio connections between the remote sites allow the doctor to explain the anatomy and students to ask questions, while being able to point at what they are talking about. This kind of collaboration works fine in any VR display. Shared VR for teaching with the Virtual Temporal Bone
CALVIN - simple articulated avatars were used in this design environment that encouraged people to work at different scales to set up a configurable room. Tracking the head and hand of the user and providing an audio link allowed people to see where each other were looking and pointing. This kind of collaboration works fine in any VR display

Shared VR for architectural design - CALVIN
NICE - more articulated avatars in an educational setting, and usage by more casual users (issues of being able to see yourself, issues of 'equality', emergent social patterns)


and a short paper about the issues of scaling this up a bit (nametags, colored costumes, etc.)


Shared VR for education - NICE
TIDE - pointers with static photographs attached. One issue with pointers (and avatars) is knowing who is who. Attaching a name to a pointer is one solution where you can quickly talk to the person with the appropriate pointer since you see their name (assuming a shared character set). If you already know the people in the collaboration you can use photos to identify the pointers.

Shared VR for Visualization - TIDE
Round Earth - participants in a shared experience may not have the same capabilities in the shared world. Here one person was on the surface of a small asteroid moving forward / back / left / right, while the other orbited above the asteroid. Having a body for the astronaut on the surface allowed the person guiding from orbit to know where the person they were guiding was located and which way they were facing (though a bright arrow made this more obvious). Negotiating and remapping directions were key to accomplishing the mission.
Shared VR for education - Round Earth

You can also scan people in and have 3D realistic, but not mobile, 'statue' avatars

Creating static realistic avatars

Creating static realistic avatars

Today you can also get yourself scanned in a 3D scanner and generate an articulated avatar of yourself

for example - https://www.youtube.com/watch_popup?v=DYllOiFmwdc

Some more avatar bodies:

- you may want to choose one that shows where you are from

example avatar representing a city (Dennis Rodman for Chicago)

- another to make you look a bit more 'professional' - note the virtual 'name tag' hovering over the avatar to help identify who this is and where he/she is looking

more professional avatar with nametag

One issue with avatars is that they tend to level hierarchies during collaborations, so the 'boss' is less obvious. This can be a serious issue as some people will not take part in these kinds of activities if it lowers their perceived amount of power. If someone has invested a lot of money in their appearance they sometimes are very unhappy when that disappears into a goofy avatar. For these people video is the only way that they will collaborate over distance.

Avatars moving through small spaces

Another issue to keep in mind when you have multiple avatars in a space is whether they will collide with each other and/or the walls and objects in the space. Trying to get a bunch of people moving down a compact hallway can be extremely challenging for people - think about a group of people all trying to get off an elevator at the same time.

Avatars in Augmented Reality

Just as one can have computer generated avatars in virtual reality, one can have computer generated avatars in augmented reality, though these avatars may be more focused on communication rather than collaboration since your physical world most likely does not match their physical world.

One can also imagine taking all of the various webcam overlays and having those appear on your real self through everyone else's AR displays (re Ghost in the Shell) though you would still look like yourself to everyone without AR.

One could also imagine people wearing holographic projection systems, or clothing made of light emitting fabric as in Samuel Delaney's Dhalgren (1975) where people without special equipment can see you as someone or something else. 

Heterogeneous Perspectives

heterogeneous perspectives in VR collaboration

People may not be seeing the same exact shared world, especially in science and engineering domains.

Asynchronous Work

While collaborative VR across continents is rather easy to do technologically, somebody has to stay late or come in early to collaborate synchronously with people on the other side of the planet. Sometimes asynchronous collaboration is better, but how do you coordinate your work?

One way is to record your voice and gestures in the virtual world and then allow others to play them back later on

Virtual Harlem / V-Mail

asynchronous VR collaboration

Some current consumer collaborative applications

Black Hat Cooperative and Keep Talking and Nobody Explodes are good examples of local cooperative games where one person wears an HMD and performs tasks in the virtual world and another person (or persons) helps that person with their tasks given a set of knowledge that the person in the HMD doesn't have -

Black Hat Cooperative

Keep Talking and Nobody Explodes


Bridge Crew is interesting because you have four seated participants sharing the same virtual space who can talk to each other and gesture, while solving a common set of tasks - https://www.youtube.com/watch_popup?v=tRFVmTAhhac

Rec Room is a collaborative VR space to play games with other people from around the planet - https://www.youtube.com/watch_popup?v=Qb0qWWVsvRA

Some worlds are just designed to be social spaces

AltSpaceVR - https://www.youtube.com/watch_popup?v=0I6QNXR0dPY

facebook spaces - https://www.youtube.com/watch_popup?v=_kGRpSd4vnc most famously used by Mark Zuckerberg to 'visit' Puerto Rico after Hurricane Maria - https://www.youtube.com/watch_popup?v=F3S-T5zgRnw

As more and more people have been using the internet to communicate with other we have seen ever increasing amounts of harassment in these shared spaces, and shared virtual worlds are not immune to this. Making this kind of communication possible isn't very hard - making it a useful and safe space for its participants will be the subject of research for many years to come.

Coming Next Time

Student Choice Presentations

last revision 8/6/2020