Project 2 - (Don't Fear) The Reaper


Date for telling me you in a 2 person group Friday September 24 at 9pm Chicago time
Application Due Saturday October 16th at 9pm Chicago time
Documentation Due Monday October 18th at 9pm Chicago time


This second project is focused on immersive Virtual Reality at room scale. This project will give you some experience with building virtual worlds for people to move around in and interact with, seeing that world at both human scale as well as other scales. It will also give you a chance to experience the world that you create through physically walking, and moving your head and hands to directly interact with the space, as well as more mediated controls allowing you to teleport or interact at a distance.

One fairly common 'serious' use of VR is to help people deal with their fears, so we are going to explore that as you help design the new offices for ScareCo, a startup that specialize in scaring people for fun and profit. I am supplying you with a starter office space that can run by default in the Oculus Quest headsets in the classroom and in a desktop Unity simulator, allowing you to walk or teleport around the space, and interact with some of the objects. Your job is to augment this space with additional models, sounds, lights, interactives to set up the space to help a specific client deal with their specific phobia.

For example if the company was helping a person with a fear of spiders they might put a really big inflatable moving spider in the main area, or if they were helping a person with a fear of heights they might make use of the ladders and upper walkways to get the person moving at height, or if they were helping a person with a fear of dodgeball they could create a court in the main area with lots of angry red balls flying around, or if its an issue with water they could put a big tank in the space, or if its polka music there might be musicians and giant accordions, etc, etc.

ScareCo
      Starter Scene



Implementing the Project

You will be implementing the project in Unity 2019.4.28f1 (as with Project 1) and VRTK version 4.

One of the biggest issues over the last 30 years in VR development has been the lack of standard higher level libraries, and the difficulty in getting code to run across multiple platforms. Right now VRTK V4 is the probably the best attempt at this, even though it has been in beta for a couple years now. VRTK also has a very nice simulator, letting you do a fair amount of development in Unity itself without a headset. However you will need to test on a real headset. Its impossible to create immersive worlds without testing them immersively as they are so tied to how our bodies inhabit and interact with space. Your projects will be graded based on how they run in the Quest headsets in class, not in the simulator so please give yourself enough time to test.

You should be familiar using Unity and GitHub from Project 1. VRTK has a variety of documentation on GitHub https://github.com/ExtendRealityLtd and various good videos on YouTube here where the braindump videos are live demonstrations of getting small projects running in Unity. There is also a fairly active VRTK community on Discord. Its important when you are looking for documentation that you stick with the V4 documentation.

There is a very good starter tutorial building a simple bowling game that I highly recommend people go through. Because VRTK is designed to run on multiple platforms there has been a lot of work abstracting concepts away from particular hardware, so it can seem rather convoluted at first, but once you get things running you get a feeling for how things work.

We will be spending some time in class helping people go through these tutorials and helping people get things running on the headsets and in the simulator.

The starter project is available here.


Before you try and open it you should install a recent version of Blender - https://www.blender.org/ - on your computer (2.93 is probably safe) as it can be helpful in importing models into Unity

You can add this project to Unity Hub and take a look at it in Unity. Grab the ScareCoOffices scene and drag it to your hierarchy. Delete the default scene. When you play the scene you should be able to move around with the WASD keys, use the mouse to look around, use the Q key to teleport around, use the E key to turn some of the lights on / off with the switch by the door. You can use the 2 key and 3 key to take control of the simulated left and right hand controls respectively. If you place one of the simulated controllers into one of the red balls or one of the mugs you can use the left mouse button to pick them up. The 1 key takes you back to controlling the player.

If you plug in one of the Quest headsets in the classroom, switch the platform to Android, turn off the CameraRigs.SpatialSimulator and turn on the CameraRigs.UnityXR, you can build and run the same scene on the headset and move around with your body and interact with the real controllers.

The numbers below are for a single person project. You can do Project 2 alone or in a 2-person team. If you work in a 2-person team then all the requirements are doubled (aside from the frame rate which remains the same).


60% of the points on the project is for creating a basic scary scenario for one phobia in the space with

40% of the points on the project are for ...


Graduate students in the class also need to be able to ...

and in all these cases the
frame rate should remain high (at least 50 frames per second in stereo on the Quest 1 headsets)

The phobias that you pick could be typical ones like spiders, heights, talking to a group of people, or they could be more entertaining ones like 'the 80s' or 'my cousin's birthday party'. If your phobia is atypical then please check with Andy first to make sure its OK. Your phobia and its scene should also be PG rated and generally not offensive - VR can be a great way to engage people in discussions about complex issues but its very difficult to explore those issues in depth in a 5 minute presentation, so please keep your topic casual. Humorous atypical phobias are encouraged. Your phobia needs to be implementable within the given office area. The main open area is a good place to deploy your phobia, but depending on what the phobia is other locations in the office area are OK as well.
You can not change the basic structure and size of the room, though you could raise the ceiling height if you want. You can put new things in front of the walls, or hanging from the ceiling. You can remove some of the cubicles if you want but the space should still be able to function as an office. You can change the colors and textures on most anything.

Be careful when you are collecting or creating models for your space as the polygons start to add up and you want to make sure you maintain a good interactive frame rate in stereo on the Quest 1 headsets. For measuring and documenting your frame rate we are going to use the OVR Metrics Tool which is discussed here and there is a nice YouTube video here. It should be pre-loaded on the main classroom Quest and you can see it as an overlay while you are testing your application.
The models that you create on your own need to made of multiple geometric parts and should look like what they represent but do not have to be AAA game quality. The laptop on the front desk is a typical example - its made of multiple simple parts, is the right shape and size, has decent colors.

Also, keep in mind that your objects can be at different scales - some of them might be the size of tables that help fill the space but others might be smaller objects on a table, but they all need to be visible at some point (i.e. they could be hidden in drawers). You also want to encourage the user to look around, so the ceiling and the floor are also good places to put objects.



Avoiding some common problems:



Getting your Unity project to run on the Quest headset:

- connect the Quest headset to your laptop via a suitable USB cable (one that passes data)
- put on the Quest headset and use the controllers to tell the Quest to accept the PC connection and always allow that connection.



If you don't see that appear in the Quest then you should make sure that developer mode is on. It should be on for all the evl headsets. Click on the Quick Settings menu on the left side of the thin menu, then click on the settings menu on the upper right, then scroll down in the menu on the left to Developer and turn on the USB Connection Dialog.

   

- in your Unity project hierarchy make sure:
    - you have CameraRigs.UnityXR ENabled
    - you have CameraRigs.SpatialSimulator DISabled

- under Unity Build Settings make sure:
    - you are building for the Android Platform
    - the particular Quest you connected should show up under the Run Device list of compatible connected devices. If the Quest headset does not show up in the list you may need to unplug and re-plug the USB cable and again tell the quest to accept the connection to the laptop

you may want to save the project and restart unity to make sure Unity reconfigures itself appropriately

- click on Build and Run

it may take a while (5-10 minutes) the first time while Unity converts all the assets to a form suitable for the Quest. Next time it will be faster.

- disconnect the USB cable from the Quest (so you don't pull your laptop off the table, or trip on the cable while in VR)
- put on the Quest headset and grab the two controllers
- make sure you have enough space around you so you wont hit anything or anyone
- click on the application menu (9 dots in a grid on the right of the thin menu) then in the upper right of the big menu click on all Applications, then Unidentified Developers at the bottom of the list and click on your application to start it

 

Once your application is running:
- make sure your hands / controllers are tracking
- make sure you can physically turn around and walk around and the world reacts correctly. If not you may still be running the simulator configuration and you need to go back to Unity and make sure you are only running the XR Camera Rig.

The Quest has Stationary (sitting in a chair) or Room Scale (walking around) tracking. While you can test in Stationary mode, the app will be demonstrated and evaluated at Room Scale in the classroom so you should do a fair amount of testing in that mode.

If you want to remove your project from the Quest then you can go back to Apps / Unidentified developers and find your application. If you click on the kebab 3-dot menu to the right you can uninstall the application.

If you want to cast your application to one of the walls in the classroom (or to your phone) to show your work to others then go to the Sharing menu on the right side of the thin menu, then choose Cast from the upper left of the Sharing Menu, and then pick the destination device. The headsets should all be on the evl-class wireless network giving them access to the Chromecast. You can check by clicking on the Quick Settings on the left as before.
 

If you want to take screenshots on the quest for your documentation you can quickly press the Oculus button on the right controller and then press any button on the left controller. You will see a dialogue telling you that a new photo was added to the collection. If you connect the Quest to your computer and you have allowed Unity to install apps on the Quest then you should be able to see the files on Windows, or use Android File Transfer (https://www.android.com/filetransfer/) on the mac. You can see all the snapshots under Oculus/Screenshots/



Turning in the Project

You should create a GitHub page for your project. You can integrate Unity with git so that it will track all your changes, or just regularly push files to git. The final project will need to be turned in via git so we know the timestamp on the files, but it can be also helpful to have regular commits in case something goes wrong so you can get partial credit. Initially this repository should be private to yourself, and then you can make it public for turning it in.

Note that there is a very big difference between getting something working and getting it working well. The first is not that hard. The second takes much more time. You are expected to have things working well.

There are two due dates for the project.

The unity source and application is due first. This will be turned in via GitHub by making your repository public to at least everyone at UIC.  Be sure to email the location of your repository to Andy before the deadline.

The second deadline is for the documentation.

You should create a public web page with multiple sections (visible to anyone for at least the duration of the course) that describes your work on the project. You can host your web page at UIC (http://people.uic.edu), GitHub, or the provider of your choice, as long as it remains publicly available to all. You can use any publicly available templates as long as you cite them, or create your own.

This page should have several sections including:
  • introduction and description of how to use your application and the things you can do with it. What is the phobia you are working with? How would this space help a person work through that phobia?
  • link to your git page that allows someone to easily download the source code to your entire project to be built and run on the Quest. This page should have instructions on how to build your application and list the supported version numbers of all relevant software (Unity, VRTK, etc.).
  • listing of the source for any assets (models, textures, sounds, music) that you used that you didn't create yourself. and a list of which models satisfy which project requirements (e.g. if 15 models are required, or 5 need to be grabbable, then please give us a numbered list of them including showing them in the scene)
  • show a screenshot, or several, showing your app running in the headset with the frame rate showing and describe how the frame rate changes in different areas of your scene or during different activities
  • link to a 5 minute video showing off your project (see below)
  • at least a one page / 500 word discussion of how viewing and interacting with your world is different in the simulator and the headset. 

all of which should have plenty of screenshots with meaningful captions. Web pages like this can be very helpful later on in helping you build up a portfolio of your work when you start looking for a job, giving you something to point potential employers at, so please put some effort into it.


My sample scene included these resources:
  • UFO model by hana - https://3dwarehouse.sketchup.com/model/273a556e078133532296d88107d065f6/UFO
  • UFO sound by Daniel Simion - https://soundbible.com/2213-Alien-Spaceship-UFO.html
  • Cow model by Max M - https://3dwarehouse.sketchup.com/model/288b4047-1c6b-43e5-9b76-eb14ff9152fa/Cow
  • Roswell Logo from the Roswell New Mexico site - https://roswell-nm.gov/FormCenter/Convention-Civic-Center-9/Submit-a-Community-Event-53
  • bounce sound by Mike Koenig - https://soundbible.com/1120-Bounce.html
  • Mars Texture from NASA - https://nasa3d.arc.nasa.gov/detail/mar0kuu2
  • Typing on Keyboard sound - https://soundbible.com/358-Typing-On-Computer-Keyboard.html
  • Office chairs and tables - https://www.allsteeloffice.com/design-resources/resources/2d-3d-models?brand=Allsteel
  • avatar idle animation from Mixamo.com


You should also create a 5 minute YouTube video showing the use of your application including narration with decent audio quality. That video should be in a very obvious place on your main project web page. Capturing video on the classroom PC while interacting with one of the headsets would be best (and get you the most points), but you can capture interaction from the desktop simulator. You can try to narrate while interacting but you will most likely find its useful to do some editing afterwards to tighten the video up. This video is helpful for me to know which parts of your project work, and may be useful in the future when you want to show off some of the projects you did but you can't get the code to compile anymore or you don't have the hardware handy.

Once you have your web page done, send the URL to Andy before the deadline. I will respond to this email as your 'receipt'.

We will be linking your web page to the course notes so please send Andy a nice representative jpg image/photo of your application running highlighting your chosen phobia for the web. This should be named p2.<your_last_name>.jpg and be roughly 1024 x 768 in size.




Presenting the Project

An important part of creating VR applications is getting feedback and using it to improve your design. This also allows you to see how others approached the problem, and maybe they solved some problems that you had issues with, and vice-versa, so it helps everyone get better for the next project.

We will be spending time in class for each person or group to show off their work live using one of the Oculus headsets streaming to the big classroom wall. Well before your presentation you should move a copy of your code including the apk file over to the class VR PC. On the Desktop there will be a CS428 Fall 2021 folder. Make yourself a new folder in there named with your name, download a copy of your code in that folder. Load up your project in Unity and make sure it runs in the headset before your time to present so we can move quickly through the presentations without having to wait for Unity to convert the files.

See the Week 9 and 10 notes for details on the in-class presentations.



Links to all of the Projects:


10/19

Name
GitHub
Web page
Photo
Chen
link

drive
link
Kobylarz
link link
Love
link link

Bui
link link
Lopez
drive link
Prince
link link



10/21

Name
GitHub
Web page
Photo
Parente
link link
Laughton
link link
Shed
link link
Cole
link link
Larco
link link
Kumar
link link
Husnic
link link



10/26 (presentations in Room 3036 ERF, 1 floor up, above where CAVE2 is located)

map
      from 2068 to 3036 ERF

Name
GitHub
Web page
Photo
Woo
&
Janowski
link link
Brandt
&
Tiemann
link link
Kao
&
Kim
link link
Danciu
&
Trufin
link
link
Kapur
link

Vukovic



Egbesemirone






10/28 back in the usual classroom

Name
GitHub
Web page
Photo
Diaz
link link
Mo
link link
Singh
link link
Shah
link link
Soliva
link
link
Channesh







last update: 10/20/2021 - added map to 3036
10/18/2021 - removed the fps screenshot requirement since there isnt an easy way to do that these days
10/16/2021 - added in GitHub links
10/7/2021 - added some more notes on uninstalling apps
10/5/2021 - added new warning about directory paths affecting your ability to grab in the simulator
10/4/2021 - added in the presentation slots
9/30/2021 - added instructions for getting your app running on the Quest
9/22/2021 - fixed GitHub link to the VRTK Bowling Tutorial