Week 2

Unity3D and Content Generation Tools

Week 1 we showed off a bunch of different sample VR / AR worlds, so this week we are going to talk about how to build them, in particular with respect to Project 1.

but first lets talk about the Course Syllabus and Project 1 in some more detail

There are various libraries that have been used to create virtual reality worlds, and most of them come and go after a few years as the technology landscape changes.

Personally, I have used a variety of languages and libraries to create VR applications:

With the newer consumer level devices a couple of the free big game development engines now support VR development.

For the first two projects in the class we will be using Unity3D so people can share experiences and issues. For the third project people will be (reasonably) free to choose your own engine and display platform and can also use unity for that project if you wish.

Unity3D runs on windows and OS-X so you can do much of your development on a laptop and then move over to a machine connected to a VR / AR display for testing. I highly suggest that you test on the actual hardware regularly so you don't go too far down a development path that wont work for the final deployment. In the lab, there is one VIVE in the main lab, and another in the classroom, both connected to Windows PCs. The one in the classroom has a webcam attached for AR work.

You should stick with Unity version 2018.2 for now so you are compatible across various platforms. You should start with the Personal Unity Installer, and have it install:

We will be presenting and grading all of the projects on the windows PC in the classroom. If you are on OS-X and forget to install the Windows Build Support, you can come back and install it later from the same Install Utility. If you want (or feel the need) to install other components that should be fine. Be prepared to lose 4-5 gigabytes of disc space and at least 10 minutes for the install. You can have multiple versions of Unity installed at the same time in different directories in case you need to have different versions running for different classes or projects.

With Unity you will be using a combination of an IDE and writing C# code, so its a good idea to go through the introductory tutorials if you haven't used Unity before (e.g. in the CS Video Game course).

The tutorials at https://unity3d.com/learn are a good place to start learning the Unity3D IDE.

AR / Vuforia (which we will use in Project 1)

The tutorial at https://library.vuforia.com/articles/Training/getting-started-with-vuforia-in-unity.html is a nice introduction to Vuforia AR in Unity.

A couple things that may trip you up in this tutorial - if you have multiple cameras attached to your computer then you may need to go to Window/VuforiaConfiguration and see which Camera Device Vuforia wants to use and make sure its the one you want it to use. When you play your application it should show the view from the camera you want.

If you dont see Assets/Resources/VuforiaConfiguration make sure you go to the build settings / player settings and check the box for Vuforia Augmented Reality Supported.

You will probably have the option to update Vuforia - the installer should automatically download and then you can point it at the directory where Unity was installed (on Windows Program Files/Unity563, let it install, and then restart Unity.

At the end of that page they show the Vuforia Astronaut on the Image Target. To start with you could just add a new GameObject / 3D Object / Cube, and scale it so something like 0.2, 0.2, 0.2 and add it as a child of the image target, then start up the app, point the camera at the image target (which you can print from Assets/Editor/Vuforia/ForPrint, or you could search for the image on google and bring it up on another device you have like a phone)

Save that off, and start a new project, turn on the Augmented reality support, make sure it is using the right camera, and then go to the asset store in unity, search for Vuforia and download and import the free Vuforia Core Samples. https://assetstore.unity.com/publishers/24484

Once you import everything you can go to Assets / Sample Resources / Scenes and drag the 0 - Splash scene into the hierarchy, which gets you access to a variety of demos of different Vuforia features. The Image target one is an enhanced version of the previous tutorial with multiple image targets including the astronaut. You can also move the individual demos to the hierarchy to run them separately. The Image Target, Multi Targer, and Cylinder Target demos may be particularly appropriate to help with Project 1. You may get an error that Vuforia wont initialize. If this happens try going to Assets / Reimport All.

This YouTube video is a pretty good starting point for working with Image Targets - https://www.youtube.com/watch?v=MtiUx_szKbI

This page has some good tips on various kinds of targets - https://gamedevelopment.tutsplus.com/tutorials/vuforia-tips-and-tricks-on-unity--cms-28744

And this is a pretty nice button tutorial - https://library.vuforia.com/articles/Solution/How-To-Implement-Virtual-Buttons.html and a video on the same topic https://www.youtube.com/watch?v=ElmzIq6stNI

You will also need to work with databases as you add your own targets - this is a nice tutorial for that - https://library.vuforia.com/articles/Solution/How-To-Work-with-Device-Databases

As part of this you will need to register and log into the Voforia Developer portal at https://developer.vuforia.com/ which gets you a license key and the ability to use the online target manager. You should create Device databases for this project so they can be downloaded and included in your unity build that you turn in.

For parts of Project 1 when you need to determine if the objects are near each other, you should make sure that you go to your AR camera and set World Center Mode to DEVICE, so then the position of all of the tracked objects are relative to the camera.

VR / VRTK (which we will use in Project 2)

VRTK - https://vrtoolkit.readme.io/ - is a nice cross-platform toolkit for VR with Unity that contains  a built in simulator as well as the ability to run with a variety of the modern headsets. It comes with a very nice set of demos to show off its capabilities.

You can download the current version from https://github.com/thestonefox/VRTK and open the Project in Unity. By default it will run with the simulator. If you have a VIVE you can import steam vr from asset store and it will start working with the headset and controllers.

This https://www.youtube.com/watch?v=sW9lxEUXfe8 is a pretty good YouTube tutorial on how to start from an empty seen and then be able to add teleportation, and grabbable objects. The tutorial goes fast so be ready to use the pause key. As the comments note, there is one missing step in the tutotial when it comes to making the green sphere grabbable, but that's fixed by just clicking the 'Is Grabbable' check box in the Grab Settings area of the VRTK_Interactable Object (Script) for the green ball.

This https://www.youtube.com/watch?v=bxxGaJg75g4 is a good video on the built-in slmulator so you can do a fair amount of your development without a headset, except, as noted in the comments, the VRSimulatorCameraRig is now in VRTK/Source.SDK.Simulator, but again remember to test on the actual hardware before you turn your project in.

Aside from the documentation (https://vrtoolkit.readme.io/docs/getting-started and https://vrtoolkit.readme.io/docs/examples ) there are a bunch of nice videos on their YouTube Channel - https://www.youtube.com/channel/UCWRk-LEMUNoZxUmY1wO7DBQ showing off various features.

We are also now starting to see browser-based VR solutions such as A-Frame and WebVR- https://mozvr.com/ so this or the Unreal engine may be things to consider for Project 3.

There are several other AR libraries available right now with Google's ARCore https://developers.google.com/ar/ and Apple's ARKit https://developer.apple.com/arkit/ and the venerable ARToolkit https://www.hitl.washington.edu/artoolkit/ and https://github.com/artoolkit/arunity5.

Content Generation Tools

Along with the main engine there are a variety of tools that can be used to create models, sounds, textures, etc. Here you are free to use the tools of your choice. The following are pretty decent free ones.

There are also various websites that provide Open Source / Royalty Free resources, just be sure to cite the creators.

Here a nice page on importing google SketchUp models into Unity3d, though grabbing the collada file usually works pretty well

For my AR demo I grabbed a UFO from here as a collada model:

and a classic 'grey' alien from here as a collada model:

for the space image on the box I used this:

In all your projects, make sure you have the rights to use the assets you find on the internet, and make sure you cite their creators.

One important thing to keep in mind with VR and AR is that, unlike other media, the size of the screen does not affect the size of the objects in the world. If you are watching a movie then  Humphrey Bogart or Lego Batman may vary from an inch tall to 100 feet tall depending on the display you are watching it on. In VR or AR the display is a window into the virtual world, and changing the size of the window just gives you a bigger or smaller window, it doesn't change the size of the things the window looks out on. Scale is absolute in VR or AR, so at some point the environment you create ends up being scaled in units of feet or meters for people to move around in. Similarly there are various image based tricks that can be used in 2D or even 3D movies, that don't work in VR or AR where the space is really 3D.

In this course you will be doing a lot of work with Unity and the content generation tools so make sure you have access to a decent computer that can run these pieces of software. For the AR assignments you will need access to a webcam for that computer - built in cameras should be OK but an external one you can move around will make your life a lot easier.

Week 2 Homework - Due Friday 9/7 at 9pm

By this Friday you should have Unity set up and running with Vuforia and have gone through the getting started with Vuforia in Unity tutorial. As part of that you should have the Vuforia astronaut mascot standing on the Astronaut image target. Take a screenshot of your astronaut standing on your table and add that to a new webpage that you add to your 491 site and a page of text on your initial thoughts about this technology, assuming you were viewing it through your phone, and then through a pair of AR glasses that you were always wearing.

Coming Next Time

History of VR and AR

last revision 8/18/18