Week 1 we showed off
a bunch of different sample VR / AR worlds, so this week we are going to
talk about how to build them, in particular with respect to Project 1.
but first lets
talk about the Course Syllabus and Project 1 in some more detail
For the first two
projects in the class we will be using Unity3D so people can share
experiences and issues. For the third project people will be
(reasonably) free to choose your own engine and display platform and can
also use unity for that project if you wish.
Unity3D runs on
windows and OS-X so you can do much of your development on a laptop and
then move over to a machine connected to a VR / AR display for testing.
I highly suggest that you test on the actual hardware regularly
so you don't go too far down a development path that wont work for the
final deployment. In the lab, there is one VIVE in the main lab, and
another in the classroom, both connected to Windows PCs. The one in the
classroom has a webcam attached for AR work.
You should stick with Unity version 2018.2 for now so you are compatible across various platforms. You should start with the Personal Unity Installer, and have it install:
will be presenting and grading all of the projects on the windows PC in
the classroom. If you are on OS-X and forget to install the Windows
Build Support, you can come back and install it later
from the same Install Utility. If you want (or feel the need) to install
other components that should be fine. Be prepared to lose 4-5 gigabytes
of disc space and at least 10 minutes for the install. You can have
multiple versions of Unity installed at the same time in different
directories in case you need to have different versions running for
different classes or projects.
With Unity you will
be using a combination of an IDE and writing C# code, so its a good idea
to go through the introductory tutorials if you haven't used Unity
before (e.g. in the CS Video Game course).
The tutorials at https://unity3d.com/learn are a good place to start learning the Unity3D IDE.
we will use in Project 1)
The tutorial at https://library.vuforia.com/articles/Training/getting-started-with-vuforia-in-unity.html is a nice introduction to Vuforia AR in Unity.
A couple things that may trip you up in this tutorial - if you have multiple cameras attached to your computer then you may need to go to Window/VuforiaConfiguration and see which Camera Device Vuforia wants to use and make sure its the one you want it to use. When you play your application it should show the view from the camera you want.
If you dont see Assets/Resources/VuforiaConfiguration make sure you go to the build settings / player settings and check the box for Vuforia Augmented Reality Supported.
You will probably
have the option to update Vuforia - the installer should automatically
download and then you can point it at the directory where Unity was
installed (on Windows Program Files/Unity563, let it install, and then
At the end of that page they show the Vuforia Astronaut on the Image Target. To start with you could just add a new GameObject / 3D Object / Cube, and scale it so something like 0.2, 0.2, 0.2 and add it as a child of the image target, then start up the app, point the camera at the image target (which you can print from Assets/Editor/Vuforia/ForPrint, or you could search for the image on google and bring it up on another device you have like a phone)
Save that off, and
start a new project, turn on the Augmented reality support, make sure it
is using the right camera, and then go to the asset store in unity,
search for Vuforia
and download and import the free Vuforia Core Samples.
Once you import everything you can go to Assets / Sample Resources / Scenes and drag the 0 - Splash scene into the hierarchy, which gets you access to a variety of demos of different Vuforia features. The Image target one is an enhanced version of the previous tutorial with multiple image targets including the astronaut. You can also move the individual demos to the hierarchy to run them separately. The Image Target, Multi Targer, and Cylinder Target demos may be particularly appropriate to help with Project 1. You may get an error that Vuforia wont initialize. If this happens try going to Assets / Reimport All.
This YouTube video is
a pretty good starting point for working with Image Targets -
This page has some good tips on various kinds of targets - https://gamedevelopment.tutsplus.com/tutorials/vuforia-tips-and-tricks-on-unity--cms-28744
And this is a pretty nice button tutorial - https://library.vuforia.com/articles/Solution/How-To-Implement-Virtual-Buttons.html and a video on the same topic https://www.youtube.com/watch?v=ElmzIq6stNI
You will also need to work with databases as you add your own targets - this is a nice tutorial for that - https://library.vuforia.com/articles/Solution/How-To-Work-with-Device-Databases
As part of this you
will need to register and log into the Voforia Developer portal at
https://developer.vuforia.com/ which gets you a license key and the
ability to use the online target manager. You should create Device
databases for this project so they can be downloaded and included in
your unity build that you turn in.
For parts of Project 1 when you need to determine if the objects are near each other, you should make sure that you go to your AR camera and set World Center Mode to DEVICE, so then the position of all of the tracked objects are relative to the camera.
VR / VRTK (which we will use in Project 2)
VRTK - https://vrtoolkit.readme.io/
- is a nice cross-platform toolkit for VR with Unity that contains
a built in simulator as well as the ability to run with a variety of the
modern headsets. It comes with a very nice set of demos to show off its
You can download the current version from https://github.com/thestonefox/VRTK and open the Project in Unity. By default it will run with the simulator. If you have a VIVE you can import steam vr from asset store and it will start working with the headset and controllers.
This https://www.youtube.com/watch?v=sW9lxEUXfe8 is a pretty good YouTube tutorial on how to start from an empty seen and then be able to add teleportation, and grabbable objects. The tutorial goes fast so be ready to use the pause key. As the comments note, there is one missing step in the tutotial when it comes to making the green sphere grabbable, but that's fixed by just clicking the 'Is Grabbable' check box in the Grab Settings area of the VRTK_Interactable Object (Script) for the green ball.
is a good video on the built-in slmulator so you can do a fair amount of
your development without a headset, except, as noted in the comments,
the VRSimulatorCameraRig is now in VRTK/Source.SDK.Simulator, but again
remember to test on the actual hardware before you turn your project in.
Aside from the
) there are a bunch of nice videos on their YouTube Channel - https://www.youtube.com/channel/UCWRk-LEMUNoZxUmY1wO7DBQ
showing off various features.
We are also now
starting to see browser-based VR solutions such as A-Frame and WebVR- https://mozvr.com/
so this or the Unreal engine may be things to consider for Project
are several other AR libraries available right now with Google's ARCore
and Apple's ARKit https://developer.apple.com/arkit/
and the venerable ARToolkit https://www.hitl.washington.edu/artoolkit/
Along with the main engine there are a variety of tools that can be used to create models, sounds, textures, etc. Here you are free to use the tools of your choice. The following are pretty decent free ones.
There are also various websites that
provide Open Source / Royalty Free resources, just be sure to cite the
Here a nice page on importing google
SketchUp models into Unity3d, though grabbing the collada file usually
works pretty well
For my AR demo I
grabbed a UFO from here as a collada model:
and a classic 'grey'
alien from here as a collada model:
for the space image
on the box I used this:
In all your projects,
make sure you have the rights to use the assets you find on the
internet, and make sure you cite their creators.
One important thing
to keep in mind with VR and AR is that, unlike other media, the size of
the screen does not affect the size of the objects in the world. If you
are watching a movie then Humphrey Bogart or Lego Batman may vary
from an inch tall to 100 feet tall depending on the display you are
watching it on. In VR or AR the display is a window into the virtual
world, and changing the size of the window just gives you a bigger or
smaller window, it doesn't change the size of the things the window
looks out on. Scale is absolute in VR or AR, so at some point the
environment you create ends up being scaled in units of feet or meters
for people to move around in. Similarly there are various image based
tricks that can be used in 2D or even 3D movies, that don't work in VR
or AR where the space is really 3D.
In this course you
will be doing a lot of work with Unity and the content generation tools
so make sure you have access to a decent computer that can run these
pieces of software. For the AR assignments you will need access to a
webcam for that computer - built in cameras should be OK but an external
one you can move around will make your life a lot easier.
Week 2 Homework - Due Friday 9/7 at 9pm
By this Friday you
should have Unity set up and running with Vuforia and have gone through
the getting started with Vuforia in Unity tutorial. As part of that you
should have the Vuforia astronaut mascot standing on the Astronaut image
target. Take a screenshot of your astronaut standing on your table and
add that to a new webpage that you add to your 491 site and
a page of text on your initial thoughts about this technology, assuming
you were viewing it through your phone, and then through a pair of AR
glasses that you were always wearing.
History of VR and AR