Project 1 - Saturday Night at the Movies

Due Monday 2/06/12 at 8:59pm Chicago time

The purpose of this project is to learn about GLSL shaders by experimenting with various GLSL shaders.

For this project you should use GLSL so everyone has a common comparable set of code - no CUDA, no OpenCL, no HLSL or other shading languages.

For this project you will use OpenCV to grab video from a webcam and then run that live video through one or more GLSL shaders to manipulate the imagery.

Your program should start off by breaking the main OpenGL window into a 2 x 2 grid with your normal video in the upper left corner. In each of the other 3 positions you should have a modified version of that video.

By clicking the mouse in any of the 4 regions (or pressing Q, W, A, or S), that region should enlarge and fill the entire window. Clicking on the window again (or pressing the Z key) should return the user to the 2 x 2 matrix.

On this first screen you will use 3 different fragment shaders to alter the image, along with a simple pass-thru vertex shader like the one we used in the image adjustment discussion. Some possibilities are:

Doing the first page correctly gets you a C on the project. To get a B you need to add a second page of shaders. The user can use the 1, 2, or 3 to move between the three pages of shaders. On page 2, in addition to your untouched video, you should have 3 more examples:

To get an A you need to add a third page of shaders. On page 3, in addition to your untouched video, you should have 3 more examples of your own choosing where at least one should involve a geometry shader

For all three pages of shaders you can draw inspiration from existing shaders (in GLSL or other shading languages) you find in the class notes, on the web, or in textbooks, but you can't just copy the code. You need to make some modifications to the code that show you understand how it works.

Note that you should not be doing any image manipulation in OpenCV; you can use OpenCV to get information about the video and then use that information to manipulate the video in GLSL.

Also note that there is a difference between getting something working and getting it working well. The first is not that hard. The second takes much more time to get the effect to look good.

In order to get you started on this, especially people who haven't played with texture maps in OpenGL before, here is a short program that demonstrates grabbing a webcam image and feeding it through GLSL.

The biggest common problem with assignment 1 in the past has been waiting too long to start coding. Try to get one of the sample GLSL / C program running by the end of the week, and then you can start adding shaders onto it. If you have trouble with particular configurations (windows x with Microsoft studio Y etc) then ask your classmates.

To turn in your project you should set up a web page or pages describing your work, including the well-commented source code and required files to be able to compile and run your program, and some screendumps showing what your application should look like when its running. You should then email andy with the location of this website before the deadline. It  would probably be a good idea to put a backup copy of the web page at a second website just in case I can't get to the first one.

You should also create a 2-3 minute YouTube video showing your application running and feature the video prominently on your project webpage. The video should be narrated and rehearsed to show off the important features of your project.

As part of this web page you should describe each shader, where you learned about it (e.g. I found this shader on www.yada.yada/etc/etc) and say what changes you made to it. Other people should be able to read your web page and understand how they can make use of all your cool shader ideas. This way we can share these methods with the rest of the class. Please use lots of images and explanatory text. Usually people do not put enough work into the website. Remember that this site may be useful to you later on when you are looking for a job and want to show off the projects you have done.

When you send andy the location of your webpage you should also email a screen snapshot of your favorite shader image that is 320 pixels wide by 240 pixels tall in jpeg format named p1.<your_last_name>.jpg. This image will be used on the class web pages along with the link to your project web page.

OpenCV can take input from a file as well as live camera input so you may want to save off a good set of camera input as a backup for your demonstration. Light levels in the room have a dramatic affect on the quality of the video and the ability of the shaders to process it.

Each student will also give a short presentation about their project in-class and answer some questions about your work. You should prepare a good presentation - you can use your project website, or create some slides. You should definitely run your code and show your shaders to the class. Be sure to practice your presentation so you finish within the allotted time so everyone has equal time to present.

last revision 1/10/12