Sangyoon (James) Lee
Assistant Professor, Computer Science
Connecticut College, New London, CT 06320, USA
PhD in Computer Science, MFA in Electronic Visualization
Electronic Visualization Lab.
University of Illinois at Chicago, Chicago, IL 60607, USA
e-mail: s.james.lee at me.com
Computer Graphics and Animation, Realistic Skin Rendering, GPU Programming, Virtual Reality, Affective Computing, Expressive Vritual Human, Autonomous Affect and Behavior Control, User Interaction & Experience Design, Physical Computing, and Interdisciplinary Collaborations with domain scientist and artists.
Ph.D. in Computer Science and Electronic Visualization Laboratory (EVL), University of Illinois at Chicago (UIC), (2006 – 2013)
Thesis “LifeLike Natural Computer Interface” involves the development of user-friendly natural computer interfaces to enable people to interact with a realistic-looking 3D virtual avatar; more specifically, integrating visualization, natural language processing and knowledge base systems to create a real human-like communication interface. LifeLike Responsive Avatar Framework (LRAF) is written in C++ and uses the Microsoft Speech interface to synchronize lip animation with synthesized voice. Character model is designed in Maya with full-body rig for gestural motion and blendshape is used for real-time facial expression. Motion database is acquired via the Vicon Motion Capture system and processed in Autodesk MotionBuilder. Intelligent Avatar behavior model is implemented using boost Python C++ binding to easily apply to other application domains.
Master of Fine Art in Electronic Visualization, UIC (2003 – 2006)
Thesis “PAN – Interactive Visual Installation” used a microcontroller with accelerometer sensor to detect user interaction with custom-built table interface and OpenGL application in C++ to generate abstract visuals. Involved network programming to communicate with multiple devices for collaborative experience.
Master of Science in Architectural Engineering, Yonsei University, Seoul, Korea (1997 – 1999)
Thesis “Development of a 3D Model House for Apartment Sales on the Internet” introduced an online mockup apartment showcase using 3D Max and Virtual Reality Modeling Language (VRML).
Bachelor of Science in Architectural Engineering, Yonsei University, Seoul, Korea (1993 – 1997)
Assistant Professor, Computer Science, Connecticut College (2013 – present)
Guest Lecturer and Hands-on Tutorials, Computer Science and School of Art & Design, UIC (2008 – present)
Gave lectures and taught students to use EVL’s Motion Capture system in Computer Science courses (Virtual Reality 09/2010, Computer Game Design 4/2009, Computer Animation 04/2008) and Art & Design courses (Advanced Electronic Visualization Critique 09/2008, Advanced 3D Modeling and Animation 02/2009); Was a guest lecturer on Advanced Character Design, Modeling, and Animation for Advanced Interactive 3D class (11/2008). Stereoscopic rendering, character modeling, and 3D prototyping for Biomedical Visualization courses (Virtual Reality and Stereoscopy in Biomedical Visualization 02/2013, 02/2012)
Adjunct Assistant Professor, School of Art & Design, UIC (2010 – 2011)
Teach interdisciplinary Interactive 3D I and II courses. Use 3DVIA Virtools and Autodesk Maya for new media art students to create interactive applications without requiring much programming. Students in the classes also used MS Kinect hack, motion capture data, and object tracking via a webcam to implement interactive art pieces.
Teaching Assistant, School of Art & Design, UIC (2003 – 2006)
Assisted the Electronic Visualization and the Design Visualization Laboratories in the UIC School of Art and Design with Windows, Linux, and SGI Virtual Reality Systems. Supported “Virtual Reality”,
Teaching Assistant, Yonsei University, Seoul, Korea (1997 – 1999)
Supported “Design and History of Modern Architecture” and “Practice & Application of Architectural CAD” courses. Taught Autodesk AutoCAD, 3D Max, and lectured on modeling and animation.
Research Assistant and Teaching Assistant, EVL, UIC (2006 – present)
Research Assistant (2006-present): Working on Project LifeLike, a computer graphics and human computer interaction research and design project. Manage EVL’s Vicon FX40 motion capture studio, design workflow pipeline, perform tutorial sessions for students, conduct motion acquisition, and process various research projects. Worked on “CoreWall,” a visualization environment for the analysis of lake and ocean cores; developed user interfaces to manipulate seamless core images in multiple display environments, including screen widget, preference, and tools.
Researcher, Electronics & Telecommunications Research Institute, Daejeon, Korea (2001 – 2002)
Did virtual-reality research, developed graphical user interfaces, created special effects, and designed prototype 3D online computer game engine software using MS DirectX library in C++ to promote the Korean computer gaming industry, which included motion transition graph and multi-height map system tools.
Researcher, R&D Department, Korea Game Promotion Center, Seoul, Korea (1999 – 2000)
Created 3D game engine software using high-quality animation data via a motion-capture hardware system, funded by Korea’s Ministry of Science and Technology (2000). Researched and wrote several proposals analyzing computer gaming industry trends for presentation to the Korean government. Surveyed gaming market trends and technologies to develop a strategic plan to promote the Korean game industry, and prepared a plan to manage the new market of digital multimedia (movies, computer animations/games, virtual-reality software, and digital music) for the Ministry of Culture & Tourism (1999).
The New Eco series at gosia koscielak studio & gallery, Chicago, Jan 2007
“PAN – A Life Force”, The first show from the NEW ECO exhibition series examining new ecology and media phenomena in an international art curated solo exhibition
“Drawing current – Electronic Visualization MFA thesis show”, Great Space, UIC, Chicago, May 2006
“PAN”, Electronic Visualization MFA Thesis Group Exhibition
“Year End Show of Electronic Visualization”, Center for Virtual Reality Arts, UIC, Chicago, May 2005
“3.D.UO.PAD” and “Bouncing Space”, Annual exhibition of Electronic Visualization program
“2005 Indiana IDEAS Festival”, Bloomington, Indiana, April 2005
“3.D.UO.PAD”, Best work award in Virtual Reality Environment session hosted by Indiana University
“Electronic Visualization Show”, Center for Virtual Reality Arts, UIC, Chicago, May 2004
“The Creation – From eight signs of divination to the world”, A part of portal application of VR Grid, Electronic Visualization show
Lee, S., LaFond, C., Johnson, A., Vincent, C., Leigh, J., Renambot, L., “A Virtual Patient to Assess Pediatric Intensive Care Unit (PICU) Nurses’ Pain Assessment and Intervention Practices,” International Conference on Intelligent Virtual Agent (IVA) 2013, Edinburgh, UK, 08/29/2013 - 08/31/2013
A. Gonzalez, R. DeMara, V. Hung, C. Leon-Barth, M. Elvir, J. Hollister, S. Kobosko, J. Leigh, A. Johnson, S. Jones, G. Carlson, S. Lee, L. Renambot, M. Brown, “Passing an Enhanced Turing Test - Interacting with Lifelike Computer Representations of Specific Individuals,” the Journal of Intelligent Systems (JISYS), (to be published. accepted on 17 Apr, 2013)
LaFond, C., Lee, S., Johnson, A., Vincent, C., “Virtual Human Vignettes for Evaluating Pediatric Nurses’ Pain Assessment and Intervention Choices,” Midwest Nursing Research Society 37th Annual Research Conference, Chicago, IL, 03/08/2013 - 03/10/2013
S. Lee, A. D. Kshemkalyani, and M. Shen, “Performance Evaluation of Incremental Vector Clocks,” in International Symposium on Parallel and Distributed Computing, Cluj-Napoca, 07/06/2011 – 07/08/2011.
Lee, S., Carlson, G., Jones, S., Johnson, A., Leigh, J., Renambot, L., “Designing an Expressive Avatar of a Real Person,” International Conference on Intelligent Virtual Agent (IVA) 2010, Philadelphia, PA, 09/20/2010 - 09/22/2010
Sun, Y., Leigh, J., Johnson, A., Lee, S., “Articulate: A Semi-automated Model for Translating Natural Language Queries into Meaningful Visualizations,” Proceedings of 10th International Symposium on Smart Graphics, Banff, Canada, 06/24/2010 - 06/26/2010
Chen, Y., Hur, H., Lee, S., Leigh, J., Johnson, A., Renambot, L., “Case Study - Designing An Advanced Visualization System for Geological Core Drilling Expeditions”, Proceedings of the ACM Conference on Human Factors in Computing Systems 2010 (CHI 2010), Atlanta, GA, 04/10/2010 - 04/15/2010
Chen, Y., Lee, S., Hur, H., Leigh, J., Johnson, A., Renambot, L., “Design an Interactive Visualization System for Core Drilling Expeditions Using Immersive Empathic Method,” Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Boston, MA, 04/04/2009 - 04/09/2009
DeMara, R., Gonzalez, A., Hung, V., Leon-Barth, C., Dookhoo, R., Jones, S., Johnson, A., Leigh, J., Renambot, L., Lee, S., Carlson, G., “Towards Interactive Training with an Avatar-based Human-Computer Interface,” Industry Training, Simulation & Education Conference (I/ITSEC) 2008, Orlando, Florida, 12/1/2008 - 12/4/2008
Rao, A., Chen, Y., Lee, S., Leigh, J., Johnson, A., Renambot, L., “Corelyzer: Scalable Geologic Core Visualization using OSX, Java and OpenGL”, Apple's Worldwide Developers Conference 2006, 07/08/2006 - 08/11/2006
Work featured on NOVA scienceNOW "Can We Live Forever?” January 2011 < http://www.pbs.org/wgbh/nova/tech/jason-leigh-avatars.html>
Work featured on Discovery Science channel “Future of: Immortal Avatars,” September 2009 <http://science.discovery.com/videos/popscis-future-of-immortal-avatars.html>
“Mr. P” is a 1/10 scale plastic figure inspired by “Garageband” project. Main guitarist singer in former project is realized via ABS 3D printer and detailed with acrylic paint and a little LED throwie gimmick to generate illumination from inside of pumpkin head. In collaboration with Hyejung Hur and Don Olmstead, “Garageband” is a short computer animation created using Autodesk Maya and MotionBuilder software. The band consists of three band members, a guitarist (pumpkin head singer), a percussionist (skeleton dancer), and drummer (monstrous beast). All three characters’ animation is acquired via a full body motion capture system and retargeted in MotionBuilder and Maya. “Motion Viewer” is small low burst motion playback and retarget utility software for Vicon motion data (v file). Viewer support full body rigged character animation using captured motion files as source of animation. Multiple character synchronization and real time animation control interface are provided. Implemented in C++ with Object-Oriented Rendering Engine (OGRE) and Crazy Eddie’s GUI libraries. “Shall We Dance?” is a short computer animation inspired by the gingerbread dancing sequence from movie Shrek 2. Lonely gingerbread character envies the Gingy dancing with Tinkerbell on TV and eventually dances with a fork on a table. Modeling and Animation is done in open source Blender tool. Character motion is manually keyframed in Blender’s animation track.
“High Stakes Poker”, Dec 2007
In collaboration with Hyejung Hur and Geoffrey Brown, “High Stakes Poker” is a full 3D single player poker game developed in Computer Game Design class. Main opponent character is designed in full 3D skeletal model with motion capture animation. Game provides smart tips and intelligent character behavior based on poker game logic. Game play uses Wiimote controller to interact with cards and each actions for poker game play. Implemented in C++ using Object-Oriented Graphics Rendering Engine library. “PAN” is an interactive visual installation that facilitates collaborative performance. It provides a space for delightful activities that invites the audience to participate in its synthetic movement. By interacting with a tangible Pan interface, the installation renders tabletops with a new aesthetic that is generative and organic. This project was Lee’s MFA thesis work.
“Rolling Ball”, Dec 2005
“ROLLING BALL” is an experimental project to develop a wireless user interface to control balance of virtual environment via small microcontroller circuit. With accelerometer sensor and RF modules, a user can interact with virtual objects to control a traditional rolling ball toy.
“Light Me Up”, Dec 2005
“LIGHT ME UP” is an interface to translate user hand motion into visual presence. A tiny physical interface represents one of instances of the composition of virtual building blocks. By moving your hand over the sculpture, one can create various height and color schemes for these blocks.
“STREAM IT”, May 2005
“STREAM IT!” is a real time media work base on virtual reality system. With two small cardboard box installations, people can send video stream data into VR environment. Primary interaction is to put various physical objects into boxes and then system will translate the video stream from webcam that is installed inside box into 3 dimensional texturing and geometry data. “BOUNCING SPACE” aims to create the relationship between the concept of sound and motion. From our expectation, moving objects usually produce sound in real, a user creates motion of the primitive objects and each object generates sound. The second concept came from the reverse of this general idea. It is “from sound to motion.” Every moving object base on simulation of physics via Open Dynamics Engine (ODE) “3.D.UO. PAD” is an interactive 3D sketchpad. In immersive 3D display systems such as CAVE or CWALL, the user can draw and carve sketchpad. One can select and assign 2D image as texture for sketchpad and morph the pad dynamically by color values of texture image pixels. Once the work is finished, the result can be saved and loaded later in real time.
“Dinner at the fairy castle: a virtual set”, May 2004
In collaboration with Helen-Nicole Kostis, Tina Shah, Ruby, “Dinner at the Fairy Castle”: A Virtual Set is an attempt to recreate the dining room of the Fairy Castle exhibit at the Museum of Science and Industry, Chicago. Since the actual exhibit is miniature in size, the artists thought it would be interesting for audience participants to actually see themselves keyed inside one of the famous elegant rooms of the castle.
“The Creation - From eight signs of divination to the World”, Dec 2003
The Creation simulates the very first stage of the Creation of the world by interacting with eight signs of divination plates that symbolize the universe. The signs become the four faces of world and holy guardian animals such as Tiger, Dragon, Phoenix, and Turtle. Once the world is created, the user can interact with these holy animals by throwing stones towards each world face.
UIC Image of Research Awards 2010 for “Avatar – A Virtual Human,” 1st place winner
Keio University’s Keio Research Institute Digital Art Awards 2006, “PAN”, Honorable Mention in Interactive category
Indiana Ideas Festival 2005, “3.D.UO.PAD”, Best work in Virtual Reality Environment session
Member, Association for Computer Machinery (ACM)
Member, Institute of Electrical and Electronics Engineers (IEEE)
Operating Systems: Microsoft Windows, MAC OSX, and Linux
Programming Languages: C/C++, Java, Python. Also Multi-threading, Network Programming
Character Rigging and Animation: Vicon Motion Capture System, Autodesk MotionBuilder and Maya, Pixologic ZBrush, Blender
Computer Graphics: OpenGL, DirectX programming for real-time application; Advanced Shader programming, nvidia CUDA for general purpose GPU programming
Volunteer, Chicago Adler Planetarium: Deploying avatar-enabled museum application (1/17/2011–present)
Demonstrated Lifelike Avatar project, National Science Foundation (NSF) I/UCRC 2010 Annual Meeting, Washington DC (1/13-15/2010) and 2009 Annual Meeting, Washington DC (1/79/2009)
Member of CCAMANG Special Interest Group in Photography, Yonsei University (1993 – present)
Vice President of Korean Graduate Student Association, University of Illinois at Chicago (8/2004 – 7/2005)
President of Student Association, Dept. of Architectural Engineering, Yonsei University (3/1995 – 2/1996)
Love photography, Sci-Fi movies, and animation. Enjoy building and playing with plastic models and Legos. Recently, more fascinated by a little war game figure paintings such as Games Workshop’s Warhammer models. Fluent in English and Korean is native language.