CSC 216/416: AR/VR Interaction Design [Spring, 2024][4 Credits]
Course Projects:
FridgeMate
VR Dating [website]
ASL Learning in the Zoo
More projects:
- Drink MixAR [website]
Contact information:
- Instructor: Dr. Zhen Bai (address as Dr. Bai)
- E-mail: zhen (dot) bai (at) rochester (dot) edu
- Office hour: Thu 2-3pm
- Location: Wegmans 3007
- Teaching Assistant: Yifan Li
- E-mail: yli287 (at) ur (dot) rochester (dot) edu
- Office hour: Wed 2:30-4:30pm
- Location: Wegmans 2507
- Teaching Assistant: Ilene Kang
- E-mail: zliu114 (at) ur (dot) rochester (dot) edu
- Office hour: Tue 2-3pm, Wed 1:30-2:30pm
- Location: Wegmans 3209
Course overview:
Human activity is embodied through all our sensorimotor capacities, immersed in our immediate physical and social surroundings. Augmented Reality (AR) and Virtual Reality (VR) technologies have shown vast potential to extend our sensorimotor and social experiences in 3D space. This undergraduate/graduate course aims to help students to develop understanding of key AR/VR concepts, principles, technologies, design practices and development tools. The students will obtain hands-on experiences of creating AR/VR applications that address real-world problems in domains such as education, collaboration and quality of life, and understand technological and ethical implications of AR/VR such as human behavior sensing and ethical challenges.
Prerequisites and Target Audience:
The course is suitable for undergraduate and graduate students with basic programming (CSC 172 - required) and mobile development (CSC 214 - required). It’s desired that students have some basic knowledge about 3D graphics and/or game development.
Learning objectives:
- Obtain an overview of AR/VR such as history, concept and main application domains
- Achieve understanding of key elements of AR/VR technologies such as display, tracking, and 3D interaction
- Identify design opportunities for real-life problems based on the special characteristics of AR/VR technologies
- Apply user-centered design practices in rapid prototyping for AR/VR applications
- Obtain hands-on programming skills of key elements of AR/VR
- Identify appropriate evaluation methods to understand AR/VR user experience
- Develop teamwork experiences in designing, developing and evaluating AR/VR applications
- Obtain insights of special topics, social implications and future directions of AR/VR
How learning objectives will be addressed:
This course will contain a combination of lectures, workshops, seminars and group projects. The lectures will cover theoretical and technical knowledge of AR/VR technologies, methods and tools for design and development, usability and user experience evaluation, the hands-on workshops will provide technical demonstration of AR/VR hardware (e.g., mobile, head-mounted display), interaction scenarios (individual, co-located, remote), and programming tools (e.g., ARCore, RealityKit and Unity). The seminars will involve student-led presentation and discussion, guest lecturers on special topics of AR/VR depending on guest lecturer availability. The group project will take place throughout the course and provide an opportunity for students to apply knowledge and skills learnt in the class to develop proof-of-concept AR/VR applications or interactions in addressing real-life problems, through contextual inquiry, low-fidelity prototyping, implementation, evaluation, writing and presentation.
Schedule:
Week | Date | Lecture |
1 | 1/18 | Course Intro |
2 | 1/23 | AR/VR Overview |
1/25 | AR/VR Application | |
3 | 1/30 | AR/VR Display |
2/1 | AR/VR Tracking | |
4 | 2/6 | Seminar: Project pitch |
2/8 | Programming Workshop: AR Mobile Applications [Studio X] | |
5 | 2/13 | 3D Interaction Techniques |
2/15 | UX Workshop: AR/VR Experience [Studio X] | |
6 | 2/20 | Project pitch, brainstorm, team forming |
2/22 | Programming Workshop: Unity [Studio X] | |
7 | 2/27 | Programming Workshop: VR 3D Interaction – 1 [Studio X] (Dr. Bai away) |
2/29 | Programming Workshop: VR 3D Interaction – 2 [Studio X] (Dr. Bai away) | |
8 | 3/5 | Designing 3D User Interface |
3/7 | Design Workshop: AR/VR Low-Fidelity Prototyping [Studio X] | |
9 | 3/12 | Spring Break |
3/14 | Spring Break | |
10 | 3/19 | Mid-term project proposal presentation -1 |
3/21 | Mid-term project proposal presentation -2 | |
11 | 3/26 | Advanced topic: AR/VR Accessibility (Dr. Dhruv Jain, Univ Michigan) |
3/28 | AR/VR Evaluation | |
12 | 4/2 | TBC |
4/4 | Future of AR/VR | |
13 | 4/9 | Advanced topic: Dr. Yukang Yan (ROCHCI): “Understand, predict, and enhance user behavior in Mixed Reality |
4/11 | Privacy, Ethical, Social Implication of AR/VR” | |
14 | 4/16 | Advanced Topic: Dr. Athena Willis (URMC) “Designing Emerging Technology for Diverse Sensory and Language Experiences Through Neuroscience” |
4/18 | Project Development/Evaluation Session | |
15 | 4/23 | Final project presentation - 1 |
4/25 | Final project presentation - 2 | |
16 | 4/30 | Public Poster & Demo [Studio X] |
Course Assessment and grading:
- Class participation (10%)
- Students will take part in lecture discussion, seminar presentation, and design workshop
- Writing assignment (10%)
- Students will complete various writing tasks such as application investigation, project pitch, and project evaluation plan. Graduate students will have extra assignments such as paper reading and quantitative human behavior analysis plan.
- Mini AR/VR programming assignments (30%)
- Students will complete three individual mini-projects, one on AR and two on VR, by extending provided tutorial and guidance with new features
- Mid-term group project (10%)
- Student will complete a mid-term group project that describes the problem, motivation of using AR/VR, proposed solution, storyboard and low-fidelity prototype
- 10-minute in-class presentation
- Final Project: 40% + 10% bonus
- Improved low-fidelity prototype
- A working prototype
- Evaluation
- Project report
- Final oral presentation
- Final poster presentation
- Media
- Website
- Demo video: a 2-3 minutes video to showcase your project. Example videos can be found in the previous course website
- (Optional) Students are encouraged to submit their work as a poster or work-in-progress paper to AR/VR related peer-reviewed conferences (e.g., ISMAR conference, RIT Frameless Symposium (fall))
Note: Late submission
The grading policy for late submission is: (1) submitted within 24 hours after the deadline, the student will get up to 80% of the total score; (2) submitted beyond 24 hours after the deadline, will get up to 50% of the total score.
Attendance:
If a student cannot attend the class due to a planned event (e.g., travel), s/he has to notify the instructor one week ahead of time. For unforeseen reasons (e.g., illness, family emergency), the student should let the instructor know before 9am on the day of the class.
For students who have to attend the class on zoom due to illness, permission is needed from the instructor before 9am on the day of the class.
Class culture:
- Student-centered, inquiry-driven, collaborative
- Only use electronic devices (e.g., laptop) for note-taking or class activity purposes (e.g., programming). No mobile phone usage is allowed during the class.
Students with special needs:
Set up a meeting with the student disability service office and meet with the course instructor to discuss necessary accommodation within the first two weeks
Academic honesty:
All assignments and activities associated with this course must be performed in accordance with the University of Rochester's Academic Honesty Policy. More information is available at: www.rochester.edu/college/honesty.
For individual course assignments, students can discuss with classmates for questions related to the assignments. However, all written reports and programming for individual projects must be done independently, represent your own work and not be completed in collaboration with others.
For the group project, students are permitted and encouraged to share project materials with group members, and write the report as a group effort. Therefore, it is important to understand that you are responsible for the academic integrity of the entire report, including contributions of other group members. To avoid potential problems with academic honesty (and to more fully engage in the project), you should be involved in various aspects of writing the report, and you should verify that citations are correct and that all text is accurately cited and not plagiarized. At the end of the project report, you are required to provide a clear statement of the contributions of each member of your group to the group activities. If you feel that problems are developing in your group project, you should come to see the instructor early, so that I can provide general guidance to group members to set your activities on the right course. As you are responsible for the entire assignment, it is incumbent upon each of you to ensure the integrity of the project.
Materials:
Class lending: 2 Oculus Rift, 1 Samsung Galaxy S8, 2 Samsung Galaxy Tab S4
Library device lending/usage:
- Studio X: Meta Quest 2, Microsoft Hololens 2, Valve Index, (limited) iPad Pro, Android phones
- Rettner library: Oculus Rift, Vive VR
Course technology:
- Blackboard: announcement, assignments, forum discussion
- Slack: communication with the instructor, TA, and classmates
- Zoom: ONLY to accommodate student’s special needs with lecture live caption, and remote attendance under special circumstances.