AR/VR Interaction Design – Fall 2021

CSC 216/416: AR/VR Interaction Design [Fall, 2021][4 Credits]

Course Projects:

EZLearn [website]

Find Philip [website]

Team Infinite Wardrobe [website]

AR for Lego Assembly [website]

Money visualizer [website]

More projects:

Contact information: 

  • Instructor: Dr. Zhen Bai (address as Dr. Bai)
    • E-mail: zbai7 (at) cs (dot) rochester (dot) edu
    • Office hour: Thu 5-6pm
    • Location: Zoom
  • Teaching Assistant: Jing Bi
    • E-mail: jing (dot) bi (at) rochester (dot) edu
    • Office hour: Tue 4:40-6:40pm
    • Location: Zoom
  • Teaching Assistant: Ilene Kang
    • E-mail: ykang19 (at) u (dot) rochester (dot) edu
    • Office hour: Thu 10am-12pm
    • Location: Zoom

Course overview:

Human activity is embodied through all our sensorimotor capacities, immersed in our immediate physical and social surroundings. Augmented Reality (AR) and Virtual Reality (VR) technologies have shown vast potential to extend our sensorimotor and social experiences in 3D space. This undergraduate/graduate course aims to help students to development understanding of key AR/VR concepts, principles, technologies, design practices and development tools. The students will obtain hands-on experiences of creating AR/VR applications that address real-world problems in domains such as education, healthcare, and collaboration, and understand technological and ethical implications of AR/VR such as human behavior sensing and ethical challenges.

Prerequisites and Target Audience:

The course is suitable for undergraduate and graduate students with basic programming (CSC 172 - required) and mobile development (CSC 214 - required). It’s desired that students have some basic knowledge about 3D graphics and/or game development.

Learning objectives:

  • Obtain an overview of AR/VR such as history, concept and main application domains 
  • Achieve understanding of key elements of AR/VR technologies such as display, tracking, and 3D interaction
  • Identify design opportunities for real-life problems based on the special characteristics of AR/VR technologies
  • Apply user-centered design practices in rapid prototyping for AR/VR applications
  • Obtain hands-on programming skills of key elements of AR/VR 
  • Identify appropriate evaluation methods to understand AR/VR user experience 
  • Develop teamwork experiences in designing, developing and evaluating AR/VR applications
  • Obtain insights of special topics, social implications and future directions of AR/VR

How learning objectives will be addressed:

This course will contain a combination of lectures, seminars, workshops and group projects. The lectures will cover theoretical and technical knowledge of AR/VR technologies, methods and tools for design and development, usability and user experience evaluation, technical demonstration of various devices (e.g., mobile, head-mounted display) and interaction scenarios (individual, co-located, remote) in class. The course focuses on design and prototyping AR/VR interfaces to address real-life problems, but will also include a brief introduction of programming tools such as ARCore and Unity. The seminars will involve student-led presentation and discussion, guest lecturers on special topics of AR/VR such as haptic interaction, audio augmentation, depending on guest lecturer availability. The group project will take place throughout the course and provide an opportunity for students to apply knowledge and skills learnt in the class to develop proof-of-concept AR/VR applications or interfaces through contextual inquiry, low-fidelity prototyping, implementation, evaluation, writing and presentation.

Schedule:

WeekDateLecture
18/26Course Intro
28/31AR/VR Overview
9/2AR/VR Application -1
39/7AR/VR Application -2
9/9Seminar: AR/VR Applications
49/14AR/VR Display
9/16AR/VR Tracking
59/21Workshop: Developing AR Applications
9/23Workshop: AR/VR Experience [Studio X]
69/28Project pitch, brainstorm, team forming
9/30Workshop: Developing Unity/VR - 1
710/5Workshop: Developing Unity/VR - 2
10/7Designing 3D User Interface
810/12Fall break
10/14Workshop: AR/VR Low-Fidelity Prototyping
910/19Mid-term project proposal presentation
10/213D Interaction Techniques + Situated Visualization
1010/26AR/VR Evaluation
10/28Special Topics – Jing bi (Human Behavior Sensing)
1111/2Situated Visualization
11/4Future of AR/VR
1211/9Special Topics - Spatial Audio (Dr. Ming-lun Lee)
11/11Privacy, Ethical, Social Implication of AR/VR
1311/16Special Topics - Haptic Interface (Dr. Roshan Peiris)
11/18Project Development Session
1411/23Project Development Session
11/25Thanksgiving break
1511/30Final project presentation - 1
12/2Final project presentation - 2
1612/7Poster presentation

 

Grading:

Direct assessment

  • Class participation (10%)
    • Students will take part in lecture discussion, seminar presentation, and design workshop 
  • Assignment (10%)
    • Students will complete various writing tasks such as individual project proposals, project evaluation plan. Graduate student will have extra assignments of paper reading and leading seminar discussion
  • Two individual mini-projects (30%)
    • Students will complete two individual mini-projects, one on AR and one on VR, by extending provided tutorial and guidance with new features
  • Mid-term group project (10%)
    • Student will complete a mid-term group project that describes the problem, motivation of using AR/VR, proposed solution, storyboard and low-fidelity prototype
    • 5-minute in-class presentation
  • Final Project: 40% + 10% bonus
    • Improved low-fidelity prototype 
    • A working prototype 
    • Evaluation
    • Project report
    • Final oral presentation 
    • Final poster presentation 
    • Media
      • Website 
      • Demo video: a 2-3 minutes video to showcase your project. Example videos can be found in the previous course website 
    • (Optional) Students are encouraged to submit their work as a poster or work-in-progress paper to AR/VR related peer-reviewed conferences (e.g., RIT Frameless Symposium, ISMAR conference)

Indirect assessment

  • Collect mid-term and end-term student surveys to evaluate students’ self-efficacy in fulfilling the learning objectives, and feedback on course content, instruction and projects

Late submission

The grading policy for late submission is: (1) submitted within 24 hours after the deadline, the student will get up to 80% of the total score; (2) submitted beyond 24 hours after the deadline, will get up to 50% of the total score.

Attendance:

If a student cannot attend the class due to a planned event (e.g., travel), s/he has to notify the instructor one week ahead of time. For unforeseen reasons (e.g., illness, family emergency), the student should let the instructor know before 9am on the day of the class. 

If the student misses more than two classes, s/he should contact the instructor to discuss course make-up.

Class culture:

  • Student-centered, inquiry-driven, collaborative
  • Only use electronic devices (e.g., laptop) for note-taking or class activity purposes (e.g., programming

Students with special needs:

Set up a meeting with the student disability service office and meet with the course instructor to discuss necessary accommodation within the first two weeks

Academic honesty: 

All assignments and activities associated with this course must be performed in accordance with the University of Rochester's Academic Honesty Policy. More information is available at: www.rochester.edu/college/honesty.  

For individual course assignments, students can discuss with classmates for questions related to the assignments. However, all written reports and coding for individual projects must be done independently, represent your own work and not be completed in collaboration with others. 

For the group project, students are permitted and encouraged to share project materials with group members, and write the report as a group effort. Therefore, it is important to understand that you are responsible for the academic integrity of the entire report, including contributions of other group members. To avoid potential problems with academic honesty (and to more fully engage in the project), you should be involved in various aspects of writing the report, and you should verify that citations are correct and that all text is accurately cited and not plagiarized. At the end of the project report, you are required to provide a clear statement of the contributions of each member of your group to the group activities. If you feel that problems are developing in your group project, you should come to see the instructor early, so that I can provide general guidance to group members to set your activities on the right course. As you are responsible for the entire assignment, it is incumbent upon each of you to ensure the integrity of the project.

Materials: 

Class lending: 2 Oculus Rift, 2 Samsung Galaxy S8, 2 Samsung Galaxy Tab S4

Library device lending/usage: 

  • Studio X: Oculus Rift, Hololens, iPad, iPhone

Rettner library: Oculus Rift, Vive VR

Course technology:

  • Blackboard: announcement, assignments, forum discussion 
  • Slack: communication with the instructor, TA, and classmates 
  • Zoom: Instructor and TA office hour