AR/VR Interaction Design

CSC 294: AR/VR Interaction Design [Fall, 2019][4 Credits]

Contact information: 

  • Instructor: Dr. Zhen Bai (address as Dr. Bai)
    • E-mail: zbai7 (at) cs (dot) rochester (dot) edu
    • Office: Wegmans Hall 3007
    • Office hour: Tue 5-6pm
  • Teaching Assistant: Jing Shi
    • E-mail: j (dot) shi (at) rochester (dot) edu
    • Office: Wegmans Hall 2403
    • Office hour: 2pm-3pm Wednesday and Friday

Prerequisites and Target Audience:

The course is suitable for undergraduate students with basic programming (CSC 172 - required), mobile development (CSC 214 - required) and 3D graphics skills (CSC 131 - desired). Instructor’s permission is needed to register for the course.

Course overview:

Human activity is embodied through all our sensorimotor capacities, immersed in our immediate physical, social and cultural surroundings. Interaction technologies such as Augmented Reality (AR) and Virtual Reality (VR) have shown vast potential to enhance the way we think, feel and behave by superimposing or substituting digital contents into our surrounding 3D space. This undergraduate course aims to provide an introductory overview of the concepts, principles, methods and special topics of AR, VR and related technologies. This course will help students to build a frame of reference of the above technologies, apply them in design and development practices that address real-world problems in domains such as education, healthcare, and collaboration, and actively reflect on social implications such as acceptance and privacy. 

Learning objectives:

  • Understand the history, concept and cognitive origins of AR/VR technologies
  • Understand AR/VR technologies
  • Explore design and evaluation methods for AR/VR applications
  • Obtain hands-on experience in developing AR/VR systems to solve real-world problems
  • Obtain insights of social implications of AR/VR technologies and reflect on future directions 
  • Obtain knowledge of special topics of AR/VR (depending on guest lecturer’s availability)

How learning objectives will be addressed:

This course will contain a combination of lectures, seminars and group projects. The lectures will cover theoretical and technical knowledge of AR/VR technologies, methods and tools for design and development, usability and user experience evaluation, and hands-on technical demonstration of various devices (e.g. mobile, head-mounted display) and interaction scenarios (individual, co-located, remote) in class. The course focuses on designing and prototyping AR/VR interfaces to address real-life problems, but will also include a brief introduction of programming tools such as Unity, ARCore, Google VR. The seminars will invite guest lecturers to introduce special topics related to these technologies such as optics design, visual-audio perception, user experience design, and computer vision technology, depending on guest lecturer availability. The group project will take place throughout the course and provide an opportunity for students to apply cutting-edge technologies and tools and user-centered design processes to develop proof of concept AR/VR interfaces through contextual inquiry, low-fidelity prototyping, implementation, evaluation, writing and presentation.

Learning outcomes:

  • Identify design opportunities for real-life problems based on the special characteristics of AR/VR technologies
  • Design and develop low-fidelity prototypes using prototyping methods suitable for AR/VR technologies 
  • Develop and deploy in-class AR/VR programming exercises and the term-wide group project in at least one hardware platform
  • Design and conduct basic analytic and evaluation methods based on usability and user experience methods
  • Relate the term-wide group project with social implications of AR/VR technologies discussed in class


18/29Class IntroShort bio and statement of interest
29/3AR/VR OverviewReview AR/VR application;

Prepare a 4-minute presentation of selected AR/VR application
9/5AR/VR Application Session
Guest speaker: Dr. Michael Hasselburg (Assistant Professor of Psychiatry and Clinical Nursing)
Review the AR/VR application slides, write project idea #1, including problem, motivation for using AR/VR to address the problem, 2-3 existing or related applications

AR mini project starts
39/10Display & Visual Perception
AR/VR Application Session (Cont.)
Project idea 1 due
9/12Tracking AR/VR
Guest speaker: Ms. Jessica Keith (Department of Psychology), Ms. Emily Isenstein (BCS)
AR mini project Ex1 due
Write project idea #2
49/17AR - Situated Visualization
Guest speaker: Dr. Michael Javis (History), Heta Gandhi (Chemical Engineering)
Project idea #2 due
Write project idea #3;
9/19AR/VR Application Session (Cont.)
Project idea discussion
Guest speaker: Dr. Lynnie Correll (Assistant Professor, Department of Anesthesiology and Perioperative Medicine)
AR mini project examine
AR mini project Ex2, Ex3 due
59/24Designing 3D User InterfaceProject idea #3 due

Write 2-page pitch project proposal: (1) 1-page of project idea #1-3 (2) 1-page of pitch project: problem, motivation of using AR/VR, solution

Prepare 3-minute presentation for project pitch
9/26Project Pitch, Brainstorm, Group formation2-page pitch project proposal due
610/1Interaction & NavigationAR mini project Ex4, Ex5 due

VR mini project starts
10/3AR/VR experience session (Rettner Hall, collaborative AR)
Document the AR/VR application

Group project member and tentative project description due
710/8Low-fidelity AR/VR prototype workshopWrite mid-term project proposal (600-800 words): title, team member, problem, motivation of using AR/VR, proposed solution, low-fidelity prototype.

Prepare for 5-min group project proposal presentation

VR mini project – Ex1 due
10/10Mid-term project proposal presentationMid-term project proposal due

Create project website and start weekly progress blog
810/15Fall break
10/17Guest Speaker: Ehsan Hoque - Virtual Human, Affective Computing)VR mini project – Ex2, Ex3 due
910/223D modeling (tbc)
10/24AR/VR Experience Write project design consideration for AR/VR experiences
1010/29Guest Speaker: Zhiyao Duan - Computer Audition
10/31Project development (instructor is away)
1111/5Guest Speaker: Yuhao Zhu - AR/VR hardware + software
11/7Evaluating 3D User InterfaceWrite evaluation Plan
1211/12Guest Speaker: Cetin Mujdat - Human-Brain Interaction
11/14Privacy, Ethical, and Social Implications of AR/VRWrite privacy, ethical and social implication analysis of the projec
1311/19Future AR/VR
11/21Guest Speaker: Duje Tadin (BCS)
1411/26Project development Prepare for demo video, document code, clean up for github
11/28Thanksgiving break
1512/3Project developmentPrepare for final presentation
12/5Final PresentationPrepare for poster, update website
1612/10Poster Presentation



  • Direct assessment
    • Class participation (10%)
      • Students will take part in lecture discussion, seminar presentation, and design workshop 
    • Assignment (10%)
      • Students will complete various writing tasks such as individual project proposals, AR/VR experience design, social implication analysis  
    • Two individual mini-projects (30%)
      • Students will complete two individual mini-projects, one on AR and one on VR, based on provided tutorial and guidance, and add at least one new feature for either visualization or interaction  
    • Mid-term group project (10%)
    • Final group project (40%)
      • Students will form a group of 2-3 students to identify the strengths of AR/VR technologies to address a specific real-life problem, design and develop a low-fidelity prototype, implement a proof-of-concept prototype with a chosen platform, conduct basic evaluation, write up an 8-10 pages project report in the template of ACM SIGCHI conference, make a formal presentation, and create a project webpage to provide weekly updates on their learning and project related progress, and demonstrate their final project outcomes
    • (Optional) Students are encouraged to submit their work as a poster or work-in-progress paper to AR/VR related peer-reviewed conferences, or local, regional or national student research consortiums 
  • Indirect assessment
    • Collect mid-term and end-term student surveys to evaluate students’ self-efficacy in fulfilling the learning objectives, and feedback on course content, instruction and projects

Class culture:

  • Student-centered, inquiry-driven, collaborative
  • Only use electronic devices (e.g. cell phone, laptop) for note-taking purposes


  • If a student cannot attend the class due to a planned event (e.g. travel), s/he has to notify the instructor one week ahead of time. For unforeseen reasons (e.g. illness, family emergency), the student should let the instructor know before 9am on the day of the class. 
  • If the student misses more than two classes, s/he should contact the instructor to discuss course make-up

Students with special needs:

  • Set up a meeting with the student disability service office and meet with the course instructor to discuss necessary accommodation within the first two weeks

Academic honesty: 

All assignments and activities associated with this course must be performed in accordance with the University of Rochester's Academic Honesty Policy. More information is available at:

  • For individual course assignments, students can discuss with classmates for questions related to the assignments. However, all written reports and coding for individual projects must be done independently, represent your own work and not be completed in collaboration with others. 
  • For the group project, students are permitted and encouraged to share project materials with group members, and write the report as a group effort. Therefore it is important to understand that you are responsible for the academic integrity of the entire report, including contributions of other group members. To avoid potential problems with academic honesty (and to more fully engage in the project), you should be involved in various aspects of writing the report, and you should verify that citations are correct and that all text is accurately cited and not plagiarized. At the end of the project report, you are required to provide a clear statement of the contributions of each member of your group to the group activities. If you feel that problems are developing in your group project, you should come to see the instructor early, so that I can provide general guidance to group members to set your activities on the right course. As you are responsible for the entire assignment, it is incumbent upon each of you to ensure the integrity of the project.


  • Class lending: 1 Oculus Rift, 2 Samsung Galaxy S8, 2 Samsung Galaxy Tab S4
  • Library lending: Samsung Gear VR, Google Cardboard

Course technology:

  • Course website: for updated syllabus
  • Blackboard: assignment announcement, assignment (blog), forum discussion (assignment and project related discussion)
  • Zoom: remote teaching sessions