AR/VR Interaction Design

CSC 216/416: AR/VR Interaction Design [Fall, 2019][4 Credits]

Contact information: 

  • Instructor: Dr. Zhen Bai (address as Dr. Bai)
    • E-mail: zbai7 (at) cs (dot) rochester (dot) edu
    • Office: Wegmans Hall 3007
    • Office hour: Tue 5-6pm
  • Teaching Assistant: Jing Shi
    • E-mail: j (dot) shi (at) rochester (dot) edu
    • Office: Wegmans Hall 2403
    • Office hour: 2pm-3pm Wednesday and Friday

Prerequisites and Target Audience:

The course is suitable for undergraduate students with basic programming (CSC 172 - required), mobile development (CSC 214 - required) and 3D graphics skills (CSC 131 - desired). Instructor’s permission is needed to register for the course.

Course overview:

Human activity is embodied through all our sensorimotor capacities, immersed in our immediate physical, social and cultural surroundings. Interaction technologies such as Augmented Reality (AR) and Virtual Reality (VR) have shown vast potential to enhance the way we think, feel and behave by superimposing or substituting digital contents into our surrounding 3D space. This undergraduate course aims to provide an introductory overview of the concepts, principles, methods and special topics of AR, VR and related technologies. This course will help students to build a frame of reference of the above technologies, apply them in design and development practices that address real-world problems in domains such as education, healthcare, and collaboration, and actively reflect on social implications such as acceptance and privacy. 

Learning objectives:

  • Understand the history, concept and cognitive origins of AR/VR technologies
  • Understand AR/VR technologies
  • Explore design and evaluation methods for AR/VR applications
  • Obtain hands-on experience in developing AR/VR systems to solve real-world problems
  • Obtain insights of social implications of AR/VR technologies and reflect on future directions 
  • Obtain knowledge of special topics of AR/VR (depending on guest lecturer’s availability)

How learning objectives will be addressed:

This course will contain a combination of lectures, seminars and group projects. The lectures will cover theoretical and technical knowledge of AR/VR technologies, methods and tools for design and development, usability and user experience evaluation, and hands-on technical demonstration of various devices (e.g. mobile, head-mounted display) and interaction scenarios (individual, co-located, remote) in class. The course focuses on designing and prototyping AR/VR interfaces to address real-life problems, but will also include a brief introduction of programming tools such as Unity, ARCore, Google VR. The seminars will invite guest lecturers to introduce special topics related to these technologies such as optics design, visual-audio perception, user experience design, and computer vision technology, depending on guest lecturer availability. The group project will take place throughout the course and provide an opportunity for students to apply cutting-edge technologies and tools and user-centered design processes to develop proof of concept AR/VR interfaces through contextual inquiry, low-fidelity prototyping, implementation, evaluation, writing and presentation.

Learning outcomes:

  • Identify design opportunities for real-life problems based on the special characteristics of AR/VR technologies
  • Design and develop low-fidelity prototypes using prototyping methods suitable for AR/VR technologies 
  • Develop and deploy in-class AR/VR programming exercises and the term-wide group project in at least one hardware platform
  • Design and conduct basic analytic and evaluation methods based on usability and user experience methods
  • Relate the term-wide group project with social implications of AR/VR technologies discussed in class


18/29Class IntroShort bio and statement of interest
29/3AR/VR OverviewReview AR/VR application;

Prepare a 4-minute presentation of selected AR/VR application
9/5AR/VR Application Session
Guest speaker: Dr. Michael Hasselburg (Assistant Professor of Psychiatry and Clinical Nursing)
Review the AR/VR application slides, write project idea #1, including problem, motivation for using AR/VR to address the problem, 2-3 existing or related applications

AR mini project starts
39/10Display & Visual Perception
AR/VR Application Session (Cont.)
Project idea 1 due
9/12Tracking AR/VR
Guest speaker: Ms. Jessica Keith (Department of Psychology), Ms. Emily Isenstein (BCS)
AR mini project Ex1 due
Write project idea #2
49/17AR - Situated Visualization
Guest speaker: Dr. Michael Javis (History), Heta Gandhi (Chemical Engineering)
Project idea #2 due
Write project idea #3;
9/19AR/VR Application Session (Cont.)
Project idea discussion
Guest speaker: Dr. Lynnie Correll (Assistant Professor, Department of Anesthesiology and Perioperative Medicine)
AR mini project examine
AR mini project Ex2, Ex3 due
59/24Designing 3D User InterfaceProject idea #3 due

Write 2-page pitch project proposal: (1) 1-page of project idea #1-3 (2) 1-page of pitch project: problem, motivation of using AR/VR, solution

Prepare 3-minute presentation for project pitch
9/26Project Pitch, Brainstorm, Group formation2-page pitch project proposal due
610/1Interaction & NavigationAR mini project Ex4, Ex5 due

VR mini project starts
10/3AR/VR experience session (Rettner Hall, collaborative AR)
Document the AR/VR application

Group project member and tentative project description due
710/8Low-fidelity AR/VR prototype workshopWrite mid-term project proposal (600-800 words): title, team member, problem, motivation of using AR/VR, proposed solution, low-fidelity prototype.

Prepare for 5-min group project proposal presentation

VR mini project – Ex1 due
10/10Mid-term project proposal presentationMid-term project proposal due

Create project website and start weekly progress blog
810/15Fall break
10/17Guest Speaker: Ehsan Hoque - Virtual Human, Affective Computing)VR mini project – Ex2, Ex3 due
910/223D modeling (tbc)
10/24AR/VR Experience Write project design consideration for AR/VR experiences
1010/29Guest Speaker: Zhiyao Duan - Computer Audition
10/31Project development (instructor is away)
1111/5Guest Speaker: Yuhao Zhu - AR/VR hardware + software
11/7Evaluating 3D User InterfaceWrite evaluation Plan
1211/12Guest Speaker: Cetin Mujdat - Human-Brain Interaction
11/14Privacy, Ethical, and Social Implications of AR/VRWrite privacy, ethical and social implication analysis of the projec
1311/19Future AR/VR
11/21Guest Speaker: Duje Tadin (BCS)
1411/26Project development Prepare for demo video, document code, clean up for github
11/28Thanksgiving break
1512/3Project developmentPrepare for final presentation
12/5Final PresentationPrepare for poster, update website
1612/10Poster Presentation



Ilene Kang is a junior studying CS. She is interested in both humans and machines and hopes to work with both in the future. She plays cello, takes photos and works at a bank part-time.


Zac Lukegord is a senior planning to graduate this spring with a BA in Computer Science. He is eager to learn about VR/AR and how these technologies can be utilized to create interesting video games.


Prikshet Sharma is a senior interested in learning new things.


Sifan Ye is a senior majoring in Computer Science.  He is interested in AI, Machine Vision and his hobbies are traveling and photography.


Sydney Dlhopolsky is in her junior year studying computer science and studio art. She is interested in using emerging technology, like AR/VR, to solve problems and help people. In her free time she enjoys crafting, spending time outdoors, hanging out with animals, and watching cartoons.


Meiwen Zhou is a senior majoring in computer science and psychology. She's here to learn fundamental design principles of AR/VR and how to make AR/VR a more affordable and sharable experience. In her spare time, she enjoys reading/drawing comics, photography, drum sets, and cooking authentic Chinese food.


Shuran Zhang is a third-year student major in Computer Science with a minor in Religions. She is very interested in exploring technologies that enable her to see her work in a direct and visual way such as mobile app development and VR/AR. Besides her academic interests, she likes playing video games and reading thrillers.


Gao Fan is an undergraduate senior majoring in Computer Science and Financial Economics. He found his interest in AR/VR after an exciting experience with Beat Saber, a VR music game. He also likes photography and travel


Shagun Bose is a senior pursuing a double degree in Psychology and Computer Science. She is interested in thinking more critically about real-world problems and the impacts of the technology we use to solve them. In her free time, she likes to pursue music (voice) and art.


Anesi Ikhianosime is an undergraduate junior studying Computer Science and Psychology. He is interested in learning more about how AR/VR can play a role in the future of consumer social apps


Dane Johnson is a junior majoring in brain and cognitive sciences and computer science. He is interested in learning about the potential for VR and AR to impact education and the way people learn. He likes to play guitar and is a member of the quidditch team.


Pooja Priya is a senior majoring in Computer Science and Digital Media. She is interested in exploring how AR/VR technology can help address the problems in design, health, and education. She enjoys biking, hiking, and most outdoor activities.


Emma Strauch is a senior studying Computer Science and Psychology. She became interested in exploring the course after working a bit with the technology in HCI. In her free time, she likes to volunteer, go rock climbing, and bake.


  • Direct assessment
    • Class participation (10%)
      • Students will take part in lecture discussion, seminar presentation, and design workshop 
    • Assignment (10%)
      • Students will complete various writing tasks such as individual project proposals, AR/VR experience design, social implication analysis  
    • Two individual mini-projects (30%)
      • Students will complete two individual mini-projects, one on AR and one on VR, based on provided tutorial and guidance, and add at least one new feature for either visualization or interaction  
    • Mid-term group project (10%)
    • Final Project: 40% + 10% bonus
      • Low-fidelity prototype (3%): Use everyday materials such as paper, plastic film, glasses to illustrate and experiment with the design intent. Include the design rationale (why choose this particular design) and at least 3 different user scenarios (e.g. reveal social information, navigate in unfamiliar scenes)
        • Design of three distinct user scenario (e.g. in an online shopping website, user scenarios may include search product, compare product, add to cart, check out)
      • Working prototype (16%+6% bonus): Implement at least two user scenarios from the low-fidelity prototype:
        • Visual implementation (4% for each user scenario)
        • Interaction implementation (4% for each user scenario)
      • Bonus:
        • 2% extra for each additional user scenario (up to 6% extra in total)
      • Evaluation (3%)
        • Evaluate the working prototype with users outside of the team
      • Final oral presentation (5%) 12/5
        • Clear project presentation:
          • Project revisited (1%): problem statement, project goal, storyboard, team member work allocation
          • Low-fidelity prototype (1%)
          • Working prototype (1%)
        • Insightful discussion:
          • Main findings and future work (1%)
          • Appropriate reference in APA style (1%)
      • Final poster presentation (3%) 12/10
        • Content (2%): align with the oral presentation and final report
        • Presentation publicly at CS atrium (1%)
      • Project report (5% + 4% bonus): submit by midnight 12/14
        • Content: Extend the mid-term report with:
          • Design and implementation of low-fidelity prototype (2%)
          • Design and implementation of working prototypes (2%)
          • Main findings, implications, future work and conclusion (1%)
        • Bonus
          • Report in a conference template – e.g. ACM SIGCHI paper (1%)
          • Frameless demo (abstract submission 1%, demo 2%)
      • Media (5%): submit by midnight 12/14
        • Website (2%): An interactive media platform to archive and share your project work.
        • Video (3%): 2-3 minute video to showcase your project. The video should include a title slide, team member, date, project goal, low-fidelity and working prototype demonstration. Showcase video examples can be found here
    • (Optional) Students are encouraged to submit their work as a poster or work-in-progress paper to AR/VR related peer-reviewed conferences, or local, regional or national student research consortiums 
  • Indirect assessment
    • Collect mid-term and end-term student surveys to evaluate students’ self-efficacy in fulfilling the learning objectives, and feedback on course content, instruction and projects

Class culture:

  • Student-centered, inquiry-driven, collaborative
  • Only use electronic devices (e.g. cell phone, laptop) for note-taking purposes


  • If a student cannot attend the class due to a planned event (e.g. travel), s/he has to notify the instructor one week ahead of time. For unforeseen reasons (e.g. illness, family emergency), the student should let the instructor know before 9am on the day of the class. 
  • If the student misses more than two classes, s/he should contact the instructor to discuss course make-up

Students with special needs:

  • Set up a meeting with the student disability service office and meet with the course instructor to discuss necessary accommodation within the first two weeks

Academic honesty: 

All assignments and activities associated with this course must be performed in accordance with the University of Rochester's Academic Honesty Policy. More information is available at:

  • For individual course assignments, students can discuss with classmates for questions related to the assignments. However, all written reports and coding for individual projects must be done independently, represent your own work and not be completed in collaboration with others. 
  • For the group project, students are permitted and encouraged to share project materials with group members, and write the report as a group effort. Therefore it is important to understand that you are responsible for the academic integrity of the entire report, including contributions of other group members. To avoid potential problems with academic honesty (and to more fully engage in the project), you should be involved in various aspects of writing the report, and you should verify that citations are correct and that all text is accurately cited and not plagiarized. At the end of the project report, you are required to provide a clear statement of the contributions of each member of your group to the group activities. If you feel that problems are developing in your group project, you should come to see the instructor early, so that I can provide general guidance to group members to set your activities on the right course. As you are responsible for the entire assignment, it is incumbent upon each of you to ensure the integrity of the project.


  • Class lending: 1 Oculus Rift, 2 Samsung Galaxy S8, 2 Samsung Galaxy Tab S4
  • Library lending: Samsung Gear VR, Google Cardboard

Course technology:

  • Course website: for updated syllabus
  • Blackboard: assignment announcement, assignment (blog), forum discussion (assignment and project related discussion)
  • Zoom: remote teaching sessions

Example course projects:

Square Breathing [website]

aware [website]

Math In Motion [website]

Vride [website]

AR Group Joining [website]

AR Style Transfer [website]

Hyperion [website]