Future User Interfaces

CSC 294/CSC 574: Future User Interfaces [Fall, 2018][4 Credits]

Instructor contact information:

  • E-mail: zbai7 at cs dot rochester dot edu
  • Office: Wegmans Hall 3017
  • In-person meeting: by appointment


  • Prior course: CSC 172. Recommended: CSC212/412, CSC 242, and CSC 249.
  • Knowledge/skills needed: basic programming skills, previous course/research experience in HCI, graphics, computer vision, AI is desired.
  • Instructor permission: Yes

Course overview:

People today connect with information and each other by typing just a few words on a smartphone, however human activity is embodied far beyond our fingertips, through all our sensorimotor capacities, immersed in our immediate physical, social and cultural surroundings. Interaction paradigms such as virtual reality, augmented reality, tangible user interface and pervasive computing, have vast potential to enhance the way we think, feel and behave in ways that traditional interfaces cannot. This seminar provides an introductory overview of embodied technologies including key theoretical frameworks, interaction technologies, and design methods, across application domains such as education, healthcare and computer-supported collaboration. Students will engage in paper review, group discussion, and critical writing, which build towards a final project involving user interface design and prototyping.

Learning objectives:

  • Prepare students with theoretical and technological knowledge of state-of-the-art embodied and intelligent user interfaces.
  • Prepare students with hands-on experience to apply user-centered design methods in creating novel embodied and intelligent interfaces to tackle real-life problems.
  • Prepare students with key research and project skills including paper review, critical thinking, project planning, and presentation through collaborative learning.

Methods of instruction:

  • Paper review
  • Class discussion
  • Group project


  • Paper Review: 40%
    • Each student will write one paper review before noon one day before each class. For example, if the class is on Tuesday, then the student should submit the paper review no later than 12pm on Monday. This leaves 24 hours for students to comment on others’ paper review.
    • The paper review should be 500-600 words, and include (1) 1-2 sentence of what you want to learn before reading the paper; (2) 2-3 sentence summary of the paper (motivation, method, result); (3) in-depth summary of key contents such as arguments, technology, system, and findings; (4) your own opinion about paper contents and reflections on challenges and opportunities of future user interface
    • For students to earn credit for upper level writing, you must also synthesize all your review papers into a coherent 25-page tech report before the last class. The report should include a coherent literature review covering theory, technology and application discussed during the class, and a brief overview of the group project, including findings and implications.based on your literature review.
  • Peer-review and discussion: 10%
    • Before noon on the day of each class, each student will read and make comments for at least 3 other students’ paper review.
    • During the class, one student will lead the discussion by first sharing his/her own thoughts of the paper and assist the follow-up group discussion. If there are multiple papers to discuss, it will be one student per paper. Students will lead the discussion following seniority. You can also volunteer to lead the discussion for a given class close to your interest, if the appointed student is willing to swap.
  • Midterm project: 10%
    • Project proposal: including problem statement, contextual inquiry, storyboard, implementation and evaluation plan.
  • Final Project: 40% + 10% bonus
    • Low-fidelity prototype (10%): Use everyday materials such as paper, plastic film, glasses to illustrate and experiment with the design intent. Include the design rationale (why choose this particular design) and at least 3 different user scenarios (e.g. reveal social information, navigate in unfamiliar scenes)
      • Visual design (4%): design rationale (1%), 3 user scenarios (3%)
      • Content (2%)
      • Interaction (4%): design rationale (1%), 3 user scenarios (3%)
    • Working prototype (10% + 7% bonus):
      • Implement at least one user scenario from the low-fidelity prototype: visualization (4%), content (2%), interaction (4%)
      • Bonus:
        • 3% extra for each additional user scenario (up to 6% extra in total)
        • Evaluation of the working prototype with users outside of the team (1%)
    • Final presentation (5% + 1% bonus)
      • Clear project presentation:
        • Project revisited (1%): problem statement, project goal, storyboard, team member work allocation
        • Low-fidelity prototype (1%)
        • Working prototype (1%)
      • Insightful discussion:
        • Main findings and implication (1%)
        • Future work and conclusion (1%)
      • Bonus (1%): Appropriate reference
    • Peer evaluation (5%): based on the same presentation criteria above
    • Project report (5% + 2% bonus): submit by midnight 12/11
      • Content: Extend the mid-term report with:
        • Design and implementation of low-fidelity prototype (2%)
        • Design and implementation of working prototypes (2%)
        • Main findings, implications, future work and conclusion (1%)
      •  Bonus
    • Media (5%): submit by midnight 12/15
      • Website (2%): An interactive media platform to archive and share your project work.
      • Video (3%): 2-3 minute video to showcase your project. The video should include a title slide, team member, date, project goal, low-fidelity and working prototype demonstration. Showcase video examples can be found here

Below is a tentative course schedule that provides an overview of the course topics. Please note that the contents may be updated due to emerging interests and availability of guest lecturer.

DateTopicReading List
8/30 Introduction, class syllabus and mechanics
9/4 Paper Reading SkillsReading a computer science research paper [read]
How to Read a Scientific Research Paper [read]
Efficient Reading of Papers in Science and Technology [read]
9/6HCI Theoretical FrameworkDistributed Cognition: Towards a New Foundation for Human-Computer Interaction [read]
9/11Augmented Reality OverviewA Survey of Augmented Reality [read]
9/13Augmented Reality Cont.
9/18Virtual Reality OverviewEnhancing Our Lives with Immersive Virtual Reality [read]
9/20Virtual Reality Cont.
9/25User-Centered Design User-Centered Design [read]
9/27Hands-on VR session
10/2Application - LearningBodyVis: A New Approach to Body Learning Through Wearable Sensing and Visualization [read]
FoldMecha: Exploratory Design and Engineering of Mechanical Papercraft [read]
An Active Tangible User Interface Framework for Teaching and Learning Artificial Intelligence [read]
TactileVR: Integrating Physical Toys into Learn and Play Virtual Reality Experiences [read]
10/4Application - Social InteractionApplying Social Psychological Theory To The Problems Of Group Work [read]
ImmerseBoard: Immersive Telepresence Experience using a Digital Whiteboard [read]
Room2Room: Enabling Life-Size Telepresence in a Projected Augmented Reality Environment [read]
10/9Application - Healthcare and wellbeingSleeveAR: Augmented Reality for Rehabilitation using Realtime Feedback [read]
VRRRRoom: Virtual Reality for Radiologists in the Reading Room [read]
Evaluating Wrist-Based Haptic Feedback for Non-Visual Target Finding and Path Tracing on a 2D Surface [read]
10/11Design for DiversityUniversal Usability [read]
Participatory Evaluation with Autistic Children [read]
The Role of Children in the Design of New Technology [read]

10/16Fall term break
10/18Mid-term proposal presentation
10/233D InteractionWatchThru: Expanding Smartwatch Displays with Mid-air Visuals and Wrist-worn Augmented Reality [read]
tactoRing: A Skin-Drag Discrete Display [read]
Halo Content: Context-aware View Management for Non-invasive Augmented Reality [read]
Sketching up the world: in situ authoring for mobile Augmented Reality [read]
10/25Internet of Things (IoT)Internet of Things (IoT): A vision, architectural elements, and future directions [read]
10/30Project Specific Readings -1 Learning Science in Immersive Virtual Reality [read]
Augmented Reality-based Indoor Navigation: A Comparative Analysis of Handheld Devices vs. Google Glass [read]
Sign Language in the Interface: Access for Deaf Signers [read]
Towards Robot Autonomy in Group Conversations: Understanding the Effects of Body Orientation and Gaze [read]
11/1Project Specific Readings -2A Literature Review on Immersive Virtual Reality in Education: State of the Art and Perspectives [read]
Bare-Handed 3D Drawing in Augmented Reality [read]
DeepASL: Enabling Ubiquitous and Non-IntrusiveWord and Sentence-Level Sign Language Translation [read]
ProCom: Designing and Evaluating a Mobile and Wearable System to Support Proximity Awareness for People with Autism [read]
11/6Project Specific Readings -3Patient Perceptions of Wearable Face-Mounted Computing Technology and the Effect on the Doctor- Patient Relationship [read]
Augmenting Multi-Party Face-to-Face Interactions Amongst Strangers with User Generated Content [read]
11/8Intelligent User Interface Mapping Machine Learning Advances from HCI Research to Reveal Starting Places for Design Innovation [read]
11/13Project Specific Readings – Self Selected Paper 1
11/15Project Specific Readings – Self Selected Paper 2
11/20Guest Speaker [tbd]
11/27Guest Speaker [tbd]
12/6Final Project Presentation
12/11Course Wrap up



Sooyoung Kim is a senior studying computer science and economics. He is excited to learn more about both virtual and augmented reality. His hobbies include traveling and taking pictures.


May Shin Lyan is a junior majoring in CS and Business. She is passionate to learn about how AR and VR could play a role in the near future of the tech industry and entrepreneurship. She likes peace and collecting magnet souvenirs.


Chester Szeen is a 2nd year MS in Data Science student, with a background in business. He became interested in the course after experience VR/AR games like Pokemon Go. In his free time, he likes to sing and practice martial arts.


Ziqiu Wu is a first year master student in computer science. She is interested in AR, VR and 3D graphics. She is good at some ball games, like badminton, ping pong and basketball.

Processed with VSCO with a6 preset

Alana Zakroczemski is in her senior year in undergrad studying Computer Science and Studio Art. She grew up in Buffalo, New York and enjoys photography, cooking, playing the xylophone, and biking. Alana wanted to take this class because she hopes to work in developing AR/VR technology.


Jesse Wang is a 4th year MD/PhD student (2nd year in the Translational Biomedical Sciences department) who is interested in developing AR applications for facilitating real-time clinical diagnoses. He is originally from Westford, MA. One thing that makes him human is that he can ride a unicycle.


Samiha Samrose is a 3rd-year PhD student in Computer Science. Her research is on behavior and emotion analysis for automated machine mediation in group conversations. She enjoys traveling and watching movies/series.


Wenyi Chu is an undergraduate senior majoring in computer science. She also works as an amateur photographer and designer. Wenyi likes building new things and she looks forward to making some cool products/ digital solutions throughout the course


He Gong is a graduate student majoring in computer science. He is an AI track student. He also has an interest in computer vision and graphics. He looks forward to developing his own app in the future and he needs to know more about the newest technique and trend.

Project Demo

Line Navigation [Website]

[He Gong, Ziqiu Wu, Junbo Hu] 

Cuenector [website]

[Samiha Samrose, Wenyi Chu, Sharfuz Sifat]

CPUniverse [website]

[Alana Zakroczemski, Sooyoung Kim]

ASL to Speech Translation

[Ashely Tenesaca]

Cauliflower AR [website]

[Jesse Wang, May Shin Lyan, Chester Szeen]

Class culture

  • Student-centered, inquiry-driven, collaborative
  • Only use electronic device (e.g. cell phone, laptop) for note taking purpose


  • If a student cannot attend the class due to a planned event (e.g. travel), s/he has to notify the instructor one week ahead of time. For unforeseen reasons (e.g. illness, family emergency), the student has to let the instructor know before 9am on the day of the class.
  • If the student misses more than two classes, s/he should contact the instructor to discuss course make-up

Student with special needs

  • Set up a meeting with the student disability service office and meet with the course instructor to discuss necessary accommodation within the first two weeks

Academic honesty:

All assignments and activities associated with this course must be performed in accordance with the University of Rochester's Academic Honesty Policy. More information is available at: www.rochester.edu/college/honesty

  • For paper reviews, you are encouraged to discuss readings with your fellow students. However, all written paper review work must be done independently, represent your own review of the paper and not completed in collaboration with others. You should only read and comment on other students’ paper reviews after you have submitted your own.
  • For the group project, you are permitted and encouraged to share project materials with group members, and you will write your report as a group effort. Therefore it is important to understand that you are responsible for the academic integrity of the entire report, including contributions of other group members. To avoid potential problems with academic honesty (and to more fully engage in the project), you should be involved in various aspects of writing the report, and you should verify that citations are correct and that all text is accurately cited and not plagiarized. At the end of the project report, you are required to provide a clear statement of the contributions of each member of your group to the group activities. If you feel that problems are developing in your group project, you should come to see me early, so that I can provide general guidance to group members to set your activities on the right course. As you are responsible for the entire assignment, it is incumbent upon each of you to ensure the integrity of the project.


  • Oculus Rift
  • Smartphone
  • Arduino kit
  • (tbd) Hololens

Course technology

  • Course website: for reading materials
  • Blackboard: assignment announcement, paper review (blog), paper review comment (blog comment), forum discussion (paper and project related discussion)