This user-centered design project invites students to conduct hands-on human-computer interaction research and design by exploring affect-aware technology. These technologies seek to account for users’ emotions, moods, and other affective phenomena in the user experience. Examples include emojis used while texting, social robots that model emotional responses, and emotionally-aware chatbots. This project is for a user-centered design and usability testing course offered to undergraduate computer science students. The course learning objectives are to use research and design methods to (1) build an empirical understanding of technology stakeholders and (2) apply that knowledge to design and evaluate an interactive prototype. By immersing themselves in the complex domain of affect-aware computing, students learn to apply user-centered design to emerging technologies. Students create and refine common user-centered design artifacts, including personas, interaction designs, and prototypes. The reader of this paper will obtain recommendations for structuring the user-centered design projectand a high-level understanding of affect-aware computing.
This project introduces students to both UCD and the domain of affect-aware
computing. The topic of emotions in technology may be unfamiliar to many
students. The instructor should give ample time to introduce affect-aware
computing scenarios and technologies. The students should be encouraged
to immerse themselves in the domain by conducting secondary research
and gathering examples of current affect-aware computing. When choosing
an affect-aware computing scenario, students can select a technology that
already incorporates emotions (e.g., a Mood Tracker app or social robots) or
a technology that currently does not incorporate emotions but could (e.g.,
streaming music service).
Ethical HCI requires UX experts to understand and be responsive to the
societal impacts of technology. This project will present the class with socio-technical scenarios related to privacy, disclosure, bias, and other ethical
considerations related to technology supporting and intervening upon people’s emotion-based experiences. Class discussion and assignments should
address ethical considerations and critical perspectives about affect-aware
computing. Normative views are often embedded in affect-aware computing making non-normative and marginalized communities vulnerable to
emotional manipulation by technology [5]. For example, a hiring algorithm
based on normative expectations of verbal and nonverbal communication
behavior is likely to mischaracterize a neurodivergent applicant and limit
job opportunities.
To address these complexities, instructors need to guide students through
conceptual analysis and practical steps for conducting ethical UCD. At a
fundamental level, students need to recognize and challenge their own
biases and deepen their appreciation for the diversity of potential users.
Instructors can scaffold discussion and design activities use toolkits, such
as Value-Sensitive Design Envisioning Cards [1], Tarot Cards of Tech,
and Black Mirror activities [3]. To support UCD processes that directly en-
gage with research participants, the instructors and students should ensure
access, cooperation, and mutual goals established with target communities. Students and research participants should be aware and prepared for
the research topics, especially potentially sensitive topics and recalling
emotionally-laden episodes. The instructor and potentially their institution’s ethics review board should review and approve research protocols,
including screening criteria and participant consent forms.
REFERENCES
[1] Batya Friedman and David Hendry. 2012. The Envisioning Cards: A Toolkit for
Catalyzing Humanistic and Technical Imaginations. In Proceedings of the SIGCHI
Conference on Human Factors in Computing Systems (CHI ’12). ACM, New York,
NY, USA, 1145–1148. https://doi.org/10.1145/2207676.2208562
[2] Alison L. Hill, David G. Rand, Martin A. Nowak, and Nicholas A. Christakis.
2010. Emotions as infectious diseases in a large social network: the SISa model.
Proceedings. Biological Sciences 277, 1701 (Dec. 2010), 3827–3835. https://doi.org/10.1098/rspb.2010.1217
[3] Shamika Klassen and Casey Fiesler. 2022. Run Wild a Little With Your Imagination: Ethical Speculation in Computing Education with Black Mirror. Proceedings of the 53rd ACM Technical Symposium on Computer Science Education 1 (March 2022), 7.https://doi.org/10.1145/3478431.3499308
[4] R. W. Picard. 1995. Affective Computing. M.I.T. Media Laboratory Perceptual
Computing Section Technical Report No. 321 (1995).
[5] Annuska Zolyomi and Jaime Snyder. 2021. Social-Emotional-Sensory Design Map for Affective Computing Informed by Neurodivergent Experiences. Proceedings of the ACM on Human-Computer Interaction 5, CSCW1 (April 2021), 38. https://doi.org/10.1145/344915
This project engages students with the user-centered design (UCD) process focusing on understanding users’ emotions and affect. The domain of emotions is not unfamiliar to UCD. User experience (UX) experts—researchersand designers—are called to develop empathy of users by attuning to research participants’ lived experience, which naturally includes emotions. By centering real users, rather than designers’ self-perception or stereotypes of users, UX experts can create technology that is meaningful to diverse, disabled, and global communities. UX experts embed a deep understanding of users’ emotions into the design process as they define personas, user journey maps, and other artifacts.
With this project, understanding users’ emotions are considered through
a different but related lens of affect-aware computing, which is technology
designed in some manner to illuminate, support, or intervene on users’
emotional experiences in some manner [4]. This technology goes beyond
technology’s incidental impact on emotions, such as the phenomenon of
social contagion on social media [2]. Instead, affect-aware computing is
explicitly affect-aware, affect-mimicking, or affect-altering in its goals. Examples include users selecting emojis in chat, and, in a more sophisticated form, artificial intelligence detecting emotions during a video call. The technology can have various goals, including informing users of their emotional
states, supporting reflection of emotions at play during an interpersonal
conflict, or role-modeling emotions. In this project, students have hands-on
experiences applying UCD principles and processes within the context of
affect-aware computing. This project allows students to make meaningful
choices that make the project culturally relevant. Key decisions include
selecting affect-aware computing user scenarios, formative research methods (e.g., diary study, survey, interview), prototype design, and scope of usability testing.
This project requires students and instructors to attune to a crucial component of user-centered design—building empathy for users by developing a
nuanced understanding of people and the context in which they use technology. Although students’ own lived experiences with emotions will influence
their choices and interpretation of data from their research, students must
learn to design for others, not themselves. The class should explore how
a researcher’s positionality, lived experiences, and empirical observations
gained through research are incorporated into user-centered design.
By conducting UCD of affect-aware computing, students will make interdisciplinary connections to CS with psychology and sociology as they consider individuals’ emotional experiences and social norms. This project
requires students to work in teams, establish clear roles and responsibilities, and synthesize their learning into design artifacts. Instructors should
scaffold team formation and performance to build effective student teams
diverse in life experiences and skills. In summary, this project’s engagement
practices are:
• Use Meaningful and Relevant Content
• Make Interdisciplinary Connections to CS
• Incorporate Student Choice
• Encourage Student Interaction
• Culturally Relevant Pedagogy