Dr. Joongi Shin

Hi, I'm a Postdoctoral researcher at Aalto University, supervised by Prof. Antti Oulasvirta (Computational Behavior Lab) and Prof. Andrés Lucero. I received my Ph.D. in Industrial Design from Korea Advanced Institute of Science and Technology, (KAIST) in 2021.

I investigate how to best use emerging technologies to support human collaboration in design activities. 

Primary Research Keywords: HCI, user-centered design, collaborative creativity, generative AI
Secondary Research Keywords: unobtrusive interaction, robotic furniture, posture manipulation, design tools for human collaborators, VR/AR.

Research Vision

My research vision is to make collaborative design activities more efficient so that collaborators can together achieve (a) high-quality outcomes and (b) satisfaction in working with other people. Collaboration has been central to diverse design activities such as co-creating solutions and acquiring insights about users. However, the high effort required in working with other people often thwarts the rigorous conduct of collaborative design practice, scaling down or omitting effortful yet critical design tasks in the process. Consequently, individuals’ creative endeavors cannot be integrated effectively, resulting in low-quality outcomes and distrust in collaborative approaches.

I aim to address this by integrating generative AI into people’s collaborative design workflows. I approach ‘integration’ not as a mere replacement of humans with AI. Instead, I approach this by (a) identifying what humans can do together better than working with AI and (b) identifying what humans like to do with other humans than AI. Similar to the Fitts humans-are-better-at / machines-are-better-at approach (HABA-MABA), I believe understanding what collaborators cannot do well together can inform what and how AI should assist the process. Similarly, understanding what collaborators can do better can help avoid the naive integration of AI in the workflow. This can further lead to identifying how AI should assist collaborators, assisting collaborators to achieve high-quality outcomes without diminishing human-centered perspectives in working with AI.

News

Selected Publications

Understanding Human–AI Workflows for Generating Personas

Joongi Shin, Michael A. Hedderich, Bartłomiej Jakub Rey, Andrés Lucero, Antti Oulasvirta

Proceedings of the 2024 ACM Designing Interactive Systems Conference

[DOI] [PAPER] [Project]

One barrier to deeper adoption of user-research methods is the amount of labor required to create high-quality representations of collected data. Trained user researchers need to analyze datasets and produce informative summaries pertaining to the original data. While Large Language Models (LLMs) could assist in generating summaries, they are known to hallucinate and produce biased responses. In this paper, we study human--AI workflows that differently delegate subtasks in user research between human experts and LLMs. Studying persona generation as our case, we found that LLMs are not good at capturing key characteristics of user data on their own. Better results are achieved when we leverage human skill in grouping user data by their key characteristics and exploit LLMs for summarizing pre-grouped data into personas. Personas generated via this collaborative approach can be more representative and empathy-evoking than ones generated by human experts or LLMs alone. We also found that LLMs could mimic generated personas and enable interaction with personas, thereby helping user researchers empathize with them. We conclude that LLMs, by facilitating the analysis of user data, may promote widespread application of qualitative methods in user research. 

Chatbots Facilitating Consensus-Building in Asynchronous Co-Design

Joongi Shin, Michael A. Hedderich, Andrés Lucero, Antti Oulasvirta

UIST '22 I4th Annual ACM Symposium on User Interface Software and Technology

[DOI] [PAPER]

Consensus-building is an essential process for the success of co-design projects. To build consensus, stakeholders need to discuss conflicting needs and viewpoints, converge their ideas toward shared interests, and grow their willingness to commit to group decisions. However, managing group discussions is challenging in large co-design projects with multiple stakeholders. In this paper, we investigate the interaction design of a chatbot that can mediate consensus-building conversationally. By interacting with individual stakeholders, the chatbot collects ideas to satisfy conflicting needs and engages stakeholders to consider others’ viewpoints, without having stakeholders directly interact with each other. Results from an empirical study in an educational setting (N = 12) suggest that the approach can increase stakeholders’ commitment to group decisions and maintain the effect even on the group decisions that conflict with personal interests. We conclude that chatbots can facilitate consensus-building in small-to-medium-sized projects, but more work is needed to scale up to larger projects.

Slow Robots for Unobtrusive Posture Correction

Joongi Shin, Eiji Onchi, Maria Jose Reyes, Junbong Song, Uichin Lee, Seung-Hee Lee, Daniel Saakes

CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems 

[DOI]

Prolonged static and unbalanced sitting postures during computer usage contribute to musculoskeletal discomfort. In this paper, we investigated the use of a very slow moving monitor for unobtrusive posture correction. In a first study, we identified display velocities below the perception threshold and observed how users (without being aware) responded by gradually following the monitor's motion. From the result, we designed a robotic monitor that moves imperceptible to counterbalance unbalanced sitting postures and induces posture correction. In an evaluation study (n=12), we had participants work for four hours without and with our prototype (8 in total). Results showed that actuation increased the frequency of non-disruptive swift posture corrections and significantly reduced the duration of unbalanced sitting. Most users appreciated the monitor correcting their posture and reported less physical fatigue. With slow robots, we make the first step toward using actuated objects for unobtrusive behavioral changes.