Jiahao Nick LI 李嘉昊
I am a Ph.D. candidate at UCLA HCI Research working with Prof. Xiang ‘Anthony’ Chen.
My research area lies in Human-AI Interaction, focusing on Egocentric Multimodal AI Agents. Specifically, I build interactive systems to provide personalized assistance for individual users. Such systems are powered by cutting-edge AI models (e.g., LLMs, VLMs) and are enhanced by incorporating: (1) external and task-specific knowledge, sourced through both crowdsourcing and targeted data collection, and (2) the retrieval of long-form multimodal memories, facilitating contextual understanding for users. Recently, I have been working on building a multimodal retrieval augmented pipeline to address the natural language query (NLQ) task on the egocentric video datasets (e.g., Ego4D).
During my Ph.D., I interned at Meta Reality Lab, Adobe Research and Palo Alto Research Center (PARC). I was also a visiting Ph.D. student at University of Tokyo working with Dr. Takeo Igarashi. Prior to that, I received my Bachelor of Engineering from Shanghai Jiao Tong University.
news
Jan 19, 2024 | Two papers (OmniActions & Human I/O) got accepted by CHI 2024 . OmniActions is from my Meta internship on user intent prediction in pervasive AR scenarios leveraging LLMs and VLMs. Human I/O is on identifying situational impairments using egocentric videos leveraging reasonings of LLMs. |
---|
selected publications
- OmniActions: Predicting Digital Actions in Response to Real-World Multimodal Sensory Inputs with LLMsIn Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (CHI ’24), 2024
- Human I/O: Towards a Unified Approach to Detecting Situational Impairments in Everyday ActivitiesIn Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (CHI ’24), 2024