The University of Houston Department of Computer Science proudly recognizes Meng-Chen (Martin) Lee, a Ph.D. student whose research in multimodal interaction has earned international distinction and industry recognition.
Lee’s path into computer science began with a fascination many can relate to: video games. As an undergraduate, he was drawn to how visual effects and movement could capture attention and evoke emotion. That interest led him to study computer graphics and, eventually, to question how non-player characters could behave in more natural and intelligent ways.
“That curiosity eventually led me to the research I pursue today,” Lee said. “I focus on multimodal conversational AI and how agents can understand and generate natural human communication signals.”
Throughout his doctoral studies at UH, Lee has demonstrated exceptional research productivity and leadership. He is the first author of five peer-reviewed papers, four of which were published at the ACM International Conference on Multimodal Interaction (ICMI), one of the leading venues in the field of human-computer interaction. His work centers on enabling machines to interpret and respond to human behavior across multiple sensory modalities, including vision, speech, gaze, and body motion.
In particular, Lee’s ACM ICMI 2024 paper, “Online Multimodal End-of-Turn Prediction for Three-Party Conversations,” earned the Best Paper Runner-up Award, an honor given to only one submission each year. The recognition highlights the originality, rigor and influence of his work.
The paper introduces a novel window-based method for real-time prediction of conversational turn-taking in multi-speaker settings by integrating pre-trained language models and recurrent neural networks. The approach improves how systems anticipate interruptions, overlaps and speaker transitions, helping conversational agents interact more smoothly and naturally in real-world applications such as virtual assistants and collaborative robotics.
Lee also presented two papers at ICMI that further emphasize the importance of multimodal signals in conversation. One explores speaker-aware gaze prediction in multi-party interactions, while the other demonstrates how learned motion cues significantly improve real-time turn-taking prediction. Together, the works underscore that human communication extends far beyond spoken words.
“Human communication is not only verbal,” Lee said. “These findings help move the field toward more realistic and human-centered conversational models.”
Beyond conference success, Lee completed a summer 2025 research internship at Microsoft Research Lab in Redmond, where he worked on a voice-based conversational agent designed to interact more naturally in real time. The project focused on enabling behaviors such as barge-in and backchanneling, allowing agents to respond and signal engagement in ways that more closely resemble human conversation. Lee continues to collaborate with Microsoft Research faculty on a joint publication stemming from the internship.
“The biggest lesson I learned was how critical communication is in industry research,” Lee said. “My experience at UH, especially working in the CGIM Lab, helped prepare me to collaborate, discuss ideas clearly and transition into that environment.”
Lee’s contributions to the research community also extend to service. He received a Certificate of Recognition for reviewing papers for IEEE Transactions on Visualization and Computer Graphics, an experience he says reshaped how he approaches his own writing and evaluation of research.
“I now pay more attention to clarity, motivation and fairness,” Lee said. “Reviewing also taught me empathy—every paper represents a significant amount of effort.”
Lee is a member of the Computer Graphics and Interactive Media Lab, advised by Zhigang Deng, Moores Professor of Computer Science. In recognition of his academic performance and research impact, he also received the 2025 Best Ph.D. Student Award from the Department of Computer Science.
Reflecting on his journey, Lee emphasizes the importance of community and persistence for students interested in research.
“Talk to professors and peers early and often, even if your work doesn’t feel ready,” he said. “A rejected paper doesn’t mean failure—it’s part of the process. A supportive research community can make all the difference.”