Alexa may not be able to read your mind -- at least not yet— but Hannah Lee ’19 is trying to help her move in that direction.
“We want to train machines so they can know by looking at people’s facial expressions or hearing their speech patterns that the machine got something wrong, even before the person tells them,” Lee said.
Lee is combining her interests in psychology and computer science as part of a new partnership between Cornell Tech and the colleges of Arts & Sciences and Engineering. Five students spent the summer in the city conducting research at Cornell Tech.
Lee is working with Andrea Cuadra, a graduate student in the lab of Wendy Ju, assistant professor in the information science program, studying human-computer interaction.
“My research revolves around designing machine interactions that are responsive to humans at a level that is now possible via machine learning,” Cuadra said. “For example, I believe that if someone is feeling happy, their voice agent's tone should match that emotion.”
“We also want to see if there are any patterns so that we could establish guidelines for what makes a successful or unsuccessful interaction between a human and a voice agent,” Lee said. “We’re planning to open-source the resulting dataset to enable a variety of learning algorithms to be used to model users’ reactions to predict errors.”
Coming to Cornell as a human development major, Lee said she transferred into the College of Arts & Sciences because she wanted to study the broader field of psychology. She then developed an interest in artificial intelligence from an internship at a tech startup, where she worked to create a conversational chatbot that can form an emotional connection with humans.
“Working with Andrea at Cornell Tech has been a perfect opportunity for me, as I’ve been able to combine my interests in machine learning and psychology.” Lee said.
Prem Talwai ‘19, a math major, is working in the field of stochastic optimal control, developing algorithms for making optimal decisions under uncertainty. The field has applications in a broad range of areas ranging from finance to transportation to energy optimization.
At Cornell Tech, he is working with Associate Professor Itai Gurvich, who is also a member of Cornell's Operations Research and Information Engineering Department. Talwai studies complex nonlinear models of decision-making, exploring ways their structural properties can be used to make them simpler and easier to solve computationally. He then applies these insights to develop novel, more efficient algorithms for a type of machine learning known as reinforcement learning.
“A lot of these techniques directly translate into insights into, for example, how to best navigate a robot,” he said. “There’s a direct correspondence between the theory we’re working on here and its application to real-world science.”
On the Ithaca campus, Talwai has been involved in research with professors Robert Strichartz and Alexander Vladimirsky in the math department. He has spent summers doing research at Cornell and UCLA and his junior year abroad at Oxford, and he’s still involved virtually in research papers in progress at each location.
Talwai said he was able to apply much of the work he did at Oxford during his Cornell Tech internship and, throughout his studies, he’s honed his interests in applied mathematics and operations research.
“I’ve had the opportunity to take many courses that I don’t think I could have taken as an undergraduate at other universities,” he said. “Cornell’s Center for Applied Math encourages interdisciplinarity, with all of these professors teaching courses that don’t really come under one category but are relevant to so many different problems.”
Both Lee and Talwai say Cornell Tech has been the perfect place to conduct summer research. While many professors are off-campus for the summer, graduate students are around, as well as interns from other colleges. They both said they’ve had lots of independence and opportunity to innovate on their projects.
“It’s rare to get the opportunity to participate in a research program and also explore the city,” Talwai said. “And the industry-focused approach is really prevalent everywhere you go at Cornell Tech, with startups actually having office space right here. The collaborative spirit among disciplines is also evident here, and infectious in my opinion.”
