Luca Yu
Luca Jiang-Tao Yu
Ph.D. Candidate

School of Computing and Data Science
The University of Hong Kong

E-mail: lucayu at connect dot hku dot hk

[CV] [Scholar] [GitHub] [LinkedIn] [Twitter]

Bio

I am a graduating Ph.D. candidate at the School of Computing and Data Science, The University of Hong Kong, advised by Prof. Chenshu Wu. I also work closely with Prof. Yiming Qiu. Currently, I am an intern at Microsoft Research Asia, advised by Dr. Yuqing Yang and Dr. Rui Ma. Previously, I was a research assistant at the Institute for Interdisciplinary Information Sciences, Tsinghua University and Shanghai Qi Zhi Institute, advised by Prof. Hang Zhao. Before that, I got my B.Eng. degree and was a research intern at Beijing University of Posts and Telecommunications, advised by Prof. Anfu Zhou.

I am actively looking for internship opportunities. I am open to collaboration and happy to discuss. If you're interested in my research, feel free to reach out via email.

Research Vision

My research focuses on systems that let AI interact with its environments: the digital world, where agents act through GUIs, tools, and APIs, and the physical world, where AI must perceive beyond text, images, and speech.

AI in the Digital World. LLMs are no longer standalone models answering one request at a time; as agents, they operate inside live digital environments by driving GUIs, calling tools, and orchestrating sub-tasks. This shift exposes two systems challenges. First, model outputs are inherently probabilistic, yet actions in real environments carry irreversible side effects: a wrong tap or command cannot simply be resampled. Second, an agent's workload is no longer a single prefill-and-decode pass, but a multi-turn trajectory interleaved with external tool execution and dynamic model invocations that request-level serving engines can neither see nor schedule as a whole. Addressing these challenges requires runtime verification that enforces safety and state invariants before actions execute, alongside trajectory-aware serving systems that schedule compute and memory across the full lifecycle of an agent task.

AI in the Physical World. Mainstream multimodal AI perceives the world through text, images, video, and speech, but the physical environment offers rich information beyond these modalities. Wireless and acoustic signals, spanning Wi-Fi, mmWave, and ultrasound, work through occlusion and in darkness, preserve privacy, and already radiate from everyday devices. Yet these signals are not human-interpretable and lack internet-scale training corpora, so turning them into practical perception calls for embedding signal physics into model architectures, transferring knowledge from conventional modalities such as vision, speech, and language, and validating systems in the wild.

News

Selected Publications

Experience

[Aug 2026 – now] Research Intern, Microsoft Research Asia
Shanghai, China; Advisors: Dr. Yuqing Yang and Dr. Rui Ma
[Nov 2021 – Sep 2023] Research Intern, Tsinghua University and Shanghai Qi Zhi Institute
Beijing and Shanghai, China; Advisor: Prof. Hang Zhao
[Jun 2020 – Sep 2021] Research Intern, Beijing University of Posts and Telecommunications
Beijing, China; Advisor: Prof. Anfu Zhou

Awards

Invited Talks

[Oct 2026] Modularized Signal Processing-Deep Learning Co-Design for RF Sensing via Deep Unrolling
Shenzhen Edge Terminal Agent Institute, Shenzhen, China
[Oct 2026] Towards Open-Sourced Datasets, Benchmarks, and Platforms for Multi-Modal Sensing
MuMoSe Tutorial, UbiComp 2026, Shanghai, China
[May 2026] How to Stay Ourselves in the Age of LLMs: A PhD Student Perspective
Shenzhen University, Shenzhen, China

Service

Template adapted from Yiming Qiu.