KVServe
A service-aware KV-cache compression framework for communication-efficient disaggregated LLM serving, designed to reduce KV transfer overhead while offering flexible compression configurations.
I am a first-year Ph.D. student at the Institute of Computing Technology, Chinese Academy of Sciences (ICT, CAS), advised by Prof. Dingwen Tao and Prof. Guangming Tan. Previously, I completed my undergraduate studies at the University of Electronic Science and Technology of China (UESTC).
My research interests primarily lie in Machine Learning Systems and Data Compression, with a particular focus on efficient large-scale distributed inference and compression for massive scientific data. Recently, I have also become interested in the co-optimization of data compression techniques and large model inference systems.
Please feel free to contact me if you’re interested in relevant research topics or would like to discuss potential collaborations!
I actively develop open-source tools for efficient AI systems. Contributions are welcome—whether through issues, feature proposals, benchmarks, documentation, or pull requests.
A service-aware KV-cache compression framework for communication-efficient disaggregated LLM serving, designed to reduce KV transfer overhead while offering flexible compression configurations.
A training-free, token-efficient reading layer for AI agents. It selectively retrieves source-grounded evidence from large documents and workspaces, with integrations across agent frameworks.