Hi there 👋

I am now working on building the cutting-edge multimodal LLM at Bytedance Seed.

I am looking for interns to work with me on multimodal LLM. Feel free to contact me if you are interested!

I obtained my Ph.D. degree at Zhejiang University, supervised by Prof. Siliang Tang（汤斯亮） and Prof. Yueting Zhuang（庄越挺）. I also collaborate with Xu Tan (谭旭) from Microsoft Research Asia and Lingfei Wu (吴凌飞) closely.

My research interest includes multimodal LLM, Text-to-Speech Synthesis, ASR Error Correction, Graph Neural Network and Neural Machine Translation. I have published 10+ papers includes ICML, ICLR, NeurIPS, EMNLP, IJCAI, et.al.

I have developed: 1. the large-scale non-autoregressive text-to-Speech synthesis system NaturalSpeech 2 and the advanced version NaturalSpeech 3 when in MSRA; 2. the SOTA audio foundation model Kimi-Audio when in Moonshot.AI.

🔥 News

• 2024.05: 🎉 NaturalSpeech 3 is accepted by ICML2024 as Oral presentation!
• 2024.03: 🎉 We are delighted to released NaturalSpeech 3, which is a advanced version of NaturalSpeech series with speech factorization.
• 2024.01: 🎉 NaturalSpeech 2 and PromptTTS 2 are accepted by ICLR2024 for Spotlight and Poster presentation!
• 2023.09: 🎉 We released PromptTTS 2, which is a large-scale TTS system using text prompt.
• 2023.04: 🎉 We are delighted to release our NaturalSpeech 2, which is the first large-scale NAR TTS system. It can generate high-quality speech with only a 3-second prompt!
• 2021/06: 🎉 Check out our most recent survey paper, titled "Graph Neural Networks for Natural Language Processing: A Survey"! First comprehensive survey on GNNs for NLP!
• 2021.06: 🔥 We are delighted to release our Graph4NLP Library (⭐️1.6k+), which is the first library for the easy use of GNNs for NLP!