I am Heejun Lee. I am a Ph.D. student at the Korea Advanced Institute of Science and Technology AI. Hello, World! 🐱
My research interests include:
- Neural Network Compression and Computation Optimization (Currently, I am focusing on an attention mechanism)
- Large-scale Machine Learning System Design, Development, and Serving (This is why I am working with DeepAuto.AI)
- General Purpose AI (NLP/CV)
[CV]
Education
- KAIST | Sep. 2024 - | Ph.D. in Artificial Intelligence
- KAIST | Mar. 2020 - Aug. 2024 | B.S. in Computer Science
Publications
[A1] InfiniteHiP: Extending Language Model Context Up to 3 Million Tokens on a Single GPU
[paper][code] Heejun Lee*, Geon Park*, Jaduk Suh, Sung Ju Hwang. Arxiv
[C5] Delta Attention: Fast and Accurate Sparse Attention Inference by Delta Correction
[paper][code][code-prod] Jeffery Willette, Heejun Lee, Sung Ju Hwang. NeurIPS 2025
[C4] HiP Attention: Sparse Sub-Quadratic Attention with Hierarchical Attention Pruning
[paper][code] Heejun Lee*, Geon Park*, Youngwan Lee*, Jaduk Suh*, Jina Kim, Wonyoung Jeong, Bumsik Kim, Hyemin Lee, Myeongjae Jeon, Sung Ju Hwang. ICLR 2025
[C3] Training-Free Exponential Extension of Sliding Window Context with Cascading KV Cache
[paper][code] Jeffrey Willette, Heejun Lee, Youngwan Lee, Myeongjae Jeon, Sung Ju Hwang. ICLR 2025
[C2] SEA: Sparse Linear Attention with Estimated Attention Mask
[paper][code] Heejun Lee, Jina Kim, Jeffrey Willette, Sung Ju Hwang. ICLR 2024
[C1] Sparse Token Transformer with Attention Back Tracking
[paper][code] Heejun Lee, Minki Kang, Youngwan Lee, Sung Ju Hwang. ICLR 2023
(*: equal contribution)
Experiences
- DeepAuto.ai. Dec. 2023 -. AI Researcher
- New York University. Mar. 2025 - May. 2025. Short-term Research Scholar (Global AI Frontier Lab)
- KAIST MLAI. Jul 2021 - Aug 2024. Undergraduate Intern (I am continuing my Ph.D. program in this lab)
- DeepAuto.ai. Jan 2023 - May 2023. Research Engineer
- Transverse. Oct 2020 - Mar 2021. AI Lead Developer
Awards
Academic Services
- ICLR 2026 Reviewer
- NIPS 2025 Reviewer
- ICML 2025 Reviewer
- ICLR 2025 Reviewer
- NeurIPS 2024 Reviewer (Top Reviewer)