Rui-Jie Zhu 朱芮捷
Ph.D. Candidate, Electrical and Computer Engineering
University of California, Santa Cruz
Advisor: Jason K. Eshraghian
I am on the 2026 job market and expect to graduate this year. My CV is here.
Previously, I worked on spiking neural networks, contributing to
snnTorch,
SpikingJelly, and building
SpikeGPT.
My research has since shifted to scalable and efficient sequence modeling architectures,
and how to scale them.
Find me on GitHub, Google Scholar, X (Twitter), and CV.
Email: ridger@ucsc.edu
Research
I am interested in building scalable and efficient sequence modeling architectures as an alternative to standard Transformers. On the architecture side, I have joined the development of linear attention and recurrent models that achieve Transformer-level quality at a fraction of the cost:
- Linear attention with expressive recurrent states (I joined RWKV and GSA)
- Matmul-free and neuromorphic language modeling (Matmul-free LM)
What I care about most is touching scaling with my own hands. My personal scaling trajectory covers three orders of magnitude in compute:
- 1×1019 FLOPs: SpikeGPT (216M params, 10B tokens, 2023)
- 2×1021 FLOPs: Matmul-free LM (2.7B params, 100B tokens, 2024)
- 4.6×1023 FLOPs: Ouro (2.6B×4, 7.7T tokens, 2025). Ouro is a looped language model where the same 2.6B parameters are reused across multiple passes, enabling a model trained from scratch to reach the quality of an 8B-scale industrial model at a fraction of the parameter count. I led the entire process, from infrastructure and pre-training to mid-training and post-training.
For each of these runs, I watched every checkpoint from the very first to the last, witnessing a model go from random to intelligence. That is what I am really enjoying. The journey is the reward.
I was also fortunate to spend a year interning at Seed, where I was deeply involved in LLM pre-training research. Across looped language models, concept models, and data synthesis (MGA), I developed a hands-on understanding of how architectures, data, and training dynamics interact at scale.
- Scaling Latent Reasoning via Looped Language Models
- A Systematic Analysis of Hybrid Linear Attention
- Dynamic Large Concept Models: Latent Reasoning in an Adaptive Semantic Space
- Reformulation for Pretraining Data Augmentation
Please refer to publications for the full list.
Talks & Media
- Scaling Latent Reasoning via Looped Language Models, ASAP Seminar #48
- LLMs Don't Need More Parameters. They Need Loops., YouTube
- 利用Loop语言模型扩展Latent Reasoning, FAI × UCSC（Bilibili）
Experience
- Jul. 2023 – 2026: Ph.D. Student, UC Santa Cruz
- Sep. 2019 – Jul. 2023: B.S., University of Electronic Science and Technology of China
This website is adapted from Tianyu Gao's design, which is in turn adapted from Gregory Gunderson.
