Rui-Jie Zhu

Biography

Hi! I’m Rui-Jie (Ridger), currently pursuing a Ph.D. in Electrical and Computer Engineering at UC Santa Cruz, where I began in Fall 2023 under the supervision of Jason K. Eshraghian. I earned my Bachelor’s degree from the University of Electronic Science and Technology of China in the same year.

My deep learning journey started with spiking neural networks. During my undergraduate studies, I contributed to several open-source neuromorphic projects, including snnTorch and SpikingJelly. As my interests expanded, I shifted focus to efficient language models. I joined the development of RWKV, the first RNN-based language model, and developed SpikeGPT and Matmul-free LM. Currently, my research focuses on scaling up models through the lens of efficient inference.

Recent News

  • Aug. 2024 - Bridging the gap between artificial intelligence and natural intelligence accepted by Nature Computational Science! [paper]
  • Jul. 2024 - Eagle and Finch: RWKV with Matrix-Valued States and Dynamic Recurrence accepted by Conference on Language Modeling! [paper][code]
  • Apr. 2024 - Optically Tunable Electrical Oscillations in Oxide‐Based Memristors for Neuromorphic Computing accepted by Advanced Materials! [paper]
  • Apr. 2024 - Tensor Decomposition Based Attention Module for Spiking Neural Networks accepted by Knowleage-based Systems! [paper][code]
  • Dec. 2023 - Recent Advances in Scalable Energy-Efficient and Trustworthy Spiking Neural networks: from Algorithms to Technology accepted by 2024 International Conference on Acoustics, Speech and Signal Processing (ICASSP)! [paper]
  • Dec. 2023 - Gated Attention Coding for Training High-performance and Efficient Spiking Neural Networks accepted by the 38th Annual AAAI Conference on Artificial Intelligence! [paper]
  • Oct. 2023 - RWKV: Reinventing RNNs for the Transformer Era accepted by 2023 EMNLP Findings! [paper][code]
  • Aug. 2023 - Towards popularity prediction of information cascades via degree distribution and deep neural networks accepted by Journal of Informetrics! [paper][code]
  • May. 2023 - Invited to give a talk at the Intel Neuromorphic Research Community! [link]
  • Apr. 2023 - When Spiking Neural Networks Meet Temporal Attention Image Decoding and Adaptive Spiking Neuron accepted by ICLR 2023 tiny paper track! [paper][code]
  • Apr. 2023 - Uni-Match: A Semantic Unified Model for Query-Product Retrieval accepted by ICLR 2023 tiny paper track! [paper]
  • Mar. 2023 - VTSNN: a virtual temporal spiking neural network accepted by Frontiers in Neuroscience! : [paper][code]

Experiences

  • July. 2023 - Now, Ph.D. student, UC Santa Cruz
  • Sept. 2019 - July. 2023, Undergradute, University of Electronic Science and Technology of China

Professional Services

Membership

IEEE Graduate Student Member

Reviewer

2024 Conference on Neural Information Processing Systems (NeurIPS)

2022/2024 European Conference on Computer Vision (ECCV)

2023 Conference on Empirical Methods in Natural Language Processing (EMNLP)

IEEE Transactions on Neural Networks and Learning Systems (TNNLS)

Neural Networks (NN)

IEEE Journal on Emerging and Selected Topics in Circuits and Systems

IEEE Transactions on Emerging Topics in Computational Intelligence

IEEE Transactions on Cognitive and Developmental Systems

IEEE International Symposium on Circuits and Systems

APL Machine Learning