Hi, I am Jian. Currently, I work at Monash University as a PhD candidate, under the supervision of Prof. Aldeida Aleti, Prof. Chunyang Chen, and Prof. Hongyu Zhang.

I was working as a PhD candidate (research assistant) at University of Zurich, supervised by Prof. Harald C. Gall. Before that, I obtained my master’s degree in machine learning, at KTH Royal Institute of Technology, supervised by Prof. Martin Monperrus. Further, I completed the bachelor’s study in computer science (the elite class), at Shandong University, supervised by Prof. Jun Ma.

My research interests are on the intersection between software engineering and machine learning. The focus is adapting the idea of program repair to language models, namely LM repair. If you are seeking any forms of academic cooperation, welcome to contact me via email.

🔥 News

• 2024.06: 🎉 Proudly completed our forth work on LM repair: transition on semantics!
• 2024.01: 🎉 Proudly completed our third work on LM repair: vocabulary-defined semantics!
• 2023.07: Happy to have our paper on top-down code generation accepted by FSE!
• 2023.03: 🎉 Proudly completed our first work on LM repair: semantic-based model repair!
• 2022.09: The LLM age is coming, I moved from DL4SE (University of Zurich) to SE4DL (Monash University).

💻 Featured Work

A Semantic-based Layer Freezing Approach to Efficient Fine-Tuning of LMs
Jian Gu, Aldeida Aleti, Chunyang Chen, Hongyu Zhang

Our semantic-based layer freezing approach improves the efficiency of language model finetuning by determining where to finetune, outperforming existing methods through a detailed semantic analysis of the model’s inference process.

Realizing Disentanglement in LM Latent Space via Vocabulary-Defined Semantics
Jian Gu, Aldeida Aleti, Chunyang Chen, Hongyu Zhang

Vocabulary-defined Semantics enhances language model understanding and adaptability through disentangled semantic analysis, improved logits computation, and neural clustering, outperforming existing techniques in diverse text understanding tasks.

Neuron Patching: Semantic-based Neuron-level LM Repair for Code Generation
Jian Gu, Aldeida Aleti, Chunyang Chen, Hongyu Zhang

MENT is an efficient and reliable model editing approach for Large Language Models in software engineering. It successfully patches 1 or 2 neurons, outperforming state-of-the-art methods in coding tasks, and demonstrates enhanced reasoning in LLMs.

Towards Top-Down Automated Development in Limited Scopes: A Neuro-Symbolic Framework from Expressibles to Executables
Jian Gu, Harald C. Gall

Deep code generation integrates neural models into software engineering for generating code but requires enhancements for project-level tasks, suggesting a taxonomy on code data and introducing a semantic pyramid framework to improve software development processes.

📝 Publications

Software Engineering for Deep Learning

• ArXiv A Semantic-based Layer Freezing Approach to Efficient Fine-Tuning of LMs
  Jian Gu, Aldeida Aleti, Chunyang Chen, Hongyu Zhang
• ArXiv Realizing Disentanglement in LM Latent Space via Vocabulary-Defined Semantics
  Jian Gu, Aldeida Aleti, Chunyang Chen, Hongyu Zhang
• Revision Focus-aware Code LLMs: Coding with Selective Attention
  Jian Gu, Aldeida Aleti, Chunyang Chen, Hongyu Zhang
• ArXiv Neuron Patching: Semantic-based Neuron-level LM Repair for Code Generation
  Jian Gu, Aldeida Aleti, Chunyang Chen, Hongyu Zhang

Deep Learning for Software Engineering

• Revision Memory Augmentation Methods for Continuous Code Summarization
  Jian Gu, Harald C. Gall
• FSE'23 Towards Top-Down Deep Code Generation in Limited Scopes
  Jian Gu, Harald C. Gall
• SANER'22 Assemble Foundation Models for Automatic Code Summarization
  Jian Gu, Pasquale Salza, Harald C. Gall
• ICSME'21 Multimodal Representation for Neural Code Search
  Jian Gu, Zimin Chen, Martin Monperrus
• TSE 2022 On the Effectiveness of Transfer Learning for Code Search
  Pasquale Salza, Christoph Schwizer, Jian Gu, Harald C. Gall
• TSE 2021 Automated Classification of Overfitting Patches with Statically Extracted Code Features
  He Ye, Jian Gu, Matias Martinez, Thomas Durieux, Martin Monperrus

“Machine intelligence is the last invention that humanity will ever need to make. Machines will then be better at inventing than we are, and they’ll be doing so on digital timescales.” – Nick Bostrom