Current Projects

Understanding & building compositionality

We study whether and how foundation models can identify compositional components across three levels: concepts, reasoning patterns, and parameter modules. Our goal is to make model capabilities more interpretable, diagnosable, and reusable, so that complex abilities can be later built by composing well-defined components.

• Concepts: extracting reusable multimodal semantic components from tokens, pixels, and latent representations. Related: Concept-Guided Tokenization; Plug-and-Play Compositionality; Visualization of Multimodal Concepts.
• Reasoning patterns: decomposing long-chain and multi-hop reasoning into structural sub-patterns. Related: LCoT2Tree / What Makes a Good Reasoning Chain?; Reasoning Pattern Discrepancy in Long-CoT SFT.
• Parameter modules: discovering sparse experts and dynamic modular routing inside foundation models. Related: Automatic Expert Discovery in LLM Upcycling.
• Skills: ongoing

Lifelong learning through compositionality

We study how foundation models and agents can evolve continuously by localizing compositional components to delete, add, consolidate, or correct. Our key idea is to use compositional components (such as concepts, reasoning patterns, modules and skills) as anchors for precise capability gap localization and targeted evolution across continual post-training and test-time / agent evolution.

• Continual post-training: localizing and adapting reusable parameter subspaces, function vectors, and LoRA modules to support efficient but stable capability evolution. Related: SD-LoRA; Function Vectors for Continual Instruction Tuning; SIMoE;
• Continual test-time / agent evolution: diagnosing reasoning failures or task-level capability gaps during inference and agent execution, then correcting reasoning paths, composing edits or adapting skills within and across tasks. Related: One-Token Verification; Scaling Reasoning Hop; Towards Compositional Model Editing. Learning to Edit.

Efficient adaptation

We develop efficient adaptation methods that adapt foundation models more effectively and efficiently. Our work approaches this problem from three complementary directions: improving the optimization dynamics of lightweight adapters (CoTo), designing adapters that better generalize across evolving domains (Terra), and compressing large models through quantization while preserving their reasoning and generation capabilities (DuQuant).

Applications

We study application scenarios where foundation models and agents must continuously improve their reasoning and decision-making in real-world settings. AI for Science is our current major focus, covering retrosynthesis modeling (ARP, RetrOchestro), drug/materials discovery, synthetic biology, and more. We also track broader complex-reasoning benchmarks, math, code, agentic tasks, and beyond, wherever current foundation models fall short and model evolution is essential.