Large language models and generative AI are transforming every domain, but their impact depends on building systems that can train and serve these models efficiently. This course provides a holistic view of LLM acceleration, showing how algorithmic choices ripple through models, training, inference, hardware, compilers, and serving systems.
The course is organized in three parts. First, we build up the model: starting from the end of Dennard scaling and the case for domain-specific acceleration, we cover scaling laws, optimization, learning paradigms and alignment, the transformer architecture and diffusion models, advanced model capabilities (mixture of experts, multimodal, reasoning), and retrieval-augmented and agentic systems. Second, we turn to training and efficiency: distributed training across thousands of GPUs, efficient inference architectures (speculative decoding, grouped-query attention, state-space models), and model compression through quantization, low-rank adaptation, and sparsity. Third, we study the systems that run it all: GPU architecture and kernel optimizations (FlashAttention), domain-specific accelerators (TPUs, Groq, processing-in-memory), compiler optimizations (TVM, MLIR), LLM serving software (continuous batching, structured generation), KV cache management and prefill-decode disaggregation, and network infrastructure including DPUs. We close with a lookback on the quantitative laws and design philosophies that tie the course together.
Throughout the course, you will develop both foundational understanding and familiarity with state-of-the-art approaches across these topics. You will read and critically analyze research papers, identify strengths and weaknesses of proposed techniques, and are expected to engage in class discussions with critical judgment. A course project will give you hands-on experience applying these ideas to a research problem of your choosing.
The course will be a combination of lectures, student presentations, and a project. For more information on the project, please refer to the project page.
This is a research-oriented advanced course for graduate students. Students are expected to have background in a) computer architecture, b) deep learning, and c) software engineering (proficiency in Python or C/C++). The course is open to both PhD and MS students. Undergraduate students require permission from the instructor.
Students will be evaluated based on the following rubric. For the details on each item please refer to its corresponding page.
| Critiques | 30% |
| Class Presentations | 20% |
| Participation | 10% |
| Projects: Proposal, Presentation, Report, and Implementation | 40% (10% each) |
Attendance is required. The TA will hold a sign-up sheet at every class. You may miss up to two classes without penalty. Beyond that, if you need to miss a class, you must communicate this via email to the TA at least one day in advance.
All course communication will be through email. Please direct questions to the TA (hax032 [AT] ucsd [DOT] edu).
There is no required textbook. All relevant materials will be made available online.
Students are expected to abide by the UC San Diego Honor Code. Honest and ethical behavior is expected at all times. All incidents of suspected dishonesty will be reported to and handled by the office of student affairs. You will have to do all assignments individually unless explicitly told otherwise. You may discuss with classmates but you may not copy any solution (or any part of a solution).
There are no restrictions on the use of AI tools (including LLMs) in this course. You are welcome to use them for critiques, presentations, projects, and any other coursework. That said, the goal of this course is for you to build deep expertise in this field. If you delegate your learning entirely to AI, the only one missing out is you.