Level 3 - AI Engineering and LLMOps

About Course

AI Engineering and LLMOps is an advanced, hands-on course for learners who want to understand how AI systems are engineered, deployed and operated in production.

You will move beyond building individual AI applications to explore what happens underneath them. The course covers LLM inference, model serving, KV caching, batching, quantisation, deployment architecture, evaluation, observability, security and cost optimisation.

Through practical exercises, you will learn how to turn an AI prototype into a reliable production system. You will examine the engineering decisions behind latency, throughput, scalability and model quality, and learn how those decisions affect users and operating costs.

By the end of the course, you will have designed and implemented a production-ready AI system, complete with evaluation, monitoring, deployment and operational considerations.

* Understand how LLM inference works, including prefill, decoding and token generation
* Explain how KV caching, batching and scheduling affect performance
* Compare model hosting, managed APIs and self-hosted deployment
* Apply quantisation techniques to reduce memory and compute requirements
* Design scalable model-serving and AI application architectures
* Measure and improve latency, throughput, reliability and cost
* Build evaluation pipelines for AI quality, safety and regression testing
* Monitor prompts, model outputs, errors, token usage and system performance
* Apply security and governance controls to production AI systems
* Design deployment, rollback and incident-response processes
* Make informed trade-offs between model quality, speed, infrastructure and cost
* Build and present a production-focused LLMOps capstone project

Course Content

Deep Learning from First Principles
- Tensors, parameters, activations and forward propagation - Loss functions, gradient descent and backpropagation - PyTorch: autograd, modules, optimisers and training loops Practical Build and train a neural network in PyTorch and inspect how its parameters change during optimisation.

Transformers from First Principles
- Tokenisation, embeddings and positional information - Query, Key and Value projections; scaled dot-product attention - Multi-head attention, residuals, normalisation and stacking Practical Implement core components of self-attention and inspect the resulting tensor operations.

Understanding LLM Architecture
- Text to tokens to embeddings to logits to next token - Hidden dimensions, layers, attention heads, MHA, MQA and GQA - Inspecting a real small open-weight model Practical Load an open model and trace data through the key components of its architecture.

LLM Inference
- Prefill, decode, autoregressive generation and sampling - Compute, memory bandwidth and arithmetic intensity - Latency, throughput, time to first token, inter-token latency Practical Build a simplified generation loop and benchmark model behaviour.

Fine-Tuning and Model Adaptation
- Prompting versus RAG versus fine-tuning - Dataset preparation and training objectives - LoRA, QLoRA, PEFT and the training trade-offs Practical Prepare a small dataset and adapt an open-weight model using parameter-efficient techniques.

Quantisation
- Precision, GPU memory, bandwidth, speed and model quality - FP32, FP16, BF16, INT8 and INT4 - Scaling, zero points, calibration, GPTQ and AWQ Practical Implement a simple quantiser from first principles, then apply existing tooling to an open model.

KV Caching and LLM Memory
- Why previously computed Keys and Values can be reused - Cache dimensions, sequence length, batch size and KV heads - Fragmentation, paging and why PagedAttention changed serving Practical Calculate KV cache requirements for different model configurations and inspect caching during generation.

Batching, Scheduling and LLM Serving
- Static, dynamic and continuous batching - Prefill versus decode workloads, queues and admission control - Chunked prefill and the architecture of engines such as vLLM Practical Run and benchmark an inference workload, and observe how batching changes latency and throughput.

Production Deployment and Scaling
- Model servers, containers, GPU deployment and observability - Benchmarking: TTFT, tokens per second, GPU and memory utilisation - Data, tensor, pipeline, context and expert parallelism Practical Deploy and benchmark a model-serving endpoint.

Evaluation, Observability and Cost
- Evaluating output quality before and after optimisation - Building test datasets and regression checks - Monitoring latency, throughput, errors and token usage - Tracing requests across an LLM system - GPU utilisation and memory monitoring - Calculating cost per request and cost per token - Balancing quality, speed, reliability and cost Practical Build a production evaluation and monitoring dashboard for an LLM service, then compare several deployment configurations using quality, performance and cost.

AI Engineering Capstone
- Defining service-level objectives - Capacity planning and performance targets - Deployment, versioning and rollback - Incident response and operational readiness - Security, governance and responsible deployment - Communicating engineering trade-offs Practical Design, deploy and present an optimised LLM system. Learners must benchmark the baseline, introduce at least one optimisation, evaluate its effect on quality and performance, and explain the final deployment architecture, monitoring strategy and operating cost.

Student Ratings & Reviews

No Review Yet

Level 3 – AI Engineering and LLMOps

About Course

What Will You Learn?

Course Content

Understanding LLM Architecture
- Text to tokens to embeddings to logits to next token - Hidden dimensions, layers, attention heads, MHA, MQA and GQA - Inspecting a real small open-weight model Practical Load an open model and trace data through the key components of its architecture.

LLM Inference
- Prefill, decode, autoregressive generation and sampling - Compute, memory bandwidth and arithmetic intensity - Latency, throughput, time to first token, inter-token latency Practical Build a simplified generation loop and benchmark model behaviour.

Fine-Tuning and Model Adaptation
- Prompting versus RAG versus fine-tuning - Dataset preparation and training objectives - LoRA, QLoRA, PEFT and the training trade-offs Practical Prepare a small dataset and adapt an open-weight model using parameter-efficient techniques.

Quantisation
- Precision, GPU memory, bandwidth, speed and model quality - FP32, FP16, BF16, INT8 and INT4 - Scaling, zero points, calibration, GPTQ and AWQ Practical Implement a simple quantiser from first principles, then apply existing tooling to an open model.

Production Deployment and Scaling
- Model servers, containers, GPU deployment and observability - Benchmarking: TTFT, tokens per second, GPU and memory utilisation - Data, tensor, pipeline, context and expert parallelism Practical Deploy and benchmark a model-serving endpoint.

About Course

What Will You Learn?

Course Content

Understanding LLM Architecture - Text to tokens to embeddings to logits to next token - Hidden dimensions, layers, attention heads, MHA, MQA and GQA - Inspecting a real small open-weight model Practical Load an open model and trace data through the key components of its architecture.

LLM Inference - Prefill, decode, autoregressive generation and sampling - Compute, memory bandwidth and arithmetic intensity - Latency, throughput, time to first token, inter-token latency Practical Build a simplified generation loop and benchmark model behaviour.

Fine-Tuning and Model Adaptation - Prompting versus RAG versus fine-tuning - Dataset preparation and training objectives - LoRA, QLoRA, PEFT and the training trade-offs Practical Prepare a small dataset and adapt an open-weight model using parameter-efficient techniques.

Quantisation - Precision, GPU memory, bandwidth, speed and model quality - FP32, FP16, BF16, INT8 and INT4 - Scaling, zero points, calibration, GPTQ and AWQ Practical Implement a simple quantiser from first principles, then apply existing tooling to an open model.

Production Deployment and Scaling - Model servers, containers, GPU deployment and observability - Benchmarking: TTFT, tokens per second, GPU and memory utilisation - Data, tensor, pipeline, context and expert parallelism Practical Deploy and benchmark a model-serving endpoint.

Student Ratings & Reviews

Understanding LLM Architecture
- Text to tokens to embeddings to logits to next token - Hidden dimensions, layers, attention heads, MHA, MQA and GQA - Inspecting a real small open-weight model Practical Load an open model and trace data through the key components of its architecture.

LLM Inference
- Prefill, decode, autoregressive generation and sampling - Compute, memory bandwidth and arithmetic intensity - Latency, throughput, time to first token, inter-token latency Practical Build a simplified generation loop and benchmark model behaviour.

Fine-Tuning and Model Adaptation
- Prompting versus RAG versus fine-tuning - Dataset preparation and training objectives - LoRA, QLoRA, PEFT and the training trade-offs Practical Prepare a small dataset and adapt an open-weight model using parameter-efficient techniques.

Quantisation
- Precision, GPU memory, bandwidth, speed and model quality - FP32, FP16, BF16, INT8 and INT4 - Scaling, zero points, calibration, GPTQ and AWQ Practical Implement a simple quantiser from first principles, then apply existing tooling to an open model.

Production Deployment and Scaling
- Model servers, containers, GPU deployment and observability - Benchmarking: TTFT, tokens per second, GPU and memory utilisation - Data, tensor, pipeline, context and expert parallelism Practical Deploy and benchmark a model-serving endpoint.