AI Engineering Blog

A guide to all the series and topics in this blog.

Probability¶

Tutorials and notes on probability concepts with visual, intuitive explanations.

• Introduction: Probability Tutorials & Notes

• Inclusion-Exclusion Principle

• Birthday Paradox

Observability¶

Production-first anomaly detection and observability systems.

• Introduction: Observability & Anomaly Detection

Neural Networks From Scratch¶

Build neural networks from first principles, starting with a single neuron.

• Introduction: Understanding Deep Learning From First Principles

• Edge Detection Intuition: A Single Neuron as Pattern Matching

• Training an Edge-Detection Neural Network from Scratch

• Building a General Neural Network from Scratch

• PyTorch Basics (Flexible MLP)

LLM From Scratch: Foundations¶

Build a GPT model from first principles, one concept at a time.

• Introduction: Building GPT From First Principles

• L01 - Tokenization: From Scratch

• L02 - Embeddings & Positional Encoding

• L03 - Self-Attention: Understanding Context

• L04 - Multi-Head Attention: Seeing in Parallel

• L05 - Normalization & Residuals

• L06 - The Causal Mask: How to Stop Cheating

• L07 - Assembling the GPT: The Grand Finale

• L08 - Architecture Design Choices: Choosing Your Hyperparameters

• L09 - Training the LLM: Learning to Speak

• L10 - Inference & Sampling: Controlling the Creativity

• L11 - Fine-tuning: From Completion to Conversation

LLM From Scratch: Production Techniques¶

Bridge from understanding to production-ready LLM workflows.

• Introduction: Bridge to Production

• L12 - Loading Pretrained Weights & Transfer Learning

• L13 - Data Loading Pipelines at Scale

• L14 - Evaluation Frameworks

• L15 - Parameter-Efficient Fine-Tuning (LoRA)

• L16 - Mixed Precision Training

LLM From Scratch: Scaling & Optimization¶

Advanced techniques for scale and performance.

• Introduction: Advanced Scale & Performance

• L17 - Attention Optimizations

• L18 - Model Parallelism

• L19 - Long Context Handling

• L20 - Quantization for Inference

• L21 - Deployment & Serving

ML/AI Algorithms¶

Standalone tutorials on core ML and AI algorithms.

• The Hidden Power of Latent Factors

• Alternating Least Squares (ALS) for Movie Recommendations

• Softmax: From Scores to Probabilities

• Tree-Based Feature Importance