Machine Learning Engineer Interview Questions & System Design

For high-recall, low-latency enterprise RAG, I implement a hybrid retrieval architecture. Documents are parsed with hierarchical chunking (small 256-token retrieval units mapped to parent 1024-token context blocks). During query time, I run dual retrieval: dense vector search via HNSW indexing in pgvector/Pinecone for semantic similarity, and BM25 sparse keyword search to capture exact domain IDs. Candidate passages (top 50) are re-ranked using a lightweight Cross-Encoder (e.g. BGE-Reranker) down to the top 5, and served with prompt prefix caching to keep TTFT under 250ms.

Full fine-tuning updates all model parameters across all layers, requiring massive GPU memory (typically 4x model weights in VRAM for optimizer states like AdamW) and causing catastrophic forgetting. LoRA (Low-Rank Adaptation) freezes the base model weights and injects low-rank trainable decomposition matrices (A and B of rank r) into the attention layers (typically Wq, Wv). This reduces trainable parameters by 99% and VRAM consumption by 80%, allowing domain adaptation on commodity GPUs while enabling multi-tenant adapter swapping at runtime.