In transformer-based LLMs, how does the use of multi-head attention improve model performance compared to...

Multi-head attention, a core component of the transformer architecture, improves model performance by allowing the model to attend to multiple aspects of the input sequence simultaneously. Each attention head learns to focus on different relationships (e.g., syntactic, semantic) in the input, capturing diverse contextual dependencies. According to "Attention is All You Need" (Vaswani et al., 2017) and NVIDIA’s NeMo documentation, multi-head attention enhances the expressive power of transformers, making them highly effective for complex NLP tasks like translation or question-answering. Option A is incorrect, as multi-head attention increases memory usage. Option C is false, as positional encodings are still required. Option D is wrong, asmulti-head attention adds parameters.