Reviewer note: Marked answer (C) is factually incorrect — ReLU is well suited to deep networks and specifically helps mitigate vanishing gradients. The genuine, well-established disadvantage is the 'dying ReLU' problem (D).
I need to flag this one as well: the marked answer (C) does not hold up, and stating otherwise would misrepresent a fairly foundational deep learning fact. ReLU (Rectified Linear Unit, f(x) = max(0, x)) is, if anything, particularly well suited to deep neural networks — it was widely adopted specifically *because* it mitigates the vanishing gradient problem that plagued earlier activation functions like sigmoid and tanh in deep architectures: ReLU's gradient is a constant 1 for all positive inputs, rather than the saturating, near-zero gradients that sigmoid/tanh produce for large-magnitude inputs, which allows gradients to propagate more effectively through many layers.
The genuine, well-documented disadvantage of ReLU is option D: the "dying ReLU" problem. Because ReLU's gradient is exactly zero for any negative input, a neuron whose weighted input becomes consistently negative — often due to a large negative gradient update or an unfavorable initialization — will always output zero and will never receive a gradient large enough to recover, effectively "dying" and no longer contributing to learning. This is a real, practically significant issue that motivated variants like Leaky ReLU, Parametric ReLU (PReLU), and ELU, which allow a small non-zero gradient for negative inputs specifically to prevent neurons from dying.
Options A and B are also factually incorrect characterizations of ReLU — it is computationally cheap (a simple thresholding operation, part of its original appeal over sigmoid/tanh) and it specifically helps *avoid* vanishing gradients rather than causing them.
