A machine learning engineer needs to deliver predictions of a machine learning model in real-time.

Batch serving is a type of model deployment where predictions are computed in advance and stored for later use. This can be useful when the feature values are known ahead of time and the predictions need to be delivered quickly. Querying stored predictions can be faster than computing predictions in real-time, especially if the model is complex or the data volume is high. Real-time serving is a type of model deployment where predictions are computed on-demand at query time. This can be useful when the feature values are not known in advance or the predictions need to be up-to-date. However, computing predictions in real-time can be slower than querying stored predictions, especially if the model is simple or the data volume is low. References: [Batch Serving], [Real-time Serving] Message has links.