Page Reader AI | Exam Professional Machine Learning Engineer topic 1 question 145 discussion

Exam Professional Machine Learning Engineer topic 1 question 145 discussion - ExamTopics

AI Summary Hide AI Generated Summary

Problem

A Deep Neural Network (DNN) regressor trained with TensorFlow to predict housing prices exhibits a serving latency of 10ms @ 90th percentile on CPUs, exceeding the production requirement of 8ms @ 90th percentile. The goal is to reduce latency while minimizing performance degradation.

Options

Switch to GPU serving.
Apply quantization to reduce precision to tf.float16.
Increase dropout rate to 0.8 and retrain.
Increase dropout rate to 0.8 in PREDICT mode by adjusting TensorFlow Serving parameters.

Solution

The suggested answer is B: Applying quantization by reducing the floating-point precision to tf.float16. This approach directly addresses the latency issue by reducing computational complexity.

You have trained a DNN regressor with TensorFlow to predict housing prices using a set of predictive features. Your default precision is tf.float64, and you use a standard TensorFlow estimator:

Your model performs well, but just before deploying it to production, you discover that your current serving latency is 10ms @ 90 percentile and you currently serve on CPUs. Your production requirements expect a model latency of 8ms @ 90 percentile. You're willing to accept a small decrease in performance in order to reach the latency requirement. Therefore your plan is to improve latency while evaluating how much the model's prediction decreases. What should you first try to quickly lower the serving latency?

A. Switch from CPU to GPU serving.
B. Apply quantization to your SavedModel by reducing the floating point precision to tf.float16.
C. Increase the dropout rate to 0.8 and retrain your model.
D. Increase the dropout rate to 0.8 in _PREDICT mode by adjusting the TensorFlow Serving parameters.

Show Suggested Answer Hide Answer

Was this article displayed correctly? Not happy with what you see?

See Archived Versions