Question 7
Domain 3: Model DevelopmentA binary classifier predicts customer churn, but only 3% of customers actually churn. Which metric is usually more informative than accuracy for judging the model?
Correct answer: A
Explanation
When churn is only 3%, accuracy can look high by predicting “no churn” for almost everyone. Precision-recall behavior is more informative because it focuses on how well the model identifies the rare positive class; the F1 score combines precision and recall into one measure.
Why each option is right or wrong
A. Precision-recall behavior such as F1 score
Under standard classification evaluation, accuracy is dominated by the majority class when prevalence is only 3%, so a trivial always-negative model would score about 97% without identifying any churners. Precision-recall metrics are preferred in this setting because they measure performance on the rare positive class directly; the F1 score is the harmonic mean of precision and recall, so it penalizes models that miss churners or generate too many false alarms.
B. Cluster silhouette score
Silhouette score evaluates clustering quality, not supervised binary classification performance.
C. Mean squared error
Mean squared error is mainly a regression loss, not the usual metric for imbalanced class detection.
D. R-squared
R-squared measures explained variance in regression, not classifier performance on churn labels.