Question 27
Domain 3: Implement Generative AI SolutionsA company ran a 6-month fine-tuning project on GPT-4 with their proprietary technical documentation. After deployment, users report the model answers questions incorrectly because the documentation has been updated since fine-tuning. What is the fundamental architectural mistake, and what should they switch to?
Correct answer: B
Explanation
Fine-tuning bakes in a static snapshot of the training data, so it cannot stay current when the documentation changes after deployment. They should switch to retrieval-augmented generation (RAG) with Azure AI Search so the model can retrieve the latest documents at answer time instead of relying on outdated weights.
Why each option is right or wrong
A. The model was over-trained; reduce the number of training epochs
B. Fine-tuning captured a static knowledge snapshot; switch to RAG with Azure AI Search for dynamic knowledge retrieval
Fine-tuning updates model weights on the documentation as it existed during training, so the deployed GPT-4 is effectively answering from a frozen 6-month-old snapshot rather than the current corpus. When the source docs change after training, the model has no built-in mechanism to fetch those updates at inference time; the architectural fix is to externalize knowledge with retrieval-augmented generation, using Azure AI Search to retrieve the latest indexed documents before generation.
C. The fine-tuned model was deployed in the wrong region
D. The prompt temperature is too high after fine-tuning; set it to 0