Question 5
Domain 1: Generative AI with LLMs and PromptingIn fine-tuning scenarios, what is catastrophic forgetting?
Correct answer: A
Explanation
Catastrophic forgetting is when a model "los[es] pre-trained knowledge during task-specific training" because new fine-tuning updates overwrite earlier learned patterns. In fine-tuning, the model adapts to the new task, but without safeguards it can degrade performance on previously learned information.
Why each option is right or wrong
A. Model losing pre-trained knowledge during task-specific training
During task-specific fine-tuning, gradient updates are applied to the same parameters that encoded the model’s earlier general representations, so those weights can be overwritten and prior capabilities degrade. In the literature this is the standard definition of catastrophic forgetting: performance on previously learned knowledge drops as the model adapts to the new objective, especially when training is narrow and no replay, regularization, or parameter-freezing is used.
B. Model forgetting the current fine-tuning task
C. Memory overflow during training process
D. Gradient explosion during optimization