Question 9
Domain 2: Core Machine Learning, AI, and Transformer FoundationsWhat is the main challenge that positional encoding solves in transformers?
Correct answer: B
Explanation
Transformers process tokens in parallel, so they do not naturally know the order of words in a sequence. Positional encoding adds information about token position to solve the “lack of inherent sequential order in parallel processing,” allowing the model to distinguish "dog bites man" from "man bites dog".
Why each option is right or wrong
A. Handling variable sequence lengths
B. Lack of inherent sequential order in parallel processing
Transformers use self-attention over all tokens at once, so without an added position signal the architecture is permutation-invariant and cannot distinguish one word order from another. Positional encoding injects location information into each token representation, enabling the model to preserve sequence structure and interpret different arrangements of the same words differently.
C. Memory limitations
D. Computational complexity