BTechibtechi.in
Loading Loading
Free IIT Madras BS notes, PYQs and mock tests at btechi.in

LLM · Transformer & self-attention

16 questions · 40 marks · 3 papers

Answer key

Indian Institute of Technology Madras

BS Degree Programme · Examination & Study Paper

Large Language Models

Transformer & self-attention

Official Paper

Examination Instructions & Guidelines

Solve on notebook/paper first before checking solutions.
MCQ: Exactly 1 option is correct. Negative marking applies.
MSQ: One or more options correct. Select all to score.
NAT: Numerical response. No negative marking.
Scientific calculator allowed for all calculations.
Official answer key is sealed at the end of this sheet.

Quiz 1 · 26 Oct 2025 (AN)Quiz 1

Q1MCQ2 Marks·Choose one option
Key ↓
1. ATo increase the numerical precision of the attention scores.
2. BTo reduce the overall computational cost of the attention mechanism.
3. CTo ensure that the sum of the attention scores for each query equals 1.
4. DTo avoid numerical issues and loss of gradients during training.
Q2MCQ2 Marks·Choose one option
Asked 2 times Key ↓
1. A
2. B
3. C
4. D
5. E
Q3MCQ3 Marks·Choose one option
Key ↓
1. A
2. B
3. C
4. D
Q4MSQ2 Marks·Choose all correct options
Key ↓
Which of the following statements about multiple attention heads in a Transformer are true?
1. AThey allow learning diverse local/global relations.
2. BThey reduce memory usage compared to one head.
3. CIncreasing the number of heads while keeping d_(model) fixed, decreases the dimensionality handled by each head.
4. DAll heads necessarily learn completely independent information about the input.
Q5MSQ4 Marks·Choose all correct options
Key ↓
Which of the following statements about the Transformer architecture are true?
1. AMasking (look-ahead mask) is applied in every decoder self-attention layer, not just the first one.
2. BToken embeddings are used in the encoder but not in the decoder.
3. CPositional encoding is required only in the encoder, not in the decoder.
4. DToken embeddings are used in both the encoder and the decoder.
5. EPositional encodings are added to the input embeddings in both the encoder and the decoder.
Q6NAT3 Marks·Write the number
Key ↓
Your answer:
Q7NAT3 Marks·Write the number
Key ↓
Your answer:
Q8NAT2 Marks·Write the number
Key ↓
Your answer:
Read the passage below, then answer Q9–Q11
Q9NAT3 Marks·Write the number
Key ↓
How many next-token prediction targets are generated during training a full batch?
Your answer:
Q10NAT2 Marks·Write the number
Key ↓
For one sequence, how many non-zero attention scores remain in the masked attention score matrix (per head)?
Your answer:
Q11NAT2 Marks·Write the number
Key ↓
Your answer:

End Term · 31 Aug 2025 (FN)End Term

Read the passage below, then answer Q12–Q13
Q12NAT3 Marks·Write the number
Asked 2 times Key ↓
For the input “you enjoy tea often”, compute the final representation of the word “tea” after the attention layer (i.e., after applying W_Q, W_K, W_V , attention weights, and W_O). Enter the sum of the elements in the resulting vector.
Your answer:
Q13NAT2 Marks·Write the number
Asked 2 times Key ↓
Suppose the input sentence is “tea you enjoy often”. Using the same matrices and processing method, what is the attention score (i.e., softmax entry from A) for the query word “tea” attending to key word “often”?
Your answer:

End Term · 31 Aug 2025 (AN)End Term

Q14MCQ2 Marks·Choose one option
Asked 2 times Key ↓
A GPT model is trained using causal language modeling. During training, for a sequence of T = 4 tokens, which of the following correctly represents the attention mask matrix applied to the attention logits?
1. A
2. B
3. C
4. D
Read the passage below, then answer Q15–Q16
Q15NAT3 Marks·Write the number
Asked 2 times Key ↓
For the input “you enjoy tea often”, compute the final representation of the word “enjoy” after the attention layer (i.e., after applying W_Q, W_K, W_V , attention weights, and W_O). Enter the sum of the elements in the resulting vector.
Your answer:
Q16NAT2 Marks·Write the number
Asked 2 times Key ↓
Suppose the input sentence is “tea you enjoy often”. Using the same matrices and processing method, what is the attention score (i.e., softmax entry from A) for the query word “tea” attending to key word “often”?
Your answer:

Official Key

Master Answer Key & Explanations

Official answer keys from IIT Madras BS past papers. Click any question to review.

Reveal Key

Q1D

Q2C

Q3A

Q4A, C

Q5A, D, E

Q61152

Q799 to 101

Q832768

Q932736

Q10524800

Q11589824

Q120.98 to 1.06

Q130.04 to 0.1

Q14A

Q150.92 to 1

Q160.04 to 0.1

Questions and answer keys are reproduced from the official IIT Madras BS papers. Topics and repeated question types are BTechi’s own grouping. BTechi is not affiliated with IIT Madras.