BTechibtechi.in
Loading Loading
Free IIT Madras BS notes, PYQs and mock tests at btechi.in

LLM · Pretraining, T5, data & scaling laws

11 questions · 29 marks · 3 papers

Answer key

Indian Institute of Technology Madras

BS Degree Programme · Examination & Study Paper

Large Language Models

Pretraining, T5, data & scaling laws

Official Paper

Examination Instructions & Guidelines

Solve on notebook/paper first before checking solutions.
MCQ: Exactly 1 option is correct. Negative marking applies.
MSQ: One or more options correct. Select all to score.
NAT: Numerical response. No negative marking.
Scientific calculator allowed for all calculations.
Official answer key is sealed at the end of this sheet.

End Term · 31 Aug 2025 (FN)End Term

Q1MSQ3 Marks·Choose all correct options
Key ↓
Which of the following are valid design choices when building a transformer-based language model?
1. ANumber of transformer layers
2. BLearning rate schedule
3. CFont style of the source code
4. DVocabulary size
5. EType of normalization (e.g., LayerNorm)
Q2MSQ3 Marks·Choose all correct options
Asked 2 times Key ↓
Which of the following preprocessing steps are commonly required when preparing data for transformer models like BERT?
1. ALowercasing text (for uncased models)
2. BSplitting into subword units using BPE or WordPiece
3. CRemoving all punctuation marks
4. DAdding position indices for each token

End Term · 31 Aug 2025 (AN)End Term

Q3MSQ3 Marks·Choose all correct options
Asked 2 times Key ↓
Which of the following preprocessing steps are commonly used when preparing data for transformer models like BERT?
1. AAdding special tokens such as [CLS] and [SEP] to the sequence
2. BSplitting tokens into subword units using a model-specific tokenizer
3. CRemoving all punctuation marks to ensure cleaner embeddings
4. DAdding positional information to each token embedding

Quiz 2 · 3 Aug 2025 (AN)Quiz 2

Q4MCQ2 Marks·Choose one option
Key ↓
What is a key implication of scaling laws in large language models?
1. ATraining longer always leads to better performance
2. BLarger models require less training data
3. CModel size and dataset size must scale together to maintain optimal performance
4. DNone of these
Q5MCQ2 Marks·Choose one option
Key ↓
What is the primary advantage of T5’s text-to-text approach for transfer learning in NLP?
1. AIt reduces the computational cost of training models significantly.
2. BIt allows a single model and architecture to be applied to a wide variety of NLP tasks without modification.
3. CIt guarantees optimal performance on all tasks without fine-tuning.
4. DIt eliminates the need for any pre-training data.
Q6MCQ2 Marks·Choose one option
Key ↓
What is the format of output the T5 model is expected to generate during its span-corruption pretraining?
1. A
2. B
3. C
4. D
Q7MCQ3 Marks·Choose one option
Key ↓
Suppose you are given 4x more compute budget to scale a Transformer-based language model. According to scaling laws, what is the best way to utilize this increased compute for optimal model performance?
1. AIncrease the model size (number of parameters) by 4x and keep the dataset size and training steps the same.
2. BTrain the same model on 4x more data for 4x more steps without changing the model size.
3. CIncrease the model size by approximately 2x and increase the amount of training data by approximately 2x.
4. DDouble the model size and halve the training data size to compensate.
Q8MSQ3 Marks·Choose all correct options
Key ↓
In a data preprocessing stage designed to prepare clean, diverse, and safe data, which of the following examples is likely to be filtered out?
1. A“All people from that xyz area are thieves. They should be jailed immediately.”
2. BKavya’s PAN number is BDZPK1234K and her registered address is 17/4, Green Park, Delhi.
3. C[“Reinforcement learning allows agents to learn optimal strategies through interaction with an environment.”, “Reinforcement learning allows agents to learn optimal strategies through interaction with an environment.”]
4. DThe Union Budget 2024 proposes increased spending on green energy infrastructure.
5. EAccording to WHO, global life expectancy has increased by 6 years since 2000.
6. FNone of these.
Q9MSQ3 Marks·Choose all correct options
Key ↓
Select all the correct input-output representations used for different pretraining objectives based on the base sentence: “The quick brown fox jumps over the lazy dog”
1. A
2. B
3. C
4. D
Q10NAT2 Marks·Write the number
Key ↓
Your answer:
Q11NAT3 Marks·Write the number
Key ↓
Your answer:

Official Key

Master Answer Key & Explanations

Official answer keys from IIT Madras BS past papers. Click any question to review.

Reveal Key

Q1A, B, D, E

Q2A, B, D

Q3A, B, D

Q4C

Q5B

Q6B

Q7C

Q8A, B, C

Q9A, D

Q1010000

Q110.914 to 0.975

Questions and answer keys are reproduced from the official IIT Madras BS papers. Topics and repeated question types are BTechi’s own grouping. BTechi is not affiliated with IIT Madras.