BTechibtechi.in
Loading Loading
Free IIT Madras BS notes, PYQs and mock tests at btechi.in

RL · All 2025 papers

16 questions · 50 marks · 1 paper

Answer key

Indian Institute of Technology Madras

BS Degree Programme · Examination & Study Paper

Reinforcement Learning

All 2025 papers

Official Paper

Examination Instructions & Guidelines

Solve on notebook/paper first before checking solutions.
MCQ: Exactly 1 option is correct. Negative marking applies.
MSQ: One or more options correct. Select all to score.
NAT: Numerical response. No negative marking.
Scientific calculator allowed for all calculations.
Official answer key is sealed at the end of this sheet.

Q1MCQ3 Marks·Choose one option
Key ↓
1. A
2. B
3. C
4. D
Q2MCQ3 Marks·Choose one option
Key ↓
Consider following assertion reason pair: Assertion: In the UCB algorithm for multi-armed bandits, replacing the upper confidence bound with a lower confidence bound (and greedily selecting actions based on that lower bound) would still promote effective exploration or lead to optimal reward maximisation. Reason: Even when lower confidence bounds are used instead of upper bounds, if the algorithm greedily selects arms based on these lower bounds, it would still encourage exploration and ultimately achieve optimal reward maximisation.
1. ABoth Assertion and Reason are true, and Reason is the correct explanation of Assertion.
2. BBoth Assertion and Reason are true, but Reason is NOT the correct explanation of Assertion.
3. CAssertion is true, Reason is false
4. DAssertion is false, Reason is false
Q3MCQ5 Marks·Choose one option
Key ↓
1. APolicy Iteration
2. BValue Iteration
3. CBoth require equal updates
4. DIt cannot be determined
Q4MCQ2 Marks·Choose one option
Key ↓
1. AArm 1
2. BArm 2
3. CArm 3
4. DArm 4
Q5MSQ5 Marks·Choose all correct options
Key ↓
Which of the following statements correctly describes the differences in computational complexity and convergence behaviour between Policy Iteration and Value Iteration algorithms in solving Markov Decision Processes?
1. AValue Iteration requires fewer iterations to converge, but each iteration is computationally more expensive due to the policy evaluation step.
2. BValue Iteration performs a combined update of value estimation and policy improvement in one step and requires more iterations to converge compared to Policy Iteration.
3. CPolicy Iteration can only be applied to small state spaces, while Value Iteration scales well to large state spaces.
4. DValue Iteration is simpler to implement because it only maintains a value function, while Policy Iteration maintains both policy and value function.
5. EBoth Policy Iteration and Value Iteration are guaranteed to converge to the optimal policy.
Q6NAT5 Marks·Write the number
Key ↓
Your answer:
Q7NAT4 Marks·Write the number
Key ↓
Your answer:
Read the passage below, then answer Q8–Q9
Suppose you face a 2-armed bandit task where, at each time step, the true action values are either (10, 20) with probability 0.7 (case A) or (90, 80) with probability 0.3 (case B).
Based on the above data, answer the given subquestions.
Q8MCQ1 Mark·Choose one option
Key ↓
If you cannot observe which case you face, what is the best expected reward per step you can achieve, and what strategy should you follow?
1. ARandomly choose between Action 1 and Action 2; expected reward is 3.
2. BChoose Action 1 always; expected reward is 34.
3. CChoose Action 2 always; expected reward is 38.
4. DAlternate between Action 1 and Action 2; expected reward is 40.
Q9MCQ1 Mark·Choose one option
Key ↓
If, at each time step, you are told whether you face case A or case B (but not the true action values), what is the best expected reward per step you can achieve,and what strategy should you follow?
1. APick Action 2 in case A, Action 1 in case B; expected reward is 41.
2. BAlways pick Action 2; expected reward is 38.
3. CAlways pick Action 1; expected reward is 34.
4. DThe expected reward stated in all the given options is incorrect.
Read the passage below, then answer Q10–Q12
Based on the above data, answer the given subquestions.
Q10MCQ2 Marks·Choose one option
Key ↓
Which of the following best explains why a contextual bandit approach is preferred over a standard multi-armed bandit in this scenario?
1. ABecause the optimal waiting time is the same for all VMs, regardless of their context.
2. BBecause contextual bandits can adaptively select actions based on the specific features of each VM failure event, maximising expected reward across diverse situations.
3. CBecause multi-armed bandits are unable to balance exploration and exploitation.
4. DBecause contextual bandits always guarantee zero regret after every round.
Q11MCQ4 Marks·Choose one option
Key ↓
1. AIt determines the penalty for rebooting or migrating the VM.
2. BIt is used to predict the expected reward for each possible waiting time,allowing the algorithm to personalize decisions for each failure event.
3. CIt is ignored by LinUCB, which only uses past rewards.
4. DIt is used to randomly select an action to ensure exploration.
Q12MCQ2 Marks·Choose one option
Key ↓
What is the main optimality goal of a contextual bandit algorithm over many rounds?
1. AMaximise the total expected reward by acting as closely as possible to the best policy mapping contexts to actions.
2. BAchieve zero regret after every individual round by always picking the empirically best action so far for each context
3. CAlways exploit the highest-reward action seen so far
4. DMinimise the loss on the worst round
Read the passage below, then answer Q13–Q14
Based on the above data, answer the given subquestions.
Q13NAT2 Marks·Write the number
Key ↓
What is the total reward if the robot collects both treasures and then exits,taking the shortest possible path and never hitting a wall?
Your answer:
Q14MSQ3 Marks·Choose all correct options
Key ↓
1. A
2. B
3. C
4. D
Read the passage below, then answer Q15–Q16
Consider a pole-balancing task where the goal is to apply forces to a cart moving along a track to keep a hinged pole from falling. A failure event occurs when the pole falls past a certain angle or the cart moves off the track, after which the pole resets to vertical.
The problem can be formulated either as an episodic task (with episodes ending at failure) or as a continuing task (after each failure, pole gets back into its initial position,
potentially infinite time horizon with discounting).Rewards can be defined as:
• Episodic: Reward +1 for each timestamp without failure, so the return is the number of steps until failure (possibly infinite if balanced forever).
• Continuing: Reward 0 for each timestamp except at each failure, where the reward is −1. Returns are discounted sums of future rewards.
Based on the above data, answer the given subquestions.
Q15MCQ4 Marks·Choose one option
Key ↓
1. A
2. B
3. C
4. D
Q16MCQ4 Marks·Choose one option
Key ↓
How does the return in the episodic, discounted pole-balancing task differ from the continuing formulation with the same reward structure?
1. AEpisodic return is a single discounted reward at failure; continuing return is an infinite sum over all future failures.
2. BBoth returns are always zero
3. CEpisodic return accumulates rewards at every time step; continuing return only at failure.
4. DThere is no difference between the two formulations.

Official Key

Master Answer Key & Explanations

Official answer keys from IIT Madras BS past papers. Click any question to review.

Reveal Key

Q1C

Q2D

Q3B

Q4C

Q5B, D, E

Q64

Q70.2

Q8C

Q9A

Q10B

Q11B

Q12A

Q1376

Q14A, D

Q15C

Q16A

Questions and answer keys are reproduced from the official IIT Madras BS papers. BTechi is not affiliated with IIT Madras.