Indian Institute of Technology Madras
BS Degree Programme · Examination & Study Paper
Business Data Management
A/B testing & e-commerce
Examination Instructions & Guidelines
- Solve on notebook/paper first before checking solutions.
- MCQ: Exactly 1 option is correct. Negative marking applies.
- Scientific calculator allowed for all calculations.
- Official answer key is sealed at the end of this sheet.
End Term · 31 Aug 2025 (FN)End Term
- Q1MCQ1 Mark·Choose one option
Personalized recommendations on e-commerce platforms primarily aim to:
- AIncrease website traffic
- BReduce server load
- CImprove conversion rates based on user behavior
- DStandardize user experience
- Q2MCQ1 Mark·Choose one option
What is the primary objective of conducting an A/B test?
- ATo test system scalability
- BTo compare performance of two or more variants
- CTo validate data quality
- DTo measure user satisfaction
- Q3MCQ2 Marks·Choose one option
- ATreatment improved both metrics
- BTreatment improved conversion but reduced transaction value
- CControl performed better on all metrics
- DNo significant difference between groups
End Term · 31 Aug 2025 (AN)End Term
- Q4MCQ1 Mark·Choose one option
In an A/B test for a mobile app redesign, the control group should receive:
- AThe new app design
- BThe current/original app design
- CA mix of old and new designs
- DNo app access
- Q5MCQ1 Mark·Choose one option
When should A/B testing ideally be conducted?
- AAfter full deployment to all users
- BBefore developing the solution
- CBefore full deployment to validate changes
- DOnly during system maintenance
- Q6MCQ1 Mark·Choose one option
E-commerce platforms use scarcity nudges by:
- AShowing unlimited stock availability
- BDisplaying 'Only 3 left in stock' messages
- CHiding product information
- DRemoving time limits on offers
- Q7MCQ2 Marks·Choose one option
E-commerce A/B Test Results: Version A (Control): 1000 visitors, 50 purchases, Avg order value Rs.1,200 Version B (Treatment): 1000 visitors, 65 purchases, Avg order value Rs.1,150 Which metric shows improvement in Version B?
- AConversion rate only
- BAverage order value only
- CBoth conversion rate and total revenue
- DNeither metric improved
End Term · 13 Apr 2025 (FN)End Term
- Q8MCQ1 Mark·Choose one option
Which metric is most important to monitor during an A/B test?
- ATotal number of users tested in the experiment
- BUptime of the system
- CKey performance indicators (KPIs) specific to the experiment
- DModel training time
- Q9MCQ2 Marks·Choose one option
PayBuddy conducted an A/B test to understand if their new Funding Instrument recommendation strategy works well. Following are the details of the control and treatment groups. From the given table, which of the following statements are true? i. Value of Credit transactions increased with the new recommendation strategy ii. Value of Debit transactions increased with the new recommendation strategy iii. Value of both Credit and Debit transactions increased with the new recommendation strategy iv. Value of Credit transactions decreased with the new recommendation strategy v. Value of Debit transactions decreased with the new recommendation strategy
- ABoth (i) & (v)
- BOnly (iii)
- COnly (i)
- DBoth (ii) & (iv)
- EOnly (v)
End Term · 22 Dec 2024 (AN)End Term
- Q10MCQ1 Mark·Choose one option
Swyft, a new taxi aggregator platform has developed a new recommendation engine called “TinSage?” and they want to test the recommendation engine by doing an A/B test. Can you identify the test and control groups?
- ATest: Random recommendation; Control: TinSage recommendation
- BTest: Old model recommendation; Control: TinSage recommendation
- CTest: TinSage recommendation; Control: Old model recommendation
- DTest: Old model recommendation; Control: Random recommendation
- Q11MCQ1 Mark·Choose one option
What is the primary goal of an A/B test?
- ATest the scalability of a model
- BCompare two or more versions of a product or model to identify the best performer
- CMeasure hardware resource utilization
- DValidate the hypothesis using historical data