AB Testing Architecture #12789

Feature Request

Problem / Opportunity

Describe: Open Library lacks a way to test layouts/features/designs partially by releasing to a portion of patrons to gather more insights or decide between two+ clashing options.

Justify: Implementing a testing framework allows us to make design decisions based on actual patron behavior (e.g., measuring if a layout increases borrowing rates or registrations). Doing this through a central system prevents layout glitches (flickering) and keeps pages loading fast.

Define Success:

• Ability to easily configure experiments / experiment splits.
• Patrons getting the same experience in different visits (sticky groups)
• Abilit to manually override the experiment group for testing.

Proposal

Brief Overview of Proposed Solution
We have set up a lightweight A/B testing framework that runs on both the backend and frontend:

• Experiment Registry: A single configuration file to list active tests and variant distributions (e.g., 50% group A, 50% group B).
• Stable User Assignment: Patrons are assigned to a group using their account identifier or session, ensuring they see the same layout on every visit.
• Frontend Integration: Webpage scripts can easily check which group the user is in to adjust UI components on the fly.
• Testing Overrides: Developers can append parameters to page URLs to easily preview different test groups.

Breakdown