Skip to content

Tweak solr weights/boosts - switch from additive boosting to multiplactive boosting - #12357

Merged
mekarpeles merged 1 commit into
masterfrom
11777/feature/tweak-solr-weights
Apr 21, 2026
Merged

Tweak solr weights/boosts - switch from additive boosting to multiplactive boosting#12357
mekarpeles merged 1 commit into
masterfrom
11777/feature/tweak-solr-weights

Conversation

@cdrini

@cdrini cdrini commented Apr 13, 2026

Copy link
Copy Markdown
Collaborator

Closes #11777

Technical

  • Previously, we used the bf parameter for our boosting, but this applied additive boosting, meaning the boosting could outweight the relevance quite drastically. Switching to boost gives us multiplicative boosting, which is recommended. See Additive Boosts vs Multiplicative Boosts
  • Note there is some performance concern since we are (1) using a lot of fields in our qf, where previously we only had a few, and (2) using more pf fields. If there is too much of a performance cost, we can switch back to using text ; there is still a big improvement from the boosting changes alone.

Testing

Improves upon all classes in our evaluation spreadsheet. C in the following tables (Note A is the same as C, but using text instead of the explicit listing of fields.)

image

You can also compare results before/after here: https://codepen.io/cdrini/full/wvJqzaK

Screenshot

Stakeholders

Copilot AI review requested due to automatic review settings April 13, 2026 16:28
@cdrini cdrini added State: Blocked Work has stopped, waiting for something (Info, Dependent fix, etc. See comments). [managed] Needs: Testing labels Apr 13, 2026
@github-actions github-actions Bot added the Priority: 2 Important, as time permits. [managed] label Apr 13, 2026
@cdrini
cdrini force-pushed the 11777/feature/tweak-solr-weights branch from fa46472 to 54c1707 Compare April 13, 2026 16:34

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adjusts Work Search’s Solr eDismax tuning to use multiplicative boosting (via boost) instead of additive boosting (via bf) to reduce over-weighting of popularity signals relative to textual relevance.

Changes:

  • Replaces bf-based additive boosts with an eDismax boost function expression.
  • Expands/adjusts qf and phrase-boost parameters (pf, pf2) to change match weighting and proximity scoring.

Comment thread openlibrary/plugins/worksearch/schemes/works.py Outdated
Comment thread openlibrary/plugins/worksearch/schemes/works.py Outdated
Comment thread openlibrary/plugins/worksearch/schemes/works.py Outdated
@cdrini
cdrini force-pushed the 11777/feature/tweak-solr-weights branch from 3594574 to ddf5888 Compare April 13, 2026 16:40
@cdrini

cdrini commented Apr 13, 2026

Copy link
Copy Markdown
Collaborator Author

Patching this out now for a bit to monitor performance.

@cdrini cdrini added the Patch Deployed This PR has been deployed to production independently, outside of the regular deploy cycle. label Apr 13, 2026
@mekarpeles mekarpeles self-assigned this Apr 13, 2026
@cdrini cdrini removed the Patch Deployed This PR has been deployed to production independently, outside of the regular deploy cycle. label Apr 14, 2026
@mekarpeles

Copy link
Copy Markdown
Member

Here's an example @cdrini showed where the search was "The secret of secrets" which didn't appear within the first 15 results on (before) and is 3rd on (after)

image

@mekarpeles mekarpeles removed State: Blocked Work has stopped, waiting for something (Info, Dependent fix, etc. See comments). [managed] Needs: Testing labels Apr 20, 2026
@mekarpeles
mekarpeles merged commit 42703ff into master Apr 21, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Priority: 2 Important, as time permits. [managed]

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Re-tune work search ranking parameters

3 participants