[ Test Instance ]

Wikimedia Commons reconciliation for OpenRefine

This web service aligns ('reconciles') file names/paths of Wikimedia Commons files with their corresponding MediaInfo entity URIs (M-ids). Via data extraction, the Wikitext, multilingual file captions and other structured data of these files can then be retrieved for further processing in OpenRefine.

This interface works with OpenRefine from 2.6 rc2 onwards. It is not compatible with Google Refine.

Use the following URL in OpenRefine: https://test-commonsreconcile.toolforge.org/en/api

Replacing en with another language code will display items and properties in that language, if they are available.

How do I use this service in OpenRefine?

Demo video in English, 3'25"

Reconciling

You can find general documentation about reconciling in OpenRefine's manual.

• Create or open a project in OpenRefine which contains a column with Wikimedia Commons file names / file paths.
• Use this column's dropdown menu to select Reconcile → Start reconciling.
• If you have not added the Wikimedia Commons Reconciliation service yet, click Add standard service... and paste in the URL: https://test-commonsreconcile.toolforge.org/en/api (you can replace en with another language code of your choice).
• Next, click Start reconciling.

Extract Wikitext and other structured data columns ("data extension")

With the Wikimedia Commons Reconciliation Service, you can create additional columns in your OpenRefine project which contain the Wikitext and various structured data of your Wikimedia Commons files.

• Use your reconciled (file paths) column's dropdown menu to select Edit column → Add columns from reconciled values...
• Next, you can manually type the names (labels) of properties you are interested in, or select from the list of suggestions.
• Type (or select) Wikitext to retrieve the files' Wikitext
• In order to retrieve multilingual file captions from the files in a given language, type C followed by the two-letter language code of this language. For instance: Cen will produce file captions in English; Cja will produce file captions in Japanese.

For developers

• Reconciliation Service API - A protocol for data matching on the Web (documentation of the protocol)
• Source code of the Wikimedia Commons Reconciliation Service (gerrit.wikimedia.org)
• Workboard and issue tracker of the Wikimedia Commons Reconciliation Service (gerrit.wikimedia.org)