Page MenuHomePhabricator

Decommission ISBN requests via WorldCat
Closed, ResolvedPublic

Description

Event Timeline

wow, this is sad :(

Curious to hear what happened here.

Change 918521 had a related patch set uploaded (by Mvolz; author: Mvolz):

[mediawiki/extensions/Citoid@master] Removed ISBN from buttons

https://gerrit.wikimedia.org/r/918521

Change 918521 merged by jenkins-bot:

[mediawiki/extensions/Citoid@master] Removed reference to ISBN from buttons

https://gerrit.wikimedia.org/r/918521

The silence around this makes this sound like a political decision. It sounds to me like OCLC has asked us to reduce our usage of their APIs, which we didnt do ‘sufficiently’ and they have since terminated our free api access….

This seems strange for a company that basically copies wikidata for viaf and gets links in half the wikipedia pages for free…..

Sorry for the relative silence on this, we were just trying to make sure we understood everything before giving further details. In short, OCLC is moving to a new WorldCat API version, and unfortunately migrating Citoid to this API would require both a new agreement with them and some additional technical work. We are exploring this further but will need to disable ISBN citation generation for now, and although I'd like for us to get this back, we can't yet predict when that will happen.

Was this a surprise announcement by OCLC that happened yesterday? It seems like either there should have been plenty of time to develop a new agreement and do the technical work necessary to make use of the new API, or OCLC could maintain access to the old API for some time for partners that did not have sufficient time to adjust. Or someone, somewhere, really dropped the ball and did not prioritize tasks correctly. This is sad.

I hope that the WMF's new focus on "prioritizing established editors" will make this sort of occurrence less frequent, and that the WMF will engage with editors in advance of important tools breaking, instead of waiting until they are disabled to begin communicating.

Did you consider using LoC or some other national library?

Most if not all big libraries should support OAI-PMH. OAI can be used as a dump (kind of) to get all book descriptions and keep them on some Wikimedia server.

Most big libraries have some API which you could use directly to search for ISBN. Many probably use Z39.50 which is not very friendly, but doable for ISBN searches. Polish national library (BN) also has a custom API that support JSON, XML and MARC (but it is a custom API, so probably better for a local tool or gadget then a global search service).

OCLC is moving to a new WorldCat API version, and unfortunately migrating Citoid to this API would require both a new agreement with them and some additional technical work.

This seems more complicated than I imagine they thought it would be when they upgraded their API.

What's needed to get a new agreement signed?
Is there any question that we will migrate Citoid to it?
If we might not migrate, than perhaps those editors who care about ISBN resolution should be hustling to find other solutions.
If we're definitely migrating, can there be an issue for the migration that is assigned to someone?

Per my Mass Message, we're "investigating several options". This is where we're at. As the situation keeps evolving, T336298 will have further details. So rest assured we're seeing your questions and suggestions and are grateful for the interest. Right now though, the folks who could be handling them are the same who are working hard on a solution: thanks for your patience and understanding while they focus on the latter.

Did you consider using LoC or some other national library?

Most if not all big libraries should support OAI-PMH. OAI can be used as a dump (kind of) to get all book descriptions and keep them on some Wikimedia server.

Most big libraries have some API which you could use directly to search for ISBN. Many probably use Z39.50 which is not very friendly, but doable for ISBN searches. Polish national library (BN) also has a custom API that support JSON, XML and MARC (but it is a custom API, so probably better for a local tool or gadget then a global search service).

Thanks for your suggestion! For the time being we have have enabled Zotero's ISBN endpoint internally which primarily uses Library of Congress (T336727)

With Zotero it would be possible to support other national libraries in addition by contributing a translator for them here: https://github.com/zotero/translators

Did you consider using LoC or some other national library?

Most if not all big libraries should support OAI-PMH. OAI can be used as a dump (kind of) to get all book descriptions and keep them on some Wikimedia server.

Most big libraries have some API which you could use directly to search for ISBN. Many probably use Z39.50 which is not very friendly, but doable for ISBN searches. Polish national library (BN) also has a custom API that support JSON, XML and MARC (but it is a custom API, so probably better for a local tool or gadget then a global search service).

Thanks for your suggestion! For the time being we have have enabled Zotero's ISBN endpoint internally which primarily uses Library of Congress (T336727)

With Zotero it would be possible to support other national libraries in addition by contributing a translator for them here: https://github.com/zotero/translators

Added support for the Polish National Library (BN):
https://github.com/zotero/translators/pull/3036

Would be great if you could add that for Wikipedia. LoC have some foreign records, but Wikipedia user reported that e.g. ISBN 8301136545 is missing (available in BN).

FWIW, K10plus ISBN appears to be superior to the service provided by the German National Library, as observed in Zotero so far. So, no need for a change.

Did you consider using LoC or some other national library?

Most if not all big libraries should support OAI-PMH. OAI can be used as a dump (kind of) to get all book descriptions and keep them on some Wikimedia server.

Most big libraries have some API which you could use directly to search for ISBN. Many probably use Z39.50 which is not very friendly, but doable for ISBN searches. Polish national library (BN) also has a custom API that support JSON, XML and MARC (but it is a custom API, so probably better for a local tool or gadget then a global search service).

Thanks for your suggestion! For the time being we have have enabled Zotero's ISBN endpoint internally which primarily uses Library of Congress (T336727)

With Zotero it would be possible to support other national libraries in addition by contributing a translator for them here: https://github.com/zotero/translators

Added support for the Polish National Library (BN):
https://github.com/zotero/translators/pull/3036

Would be great if you could add that for Wikipedia. LoC have some foreign records, but Wikipedia user reported that e.g. ISBN 8301136545 is missing (available in BN).

That's amazing! Once it's merged upstream we can update to use it.

Change 929322 had a related patch set uploaded (by Mvolz; author: Mvolz):

[mediawiki/services/citoid@master] Remove support for worldcat search API

https://gerrit.wikimedia.org/r/929322

Change 929322 merged by jenkins-bot:

[mediawiki/services/citoid@master] Remove support for worldcat search API

https://gerrit.wikimedia.org/r/929322

Change 932194 had a related patch set uploaded (by Mvolz; author: Mvolz):

[operations/deployment-charts@master] Remove WorldCat references

https://gerrit.wikimedia.org/r/932194

Change 932194 merged by jenkins-bot:

[operations/deployment-charts@master] Remove WorldCat references

https://gerrit.wikimedia.org/r/932194

Mvolz updated the task description. (Show Details)