Remove Selenium dependency #2
Conversation
The new code uses the (internal) BGG JSON API directly. This removes many dependencies, speeds up the process, makes it more reliable and makes it possible to handle several copies of a game at the same time. When "collid" is empty or --force-new is given, the importer creates new copies of the games. Without --force-new and with non-empty "collid" matching existing copies of the games will be updated.
|
Thanks, this seems great, I'll have a look and will test later! Although I could have a concern regarding legal aspects, because BGG ToS state:
Relying on a web browser allows me to conform to these terms, whereas relying on batch HTTP calls would probably violate them and would need an explicit permission from BGG. I'll double check that. Thanks again for sharing. |
There might be other viable alternatives:
- Create new copies by default, and introduce some --update switch
to update the existing collection.
- Add a completely new command for updating the collection, like e.g.
collection-update.
- Detect if the "collid" to be imported is already in the collection.
Then update if it is, or add a copy if it is not. This solution
probably exhibits the least surprising behavior for the user, but
it requires an additional request per imported record or a complete
export beforehand.
In that case it might be in order to throttle the requests, with a default delay in line with the ToS. If it is configurable, users could then lower the delay (possibly violating the ToS) at their own discretion. |
The new code uses the (internal) BGG JSON API directly.
This removes many dependencies, speeds up the process, makes it more
reliable and makes it possible to handle several copies of a game at
the same time.
When "collid" is empty or --force-new is given, the importer creates
new copies of the games. Without --force-new and with non-empty
"collid" matching existing copies of the games will be updated.