Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pixiv strategy - handle unlisted pages #4958

Open
bipface opened this issue Jan 15, 2022 · 4 comments · May be fixed by #5562
Open

pixiv strategy - handle unlisted pages #4958

bipface opened this issue Jan 15, 2022 · 4 comments · May be fixed by #5562
Labels
Source Support Upload/Source support

Comments

@bipface
Copy link
Contributor

bipface commented Jan 15, 2022

Example:

  • page urltestbooru
    • https://www.pixiv.net/en/artworks/unlisted/{alphanumeric-id}
  • file urltestbooru
    • https://i.pximg.net/img-original/img/{timestamp}/{digits}-{32-hex-digits}_p{page}.png

I was able to upload using the direct file URL without issues, so it should be feasible to implement support for the page URLs.
There's a number of angles to consider though — such as whether there's a way to map from file url to page url (normalisation) like with ordinary pixiv works, and whether the pixiv: metatag needs to be extended to cater for the alphanumeric IDs.
Presumably they've used these non-integer IDs to prevent scrapers discovering them by simply iterating through artwork IDs sequentially.

@BrokenEagle
Copy link
Collaborator

Do you have an example that can be used for testing purposes? If needed, you can post that information in #technical-private on Discord.

@bipface
Copy link
Contributor Author

bipface commented Jan 19, 2022

is there a problem with the example above ?

@BrokenEagle
Copy link
Collaborator

Ah, sorry. I didn't notice the links you had included above.

@nottalulah
Copy link
Contributor

Posts with these types of URLs are being tagged bad_source automatically now. See this post (NSFW)

@nonamethanks nonamethanks added Source Support Upload/Source support and removed Feature labels Feb 1, 2023
@nonamethanks nonamethanks linked a pull request Aug 13, 2023 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Source Support Upload/Source support
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants