Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improves detection of end of library #26

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

nielsabels
Copy link

Issue

In my efforts to run this tool to export my Google Photos library, I ran into some issues. I found that the tool was not able to find the last picture in my library successfully, and resorted to hit right-arrow continually until it would reach the end of the library.

Background

My library has a vast amount of images and videos, and given some transient errors while navigating, I figured the script would never find the oldest picture (it would have taken days/weeks). Through reading the source code I found out that this is actually not the intended behavior. The script actually hits page-down and end on the landing page of the library, and finds the oldest photo, after which it navigates to the left until it's done exporting.

Implementation

I've made the way the script determines whether the end of the gallery has been reached a little bit more rigid. For your convenience, I've put a console log below these messages so you can clearly see the effect this change has. I've masked the ids a bit (I have no idea whether they are sensitive), but reading the last line for both outputs clearly leads to different ids. You can also see that for that particular run, there were 2 false positives until it actually hit the end of my gallery.

I hope this submitted code is in line with your coding standards/preferences. Let me know in case there is anything I can adjust.

Console logs

normal run

λ .\gphotos-cdp.exe -dev=true -v -dldir c:\temp\google-photos\without-threshold
2020/09/27 18:03:05 Session Dir: C:\Users\Niels\AppData\Local\Temp\gphotos-cdp
2020/09/27 18:03:06 pre-navigate
2020/09/27 18:03:10 post-navigate
2020/09/27 18:03:10 Page loaded, most recent item in the feed is: AF1QipPGBGoeq********************************
2020/09/27 18:03:11 Successfully jumped to the end
2020/09/27 18:03:17 Nav to the end sequence is started because location is https://photos.google.com/photo/AF1QipPGBGoeqVPRzM4pw4C8********************************

with the supplied fix

λ .\gphotos-cdp.exe -dev=true -v -dldir c:\temp\google-photos\with-threshold
2020/09/27 18:04:22 Session Dir: C:\Users\Niels\AppData\Local\Temp\gphotos-cdp
2020/09/27 18:04:23 pre-navigate
2020/09/27 18:04:31 post-navigate
2020/09/27 18:04:31 Page loaded, most recent item in the feed is: AF1QipPGBGoeq********************************
2020/09/27 18:04:34 Screen is equal to previous screen, waiting to hit threshold [1/3]
2020/09/27 18:04:37 Screen is equal to previous screen, waiting to hit threshold [1/3]
2020/09/27 18:04:40 Screen is equal to previous screen, waiting to hit threshold [1/3]
2020/09/27 18:04:41 Screen is equal to previous screen, waiting to hit threshold [2/3]
2020/09/27 18:04:42 Screen is equal to previous screen, waiting to hit threshold [3/3]
2020/09/27 18:04:42 Successfully jumped to the end
2020/09/27 18:04:43 Nav to the end sequence is started because location is https://photos.google.com/photo/AF1QipPtyB3Qykpbg86tElI63UH********************************

@googlebot
Copy link

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here with @googlebot I signed it! and we'll verify it.


What to do if you already signed the CLA

Individual signers
Corporate signers

ℹ️ Googlers: Go here for more info.

@nielsabels
Copy link
Author

@googlebot I signed it!

@googlebot
Copy link

We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for all the commit author(s) or Co-authors. If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA (login here to double check)? If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google.
In order to pass this check, please resolve this problem and then comment @googlebot I fixed it.. If the bot doesn't comment, it means it doesn't think anything has changed.

ℹ️ Googlers: Go here for more info.

@nielsabels nielsabels force-pushed the improve-finding-last-photo-in-larger-gallery branch from a122a9a to c16b5d3 Compare September 27, 2020 16:29
@nielsabels nielsabels force-pushed the improve-finding-last-photo-in-larger-gallery branch from c16b5d3 to 3205ed5 Compare September 27, 2020 16:32
@googlebot
Copy link

CLAs look good, thanks!

ℹ️ Googlers: Go here for more info.

@mpl
Copy link
Collaborator

mpl commented Sep 29, 2020

Hi,

the screenshot method in navToEnd is pretty gross, and I was hoping someday to get rid of it.
I was thinking maybe now that chromedp.RunResponse (which I've already started to use in gphotos-cdp) exists, it could help with that.
Would you be interested in investigating that?

@nielsabels
Copy link
Author

Hi there!

Thanks a lot for considering my pull request and offering me a chance to implement a more robust method to run to the end of the gallery.

However, I think that that would take quite some investment from my side and at the moment I'm not really looking into developing skills into the chromedp area. I created the PR because I do want to give back to the community and therefore am submitting my - probably pretty crude - bugfix.

So at the moment I'm not interested in that - but thanks again for the offer. I'm willing to amend/fix anything about this PR though. Please give a shout if there's anything else I can do.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants