Google Prevents Login from Automated Test Software

[mullender]

Steps to repro:

1. go run main.go -dev -v
2. chrome opens up
3. navigate to the login, and enter email address
4. observe an error message returned by Google servers

In another attempt I launched chrome without automation but with the data dir set to the same location as gphoto-cdp uses:

1. launch: /Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --user-data-dir=/var/folders/tt/5skfg0yd4q1bwhr_fh_0t39h0000gn/T/gphotos-cdp
2. Complete authentication successfully!
3. go run main.go -dev -v
4. observe that the same user data dir is used
5. observe that the browser is not authenticated
6. Somehow Google server detects the browser is automated and logs the browser out. OR Chrome itself logs the user out when running automated.

And my apologies for yesterday's issue.

[mullender]

I just found this issue: chromedp/chromedp#996 (comment)
Which suggests that Google is indeed blocking automation.

[mholt]

You can get this to work if you disable web security (you will have to specify a user data directory). (I finally got this working last night.)

[mullender]

Thanks @mholt for digging into this, is this what you are referring to: https://medium.com/@dmadan86/run-chrome-browser-without-cors-by-disabling-web-security-d124ad4dd2cf

I am going to try this out later this evening!

[mholt]

Yep, those are the flags I set. Obviously, only access trusted sites with that flag.

[CaptainAwesomer]

@mholt / @mullender , where/how do you actually set the --disable-web-security flag?

[mholt]

When you run Chrome. It's a command line flag.

If you're using a headless browser, the automation lib should let you specify flags.

[mullender]

I added the following flags to the retrieval script, and it still failed to authenticate

diff --git a/main.go b/main.go
index 5b457b2..bcf3dcc 100644
--- a/main.go
+++ b/main.go
@@ -158,6 +158,9 @@ func (s *Session) NewContext() (context.Context, context.CancelFunc) {
        opts := append(chromedp.DefaultExecAllocatorOptions[:],
                chromedp.DisableGPU,
                chromedp.UserDataDir(s.profileDir),
+               chromedp.Flag("enable-automation", true),
+               chromedp.Flag("disable-web-security", true),
+               chromedp.Flag("allow-running-insecure-content", true),
        )

        if !*headlessFlag {
        diff --git a/main.go b/main.go
index 5b457b2..bcf3dcc 100644
--- a/main.go
+++ b/main.go
@@ -158,6 +158,9 @@ func (s *Session) NewContext() (context.Context, context.CancelFunc) {
        opts := append(chromedp.DefaultExecAllocatorOptions[:],
                chromedp.DisableGPU,
                chromedp.UserDataDir(s.profileDir),
+               chromedp.Flag("enable-automation", true),
+               chromedp.Flag("disable-web-security", true),
+               chromedp.Flag("allow-running-insecure-content", true),
        )

        if !*headlessFlag 

[mholt]

Ah, to clarify, I don't use the default flags:

opts := []chromedp.ExecAllocatorOption{
	chromedp.NoFirstRun,
	chromedp.NoDefaultBrowserCheck,
	chromedp.UserDataDir("/home/matt/.config/google-chrome"),
	chromedp.Flag("enable-automation", true),
	chromedp.Flag("disable-web-security", true),
	chromedp.Flag("allow-running-insecure-content", true),
}

[lucwillems]

its also failing for me, even with settings of @mholt
what is notice that :

• fails for my xxx@gmail.com account
• works for my google for business account
  using google chrome Version 115.0.5790.110 (Official Build) (64-bit)
  also tried incognito mode , but no result.

[pmorch]

It also fails for me with the screenshot in the first comment. I wonder if it depends on whether we have 2FA enabled. I do.

Does authentication work out of the box for anyone?

[pmorch]

Could someone describe how to succeed with authentication from scratch?

I've cloned the repo and installed golang.

$ go run main.go

shows the "Couldn't sign you in" screen.

$ go run main.go --help

doesn't really point how to set "the flags". Could anybody who is able to log in with a @gmail.com account describe what they did in a little more detail?

[mholt]

@pmorch This worked for me (see above): #37 (comment)

[pmorch]

Ok, thanks.

I have multiple profiles in the default user-data-dir, so I created a brand new one with:

$ google-chrome --user-data-dir=/home/pmorch/work/gphotos-cdp/google-chrome

and then applied this patch:

diff --git a/main.go b/main.go
index 5b457b2..fb45df8 100644
--- a/main.go
+++ b/main.go
@@ -155,10 +155,14 @@ func NewSession() (*Session, error) {
 
 func (s *Session) NewContext() (context.Context, context.CancelFunc) {
        // Let's use as a base for allocator options (It implies Headless)
-       opts := append(chromedp.DefaultExecAllocatorOptions[:],
-               chromedp.DisableGPU,
-               chromedp.UserDataDir(s.profileDir),
-       )
+       opts := []chromedp.ExecAllocatorOption{
+               chromedp.NoFirstRun,
+               chromedp.NoDefaultBrowserCheck,
+               chromedp.UserDataDir("/home/pmorch/work/gphotos-cdp/google-chrome"),
+               chromedp.Flag("enable-automation", true),
+               chromedp.Flag("disable-web-security", true),
+               chromedp.Flag("allow-running-insecure-content", true),
+       }
 
        if !*headlessFlag {
                // undo the three opts in chromedp.Headless() which is included in DefaultExecAllocatorOptions

And then ran:

$ go run main.go -n 20 -dldir /home/pmorch/work/gphotos-cdp/photos

Now it works. I see the photos in /home/pmorch/work/gphotos-cdp/photos.

[presto8]

Thanks @pmorch for the patch! It worked for me following your instructions, except I had to log in first with the profile and authenticate, before running gphotos-cdp. After pre-authenticating, then it worked.