Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HttpClient and AcceptEncoding gzip on Android stoped working after latest SR4 release. #8928

Open
PureWeen opened this issue May 7, 2024 · 17 comments
Assignees
Labels
Area: HTTP Issues with sockets / HttpClient.

Comments

@PureWeen
Copy link

PureWeen commented May 7, 2024


Issue moved from dotnet/maui#21786


From @JoacimWall on Thursday, April 11, 2024 6:58:03 PM

Description

After I updated to latest Dotnet SDK and workloads on my dev machine my app start to return error on some API calls. The same thing happened in Azure Pipeline even though I didn't make any changes to the script.

The app works flawlessly under Ios.
So the unchanged code that target MAUI 8.0.6( Also tested target 8.0.20) return the same error.

if I just check de content of the return it's still gzip I think.
var test = result.Content.ReadAsStringAsync()
on Ios = "{"authentication":{"logintoken":
on Android = "��\b\0\0\0\0\0�\0��n�8��_�0��g�T������-ǒc[�,-��)Q�e�"ɲ�A�f�a_`^l)'��N����3h����!�(�>�������UHҊz��Y�y�s'�

//Joacim

Steps to Reproduce

If I check the logs of the Mac build host in Azure pipeline
Pool: Azure Pipelines
Image: macOS-13
Agent: Azure Pipelines 3
Started: Today at 20:01
Duration: 10m 39s

I see that
Successfully installed .NET Core sdk version 8.0.201.
Creating global tool path and pre-pending to PATH.
Finishing: Use .Net 8
and the
dotnet workload install maui
Now installs newer versions of the workload and other parts than it did before, Maybe the error is there somewhere

Link to public reproduction project repository

https://github.com/JoacimWall/Joacim_Bug_Report/tree/main/GzipBug

Version with bug

8.0.20 SR4

Is this a regression from previous behavior?

Yes, this used to work in .NET MAUI

Last version that worked well

8.0.6 SR1

Affected platforms

Android

Affected platform versions

No response

Did you find any workaround?

Yes
turn of the gzip for Android
#if IOS
this.client.DefaultRequestHeaders.AcceptEncoding.Add(new StringWithQualityHeaderValue("gzip"));
#endif

Relevant log output

Writing workload pack installation record for Microsoft.NETCore.App.Runtime.AOT.Cross.net7.iossimulator-x86 version 7.0.18...
Installing pack Microsoft.NETCore.App.Runtime.Mono.ios-arm version 7.0.18...
Writing workload pack installation record for Microsoft.NETCore.App.Runtime.Mono.net7.ios-arm version 7.0.18...
Installing pack Microsoft.NETCore.App.Runtime.Mono.ios-arm64 version 7.0.18...
Writing workload pack installation record for Microsoft.NETCore.App.Runtime.Mono.net7.ios-arm64 version 7.0.18...
Installing pack Microsoft.NETCore.App.Runtime.Mono.iossimulator-arm64 version 7.0.18...
Writing workload pack installation record for Microsoft.NETCore.App.Runtime.Mono.net7.iossimulator-arm64 version 7.0.18...
Installing pack Microsoft.NETCore.App.Runtime.Mono.iossimulator-x64 version 7.0.18...
Writing workload pack installation record for Microsoft.NETCore.App.Runtime.Mono.net7.iossimulator-x64 version 7.0.18...
Installing pack Microsoft.NETCore.App.Runtime.Mono.iossimulator-x86 version 7.0.18...
Writing workload pack installation record for Microsoft.NETCore.App.Runtime.Mono.net7.iossimulator-x86 version 7.0.18...
Installing pack Microsoft.MacCatalyst.Sdk version 17.2.8053...
Writing workload pack installation record for Microsoft.MacCatalyst.Sdk.net8 version 17.2.8053...
Installing pack Microsoft.MacCatalyst.Sdk version 16.4.7142...
Writing workload pack installation record for Microsoft.MacCatalyst.Sdk.net7 version 16.4.7142...
Installing pack Microsoft.MacCatalyst.Ref version 17.2.8053...
Writing workload pack installation record for Microsoft.MacCatalyst.Ref version 17.2.8053...
Installing pack Microsoft.MacCatalyst.Runtime.maccatalyst-x64 version 17.2.8053...
Writing workload pack installation record for Microsoft.MacCatalyst.Runtime.maccatalyst-x64 version 17.2.8053...
Installing pack Microsoft.MacCatalyst.Runtime.maccatalyst-arm64 version 17.2.8053...
Writing workload pack installation record for Microsoft.MacCatalyst.Runtime.maccatalyst-arm64 version 17.2.8053...
Installing pack Microsoft.MacCatalyst.Templates version 17.2.8053...
Writing workload pack installation record for Microsoft.MacCatalyst.Templates.net8 version 17.2.8053...
Installing pack Microsoft.NETCore.App.Runtime.AOT.osx-x64.Cross.maccatalyst-arm64 version 8.0.4...
Writing workload pack installation record for Microsoft.NETCore.App.Runtime.AOT.Cross.maccatalyst-arm64 version 8.0.4...
Installing pack Microsoft.NETCore.App.Runtime.AOT.osx-x64.Cross.maccatalyst-x64 version 8.0.4...
Writing workload pack installation record for Microsoft.NETCore.App.Runtime.AOT.Cross.maccatalyst-x64 version 8.0.4...
Installing pack Microsoft.NETCore.App.Runtime.Mono.maccatalyst-arm64 version 8.0.4...
Writing workload pack installation record for Microsoft.NETCore.App.Runtime.Mono.maccatalyst-arm64 version 8.0.4...
Installing pack Microsoft.NETCore.App.Runtime.Mono.maccatalyst-x64 version 8.0.4...
Writing workload pack installation record for Microsoft.NETCore.App.Runtime.Mono.maccatalyst-x64 version 8.0.4...
Installing pack Microsoft.NETCore.App.Runtime.AOT.osx-x64.Cross.maccatalyst-arm64 version 7.0.18...
Writing workload pack installation record for Microsoft.NETCore.App.Runtime.AOT.Cross.net7.maccatalyst-arm64 version 7.0.18...
Installing pack Microsoft.NETCore.App.Runtime.AOT.osx-x64.Cross.maccatalyst-x64 version 7.0.18...
Writing workload pack installation record for Microsoft.NETCore.App.Runtime.AOT.Cross.net7.maccatalyst-x64 version 7.0.18...
Installing pack Microsoft.NETCore.App.Runtime.Mono.maccatalyst-arm64 version 7.0.18...
Writing workload pack installation record for Microsoft.NETCore.App.Runtime.Mono.net7.maccatalyst-arm64 version 7.0.18...
Installing pack Microsoft.NETCore.App.Runtime.Mono.maccatalyst-x64 version 7.0.18...
Writing workload pack installation record for Microsoft.NETCore.App.Runtime.Mono.net7.maccatalyst-x64 version 7.0.18...
Garbage collecting for SDK feature band(s) 8.0.200...

Successfully installed workload(s) maui.


Finishing: Install Maui workload
@PureWeen
Copy link
Author

PureWeen commented May 7, 2024


Issue moved from dotnet/maui#21786


From @PureWeen on Thursday, April 11, 2024 10:18:56 PM

@jonathanpeppers thoughts?

@PureWeen
Copy link
Author

PureWeen commented May 7, 2024


Issue moved from dotnet/maui#21786


From @jonathanpeppers on Friday, April 12, 2024 2:15:19 AM

@JoacimWall can you share more complete sample code?

You might be able to test with: https://httpbin.org/

@simonrozsival @grendello could this be related to some of the changes here:

@PureWeen
Copy link
Author

PureWeen commented May 7, 2024


Issue moved from dotnet/maui#21786


From @JoacimWall on Friday, April 12, 2024 6:25:07 AM

@JoacimWall can you share more complete sample code?
You might be able to test with: https://httpbin.org/
@simonrozsival @grendello could this be related to some of the changes here:

Hi I am not responsible for the backend so I can't share the code for that. I have now shared how the front end logic works and a dummy backend but this is not the real backend.

https://github.com/JoacimWall/Joacim_Bug_Report/tree/main/GzipBug

I ma running on a Mac and i have not bin able to get the simulator to accept connect to local server.

Hi I have now published a dummy API on Azure (the code is in the repo also).
if I debug on real Android device(samsung sm-g96OF (API29)) I get the error now.
If you use iOS the api works.
//Joacim

@PureWeen
Copy link
Author

PureWeen commented May 7, 2024


Issue moved from dotnet/maui#21786


From @simonrozsival on Friday, April 12, 2024 7:27:32 AM

@jonathanpeppers @grendello This change is probably relevant: #8753

@JoacimWall could you please try setting handler.AutomaticDecompression = DecompressionMethods.None; or handler.AutomaticDecompression = DecompressionMethods.GZip; here: https://github.com/JoacimWall/Joacim_Bug_Report/blob/main/GzipBug/GzipClient/HttpsClientHandlerService.cs#L9?

@PureWeen
Copy link
Author

PureWeen commented May 7, 2024


Issue moved from dotnet/maui#21786


From @jonathanpeppers on Friday, April 12, 2024 1:40:41 PM

It seems like the BCL doesn't have automatic decompression by default:

So, was AndroidMessageHandler auto decompressing in the past?

It does seem like we should match BCL behavior, just unfortunate it changed in a service release.

@PureWeen
Copy link
Author

PureWeen commented May 7, 2024


Issue moved from dotnet/maui#21786


From @JoacimWall on Thursday, May 2, 2024 5:53:32 AM

Hello
I do not understand what you need for more information. I think that I can't do more to solve this bug. I reported this bug when something that worked in the previous version of MAUI now stopped working, so that others that encounter the same error could know what the issue is. We have 3 MAUI customer apps in production that are still on version 8.0.6 because there are new bugs in the new SRs that mean we cannot update.

//Regards Joacim

@PureWeen
Copy link
Author

PureWeen commented May 7, 2024


Issue moved from dotnet/maui#21786


From @simonrozsival on Thursday, May 2, 2024 8:45:08 AM

@JoacimWall I noticed that in the GzipBug repro project you set the default Accept-Encoding headers here:
https://github.com/JoacimWall/Joacim_Bug_Report/blob/main/GzipBug/GzipClient/RestClient.cs#L68-L70. The better way to do this is by setting the AutomaticDecompression property:

var handler = new HttpClientHandler
{
    AutomaticDecompression = DecompressionMethods.Gzip | DecompressionMethods.Deflate,
};
var client = new HttpClient(handler);
// ...

When I change the default AcceptEncoding headers for AutomaticDecompression, the repro app works correctly on Android. Does this solve your issue?

I'm slightly confused that you report that this used to work on Android in MAUI 8.0.6. I thought I might have broken the default behavior in #8753 but after revisiting it, I can't find a way that this could have been affected.

@PureWeen
Copy link
Author

PureWeen commented May 7, 2024


Issue moved from dotnet/maui#21786


From @JoacimWall on Thursday, May 2, 2024 9:42:24 AM

@JoacimWall I noticed that in the GzipBug repro project you set the default Accept-Encoding headers here: https://github.com/JoacimWall/Joacim_Bug_Report/blob/main/GzipBug/GzipClient/RestClient.cs#L68-L70. The better way to do this is by setting the AutomaticDecompression property:

var handler = new HttpClientHandler
{
    AutomaticDecompression = DecompressionMethods.Gzip | DecompressionMethods.Deflate,
};
var client = new HttpClient(handler);
// ...

When I change the default AcceptEncoding headers for AutomaticDecompression, the repro app works correctly on Android. Does this solve your issue?

I'm slightly confused that you report that this used to work on Android in MAUI 8.0.6. I thought I might have broken the default behavior in xamarin/xamarin-android#8753 but after revisiting it, I can't find a way that this could have been affected.

Hi
As I write above, this is from a production app where everything works, then we update to the latest version of MAUI no code changes and it stops working, so some type of breaking change. It's good that you can get around this by defining a specific HttpHandler or remove the GZIP row. But as you can see, this is not done in the example code, instead we run the standard declaration new Httpclient();
I think that I am not the only one that has built an app where we just do
var client = new HttpClient();
and try to add
this.client.DefaultRequestHeaders.AcceptEncoding.Add(new StringWithQualityHeaderValue("gzip"));

I spent some hours to find what it was that generate the error create I sample and report this issue. So others could skip spending the hours to find the root issue.

//Joacim

@PureWeen
Copy link
Author

PureWeen commented May 7, 2024


Issue moved from dotnet/maui#21786


From @simonrozsival on Thursday, May 2, 2024 12:58:27 PM

@JoacimWall thanks for the additional details.

@grendello do you have an idea if we could have broken this with #8753 or what else could have caused this regression?

@grendello
Copy link
Member

@PureWeen yes, I think the change might have broken the sample code. In particular, this line might be overriding the headers, if the AutomaticDecompression is set by some code to DecompressionMethods.None. This would force us to set the identity encoding, thus overwriting the headers set by the sample code.

@simonrozsival
Copy link
Contributor

@grendello I'm having problems reproducing the original working solution. I'm not getting any automatic decompression when I set just the default headers. The Mono.Android.dll that's being used is "34.0.1.78; git-rev-head:87e4a6d; git-branch:release/8.0.1xx" so it shouldn't be affected by #8753.

@jpobst jpobst added Area: HTTP Issues with sockets / HttpClient. and removed needs-triage Issues that need to be assigned. labels May 15, 2024
@jamesingreersc
Copy link

I had the exact same problem, http response was just gibberish. Had to roll VS back to previous version which fixed it.

@AlleSchonWeg
Copy link

I also have the same problem. My code is from a old XF App, which i triy to migrate.
My question is: What is correct/new way to enable compression with MAUI for Android and iOS?

This:

var handler = new HttpClientHandler {
				AutomaticDecompression = DecompressionMethods.GZip,
			};
var client = new HttpClient(handler);

This handler add the AcceptEncoding Header for outgoing requests?

Thank you

@simonrozsival
Copy link
Contributor

@AlleSchonWeg yes, this is the preferred way. The handler will automatically add the appropriate encoding headers and if the server responds with compressed data, it will automatically decompress it.

@AlleSchonWeg
Copy link

@simonrozsival Thank you.
I tested it with Android:

var response = client .GetAsync("https://httpbingo.org/headers").Result;
var resultString = client.Content.ReadAsStringAsync().Result;

This was the reponse:

{
  "headers": {
    "Accept": [
      "application/json"
    ],
    "Accept-Encoding": [
      "gzip"
    ],
    "Accept-Language": [
      "en"
    ],
    "Host": [
      "httpbingo.org"
    ],
    "User-Agent": [
      "XXX0 (DEV; Phone; Google; sdk_gphone64_x86_64; Android 14)"
    ],
    "Via": [
      "1.1 fly.io"
    ],
    "X-Forwarded-For": [
      "YYY, ZZZ"
    ],
    "X-Forwarded-Port": [
      "443"
    ],
    "X-Forwarded-Proto": [
      "https"
    ],
    "X-Forwarded-Ssl": [
      "on"
    ],
    "X-Request-Start": [
      "t=1716456090916385"
    ]
  }
}

The Accept-Encoding was successfully set. Can't test iOS.

@simonrozsival
Copy link
Contributor

@grendello @PureWeen I have tried again and I wasn't able to get the original described behavior to work. I'm not sure what caused this problem and how to fix it. Moreover, I'm not sure if we want to even support this behavior of enabling automatic decompression by setting the Accept-Encoding header instead of setting the AutomaticDecompression property.

@grendello
Copy link
Member

@simonrozsival I looked at it a bit too and I have absolutely no clue why it worked before and stopped now. Clearly, it's some change in the http handler inheritance tree or thereabouts, but I think I agree with you that the way to do it by setting the property is the correct one. Especially that we're dealing with a "proxy" handler in this case, and most of the magic somehow happens in the Java backend we have no control over. As far as I can tell, we did nothing special before to make the scenario work, so I wouldn't exclude the possibility that the behavior might be different also because of a different target SDK when the app is built.

All in all, I'm in favor of documenting the issue and specifying what is the correct and supported way to achieve the desired outcome.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area: HTTP Issues with sockets / HttpClient.
Projects
None yet
Development

No branches or pull requests

6 participants