x64 build and ipv6 only filtering hangs on WinDivertOpen

[PhyxionNL]

With the MSVC build (same problem with WDDK):

HANDLE handle = WinDivertOpen("ipv6", WINDIVERT_LAYER_NETWORK, 0 ,0L);

This freezes with an x64 build (in fact, Visual Studio sometimes crashes with this command), whereas it does work with an x86 build? I used the proper files for the x64 build, running under Visual Studio 2012 and 2013. I also tried older versions of WinDivert (1.1.4 and 1.1.5) and they have the same problem.

Tried both Windows 7 and 8 (x64).

[basil00]

Hi, I was unable to reproduce the problem (WinDivertOpen behaved as expected). Can you please send a complete sample program that exhibits the issue?

[PhyxionNL]

Alright, here it is. This is the one I run at Windows 8.1 (x64). I found out that happens with any filter as long as 'ipv6' is in it. Anything without the ipv6 filter seems to work. With the 32-bit version this doesn't happen and everything works flawlessly. I also tried it in C# with pinvoke, same results (as expected, though).

I only run ESET Smart Security, but disabling it didn't help either. Sometimes (often the initial run) it takes a while (1-2 minutes) but eventually the app crashes and/or the debugger crashes.

[basil00]

I tried the sample program using Win7 x64 and it seems to work fine.
What exactly do you mean by "freeze"? E.g. program freezes, system freezes, etc.

[PhyxionNL]

The system continues to work fine, the application freezes/crashes and the attached debugger nearly always stops with an error message. This happens on all our systems with the 64-bit dll (32-bit dll works fine). Here and here are two movies (download them first, doesn't seem to work directly in the browser) I've recorded of what usually happens. In the last movie I've pressed F10 (continue to next line). Hope it helps, if you need more information, please ask.

[PhyxionNL]

Apparently, if you run it without a debugger it works fine with the 64-bit dll. But with a debugger attached it stops at the WinDivertOpen method. The 32-bit dll can be debugged fine, any idea why this would happen and how to fix this (maybe something is wrong in the 64-bit dll?)?

[basil00]

I am not familiar with VS2012 or its debugger (I exclusively use MinGW). The WinDivert DLL is not compiled in debug-mode so perhaps that is the problem? This is probably a question to ask a VS guru.

[PhyxionNL]

I'm not sure if that is the reason as the 32-bit dll works fine with debugging. I've also never had this with any other dll so far.

[basil00]

I can't think any obvious explanation. Since the problem only appears with the debugger attached, it likely requires knowledge of VS debugging which is simply beyond my expertise.

The only thing I can suggest is trying the LoadLibrary() approach and maybe the problem will go away.

[rafael-santiago]

Excuse me but I think it is related with the way that you're debugging. If you're using remote debugger it's possible that the debugger packets are being dropped. Take a look in your defined filter:

WinDivertOpen("ipv6", WINDIVERT_LAYER_NETWORK, 0 ,0L);

I'm suggesting it because I face this problem when I need to use the remote debugger.

[jagt]

I want to add that I had exactly the same issue with WinDivert 1.0. When debugging in x64 build it would always hang on WinDivertOpen.

[basil00]

If you're using remote debugger it's possible that the debugger packets are being dropped.

If the debugger uses ipv6 then this definitely will be a problem. In fact, you may deadlock the debugger, e.g. if the debugger pauses the program to wait on some IPv6 communication, but this communication is blocked because the program is paused. It's hard to say since the error message returned is too generic.

[PhyxionNL]

It's not a remote debugger but just the local VS debugger. The x86 dll works fine (also with ipv6 filter) so that doesn't make much sense anyway.

We can debug the x86 but it's still very strange that this happens, it also only happens when 'ipv6' is in the filter, with all the other filter options the debugger works perfectly with the x64 dll.

@jagt, did you manage to somehow resolve this?

[TechnikEmpire]

I can confirm that, with VS 2013 pro, I cannot be debugging any program at all while I have my app that uses WinDivert running. You're not remote debugging, but there is definitely something the debugger + dev environment is doing that relies on network traffic which, when interrupted, causes the debugger process to "hang" and VS will notify you of this, giving you only the option to terminate it.

I can say this absolutely because my application detects programs that make outbound port 80 TCP connections as possible browsers and intercepts them for filtering using WinDivert. When this happens, it will automatically push the exe name to a list of apps being filtered so you can toggle it on or off. In this list will come things like devenv and other visual studio processes. Because they are using HTTP ports but not actually communicating with the HTTP protocol, my application will fail to intercept and parse requests, causing the packets to get dropped, leading to the message from VS that the debugger is hanging.

It's not a WinDivert issue, it's yet another case of over-paid "professional" developers completely disregarding the concept of reserved ports (in my case anyway) and for everyone else, it's just the fact that your use of WinDivert is intercepting and somehow dropping comms that VS apparently needs when running, specifically debugging.

[PhyxionNL]

@TechnikEmpire Your issue sounds very different from what I describe. The x86 dll works fine, only the x64 causes problems (and only with ipv6 traffic).

[TechnikEmpire]

Can you upload those videos to youtube? You could just make them unlisted so they only appear here and people that aren't comfortable downloading files from peoples personal servers could view them.

[PhyxionNL]

It's dropbox and mp4 :)

[TechnikEmpire]

But I'm scurred. Edit - I'm not sure why they're movies, they're just still frame screenshots of your code... Anyway I'd have to study the problem I describe further but I'm pretty sure my problem also goes away when debugging x86 version, I'm just never debugging in x86. I'm also using Win 8.1 x64.

[basil00]

It should be possible to use the netdump tool to see what (if any) traffic the VS debugger is generating.

[TechnikEmpire]

In vs2015 the debugger immediately opens connections to Microsoft servers. I noticed this yesterday.

[Mpx1]

Problem is that Visual Studio (VS) is 32 bit process and can't directly debug 64 bit processes. VS uses 64 bit helper process "msvsmon.exe" to locally debug application. Communication between VS and "msvsmon.exe" is using TCP over IPv6 protocol with random TCP port - you can see it in open connections using: "netstat -n -b". To solve problem update filter to exclude this connection (e.g. conditionally depending on Debugger.IsAttached property).