How To Fix Windows Live Messenger 2011 Forefront TMG Login Issues

On July 22nd 2012 Microsoft pushed all users to the latest Live Messenger version (Live Messenger 2011).

For most users, everything worked fine, but for many organizations working behind a proxy solution such as Forefront TMG, it was impossible to log in.

After typing credentials to log into Windows Live Messenger 2011, users received the message below (in our case in portguese):

Translation: We can't sign you in to Windows Live Messenger. Signing in to Windows Live Messenger failed because the service is temporarily unavailable. Please try again later. Error code: 8100030d

Older versions, such as Live Messenger 2009 did not have these login problems, only Live Messenger 2011.

Troubleshooting Live Messenger Login Issues

When our Network Operations Center started receiving complaints from customers about their Live Messenger connection, I started doing some lab tests to troubleshoot the issue. The tests revealed some weird behavior as described below:

1. There was no way to log in to Live Messenger 2011 when working from an environment with a proxy solution, but once a successful login is made (without a proxy) the problem disappears. I logged in normally using a workstation that was not going through the corporate proxy, and then when trying to log in again using a proxy server, it worked fine.

I realized there was a process trying to send user credentials only on the first logon attempt. Once the credentials were authenticated, this process would no longer be started.

2. When testing access using a workstation with the Forefront TMG Firewall Client, we could see several exe files appear in the Forefront TMG logs. These files were msnmsgr.exe, VCSoapClient, VpxClient and wlcomm.exe.

Allow Access to Live Messenger Sites

Our first troubleshooting step was to create an access rule in Forefront TMG allowing user access to Live Messenger websites. For this we created a Domain Set called that included *.msn.com, *.hotmail.com and *.live.com. We then created an access rule to allow the Internal Network to the Domain Set.

Add Application Entry Settings for Live Messenger Exes

The second troubleshooting step was to add an Application Entry Setting for the Live Messenger EXE files into the Forefront TMG Client Settings, so as we could avoid connection problems isolating these files:

file - msnmsgr.exe key - DisableEx value – 1

file - VCSoapClient key – Disable value – 1

file - VpxClient key – Disable value – 1

file - wlcomm.exe key – DisableEx value – 1

To isolate files in Forefront TMG, go to "Networks" and "Forefront TMG Client Settings" on the right pane.

After following all the steps above, I got back to my workstation to make another attempt but it failed again!

Understanding Live Messenger Traffic

My next step was to better understand how the Live Messenger EXE files were connecting through Forefront TMG. For this we used the TCPView tool. TCPView collects all network traffic from a workstation or server to understand where connections start and finish.

The TCPView report showed that all files listed above were successfully connecting to Forefront TMG, but another EXE appeared that did not show in TMG's logs.  Here's an extract from the TCPView report showing the exe in question:

WLIDSVC.EXE  4376  TCP  uilson_wks.uilson.com.br  50992  proxy.uilson.com.br  8080  ESTABLISHED WLIDSVC.EXE  4376 UDP uilson_wks.uilson.com.br 59673 * * WLIDSVC.EXE 4376 TCP uilson_wks.uilson.com.br 51134 65.54.186.107 https SYN_SENT

As you can see, the WLIDSVC.EXE file establishes a connection to the Forefront TMG Server, but for some reason it does not complete the three-way-handshake and tries to bypass the proxy server.

The Problem

We had a separate edge firewall blocking Internet access that was not from the Forefront TMG IP address, and as this exe was trying to access the internet directly, it failed to send the Windows Live Messenger login credentials.

The Work Around

As a workaround, a rule was created on the edge firewall allowing traffic from the WLIDSVC.EXE  to the IP address range of the Live Messenger Servers (65.54.0.0 to 65.54.255.255) in order to send the first time credentials. It was a temporary solution until I could find a better way to connect new users to Live Messenger 2011.

Once this change was made, the Live Messenger login succeeded!

Microsoft doesn't explain the WLIDSVC.EXE behavior, but instead guide Forefront TMG admins to create an access rule as I did first. They don't have an official KB about this issue, but in the link below you can find some support engineers directions:

https://social.technet.microsoft.com/Forums/en-US/Forefrontedgegeneral/thread/ec7c9ee3-409e-4d89-8ac6-02e63330bc25

With the problem temporarily fixed, I started a TechNet forum thread with some colleagues here in Brazil who had the same problem. After some time discussing (unfortunately in portuguese), we found a definitive workaround for this issue.

The Solution

To avoid this behavior you need to change the Live Messenger 2011 Compatibility mode to Windows XP SP3, as in the picture below:

All these tests were done with a workstation using the Forefront TMG Firewall client but the same problem will occurs on workstations connecting by webproxy. Changing the compatibility mode will also solve the problem in this scenario.

If you want to learn know more about this issue (and improve your portuguese), here is a link to the Technet forum: https://social.technet.microsoft.com/Forums/pt-BR/forefrontpt/thread/c2d7ab89-49ee-4085-a59b-0a25ddd4d120

If you have also encountered this issue and solved it a different way, please let us know in the comments below.