Eligible Product: Windows Server 2003 SP1

Problem Product: Windows Server 2003 SP1

 

Problem Description

============================================

It is an Active Directory domain. When users/clients log into Citrix server in child domain, then if they log in with Active Directory account it is successful. Otherwise, it gets kicked out if they login through TS session.

Windows Server 2003

 

Subjective

===============

Customer has terminal server (Windows Server 2003) in the NAEAST domain (child domain). NAEAST domain has 2 way external trust with RPSLAN domain. When a user from RPSLAN domain login to the box (Terminal Server, RDP), we get the applying personal settings dialogue box and then the user gets kicked out of the session.

At the same time, users from the NAEAST domain can successfully log in to the box. When we do mstsc/console and login from with RPSLAN, it works fine.

 

We have 8 DC in this AD site, 3 are physically on the site (in local data center), all 3 are Windows Server 2000, and Other DCs are Windows Server 2003.

When Customer connects Terminal Server (through nltest /sc_rest) to any DC in the datacenter, we see the problem.

But we rest SC to any 5 dc outside datacenter, it works fine

mstsc /console always works, does not matter to which DC we have SC with

 

 

Assessment

==============

There are no errors in the event log about profile failure

Took userenv log, the problem snippet is

-------------------------------------

USERENV(1fec.e0c) 15:26:19:133

=========================================================

USERENV(1fec.e0c) 15:26:19:133 LoadUserProfile: Entering, hToken = <0x2ac>,

lpProfileInfo = 0x6e5e8

USERENV(1fec.e0c) 15:26:19:133 LoadUserProfile: lpProfileInfo->dwFlags = <0x0>

USERENV(1fec.e0c) 15:26:19:133 LoadUserProfile: lpProfileInfo->lpUserName =

<jpmctrx>

USERENV(1fec.e0c) 15:26:19:133 LoadUserProfile: NULL central profile path

USERENV(1fec.e0c) 15:26:19:133 LoadUserProfile: lpProfileInfo->lpDefaultPath =

<\\DMO9300RPS01\netlogon\Default User>

USERENV(1fec.e0c) 15:26:19:133 LoadUserProfile: NULL server name

USERENV(1fec.e0c) 15:26:19:133 GetInterface: Returning rpc binding handle

USERENV(2e4.1a0c) 15:26:19:133 IProfileSecurityCallBack: client authenticated.

USERENV(2e4.1a0c) 15:26:19:133 DropClientContext: Got client token 00000840, sid =

S-1-5-18

USERENV(2e4.1a0c) 15:26:19:133 MIDL_user_allocate enter

USERENV(2e4.1a0c) 15:26:19:133 DropClientContext: load profile object successfully

made

USERENV(2e4.1a0c) 15:26:19:133 DropClientContext: Returning 0

USERENV(1fec.e0c) 15:26:19:133 LoadUserProfile: Calling DropClientToken (as self)

succeeded

USERENV(1fec.e0c) 15:26:19:133 CProfileDialog::Initialize : Cookie generated

<285E25A23085D85DF172E48495BF67E3>

USERENV(1fec.e0c) 15:26:19:133 CProfileDialog::Initialize : Endpoint generated

<IProfileDialog_4F9A9C4A5C46A07A119F79494D45F1B7>

USERENV(2e4.e18) 15:26:20:367 IProfileSecurityCallBack: RpcBindingInqAuthClient

failed with 534

USERENV(1fec.e0c) 15:26:20:367 LoadUserProfile: Calling LoadUserProfileI took

exception. err = 5

USERENV(1fec.e0c) 15:26:20:367 LoadUserProfile: Running as self

USERENV(1fec.e0c) 15:26:20:367 LoadUserProfile: Calling LoadUserProfileI failed.

err = 5

USERENV(2e4.1774) 15:26:20:367 IProfileSecurityCallBack: client authenticated.

USERENV(2e4.1774) 15:26:20:367 ReleaseClientContext: Releasing context

USERENV(2e4.1774) 15:26:20:367 ReleaseClientContext_s: Releasing context

USERENV(2e4.1774) 15:26:20:367 MIDL_user_free enter

USERENV(1fec.e0c) 15:26:20:367 ReleaseInterface: Releasing rpc binding handle

USERENV(1fec.e0c) 15:26:20:367 LoadUserProfile: Returning FALSE. Error = 5

USERENV(1b44.1d58) 15:26:55:554 InitializePolicyProcessing: Initialised Machine

Mutex/Events

USERENV(1b44.1d58) 15:26:55:554 InitializePolicyProcessing: Initialised User

Mutex/Events

USERENV(1b44.1d58) 15:26:55:554 LibMain: Process Name:

\??\U:\WINNT\system32\winlogon.exe

USERENV(2e4.2e8) 15:27:19:195 LoadUserProfile: Yes, we can impersonate the user.

Running as self

USERENV(2e4.2e8) 15:27:19:195

=========================================================

-------------------------------------

We are not able to find any related links for error 534

It is interesting to note, that the moment we change the secure channel of the Terminal Server to good Domain Controller, we are able to login with a user account from RPSLAN domain.

My initial judgment says, the problem DC is ...cmc07

Then I verified the trust from cmc07; checked the SMB signing and matched it with working DC.

We then created new user account in the RPSLAN domain, but we are still facing the same issue.

userenv is showing the same error.

Then, Customer took some netmon traces.

We analyzed the traces and found cmc07 is throwing STATUS_TRUSTED_DOMAIN_FAILURE .

We ran the command for sc_query on cmc07 for the RPSLAN domain, ...rps01 is the DC on the RPSLAN side, which is communicating with cmc07.

 

We then took the port query from cmc07 to rps01, and saw that all required ports seem to be open.

We then took MPS reports from cmc07, terminal server and the DC on the RPSLAN side.

We also checked that the SMB signing matches on both trusted DC.

We ran netmon trace simultaneously on both cmc07 and rps01 and reproduced the issue; we found that rps01 is giving "access is denied" to cmc07, the component is lsarpc.

 

Searched and found SOX060227700063, which address the same issue

Checked the nullsessionpipe under HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\lanmanserver\parameters on the cmc07, and found that we don't have lsarpc pipe included in it.

Checked default Windows Server 2000 values on my lab machine, we don't have this value on that either.

Checked the same key on RPSLAN Domain Controller (..rps01), the value is blank there

Customer is not sure, why it's blank

Added "lsarpc" through group policy, the location is

Computer Configuration\Windows Settings\Security Settings\local policies\security

options\Network access: Named Pipes that can be accessed anonymously

Ran gpupdate /force

Restart the server service

Tried login now, and we are able to login.

Issue Resolved

Advised Customer to add named pipe exceptions through default domain controller policy.

Customer will do it later

Answered few more questions

 

Later...

Called Customer

Customer confirmed that the issue is resolved.

 

 

Resolution

==============

 

I am summarizing the key points of the call

 

Problem:

When a user from RPSLAN login to the box(TS, RDP), we get the applying personal settings and got kicked out of the session.

 

Troubleshooting:

Took Userenv log from the TS, but error in the log could not explain the cause of the issue

SMB signing is same on Domain Controllers on both domains.

Trace from TS shows the following error

---------------------------------------

No. Time Source Destination

Protocol Info

1086 00:18:11.696061 10.3.25.23 10.3.85.218 LSA

LsarLookupSids2 response, STATUS_TRUSTED_DOMAIN_FAILURE

---------------------------------------

Reproduced the error, now took simultaneous trace on DC of both domain

CMC07 was getting access denied at the time of failure. Below netmon frame shows the issue

----------------------------------------

No. Time Source Destination

Protocol Info

4381 03:22:21.252685 10.3.25.23 169.85.141.126 SMB

NT Create AndX Request, Path: \lsarpc

4391 03:22:21.326338 169.85.141.126 10.3.25.23 SMB

NT Create AndX Response, Error: STATUS_ACCESS_DENIED

----------------------------------------

Checked the nullsessionpipe under

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\lanmanserver\parameters on RPS01.

This value was blank; by default we should have named pipe exception in there such

as "COMNAP,COMNODE,SQL\QUERY,SPOOLSS,netlogon,lsarpc,samr,browser"

Edit local group policy on the RPS01 to add lsarpc in it

The location of the key is

Computer Configuration\Windows Settings\Security Settings\local policies\security

options\Network access: Named Pipes that can be accessed anonymously

Forced group policy by running "Gpupdate /force"

Restarted Server service

Able to login on Terminal Server with account from RPSLAN domain.

 

 

Research Log

==============

 

Mail from Customer says to close the Support Case and to provide additional in depth information regarding the following items

 

Explanation of what the

 

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\lanmanserver\parameters

registry key/settings is used for.

Why would the SUNECMCXML01 server have issue with W2K domain controllers and not W2K3 domain controllers?

What does the fix Jerrod performed on the RPSLAN DC do from a security perspective (tighten or loosen)?

 

Once again thank you for your assistance.

 

Research Log

==============

 

Further Explanation

Q. Why would the SUNECMCXML01 server have issue with W2K domain controllers and not W2K3 domain controllers.

A. Because the products are different. Windows Server 2000 required anonymous connection and windows Server 2003 does not.

 

Q. What does the fix Jerrod performed on the RPSLAN DC do from a security perspective (tighten or loosen)?

A. By default anonymous access is disabled on win 2xxx servers (For security reasons). But few Microsoft functions (trust) need anonymous access to named pipe like lsaRPC. That's why we have to add exceptions in the policy. By adding the entries we loosen the security a bit. But we need those exceptions to achieve the correct functionality of various applications.

 

Please let me know if you have further questions for me.

 

Research Log

==============

 

Here is a summary of the key points of the case.

 

PROBLEM:

When a user from RPSLAN login to the box(TS, RDP), we get the applying personal settings and got kicked out of the session.

 

CAUSE:

Named pipe exceptions were missing from the trusted RPSLAN Domain DC.

 

RESOLUTION:

Added lsarpc named pipe exception in the below policy on RPS01(DC in the RPSLAN domain)

Computer Configuration\Windows Settings\Security Settings\local policies\security

options\Network access: Named Pipes that can be accessed anonymously