ADSI corrupt ADAM SID in ACE

ADAM Sid in newly or modified ACEs are corrupted by ADSI during the transfert of these ACEs to the ADAM server. The buggy component is probably the ActiveDs.dll of Windows XP SP2: Windows XP Professional - 5.1.2600.2180 (xpsp_sp2_rtm.040803-2158).

This behaviour is reproductible and happen on all ACLs containing ADAM Sid manipulated throught ADSI with Windows XP. All windows XP users that manipulate ACLs containing ADAM sid are affected.

When the problem occurs, the server respond with a:
System.Runtime.InteropServices.COMException (0x80072035): The server is unwilling to process the request.
when the security descriptor is transmitted for update to the server.

A hotfix is now available, the knowledge base article regarding this issue is KB 896354.

Failed to execute the [code] macro

This sample code try to update an ACLs in an ADAM repository. A new ACE is created for an ADAM user and added to the ACL of an existing entry. The user SID as been retrieved from the objectSID attribute of the user entry:
01 05 00 00 11 C0 7D D7 C3 CD ....
and converted to string using Win32 API:
S-1-297827799-211...
It is hardcoded in the sample code for simplicity.

Unhandled Exception: System.Runtime.InteropServices.COMException (0x80072035): T
he server is unwilling to process the request.
   at System.DirectoryServices.Interop.IAds.SetInfo()
   at System.DirectoryServices.DirectoryEntry.CommitChanges()
   at Test.SdTest.Main() in c:\test\src\sdtest.cs:line 115

When committing the change of ACLs to the ADAM instance the server reply with a:
System.Runtime.InteropServices.COMException (0x80072035): The server is unwilling to process the request.

After many hours of research, I have added some code (#define MakeBinaryRoundTrip in the sample) to convert the SecurityDescriptor into an SDDL string, which require first to convert the IID SD into a RAW structure self-relative descriptor using an IADsSecurityUtility object.

Raw SD: D:AI(A;CI;LO;;;S-1-297827591-2113850819-3783973228-1289696667-1737617808
-1218521112)(A;CIID;LCRPLORC;;;S-1-297827799-2113850819-514)(A;CIID;CCDCLCSWRPWP
DTLOCRSDRCWDWO;;;S-1-297827799-2113850819-512)
Original Trustees:

   S-1-297827799-2113850819-514
   S-1-297827799-2113850819-512
   S-1-297827799-2113850819-3783973228-1289696667-1737617808-1218521112
Cloned Trustees:
   S-1-297827591-2113850819-3783973228-1289696667-1737617808-1218521112
   S-1-297827799-2113850819-514
   S-1-297827799-2113850819-512

Unhandled Exception: System.Runtime.InteropServices.COMException (0x80072035): T
he server is unwilling to process the request.
   at System.DirectoryServices.Interop.IAds.SetInfo()
   at System.DirectoryServices.DirectoryEntry.CommitChanges()
   at Test.SdTest.Main() in c:\test\src\sdtest.cs:line 115

As the sample shows, when making the conversion of the IID SD into a RAW SD and back into a IID SD, the SID of the newly created ACL change ! It became:
S-1-297827591-211...
notice the 591 in place of 799. Note that the corruption is already present in the RAW form as shown by the convertion of the RAW SD to SDDL string:
01 05 00 00 11 C0 7D 07 C3 CD ....
notice the 4 most significant bit of the 8th byte has been cleared (07 in place of D7). Which means that either the SD converted is wrong or that the conversion from IID to RAW does not perform correctly.

Moreover, if I corrupt these 4bits in the user SID (#define CorruptSid to have E7 (...815...) in place of D7 (...799...)), the SID received back from the IID => RAW => IID convertion is the same again. 

Raw SD: D:AI(A;CI;LO;;;S-1-297827591-2113850819-3783973228-1289696667-1737617808
-1218521112)(A;CIID;LCRPLORC;;;S-1-297827799-2113850819-514)(A;CIID;CCDCLCSWRPWP
DTLOCRSDRCWDWO;;;S-1-297827799-2113850819-512)

Original Trustees:
   S-1-297827799-2113850819-514
   S-1-297827799-2113850819-512
   S-1-297827815-2113850819-3783973228-1289696667-1737617808-1218521112
Cloned Trustees:
   S-1-297827591-2113850819-3783973228-1289696667-1737617808-1218521112
   S-1-297827799-2113850819-514
   S-1-297827799-2113850819-512

Unhandled Exception: System.Runtime.InteropServices.COMException (0x80072035): T
he server is unwilling to process the request.
   at System.DirectoryServices.Interop.IAds.SetInfo()
   at System.DirectoryServices.DirectoryEntry.CommitChanges()
   at Test.SdTest.Main() in c:\test\src\sdtest.cs:line 115

This confirm that these bits are effectively ignored and dropped during the conversion. More tests has shown that this only happen when the SID has been created or change using put_Trustee from the IADsAccessControlEntry interface. If I keep the same SID in an ACE created with the dsacl tool, and just change the AccessMask for example, the ADAM update the ACE correctly.

My final test was to patch the SID in its RAW form (#define PatchSid and undefine #CorruptSid no more needed). So before converting back the RAW SD into IID, I replace the 8th byte of the SID in the ACE by D7, fixing the wrong 07.

Raw SD: D:AI(A;CI;LO;;;S-1-297827591-2113850819-3783973228-1289696667-1737617808
-1218521112)(A;CIID;LCRPLORC;;;S-1-297827799-2113850819-514)(A;CIID;CCDCLCSWRPWP
DTLOCRSDRCWDWO;;;S-1-297827799-2113850819-512)

Corrupted byte patch from 7 to D7
Patched SD: D:AI(A;CI;LO;;;S-1-297827i799-2113850819-3783973228-1289696667-173761
7808-1218521112)(A;CIID;LCRPLORC;;;S-1-297827799-2113850819-514)(A;CIID;CCDCLCSW
RPWPDTLOCRSDRCWDWO;;;S-1-297827799-2113850819-512)
Original Trustees:
   S-1-297827799-2113850819-514
   S-1-297827799-2113850819-512
   S-1-297827799-2113850819-3783973228-1289696667-1737617808-1218521112

Cloned Trustees:
   S-1-297827799-2113850819-3783973228-1289696667-1737617808-1218521112
   S-1-297827799-2113850819-514
   S-1-297827799-2113850819-512

The converted back IID SD now correctly report the SID 799, and is the same than the original SD. Using this patched SD, ADAM accept the change and update the ACL correctly.

My conclusion is that during the transfert of the SD to the ADAM directory, a convertion to RAW form is made and that conversion fails the same way mine fail. The ADAM server obviously refuse to update its ACLs with an unknown SID and is therefore unwilling to process.

A hotfix is now available, the knowledge base article regarding this issue is KB 896354.