Scenario
Novell eDirectory and Microsoft Active Directory are connected using Novell (NetIQ) Identity Manager. Bidirectional password synchronization is enabled, but synchronization from AD to eDir is not working. The password filter (pwfilter.dll) is installed on all the writable domain controllers and show as Running. Users change their password in AD, then call the help desk because their password hasn't changed in eDir and all the other connected systems. The help desk changes their password in eDir and all seems well.
But... It's not. There's a hidden danger lurking on the domain controllers, and one wrong move can unleash a torrent of stale passwords just waiting to synchronize to eDirectory, and by extension to all connected systems. Has it happened to you? It's happened to me, to the tune of 2,000 passwords suddenly changing to values set many months ago.
I've done a lot of research trying to resolve the AD to eDir password sync issue for the customer, and I didn't find anything warning me about the dangers of stale cached passwords. Now that I've searched for specifically that possibility, I've found a few that briefly mention it and one TID that specifically addresses the issue, but none that has what I consider an appropriate warning. Every article addressing AD to eDir password synchronization issues should include something like:
Let me give a quick explanation of how AD to eDirectory password synchronization works. A user could potentially change their password on any DC, so every DC runs a password filter that captures the changes and stores them in the registry. The security settings prevent anyone - even Enterprise Administrators - from seeing these entries (but keep reading for a way to do it without meddling with permissions). The password changes are sent to the DC running the remote loader, which stores them in its registry (or, if there is no remote loader, the DC running IDM). That DC reads its own registry, then sends the passwords on to eDirectory.
A little aside here. I realize I haven't posted screenshots or snippets of code or log files. This is primarily because I'm posting what I've encountered at customer sites, and I don't want to identify my customers nor their system configurations. I apologize if this makes things a bit more difficult to follow, but feel free to contact me directly and I'll try to clarify anything or help you with your specific issues.
Until recently, cached passwords never timed out (and even with current code that isn't the default setting). If a DC wasn't able to send its cached passwords to IDM, they stayed there indefinitely. If somehow the DC was able to communicate with IDM, the cached passwords would be processed as if they were new, changing the password in eDirectory, and by extension every other attached system. In addition to the password itself, the cached data contains the password change time and expiration date.
How to see cached passwords in the registry without changing permissions
They are visible only to the local SYSTEM account, so the trick is to run regedit.exe as that account. You can do this using psexec.exe from SysInternals, which can be downloaded here: http://technet.microsoft.com/en-us/sysinternals/bb842062.aspx.
Extract the download and then copy ONLY psexec.exe to each of your writable domain controllers. Run CMD as Administrator (Start | Search for CMD | Right-click and choose Run As Administrator). Run "psexec -i -s -d CMD", which will launch a new CMD as SYSTEM. From there, run regedit.exe and navigate to HKLM/Software/Novell/PwFilter/Data. You'll see one key per cached password, i.e. HKLM/Software/Novell/PwFilter/Data/ABRADLEY. If the user has changed their password on the DC more than once, only the most recent password will be stored. BUT - if they've changed their password on more than one DC, there will be a separate entry for the user on each of those DCs. For the Remote Loader DC, the path is HKLM/Software/Novell/PassSync/Data.
To see WHEN the password was last changed on that DC, export that registry key, save as .txt, and then open the export with a text editor.
How I "Fixed" Password Synchronization
In this instance, it was the Windows Firewall on the Remote Loader running on Windows Server 2008 R2. When I disabled the firewall, the communication between the other DCs and the RL was open and the cached passwords came flooding in. I could see what users were impacted by looking at the log files, as it showed each username, first in a long list of arriving cached passwords, then again when processing each password.
References
How to Prevent Syncing Old Password Changes from Active Directory
View Registry Key Timestamp
Getting a CMD prompt as SYSTEM
Remote Loader on Windows 2008 R2 and PWFilter Firewall settings
IDM Password Sync Components
DC Passwords TimeToLive (Cached Password Timeout)
No comments:
Post a Comment