Article Applicability - Windows Server 2008 \ 2008 R2 \ 2012 \ 2012 R2 \ 2016
If we use the Windows server backup tool to restore an Active Directory system state backup, we see a “Perform an authoritative restore of Active Directory files” option as shown below;
The option states that if we select the checkbox, it will reset all replicated content on this domain controller including SYSVOL.
This is a misleading description. One would think that the entire AD can be restored authoritatively, however, this is not the case.
If you look at the Command line equivalent for the above option:
Wbadmin.exe start SystemStateRecovery –version:<version Identifier> -authsysvol
What Microsoft wanted to tell you is that if you select that option, the Active Directory database will get restored non-authoritatively, but the contents of the SYSVOL share folder will get restored authoritatively. Note that the DC must be rebooted in Directory Service Restore Mode (DSRM) for any AD system state restore.
We need to understand exactly what happens when we use the above option, as well as its implications.
Normally an FRS Sysvol Authoritative Restore (Burgflag D4) OR DFSR Sysvol Authoritative Restore (msDFSR-Options) does clear Journal wrap issues. However, AD system state recovery also restores/repairs SYSVOL broken structure and policies additionally.
AD System State Recovery with Authoritative Sysvol - DFSR SYSVOL |
AD System State Recovery with Authoritative Sysvol - FRS SYSVOL |
With 2008 Server, Microsoft has introduced the DFS replication engine (DFSR) for SYSVOL. Due to a basic difference between FRS and DFSR replication engine, FRS SYSVOL and DFSR SYSVOL authoritative recovery produce different results unless the rest of DCs are set for non-authoritative SYSVOL recovery in advance |
|
Default Behaviour:
If Sysvol is replicated with DFSR and you attempted a system state recovery along with an authoritative Sysvol, Active directory gets restored non-authoritatively and data restored in the Sysvol share is considered as primary/authoritative and after you reboot the DC in normal mode, data stored in the local Sysvol folders (Policies and scripts folders) of the DC gets forcefully replicated to the other DCs with overwrite mode, regardless if they are configured for non-authoritative recovery or not. This is one-way replication from restored DC to other DCs. It means Sysvol data on the other domain controllers will get overwritten (replaced) by Sysvol data in the restored DC any how.
One-way replication continues until all existing Sysvol data conflicts on the other DCs get resolved. You can see DFSR event IDs 4412 on other DCs wrt data conflicts and data deletion.
For Example:
If you have created new GPOs post system state backup, after restoring system state with an authoritative Sysvol option, those GPOs folders will get deleted from the restored DC and deletion gets replicated to the other DCs SYSVOL folder.
The GPO entry still remains in GPMC as AD gets restored non-authoritatively and hence the GPOs entry present on the other DCs gets replicated to this DC. The GPO can still be viewed through GPMC, however, it is a blank entry as associated GPO folder is deleted.
In that case, the GPO folder in Sysvol forcefully gets replicated to all DCs Sysvol folder but the GPO entry in AD gets deleted again as a replication update. To retain the GPO entry in AD, we must authoritatively restore the policies container in the domain partition with Ntdsutil tool immediately post recovery operation completed and before we reboot the DC
|
Default Behaviour:
If Sysvol is replicated with FRS and an AD system state restore with authoritative sysvol is attempted, active directory gets restored non-authoritatively and Sysvol contents are restored authoritatively as the BurFlags registry is set to D4 during the recovery operation and is further reset to 0 post reboot, hence the restored Sysvol folder replicates two way unless the replicating partners are set for a non-authoritative (BurFlags D2) restore in advance.
In short, if you do not set the other DCs as non-authoritative before attempting a restore operation, any GPO addition/deletion that happened post system state backup will get replicated to the restored DC and it will defeat the purpose of an authoritative Sysvol restore.
For Example: If you have created new GPOs post system state backup, after restoring the system state with an authoritative Sysvol, those new GPO folders will get replicated to the restored DC from other DCs as replication updates unless you set the other DCs for D2 recovery. GPO entries in AD still replicate to the restored DC as an AD replication update. Likewise, if the restore operation restored any deleted GPO, it actually restores the GPO entry in AD and the GPO folder in Sysvol, both non-authoritatively. The GPO folder will again get deleted as a replication update by other DCs unless they are set for a D2 recovery.
The GPO entry from Active directory also gets deleted as a replication update from other DCs. To retain a GPO entry in AD, we must authoritatively restore the policies container in the domain partition with Ntdsutil tool immediately post recovery operation completed and before we reboot the DC
|
This type of recovery should be attempted on any (one) DC in a domain (PDC master preferably) and only if any one of the conditions mentioned below is true
|
|
Prerequisites:
|
|
GPO and GPO links restoration: An AD system state recovery with the AuthSysvol option restores Sysvol data (Sysvol share) on physical drive authoritatively and active directory data non-authoritatively. GPOs are stored in active directory under domain.com\System\Policies container and GPO links are stored on OUs as a GPLink attribute. In case GPOs are deleted from AD as well, we need to restore the GPO entry in AD and GPO links on OUs, we need to restore the policies container and OUs authoritatively with the Ntdsutil tool post system state recovery and before rebooting the server |
Recovery Procedure - DFSR SYSVOL
The process should be carried out during off business hours as the logon process will get halted for DCs during Sysvol initialization period.
Decide which date system state backup copy should be restored (Sysvol structure should be intact with policies) in case of a broken sysvol structure or GPO deletions case
Note that a System state restore should be carried out on the PDC master server as far as possible
Step 1
Before we attempt this type of Sysvol authoritative restore, we need to ensure that all other DCs except one being recovered are set for a Sysvol non-authoritative restore
From Adsiedit.msc, load the domain directory partition and navigate to the following DN and edit the single attribute on all other domain controllers one by one in that domain
CN=SYSVOL Subscription,CN=Domain System Volume,CN=DFSR-LocalSettings,CN=ADC,OU=Domain Controllers,DC=domain,DC=com
(Replace ADC with other additional DCs one by one)
msDFSR-Enabled= FALSE
Step 2
A). Force AD replication across the domain (repadmin /AdeP) from PDC server
B). Run "dfsrdiag pollad" on every DC from elevated command prompt. You need to install DFS management tools on 2012 R2 and above DCs to install the "dfsrdiag" tool. On all DCs except PDC, check if DFSR event ID 4114 is generated which states that Sysvol replication is disabled on the domain controller.
C). Once you found event ID 4114 under DFSR events, stop DFSR service on all DCs except PDC
Step 3
Run "bcdedit.exe /set safeboot dsrepair" from an elevated command prompt on the server (PDC master preferably) being restored with system state backup, so that after reboot, the server will start in Directory Service Restore Mode (DSRM).
Step 4
Now reboot DC server in DSRM and run the wbadmin get versions command to locate available backups as shown below
Step 5
Once you find the desired backup, run the following command
wbadmin start SystemStateRecovery -version:12/15/2018-20:52 -authsysvol
Replace the version identifier with yours
Side Note:
What authoritative restore will do Is actually overwrite the Sysvol data from the system state backup to a local Sysvol folder and create the registry keys shown below
String (REG_SZ) key – SYSVOL
Value – Authoritative
Under path [HKLM\SYSTEM\CurrentControlSet\Services\DFSR\Restore]
AND
REG_SZ value "LastRestoreId"="b56d312b-5986-47aa-bd4b-13adb72f89df" at below path
[HKLM\SYSTEM\CurrentControlSet\Control\BackupRestore\SystemStateRestore]
The value of LastRestoreId is random 32-digit value and dynamically generates every time you attempt system state restore. The purpose is to inform system that restore operation has been attempted.
Step 6
Once restoration is completed, the system asks you to reboot the server. If GPOs are deleted from AD, those also need to be restored, do not reboot the server and open an elevated command prompt. Now enter the commands shown below, one by one
Ntdsutil
Activate instance ntds
Auth restore
Restore subtree CN=Policies,CN=System,DC=domain,DC=com
replace domain with yours
When prompted click yes for authoritative restore
This will increase USN of all group policies including restored under Policies container by 100000 numbers so that those policies will get replicated to other DCs as a directory update
Note - If you skip this step, restored GPOs will get deleted from AD again when DC comes online after reboot as part of replication update received from other DCs
Further, you need to restore the GPO links for restored GPOs, OUs needs to be restored authoritatively with Ntdsutil tool as a GPLink attribute is associated on OUs
Run the following commands from an elevated command prompt for each OU where GPOs was applied
Ntdsutil
activate instance ntds
auth restore
restore object “OU=Test,DC=domain,DC=com”
Once all required OUs are restored, quit Ntdsutil
Step 7
Now run "bcdedit.exe /deletevalue safeboot" from an elevated command prompt and reboot the server. The command is necessary, else post reboot server will again start in DSRM mode only
Step 8
Once the server is rebooted, login with a domain administrator account on the server, upon login, you will get a command prompt saying that the system state restore operation has successfully completed
Hit enter to continue.
Step 9
Now open event viewer and locate the DFSR events
Event ID 2109 (Warning) - The DFS Replication detected the System State Restore Id has changed on volume C:. Replication has been stopped for all system replicated folders on this volume
Event ID 2110 (Information) - The DFS Replication service successfully recovered from the System State Restore Id change on volume C:. Replication has resumed on system replicated folders on this volume
Event ID 4106 (Information) - The DFS Replication service detected that an authoritative restore was performed on the replicated folder at local path C:\Windows\SYSVOL\domain and is processing the restore. Replication has been paused until the restore completes
Event ID 4108 (Information) - The DFS Replication service successfully processed an authoritative restore for the replicated folder at local path C:\Windows\SYSVOL\domain
DFSR sysvol authoritative restore completed successfully. Both registry entries created by the backup application during restore remain AS IS on the restored server and can be removed now
Step 10
Open cmd and run Net Share to check if Sysvol and Netlogon shares are present. They must be present.
Locate the Sysvol folder structure and junction points are restored as appropriate including restored GPOs from GPMC if any
The restoration process will also restore default permissions on the SYSVOL folder tree
Step 11
Now it’s time to restore Sysvol non-authoritatively on the other DCs
If the SYSVOL folder tree structure is intact on these DCs, then skip this step and jump to Step 12
Else follow the steps below
If the SYSVOL folder tree structure is broken or did not exist on the DCs being restored non-authoritatively, the non-authoritative restore will fail and you will get DFSR error events
You need to manually create the SYSVOL folders structure with the steps below
A). Logon to the specified DC and ensure dfsr service is stopped and "msDFSR-Enabled" attribute is already set to False from step 2 & step 1 respectively
B). Create the SYSVOL folder structure under the %systemroot% folder as below
\SYSVOL
\SYSVOL\domain
\SYSVOL\domain\Policies
\SYSVOL\domain\scripts
\SYSVOL\staging
\SYSVOL\staging\domain
\SYSVOL\staging areas
\SYSVOL\SYSVOL
C). Navigate to the SYSVOL\Sysvol folder from an elevated prompt and create a junction point to redirect the policies and scripts folder. Run the following command
mklink domain.com /J C:\windows\SYSVOL\domain
D). Navigate to SYSVOL\Staging areas folder from an elevated prompt and create a junction point to redirect to the staging folder. Run the following command
mklink domain.com /J C:\windows\SYSVOL\staging\domain
E). Download INF file sysvol.zip or Copy INF contents from Microsoft links and create a sysvol.inf file under %systemroot%\security\templates folder and run the following command from an elevated prompt to restore default SYSVOL permissions
secedit.exe /Configure /cfg %systemroot%\security\templates\sysvol.inf /db %systemroot%\security\templates\sysvol.db /overwrite
Now proceed to step 12
Step 12
This step needs to be carried on all other domain controllers one at a time in that domain:
Start DFSR service
Navigate to same DN as Step 1 from Adsiedit.msc and edit single attribute
msDFSR-Enabled= TRUE
Force AD replication and poll active directory with “dfsrdiag pollad”. You may need to install DFS management tools on DC for command to work.
For the 1st targeted DC, this step will ensure that DC will delete contents of local SYSVOL (Policies and scripts) folder and fetch fresh copy from other authoritative DC which is nothing but restored one from system state.
Check DFSR event logs on the DC
Event ID 4614 (warning) - The DFS Replication service initialized SYSVOL at local path C:\Windows\SYSVOL\domain and is waiting to perform initial replication. The replicated folder will remain in the initial synchronization state until it has replicated with its partner. If the server was in the process of being promoted to a domain controller, the domain controller will not advertise and function as a domain controller until this issue is resolved.
Event ID 4604 (Information) - The DFS Replication service successfully initialized the SYSVOL replicated folder at local path C:\Windows\SYSVOL\domain. This member has completed initial synchronization of SYSVOL with partner ROOTDC.race.net. To check for the presence of the SYSVOL share, open a command prompt window and then type "net share".
Run net share from a command prompt and ensure Sysvol is shared on the DC
Open cmd and run Net Share to check if Sysvol and Netlogon shares are present, you may need to restart the netlogon service once if shares are not present
Open GPMC on the server and check if all GPOs are visible and accessible. You may get the prompt shown below while accessing GPOs
Click OK
This way attempt all DCs one by one, one at a time and they can complete their non-authoritative restore by fetching Sysvol contents from other authoritative DCs which is nothing but the restored one or other DCs that have completed a non-authoritative restore
Step 13
Once all DC servers have completed non-authoritative recovery, create a test GPO and check if it’s able to replicate on all DCs
Recovery Procedure - FRS SYSVOL
The following process should be carried out during off business hours as the logon process will get halted for DCs during the Sysvol initialisation period.
Decide which date system state backup copy should be restored (Sysvol structure should be intact with policies) in case of a broken Sysvol structure or GPO deletions case
Note that System state restore should be carried out on PDC master server as far as possible
Step I
Before we attempt this type of Sysvol authoritative restore, we need to ensure that all other DCs except the one being recovered are set for Sysvol non-authoritative restore.
On each DC:
Locate “BurFlags” (REG_DWORD) key and set its value “D2” (hex). If the key does not exist, create it.
Key is located at HKLM\System\CurrentControlSet\Services\NtFrs\Parameters\Backup/Restore\Process at Startup
AND
Stop File replication service (ntfrs)
Step II
Run "bcdedit.exe /set safeboot dsrepair" from an elevated command prompt on the server (PDC master preferably) being restored with a system state backup, so that after reboot the server will start in Directory Service Restore Mode (DSRM).
Step III
Now reboot the DC server in DSRM and run the wbadmin get versions command to locate available backups as shown below
Step IV
Run the command shown below for the desired backup version
wbadmin start SystemStateRecovery -version:12/15/2018-09:04 -authsysvol
Side Note:
What authoritative restore will do, it actually overwrites Sysvol data from system state backup to local Sysvol folder and sets below registry keys
REG_DWORD key “BurFlags” with value “D4” (hex) as in Authoritative restore at below path
HKLM\System\CurrentControlSet\Services\NtFrs\Parameters\Backup/Restore\Process at Startup
AND
REG_SZ value "LastRestoreId"="b56d312b-5986-47aa-bd4b-13adb72f89df" at below path
[HKLM\SYSTEM\CurrentControlSet\Control\BackupRestore\SystemStateRestore]
The value of LastRestoreId is random 32-digit value and dynamically generates every time you attempt system state restore. The purpose is to inform system that restore operation has been attempted.
Step V
Once the restoration is completed, the system will ask for a reboot. If you need to recover GPOs and GPO links, do not reboot server. Follow Step 6 under Recovery Procedure - DFSR SYSVOL
Step VI
Now run "bcdedit.exe /deletevalue safeboot" from a command prompt and reboot the server. The command is necessary, else post reboot, the server will again start in DSRM mode only
Step VII
Now reboot the server normally and logon as domain administrator on the server, upon logon, you will get a command prompt saying that the system state restore operation has successfully completed
Hit enter to continue.
Note - Post reboot, as NTFRS service started, BurFlags registry reset to 0. It means SYSVOL authoritative restore is attempted.
Step VIII
Now open event viewer and locate NTFRS events 13553, 13554 and 13516
Event 13553 states that local DC is added to “Domain System Volume (Sysvol Share)” replica set
Event 13554 states that local server added connection object with another member (DC) in replica set
Event 13516 states that Sysvol is successfully initialized and advertising local server as active domain controller,
Step IX
Open cmd and run Net Share to check if Sysvol and Netlogon shares are present. They must be present.
Locate Sysvol folder structure and junction points are restored as appropriate including restored GPOs from GPMC if any
The restoration process will also restore default permissions on the SYSVOL folder tree
Step X
Now it’s time to restore Sysvol non-authoritatively on the other DCs. Target one DC at a time to avoid conflicting updates to be flown
If the SYSVOL folder tree structure is intact on DC, then skip this step and jump to Step XI
Else, ensure that "BurFlags" is set to D2 as well as the ntfrs service is stopped (as mentioned in Step I) and follow the steps from Step 11 (from B to E) under heading Recovery Procedure - DFSR SYSVOL
This way once Sysvol folder structure is recreated, go to Step XI below
Step XI
For the 1st targeted DC, Start the file replication service. Once started, ”BurFlags” registry value reset to "0" from "D2" and DC will delete local SYSVOL contents (Policies and Scripts) and fetch Sysvol contents from the other authoritative DC which is nothing but the restored one from system state. Below events get logged in NTFRS event logs on that DC
Event 13553 states that local DC is added to “Domain System Volume (Sysvol Share)” replica set
Event 13554 states that local server added connection object with another member (DC) in replica set
Event 13516 states that Sysvol is successfully initialized and advertising local server as active domain controller,
Open cmd and run Net Share to check if Sysvol and Netlogon shares are present, you may need to restart the netlogon service once if the shares are not present.
Open GPMC on the server and check if all GPOs are visible and accessible. You may get below prompt while accessing GPOs
Click OK
Step XII
This way attempt all DCs one by one, one at a time and they can complete their non-authoritative restore by fetching Sysvol contents from other authoritative DCs which is nothing but the restored one or other DCs that have completed a non-authoritative restore
Step XIII
Once all DC servers have completed a non-authoritative recovery, create a test GPO and check if it’s able to replicate on all DCs
I hope this article will help understand SYSVOL recovery from backup
If you liked this article, please click the Thumbs-up icon below.
Have a question about something in this article? You can receive help directly from the article author. Sign up for a free trial to get started.
Comments (0)