Schyler Ryan.

The problem

A domain controller was not making its SYSVOL and NETLOGON shares available, even though Active Directory replication checks were healthy. SYSVOL carries the files used by Group Policy and logon scripts, so directory replication alone was not enough to establish that the controller was ready.

A separate file-access problem involved a Distributed File System (DFS) namespace. The work needed to distinguish a replication problem from a namespace dependency problem, rather than treating every symptom as the same failure.

The decisions

I used replication state and event evidence to establish the recovery sequence. One controller was designated as the authoritative SYSVOL source, and the affected partner was re-enabled non-authoritatively. That made the direction of recovery explicit.

I also kept the namespace investigation separate. A successful SYSVOL recovery would not, by itself, demonstrate that a mapped file path or its namespace targets were healthy.

The technical work

1. Establish the actual replication state. Correlated healthy directory replication with missing shares, an Initial Sync state, and a DFS Replication event indicating that the controller was waiting for its partner.
2. Recover SYSVOL with a defined source. Worked through the authoritative and non-authoritative recovery sequence, checked the primary initialization event, and confirmed the affected controller subsequently reached State 4, Normal.
3. Trace the remaining access dependency. Checked the namespace object and running services, then investigated RPC failures and target connectivity. Name-resolution, network, RPC, and SMB checks exposed an unavailable server still referenced by the namespace.

What the evidence showed

Event and recovery-state reference: Microsoft’s SYSVOL troubleshooting guidance (opens in a new tab).

The problem

Adding multi-factor authentication (MFA) to server logons introduced questions that an agent installation could not answer: which account was being matched, what happened if the authentication service became unreachable, and how an administrator would access the system during recovery.

The environment included systems with different security and availability requirements. Existing Remote Desktop access also needed to remain usable, without restructuring directory groups or organizational units as part of the rollout.

The decisions

• Configure by server role. Treat authentication-failure behavior as an explicit policy decision, with different system responsibilities considered separately.
• Resolve identity matching. Account for Windows username formats and Duo account matching, so naming assumptions did not become an enrollment problem.
• Plan offline activation separately. Include hardware-backed offline authentication in the administrator access plan, while distinguishing a registered account from an account activated for offline access on a particular system.
• Respect the access method. Keep console and Remote Desktop scenarios separate when documenting the recovery path.

The important technical distinction

The selected security-key workflow required offline activation for each account on each Windows system. A key being available to an administrator did not mean offline access was ready everywhere.

Duo’s password-based offline U2F security-key flow supports local console logons. A key attached to a standard Remote Desktop client is not an equivalent offline authentication path to the remote server. That distinction shaped the recovery-access planning.

Product behavior reference: Duo’s Windows offline-access requirements (opens in a new tab).

Rollout checkpoints

The plan separated three states that can otherwise be mistaken for the same thing:

Policy configured

The intended server-role and authentication behavior is defined.

Account prepared

The account matches the expected identity, with the required enrollment and system-specific offline activation.

Access validated

The intended login method is exercised under the conditions it needs to support.

These are acceptance criteria, not a claim that every account and outage scenario was tested.

The problem

Validation was moving out of GitHub Actions into a local workflow. The goal was to change where checks ran while retaining the coverage and release controls that already mattered: source contracts, compatibility, packaging, infrastructure, rollback, and signing.

The boundary between validation and execution was equally important. A successful test run needed to be useful on its own, without publishing a package, triggering a release, or changing live infrastructure.

The implementation

I implemented a local wrapper, updated the tests and documentation, and retained manual packaging, signing, publishing, and deployment workflows.

Parallel checks

Run independent validation work concurrently within the wrapper.

Pinned dependencies

Use defined dependency versions so the execution environment is reproducible.

Isolated environments

Keep test execution separate from the working environment and include cleanup.

Timing reports

Make execution time visible rather than assuming an optimization produced a measurable gain.

Preserving the checks

The wrapper retained multiple kinds of evidence instead of reducing validation to a single build command:

• Source and compatibility: source-contract checks and Windows PowerShell compatibility checks.
• Packaging and operations: package and runbook checks, alongside retained release, signing, and rollback coverage.
• Behavior and infrastructure: mock-based tests and Terraform validation across multiple stacks.

Expected offline-dependent skips were documented separately. A skipped check was not counted as a pass, and Terraform validation was not treated as evidence that infrastructure had been deployed.

Closing the verification loop

1. Validate the integrated source. Incorporate the current main branch and run the relevant checks against that combined state.
2. Merge the completed work. Keep the validation change separate from the manual release and deployment workflows.
3. Compare after the merge. Verify that the merged source matched the source that had actually been tested.

This last comparison mattered: a passing pre-merge run would not, on its own, establish that the final merged source was the same.