powershelldba.de · Uwe Janke

How AlwaysOn Really Works: The Log Flow Behind Every Commit

"AlwaysOn" is a marketing name for a stack of much older, much more mechanical parts: Windows Server Failover Clustering, a database mirroring endpoint, and a REDO thread that doesn't care what your commit already promised. Here's what actually happens between COMMIT and the acknowledgment your application receives, with an animated walk-through.

The Building Blocks

An Availability Group is a logical container, not a single moving part. Four pieces work together:

None of this replicates data by shipping backups or comparing tables. It moves the transaction log: the same stream SQL Server already writes for crash recovery, mirrored to another instance in near real time.

Watching a Synchronous Commit

This is the part most explanations skip: what a client's COMMIT actually waits on when the database is in synchronous-commit mode. Hover the diagram to pause it at any step.

DATABASE MODE: SYNCHRONOUS-COMMIT APPLICATION PRIMARY REPLICA TRANSACTION LOG local hardening DATA PAGES SECONDARY REPLICA TRANSACTION LOG hardened copy DATA PAGES now current ① COMMIT ② log block mirroring endpoint — TCP 5022, encrypted ③ ACK: hardened ④ committed ⑤ REDO async, can lag
Steps 1–4 are what the commit waits on. Step 5 (REDO) happens after the client already has its acknowledgment.

The sequence:

  1. COMMIT arrives at the primary. The log record is generated and written to the primary's own log buffer.
  2. The primary hardens the log record — flushed to disk locally, same as any standalone instance.
  3. The log block is shipped to the secondary over the database mirroring endpoint (TCP port 5022 by default, encrypted, using the same wire protocol SQL Server has used for mirroring since 2005).
  4. The secondary hardens its own copy of the log record and sends an acknowledgment back.
  5. Only once that ACK arrives does the primary consider the transaction committed and return control to the application.

That round trip — steps 1 through 4 — is the entire cost of synchronous-commit mode. It's also the entire guarantee: a synchronized secondary's log is durable before the application ever finds out the transaction succeeded.

Synchronous vs. Asynchronous Commit Mode

Aspect Synchronous-commit Asynchronous-commit
Primary waits for Secondary's log-hardened ACK Nothing — commits locally and moves on
Data loss on failover (RPO) Zero, if the secondary was SYNCHRONIZED Possible — whatever hadn't shipped yet
Commit latency impact Adds one network round trip per commit None
Eligible for automatic failover Yes, if configured and SYNCHRONIZED No — manual (forced) failover only
Typical use Local/same-datacenter replicas, DR partner in a failover cluster instance role Cross-region replicas, readable secondaries, reporting offload

Distributed Availability Groups always link their two AGs asynchronously, even if every replica inside each AG is synchronous internally — see Distributed Availability Groups for that topology.

The REDO Queue: Why "Synchronized" Doesn't Mean "Current"

Step 5 in the diagram — REDO — is the detail that trips people up. Log hardening and data-page REDO are two separate jobs on the secondary, and only the first one gates the commit.

A dedicated REDO thread on every secondary continuously reads its own hardened log and applies each record to the data pages, exactly like crash recovery replaying a log after a restart. That thread runs independently of commit acknowledgment. A secondary can be fully SYNCHRONIZED (every log record safely on disk) while its REDO queue is still minutes behind on applying those records to the actual pages — under a large batch load, an index rebuild on the primary, or simply a secondary with slower storage.

Two DMV columns make this visible:

SELECT database_id, redo_queue_size, redo_rate
FROM sys.dm_hadr_database_replica_states
WHERE is_local = 0;

redo_queue_size (KB of hardened log not yet applied) and redo_rate (KB/sec being applied) tell you how far behind a readable secondary's actual data is — independent of whether Windows or SSMS is showing you a reassuring green "Synchronized" state. This is exactly why a readable secondary can return stale results even when the AG dashboard looks perfectly healthy; see AG Readable Secondaries for Reporting Offload for what that means for routed queries.

Two Independent Timers Decide When a Replica Gets Cut Loose

Health monitoring runs on two clocks that most people conflate into one:

Health Check Timeout

Every SQL Server instance in the AG continuously runs sp_server_diagnostics, reporting GOOD/WARNING/ERROR states for system, resource, query processing, I/O subsystem, and events roughly every third of the HealthCheckTimeout interval (default 30 seconds, so roughly every 10 seconds). The AG's Resource DLL (hadrres.dll) inside WSFC watches these reports. If a replica configured for automatic failover crosses the FAILURE_CONDITION_LEVEL threshold, WSFC initiates failover.

Lease Timeout

Separately, each replica holds a lease with the WSFC resource for the AG (default 20 seconds). This lease has nothing to do with query health — it's a heartbeat between the SQL Server process and the cluster. If the primary can't renew its lease in time (a paused VM, a stalled network adapter, extreme CPU starvation), the instance takes its own databases offline before WSFC even acts, specifically to prevent two replicas from both believing they're primary at once.

That's why a primary can go dark with no visible failover event in the cluster log first: the lease expiring is the instance protecting itself, not WSFC promoting a secondary. The promotion, if one happens, follows afterward and depends on quorum — see Quorum and Witness: Why Failover Fails and How to Fix It.

FAILURE_CONDITION_LEVEL, in short:
1 — SQL Server service down
2 — plus: not responding to WSFC health checks
3 (default) — plus: critical internal errors (thread pool exhaustion, etc.)
4 — plus: any error with a moderate severity that would restart the service
5 — plus: any sp_server_diagnostics "ERROR" state, including things like a full transaction log

What Automatic Failover Actually Does

  1. WSFC determines quorum still holds after removing the failed primary from the vote.
  2. A SYNCHRONIZED secondary (per the AG's failover configuration) is selected.
  3. That secondary's databases are promoted — REDO is forced to catch up completely before the role change finishes.
  4. The listener's virtual network name is repointed to the new primary's IP.
  5. The old primary, if it comes back, is fenced off and rejoins as a secondary; it does not fight for the primary role.

Automatic failover only ever promotes a replica that was fully SYNCHRONIZED, which is exactly why asynchronous replicas are excluded from the automatic policy — promoting one could mean promoting stale data without anyone choosing to accept that risk.

🔧 sqmSQLTool wraps the AG lifecycle end to end: New-sqmAvailabilityGroup and Add-sqmDatabaseToAg handle setup, Sync-sqmAgNode catches up a replica that's fallen behind, Invoke-sqmFailover runs a controlled failover, and Get-sqmAlwaysOnFailoverHistory gives you a timeline of what actually happened and when. For the failure mode where a database gets stuck outside the AG after an unplanned event, Repair-sqmAlwaysOnDatabases (and New-sqmAlwaysOnRepairJob for a standing Agent job that does it automatically) re-adds it without manual intervention, and Get-sqmAlwaysOnHealthReport gives you the redo-queue and synchronization picture from the section above in one call instead of hand-rolled DMV queries.

Full command reference: sqmSQLTool commands.

Common Misconceptions

Summary

AlwaysOn's guarantees come from where the primary actually waits: a synchronous commit is only as safe as the ACK it blocks on, and that ACK only covers the log, not the data pages. Everything downstream — REDO lag on readable secondaries, the lease-timeout replica that vanishes with no cluster event, the health-check threshold that decides whether WSFC even reacts — follows from that one mechanical fact.

For the setup side of this, see AlwaysOn Setup: Traditional vs. Automated and Restoring Databases into an Existing AlwaysOn Availability Group.