Diagnosing HADR_SYNC_COMMIT: Isolating AG, Network, Storage, or Application

What HADR_SYNC_COMMIT Actually Means

In a synchronous-commit Availability Group, a transaction on the primary isn't considered committed until every synchronous secondary has hardened the log record, written it to disk and acknowledged it back. The primary session sits in HADR_SYNC_COMMIT for exactly that round trip: send log block, secondary writes it, secondary acks, primary proceeds.

That's also why the symptom is asymmetric. SELECT doesn't touch the commit path at all, so it's unaffected by secondary latency. Only INSERT, UPDATE, DELETE, and explicit transaction commits wait on HADR_SYNC_COMMIT, because only they generate log records that need to be hardened remotely before the client gets control back. A workload that's "mostly reads, occasionally slow writes" pointing at climbing HADR_SYNC_COMMIT is close to a textbook signature.

The Detail Everyone Skips: Check the Secondary, Not Just the Primary

HADR_SYNC_COMMIT is a wait on the primary, but it's almost never caused by the primary. The primary's CPU, memory, and storage being clean, which the scenario already confirms, is exactly what you'd expect: the primary isn't doing the slow part. It's waiting on somebody else. The place to look is the secondary's ability to receive, write, and acknowledge log records, which depends on three independent things: the network path to the secondary, the secondary's own log-write throughput, and whether the secondary's redo thread is keeping up at all.

This is the single most common mistake in this scenario: spending the first twenty minutes re-checking the primary's DMVs, wait stats, and resource governor, when the primary was never the bottleneck to begin with.

What to Check FIRST

Normally the first move in an AG synchronous-commit slowdown is straight to sys.dm_hadr_database_replica_states on the primary. Here it isn't, because of one detail in the scenario that changes the whole triage order: Azure SQL Database, a completely separate PaaS service with its own architecture, is experiencing intermittent latency at the same time.

Azure SQL Database and an AG on IaaS VMs share almost nothing at the software level; different engine hosting model, different storage stack, different network path. If both are degraded simultaneously, an AG-specific misconfiguration or a workload-specific problem can't explain both. The one thing they plausibly share is the underlying Azure platform: the region's network fabric, a storage scale unit, a host cluster, or an ongoing Azure incident.

If Service Health is clean, the correlation itself is still a useful data point: it pushes the working theory toward shared infrastructure (regional network path, a specific Availability Zone, a storage scale unit) over "the AG is misconfigured" or "the application changed," and shapes steps 2 onward below.

Step-by-Step Isolation

Step 1: Quantify How Far Behind the Secondaries Are

On the primary, this is the single query that tells you whether the AG itself is the bottleneck and, if so, which secondary:

SELECT
  ar.replica_server_name,
  drs.database_id,
  db_name(drs.database_id)               AS database_name,
  drs.is_local,
  drs.synchronization_state_desc,
  drs.log_send_queue_size,               -- KB not yet sent to this secondary
  drs.log_send_rate,                     -- KB/sec actually being sent
  drs.redo_queue_size,                   -- KB received but not yet replayed
  drs.redo_rate,                         -- KB/sec being replayed on the secondary
  drs.last_commit_time,
  drs.last_hardened_time
FROM sys.dm_hadr_database_replica_states drs
JOIN sys.availability_replicas ar ON drs.replica_id = ar.replica_id
ORDER BY ar.replica_server_name, database_name;

A large log_send_queue_size with a low log_send_rate points at the network path. A large redo_queue_size on a secondary that's receiving log fine points at that secondary being unable to write/replay fast enough, its own storage or CPU, not the network. A growing gap between last_commit_time on the primary and last_hardened_time on the secondary is the stall, quantified in seconds instead of inferred from user complaints.

Step 2: Confirm Who's Actually Waiting, and How Long

-- Cumulative, since last restart or stats clear
SELECT wait_type, waiting_tasks_count, wait_time_ms, signal_wait_time_ms,
       wait_time_ms - signal_wait_time_ms AS resource_wait_ms
FROM sys.dm_os_wait_stats
WHERE wait_type LIKE 'HADR%'
ORDER BY wait_time_ms DESC;

-- Live, right now: which sessions are actually stuck in the wait
SELECT r.session_id, r.wait_type, r.wait_time, r.command,
       r.start_time, s.host_name, s.program_name
FROM sys.dm_exec_requests r
JOIN sys.dm_exec_sessions s ON r.session_id = s.session_id
WHERE r.wait_type = 'HADR_SYNC_COMMIT';

A near-zero signal_wait_time_ms relative to total wait confirms the primary isn't CPU-starved and is genuinely waiting on an external acknowledgment, which lines up with "CPU and memory look normal" from the scenario and closes the loop on the primary as a suspect.

Step 3: Check the Secondary's Own Health, Not the Primary's View of It

Everything so far has been observed from the primary. The actual bottleneck lives on the secondary, so connect to it directly:

• Log write latency on the secondary's own log disk. The secondary has to physically harden every log block before it can ack. If its log volume is throttled (a common Azure Managed Disk behavior once IOPS/throughput credits are exhausted), redo and hardening both slow down even though the secondary VM's aggregate CPU/memory metrics look fine.
• Whether redo is blocked on the secondary. A readable secondary running reporting queries can hold locks that stall the redo thread; check sys.dm_exec_requests on the secondary for a blocked redo session, and sys.dm_hadr_database_replica_states.is_suspended for a suspended database.
• Concurrent maintenance on the secondary: automatic seeding of another database, a log backup contending for the same disk, antivirus or backup-agent I/O, all consume the same log-write bandwidth the AG needs.

Step 4: Measure the Network Path Directly

Don't infer network health from "Azure VMs show normal CPU" — that tells you nothing about inter-VM latency. Measure it directly between the primary and each secondary:

# From the primary VM, against each secondary
Test-NetConnection -ComputerName SECONDARY01 -Port 5022 -InformationLevel Detailed
ping SECONDARY01 -n 50   # look for latency spikes and any loss, not just the average

Sub-millisecond, stable latency inside the same Availability Zone; low single-digit milliseconds cross-zone within a region; noticeably higher and more variable cross-region. If replicas were placed for DR across zones or regions, confirm that's still the actual topology, and that nothing changed it. A Proximity Placement Group or an Accelerated-Networking setting silently disabled on one NIC is a realistic, easy-to-miss cause of exactly this symptom.

Step 5: Rule the Application Workload In or Out

Even with healthy AG plumbing, the application can manufacture this symptom by making every commit more expensive to synchronize: very large transactions, wide inserts, or a burst of concurrent writers all inflate the amount of log that has to cross the wire and harden per commit.

-- Look for unusually large or long-running write transactions around the incident window
SELECT s.session_id, s.login_name, r.start_time, r.total_elapsed_time,
       t.database_transaction_log_bytes_used,
       t.database_transaction_log_bytes_reserved
FROM sys.dm_tran_database_transactions t
JOIN sys.dm_tran_session_transactions st ON t.transaction_id = st.transaction_id
JOIN sys.dm_exec_sessions s ON st.session_id = s.session_id
LEFT JOIN sys.dm_exec_requests r ON r.session_id = s.session_id
ORDER BY t.database_transaction_log_bytes_used DESC;

A batch job, a bulk import, or a runaway loop of single-row inserts without batching all show up here as abnormal log volume per transaction, and all make HADR_SYNC_COMMIT worse without the AG itself being unhealthy.

Reading the Pattern

Best Practices Checklist

• ☐ When two otherwise-unrelated services degrade at the same time, check Azure Service Health / Resource Health before anything engine-internal
• ☐ Query sys.dm_hadr_database_replica_states on the primary first to separate a network problem (queue building, low send rate) from a secondary-compute problem (redo queue building)
• ☐ Always confirm the secondary's own log-disk latency and redo status directly; never infer it from the primary's or the VM host's aggregate metrics
• ☐ Check for readable-secondary query load or concurrent maintenance (seeding, log backups) contending with redo on the secondary
• ☐ Measure inter-replica network latency directly instead of assuming "Azure VMs look fine" covers it
• ☐ Rule out oversized or unbatched application transactions inflating the log volume per commit
• ☐ Use signal_wait_time_ms to confirm the primary is genuinely waiting on an external ack, not CPU-starved, before ruling the primary out
• ☐ Document which replica lagged and by how much (queue sizes, timestamps) before the incident closes; "it recovered on its own" without numbers makes the next occurrence just as opaque

The Bottom Line

HADR_SYNC_COMMIT tells you the primary is waiting, not why. Because synchronous commit only gates writes, the SELECT/INSERT asymmetry in the symptom is expected, not a clue by itself. The real diagnostic path runs through the secondary: is the network delivering log fast enough, is the secondary's storage hardening it fast enough, and is anything blocking redo once it arrives. The detail that reorders the whole triage here is the unrelated Azure SQL Database workload lagging at the same moment: that correlation, checked first via Service Health, either closes the investigation in minutes with a platform incident, or rules out the platform and sends you straight to the secondary's log-disk and network metrics with confidence instead of guesswork.