SQL Server – Capturing Blocking Information (Enhanced SQL Script)

When the database that I’m handling was experiencing excessive blockings, below is the script I use too see what are being blocked and which stored procedure is the blocker. Blocking​​ is when one connection needs access to a piece of data and has to wait for another connection’s lock to clear.

  SELECT  blocking.session_id AS blocking_session_id,
         blocked.session_id AS blocked_session_id ,
         waitstats.wait_type AS blocking_resource ,
         waitstats.wait_duration_ms ,
         waitstats.resource_description ,
         DB_NAME(tl.resource_database_id) AS DatabaseName,
         blocked_cache.text AS blocked_text ,
         blocking_cache.text AS blocking_text
       FROM 
         sys.dm_exec_connections AS blocking
         INNER JOIN sys.dm_exec_requests blocked
         ON blocking.session_id = blocked.blocking_session_id
         CROSS APPLY sys.dm_exec_sql_text(blocked.sql_handle)
         blocked_cache
         CROSS APPLY sys.dm_exec_sql_text(blocking.most_recent_sql_handle)
         blocking_cache
         INNER JOIN sys.dm_os_waiting_tasks waitstats
         ON waitstats.session_id = blocked.session_id
         INNER JOIN sys.dm_tran_locks tl
         ON tl.lock_owner_address = waitstats.resource_address
WHERE waitstats.wait_duration_ms >= 2000

The above query is okay but we were able to enhanced the script by adding columns to see what objects that are being blocked. Below is the enhanced version of the script.

 ;WITH T1 AS (
         SELECT blocking.session_id AS blocking_session_id ,
                blocked.session_id AS blocked_session_id ,
                waitstats.wait_type AS blocking_resource ,
                waitstats.wait_duration_ms ,
                waitstats.resource_description ,
                DB_NAME(tl.resource_database_id) AS DatabaseName,
                blocked_cache.text AS blocked_text ,
                blocking_cache.text AS blocking_text
        FROM 
                sys.dm_exec_connections AS blocking
                INNER JOIN sys.dm_exec_requests blocked
                ON blocking.session_id = blocked.blocking_session_id
                CROSS APPLY sys.dm_exec_sql_text(blocked.sql_handle)
                blocked_cache
                CROSS APPLY sys.dm_exec_sql_text(blocking.most_recent_sql_handle)
                blocking_cache
	            INNER JOIN sys.dm_os_waiting_tasks waitstats
                ON waitstats.session_id = blocked.session_id
                INNER JOIN sys.dm_tran_locks tl
                ON tl.lock_owner_address = waitstats.resource_address
		WHERE waitstats.wait_duration_ms >= 2000
), T2A AS (     
       SELECT blocking_session_id, resource_description,
       CHARINDEX('hobtid', resource_description) AS StartPos,
             SUBSTRING(resource_description, CHARINDEX('hobtid', resource_description), LEN(resource_description)) AS StartText
      FROM T1
), T2B AS (
      SELECT blocking_session_id, resource_description, StartPos, StartText, SUBSTRING(StartText, 0, CHARINDEX(' ', StartText)) AS hotbid_text
      FROM T2A
), T2C AS (
       SELECT blocking_session_id, resource_description, CAST(SUBSTRING(hotbid_text, CHARINDEX('=', hotbid_text)+1, LEN(hotbid_text)) AS BIGINT) AS hobt_id
       FROM T2B
), T2D AS (
       SELECT T2C.blocking_session_id, T2C.resource_description, OBJECT_SCHEMA_NAME(object_id) AS schema_name, object_name(object_id) as object_name, object_id, partition_id, index_id, partition_number, p.hobt_id, rows
       FROM sys.partitions p INNER JOIN T2C ON p.hobt_id = T2C.hobt_id
)
SELECT GETDATE() AS DateTimeCaptured, T1.blocking_session_id, T1.blocked_session_id, T1.blocking_resource, T1.wait_duration_ms, T1.resource_description, T1.DatabaseName, T1.blocking_text, T1.blocked_text, 
T2D.schema_name + '.' + T2D.object_name AS blocked_object_name, T2D.object_id AS blocked_object_id, T2D.index_id as blocked_index_id, i.name AS blocked_index_name, i.type_desc AS blocked_index_type
FROM T1 INNER JOIN T2D ON T1.blocking_session_id = T2D.blocking_session_id AND T1.resource_description = T2D.resource_description
INNER JOIN sys.indexes i ON i.object_id = T2D.object_id AND i.index_id = T2D.index_id

 

Cheers!

 

 

Knowledge worth sharing...Share on linkedin
Linkedin
Share on facebook
Facebook
Share on google
Google
Share on twitter
Twitter

SQL Server – AlwaysOn Availability Group Errors

error

severity

description

 

957

17

Database '%.*ls' is enabled for database mirroring or has joined an availability group. The name of the database cannot be changed.

 

976

14

The​​ target database, '%.*ls', is participating in an availability group and is currently not accessible for queries. Either data movement is suspended or the availability replica is not enabled for read access. To allow read-only access to this and other​​ 

  • If dm_hadr_availability_replica_states.role_desc=3(INVALID), then check service broker. Because Availability group is using Service Broker to communicate between SQL Server then need to check Service Broker first. Open SSMS>“Server Objects” > “Endpoints”, there are two “Service Broker” folder>Expand both, and if the state is Stopped or Disabled, then​​ can change the state to Started.

978

14

The target database ('%.*ls') is in an availability group and is currently accessible for connections when the application​​ intent is set to read only. For more information about application intent, see SQL Server Books Online.

 

979

14

The target database ('%.*ls') is in an availability group and currently does not allow read only connections. For more information about application intent, see SQL Server Books Online.

 

982

14

Unable to access the '%.*ls' database because no online secondary replicas are enabled for read-only access. Check the availability group configuration to verify that at least one secondary replica is​​ configured for read-only access. Wait for an enabled r

 

1408

16

The remote copy of database "%.*ls" is not recovered far enough to enable database mirroring or to join it to the availability group. You need to apply missing log records to the remote database by restoring the​​ current log backups from the principal/prim

 

1409

16

Database "%.*ls" requires database logs to be restored either on the future mirror database before you can enable database mirroring or on a secondary availability database before you can join it to the availability group. Restore current log backups from

 

1465

16

Database "%.*ls" database is not in full recovery mode on each of the server instances. The full recovery model is required for a database to participate in database​​ mirroring or in an availability group.

 

1466

16

Database "%.*ls" is read-only on one of the server instances which is incompatible with participating in database mirroring or in an availability group. Set the database to read-write mode, and retry the​​ operation.

 

1467

16

Database "%.*ls" database is in emergency or suspect mode on one of the partners which is incompatible with participating in database mirroring or in an availability group.

 

1468

16

The operation cannot be performed on database​​ "%.*ls" because it is involved in a database mirroring session or an availability group. Some operations are not allowed on a database that is participating in a database mirroring session or in an availabilit

 

1469

16

Database "%.*ls" is an auto-close database on one of the partnerswhich is incompatible with participating in database mirroring or in an availability group.

 

1480

10

The %S_MSG database "%.*ls" is changing roles from "%ls" to "%ls" because the mirroring session or availability group failed​​ over due to %S_MSG. This is an informational message only. No user action is required.

 

1488

16

Database "%.*ls" database is in single user mode which is incompatible with participating in database mirroring or in an availability group. Set database to​​ multi-user mode, and retry the operation.

 

1833

16

File '%ls' cannot be reused until after the next BACKUP LOG operation. If the database is participating in an availability group, a dropped file can be​​ reused only after the truncation LSN of the primary​​ availability replica has passed the drop LSN of the

 

3104

16

RESTORE cannot operate on database '%ls' because it is configured for database mirroring or has joined an availability group. If you intend to restore the database, use ALTER DATABASE to remove​​ mirroring or to remove the database from its availability gro

 

3752

16

The database '%.*ls' is currently joined to an availability group. ​​ Before you can drop the database, you need to remove it from the availatility group.

  • Before you can drop the database, you need to remove it from the availability group. If AG exists, try ALTER AVAILABILITY GROUP [AGname] REMOVE DATABASE [DbName]. If AG does not exist, try removing with ALTER DATABASE​​ [DbName] SET HADR OFF.

  • If replica_id and group_database_id in sys.databases show as null, then this may be due to earlier meta-data when database was part of an AG so stop SQL Server, rename the database files and start SQL Server. Then you should be able to remove the database.

5529

16

Failed to remove a FILESTREAM file.​​ The database is a primary database in an availability group. Wait for the FILESTREAM data files to be hardened on every secondary availability replica. Then retry the drop file operation.

 

10786

16

The ALTER AVAILABILITY GROUP command failed because it​​ contained multiple MODIFY​​ REPLICA options: %ls. ​​ Enter a separate ALTER AVAILABILITY GROUP ... MODIFY REPLICA command for each replica option that you want to modify.

 

13251

10

availability group

 

19401

16

The READ_ONLY_ROUTING_URL '%.*ls' specified for​​ availability replica '%.*ls' is not valid. It does not follow the required format of 'TCP://system-address:port'. For information about the correct routing URL format, see the CREATE AVAILABILITY GROUP docum

 

19403

16

The availability replica '%.*ls'​​ specified in the READ_ONLY_ROUTING_LIST for availability replica '%.*ls' does not exist. Only availability replicas that belong to the specified availability group '%.*ls' can be added to this list. To get the names of ava

 

19405

16

Failed to create, join​​ or add replica to availability group '%.*ls', because node '%.*ls' is a possible owner for both replica '%.*ls' and '%.*ls'. If one replica is failover cluster instance, remove the overlapped node from its possible owners and try ag

 

19406

10

The state of the local availability replica in availability group '%.*ls' has changed from '%ls' to '%ls'. The replica state changed because of either a startup, a failover, a communication issue, or a cluster error. For more information, see the availabi

 

19407

16

The lease between availability group '%.*ls' and the Windows Server Failover Cluster has expired. A connectivity issue occurred between the instance of SQL Server and the Windows Server Failover Cluster. To determine whether the availability group is fail

 

19452

16

The availability group listener (network name) with Windows Server Failover Clustering resource ID '%s', DNS name '%s', port %hu failed to start with a permanent error: %u. Verify port numbers, DNS names and other related network configuration,​​ then retry

 

19453

16

The availability group listener (network name) with Windows Server Failover Clustering resource ID '%s', DNS name '%s', port %hu failed to start with this error: %u. Verify network and cluster configuration and logs.

 

19454

16

The​​ availability group listener (network name) with Windows Server Failover Clustering resource ID '%s', DNS name '%s', port %hu failed to stop with this error: %u. Verify network and cluster configuration and logs.

 

19455

16

The WSFC cluster does not have a​​ public cluster network with an IPv4 subnet. This is a requirement to create an availability group DHCP listener. Configure a public network for the cluster with an IPv4 subnet, and try to create the listener.

 

19456

16

None of the IP addresses configured​​ for the availability group listener can be hosted by the server '%.*ls'. Either configure a public cluster network on which one of the specified IP addresses can be hosted, or add another listener IP address which can b

  • Added a new IP address (with different subnet) manually to the existing Listener [to be tested].

19458

16

The WSFC nodes that host the primary and secondary replicas belong to different subnets. DHCP across multiple subnets is not supported for availability replicas. Use the static IP option to configure the availability group listener.

 

19460

16

The availability group listener with DNS name '%.*ls' is configured to use DHCP. For​​ listeners with this configuration, IP addresses cannot be added through SQL Server. To add IP addresses to the​​ listener, drop the DHCP listener and create it again config

 

19468

16

The listener with DNS name '%.*ls' for the availability group '%.*ls' is already listening on the TCP port %u. Please choose a different TCP port for the listener. If there is a problem​​ with the listener, try restarting the listener to correct the problem

 

19469

16

The specified listener with DNS name, '%.*ls', does not exist for the Availability Group '%.*ls'. Use an existing listener, or create a new listener.

 

19477

16

The​​ availability group '%.*ls' already has a listener with DNS name '%.*ls'. ​​ Availability groups can have only one listener. Use the existing listener, or drop the existing listener and create a new one.

 

19486

16

The configuration changes to the​​ availability group listener were completed, but the TCP provider of the instance of SQL Server failed to listen on the specified port [%.*ls:%d]. This TCP port is already in use. Reconfigure the availability group listener

 

21872

16

The availability group​​ associated with Virtual Network Name '%s' has no replicas.​​ 

 

21880

16

The virtual network name '%s' has been used to identify the redirected publisher for original publisher '%s' and database '%s'. The availability group associated with this virtual​​ network name, however, does not include the publisher database. ​​ 

 

21882

16

The database '%s' at the redirected publisher '%s' for original publisher '%s' and database '%s' belongs to a HADRon availability group and must be redirected to its associated​​ HADRon Virtual Network Name.​​ 

 

21883

16

The query at the redirected publisher '%s' to determine whether the publisher database '%s' belonged to an availability group failed with error '%d', error message '%s'.​​ 

 

21884

16

The query at the redirected​​ publisher '%s' to determine the health of the availability group associated with publisher database '%s' failed with error '%d', error message '%s'.​​ 

 

21887

16

The query at the redirected publisher '%s' to determine whether the publisher database​​ '%s' belonged to an availability group failed with error '%d', error message '%s'.​​ 

 

21892

16

Unable to query sys.availability_replicas at the availability group primary associated with virtual network name '%s' for the server names of the member replicas: error​​ = %d, error message = %s.',

 

33445

16

The database '%.*s' is a readable secondary database in an availability group and cannot be enabled for FILESTREAM non-transacted access.

 

33446

16

The FILESTREAM database configuration cannot be changed for database​​ '%.*s'. ​​ The database is either a mirror database in Database Mirroring, or is in a secondary replica of an AlwaysOn availability group. ​​ Connect to the server instance that hosts the pr

 

33449

10

FILESTREAM File I/O access is enabled, but no listener​​ for the availability group is created. A FILESTREAM PathName will be unable to refer to a virtual network name (VNN) and, instead, will need to refer to a physical Windows Server Failover Clustering (

 

33450

10

FILESTREAM File I/O access is enabled. One​​ or more availability groups ('%ls') currently do not have a listener. ​​ A FILESTREAM PathName will be unable to refer to a virtual network name (VNN) and, instead, will need to refer to a physical Windows Server F

 

35202

10

A connection for availability​​ group '%ls' from availability replica '%ls' with id ​​ [%ls] to '%ls' with id [%ls] has been successfully established. ​​ This is an informational message only. No user action is required.

 

35205

16

Could not start the AlwaysOn Availability Groups transport manager. This failure probably occurred because a low memory condition existed when the message dispatcher started up. If so, other internal tasks might also have experienced errors. Check the SQL

 

35206

10

A connection timeout has occurred on a previously​​ established connection to availability replica '%ls' with id [%ls]. ​​ Either a networking or a firewall issue exists or the availability replica has transitioned to the resolving role.

  • If there is a login failed for domain\node$ account around same time, then it may be because NT AUTHORITY\SYSTEM account lacked the​​ required permissions to failover the availability group. To failover the group to the other node, NT AUTHORITY\SYSTEM must have permission to connect to SQL (CONNECT SQL), failover the availability group (ALTER ANY AVAILABILITY GROUP) and execute sp_server_diagnostics (VIEW SERVER STATE). The NT AUTHORITY\SYSTEM account is used to automatically execute sp_server_diagnostics. Another option is to​​ add the account to sysadmin role in SQL Server.

CREATE LOGIN [NT AUTHORITY\SYSTEM] FROM WINDOWS WITH DEFAULT_DATABASE=[master], DEFAULT_LANGUAGE=[us_english];

ALTER SERVER ROLE [sysadmin] ADD MEMBER [NT AUTHORITY\system];

35207

16

Connection attempt on availability group id '%ls' from replica id '%ls' to​​ replica id '%ls' failed because of error %d, severity %d, state %d.

 

35210

16

Failed to modify options for availability replica '%.*ls' in availability group '%.*ls'. The specified availability group does not contain an availability replica with specified​​ name. Verify that​​ availability group name and availability replica name are c

 

35212

16

The %ls operation is not allowed by the current availability-group configuration. ​​ This operation would exceed the maximum number of %d synchronous-commit​​ availability replicas in availability group '%.*ls'. ​​ Change one of the existing synchronous-commit​​ 

 

35213

16

The %ls operation is not allowed by the current availability-group configuration. ​​ This operation would exceed the maximum number of %d automatic failover targets in availability group '%.*ls'. ​​ Change one of the existing synchronous-commit replicas to th

 

35217

16

​​ The thread pool for AlwaysOn Availability Groups was unable to start a new worker thread because there are not enough available​​ worker threads. ​​ This may degrade AlwaysOn Availability Groups performance. ​​ Use the "max worker threads" configuration option

 

35220

16

Could not process the operation. AlwaysOn Availability Groups replica manager is waiting for the host computer to​​ start a Windows​​ Server Failover Clustering (WSFC) cluster and join it. Either the local computer is not a cluster node, or the local cluster​​ 

 

35221

16

Could not process the operation. AlwaysOn Availability Groups replica manager is disabled on this​​ instance of SQL Server. Enable AlwaysOn Availability Groups, by using the SQL Server Configuration Manager. Then, restart the SQL Server service, and retry t

 

35222

16

Could not process the operation. AlwaysOn Availability Groups does not have permissions​​ to access the Windows Server Failover Clustering (WSFC) cluster. ​​ Disable and re-enable AlwaysOn Availability Groups by using the SQL Server Configuration Manager. The

 

35223

16

Cannot add %d availability replica(s) to availability group '%.*ls'. ​​ The​​ availability group already contains %d replica(s), and the maximum number of replicas supported in an availability group is %d.

 

35224

16

Could not process the operation. ​​ AlwaysOn Availability Groups failed to load the required Windows Server Failover​​ Clustering (WSFC) library. Verify​​ that the computer is a node in a WSFC cluster. ​​ You will need to restart the SQL Server instance to reload

 

35225

16

Could not process the operation. The ​​ instance of SQL Server is running under WOW64 (Windows 32-bit on Windows 64-bit), which does not support AlwaysOn Availability Groups. ​​ Reinstall SQL Server in the native 64-bit edition, and re-enable AlwaysOn Availab

 

35226

16

Could not process the operation. ​​ AlwaysOn Availability Groups has not started because the​​ instance of SQL Server is not running as a service. Restart the server instance as a service, and retry the operation.

 

35228

16

The attempt to set the failure condition level for availability group '%.*ls' failed. The specified level value is out of the​​ valid range [%u, %u]. Reenter the command specifying a valid failure condition level value.

 

35229

16

The attempt to set the health check timeout value for availability group '%.*ls' failed. The specified timeout value is less than %u milliseconds. Reenter the command specifying a valid health check timeout value.

 

35233

16

Cannot create an availability group containing %d availability replica(s).

 

35237

16

None of the specified replicas for availability group %.*ls maps to the instance of SQL Server to​​ which you are connected. Reenter the command, specifying this server instance to host one of the replicas. This replica will be the initial primary replica.​​ 

 

35238

16

Database '%.*ls' cannot be added to availability group '%.*ls'. ​​ The database does not​​ exist on this SQL Server instance. ​​ Verify that the database name is correct, then retry the operation.

 

35239

16

The ALTER DATABASE <database-name> SET HADR SUSPEND (or SET HADR RESUME) statement failed on database '%.*ls' of availability group '%.*ls''.​​ Either the availability group does not contain the specified database, or the database has not joined the availab

 

35240

16

Database '%.*ls' cannot be joined to or unjoined from availability group '%.*ls'. This operation is not supported on the primary​​ replica of the availability group.​​ 

  • TSQL to unjoin is “ALTER DATABASE db SET HADR OFF”. Note that failing over thus making it a​​ secondary, and then retrying ALTER DATABASE HADR is not an option, because the alter database will then give error 921 that database has not been recovered yet.

  • If this replica can be made a secondary, then fail over thus making it a secondary, and then drop replica when in this secondary role. TSQL is “ALTER AVAILABILITY GROUP AgName REMOVE REPLICA ON ‘ReplicaName’;”.

  • If no secondary​​ replica for failover and getting this error on primary, then “DROP AVAILABILITY GROUP AgName”. This will automatically drop listener. If do not want listener dropped, then create a temporary AG adding same name, then try the “DROP AVAILABILITY GROUP”.

35242

16

Cannot complete this ALTER DATABASE <database-name> SET HADR operation on database '%.*ls'. The database is not joined to an availability group. After the database has joined the availability group, retry the command.

 

35243

16

Failed to set​​ resource property '%.*ls' for availability group '%.*ls'. ​​ The operation encountered SQL Server error %d. ​​ When the cause of the error has been resolved, retry the ALTER AVAILABILITY GROUP command later.

 

35244

16

Database '%.*ls' cannot be added to availability group '%.*ls'. ​​ The database is currently joined to another availability group. ​​ Verify that the database name is correct and that the database is not joined to an availability group, then retry the operati

 

35246

16

Failed to create the​​ availability group. A SQL Server instance name could not be validated because the dynamic link library (DLL) file '%ls' could not be located (Windows System Error %d). Verify that the specified server instance exists. If it exists, th

 

35247

16

Failed to​​ create the availability group. A SQL Server instance name could not be validated because the dynamic link library (DLL) file '%ls' could not be loaded (Windows System Error %d).

 

35249

16

An attempt to add or join a system database, '%.*ls', to an​​ availability group failed. Specify only user databases for this operation.

 

35254

16

An error occurred while accessing the availability group metadata. Remove this database or replica from the availability group, and reconfigure the availability group to​​ add the database or replica again. For more information, see the ALTER AVAILABILITY G

 

35257

16

​​ AlwaysOn Availability Groups Send Error (Error code 0x%X, "NOT OK") was returned when sending a message for database ID %d. If the partner is running and​​ visible over the network, retry the command using correctly configured partner-connection parameters

 

35259

16

​​ Database '%.*ls' is already participating in a different availability group. ​​ 

 

35260

16

​​ During an attempted database recovery, an​​ availability database manager was not found for database id %d with availability group ID %d and group database ID %ls. ​​ Recovery was terminated. The most likely cause of this error is that the availability group

 

35261

16

​​ Attempt to perform an AlwaysOn​​ Availability Groups operation on a system database, '%ls', failed. System databases are not supported by AlwaysOn Availability Groups.

 

35262

17

Skipping the default startup of database '%.*ls' because the database belongs to an availability group (Group​​ ID: ​​ %d). The database will be started by the availability group. This is an informational message only. No user action is required.

  • Check SQL and Windows logs for additional messages if expecting database to start.

35264

10

AlwaysOn Availability Groups​​ data movement for database '%.*ls' has been suspended for the following reason: "%S_MSG" (Source ID %d; Source string: '%.*ls'). To resume data movement on the database, you will need to resume the database manually. For infor

35265

10

AlwaysOn Availability Groups data movement for database '%.*ls'​​ has been resumed. This is an informational message only. No user action is required.

 

35266

10

AlwaysOn Availability Groups connection with %S_MSG database established for %S_MSG database '%.*ls' on the availability replica with Replica ID:​​ {%.8x-%.4x-%.4x-%.2x%.2x-%.2x%.2x%.2x%.2x%.2x%.2x}. This is an informational message only. No user action is​​ 

 

35267

10

AlwaysOn Availability Groups connection with %S_MSG database terminated for %S_MSG database '%.*ls' on the availability replica with​​ Replica ID: {%.8x-%.4x-%.4x-%.2x%.2x-%.2x%.2x%.2x%.2x%.2x%.2x}. This is an informational message only. No user action is r

 

35275

16

A previous RESTORE WITH CONTINUE_AFTER_ERROR operation or being removed while in the SUSPECT state from an availability group left the '%.*ls' database in a potentially damaged state. The database cannot be joined while in this state. Restore the database

 

35276

17

Failed to allocate and schedule an AlwaysOn Availability Groups task for database '%ls'. Manual intervention​​ may be required to resume synchronization of the database. If the problem persists, you might need to restart the local instance of SQL Server.

 

35279

16

The attempt to join database '%.*ls' to the availability group was rejected by the primary database with error '%d'. For more information, see the SQL Server error log for the primary replica.

 

35280

16

Database '%.*ls' cannot be added to availability group '%.*ls'. ​​ The database is already joined to the specified availability group. ​​ Verify that the​​ database name is correct and that the database is not joined to an availability group, then retry the ope

 

35281

16

Database '%.*ls' cannot be removed from availability group '%.*ls'. ​​ The database is not joined to the specified availability group. ​​ Verify that the database name and the availability group name are correct, then retry the operation.

 

35282

16

Availability replica '%.*ls' cannot be added to availability group '%.*ls'. ​​ The availability group​​ already contains an availability replica with the​​ specified name. ​​ Verify that the availability replica name and the availability group name are correct, t

 

35283

16

Availability replica '%.*ls' cannot be removed from availability group '%.*ls'. ​​ The availability group does not contain an availability replica with the specified name. ​​ Verify that the availability replica name is correct, then retry the operation.

 

35284

16

Availability replica '%.*ls' cannot be removed from availability group '%.*ls', because this replica is on the local instance of SQL​​ Server. ​​ If the local availability replica is a secondary replica, connect to the server instance that is currently hostin

 

35287

16

AlwaysOn Availability Groups transport for availability database "%.*ls" failed to decompress the log block whose LSN is​​ %S_LSN. ​​ This error can be caused by a corrupt network packet or a compression version mismatch. ​​ The database replica has been put in

 

35288

16

AlwaysOn Availability Groups log apply for availability database "%.*ls" has received an out-of-order log​​ block. ​​ The expected​​ LSN was %S_LSN. ​​ The received LSN was %S_LSN. ​​ The database replica has been put into the SUSPENDED state. ​​ Resume the availabil

 

35299

10

Nonqualified transactions are being rolled back in database %.*ls for an AlwaysOn Availability​​ Groups state change. Estimated rollback completion: %d%%. This is an informational message only. No user action is required.

 

41039

16

An availability group replica already exists on the Windows Server Failover Clustering (WSFC) node '%.*ls'. ​​ Each WSFC​​ node can contain only one replica of an availability group. ​​ Please choose another WSFC node to host the new replica.

 

41040

16

Failed to remove the availability group replica '%.*ls' from availability group '%.*ls'. ​​ The availability group does not contain a replica with the specified name. ​​ Verify the availability group and replica names and then retry the operation.

 

41042

16

The availability group '%.*ls' already exists. ​​ This error​​ could be caused by a previous failed CREATE AVAILABILITY GROUP or​​ DROP AVAILABILITY GROUP​​ operation. ​​ If the availability group name you specified is correct, try dropping the availability group a

  • If AG still exists, DROP AVAILABILITY GROUP.

  • Open regedit, backup the entry from the old Availability Group that​​ starts with​​ HADR_Ag_???? key, then delete this key [to be tested].

  • Try with new name for Availability Group.

41043

16

For availability group '%.*ls', the value of the name-to-ID map entry is invalid. ​​ The binary value should contain a Windows Server Failover Clustering (WSFC) resource ID, a WSFC group ID, and their corresponding lengths in characters. ​​ The availability g

 

41044

16

Availability group name to ID map entry for availability group '%.*ls' cannot be found in the Windows Server Failover Clustering (WSFC)​​ store. ​​ The availability group name may be incorrect, or the availability group may not exist in this Windows Server Fa

 

41045

16

Cannot add database '%.*ls' to the availability group '%.*ls', because there is already a database with the same name in the​​ availability group. ​​ Please verify that the database and availability group names specified are correct.

 

41046

16

Cannot add replica '%.*ls' to the availability group '%.*ls', because there is already a replica with the same name in the availability​​ group. ​​ Please verify the replica​​ and availability group names specified are correct.

 

41048

10

AlwaysOn Availability Groups: Local Windows Server Failover Clustering service has become unavailable. This is an informational message only. No user action is​​ required.

 

41049

10

AlwaysOn Availability Groups: Local Windows Server Failover Clustering node is no longer online. This is an informational message only. No user action is required.

 

41050

10

AlwaysOn Availability Groups: Waiting for local Windows​​ Server Failover Clustering service to start. This is an informational message only. No user action is required.

 

41051

10

AlwaysOn Availability Groups: Local Windows Server Failover Clustering service started. This is an informational message only. No​​ user action is required.

 

41052

10

AlwaysOn Availability Groups: Waiting for local Windows Server Failover Clustering node to start. This is an informational message only. No user action is required.

 

41053

10

AlwaysOn Availability Groups: Local Windows​​ Server Failover Clustering node started. This is an​​ informational message only. No user action is required.

 

41054

10

AlwaysOn Availability Groups: Waiting for local Windows Server Failover Clustering node to come online. This is an informational message​​ only. No user action is required.

 

41055

10

AlwaysOn Availability Groups: Local Windows Server Failover Clustering node is online. This is an informational message only. No user action is required.

 

41056

16

Availability replica '%.*ls' of availability​​ group '%.*ls' cannot be brought online on this SQL Server instance. ​​ Another replica of the same availability group is already online on the local Windows Server Failover Clustering (WSFC) node. ​​ Each WSFC node

 

41058

10

AlwaysOn: The local replica of​​ availability group '%.*ls' is starting. This is an informational message only. No user action is required.

 

41059

10

AlwaysOn: Availability group '%.*ls' was removed while the availability replica on this instance of SQL Server was offline. ​​ The local​​ replica will be removed now. This is an informational message only. No user action is required.

 

41060

16

The Cyclic Redundancy Check (CRC) value generated for the retrieved availability group configuration data from the Windows Server Failover Clustering​​ (WSFC) store does not match that stored with the data for the availability group with ID '%.*ls'. ​​ The av

 

41061

10

AlwaysOn: The local replica of availability group '%.*ls' is stopping. This is an informational message only. No user action is required.

 

41062

16

The ID of availability group '%.*ls' in local data store is inconsistent with that in the Windows Server Failover Clustering (WSFC) data store. ​​ The availability group may have been dropped and recreated while the SQL Server instance was​​ offline, or while

 

41063

16

Windows Server Failover Clustering (WSFC) detected that the availability group resource with ID '%.*ls' was online when the availability group was not actually online. ​​ The attempt to synchronize the WSFC resource state with​​ the availability group state f

 

41066

16

Cannot bring the Windows Server Failover Clustering (WSFC) resource (ID '%.*ls') online (Error​​ code %d). ​​ The WSFC service may not be running or may not be accessible in its current state, or the WSFC resource may​​ not be in a state that could accept the r

  • In DOS prompt, run “Net helpmsg” on the %d value, and proceed accordingly.

 

41069

16

The existence of availability group data for the availability group '%.*ls' in the Windows Server Failover Clustering (WSFC)​​ store could not be determined. ​​ The local WSFC node may be down, or a previous CREATE AVAILABILITY GROUP or DROP AVAILABILITY GROU

 

41070

16

Configuration data for the availability group with Windows Server Failover Clustering (WSFC) resource ID '%.*ls'​​ is not found in the WSFC data store. ​​ The availability group may have been dropped, or a previous CREATE AVAILABILITY GROUP or DROP AVAILABILI

 

41071

16

Cannot read the persisted configuration of AlwaysOn availability group with corresponding Windows Server Failover Clustering (WSFC) resource ID '%.*ls'. ​​ The persisted configuration is written by a higher-version SQL Server that hosts the primary availabi

 

41072

16

The ID of availability group '%.*ls' in local data store does not exist in the Windows​​ Server Failover Clustering (WSFC) data store. ​​ The availability group may have been dropped but the current WSFC node was not notified. ​​ To resolve this error, try to r

 

41073

16

The database '%.*ls' cannot be removed from availability group '%.*ls'. ​​ This database does not belong to the availability group.

 

41074

10

AlwaysOn: The local replica of availability group '%.*ls' is preparing to transition to the primary role​​ in response to a request from the Windows Server Failover Clustering (WSFC) cluster.​​ This is an informational message only. No user action is required

  • This is an informational message only. No user action is required

41075

10

AlwaysOn: The local replica of availability group '%.*ls' is preparing to transition to the resolving role in​​ response to a request from the Windows Server Failover Clustering (WSFC) cluster. This is an informational message only. No user action is requir

 

41076

10

AlwaysOn: Availability group '%.*ls' is going offline because it is being removed. This is an informational message only. No user action is required.

 

41080

16

Failed to delete SQL Server instance name to Windows Server Failover Clustering node name map entry for the local availability replica of availability group '%.*ls'. ​​ The operation encountered​​ SQL Server error %d and has been terminated. ​​ Refer to the SQL

 

41081

16

Failed to destroy the Windows Server Failover Clustering group corresponding to availability group '%.*ls'. ​​ The operation encountered SQL Server error %d and has been terminated. ​​ Refer to the SQL Server error log for details about this SQL Server error​​ 

 

41089

10

AlwaysOn Availability Groups startup has been cancelled, because SQL Server is shutting down. This is an informational message only. No user action is required.

 

41091

10

AlwaysOn: The local replica of availability group '%.*ls' is going offline because either the lease expired or lease renewal failed. This is an informational message only. No user action is required.

 

41092

10

AlwaysOn: The availability replica manager​​ is going offline because %ls. This is an informational message only. No user action is required.

 

41093

10

AlwaysOn: The local replica of availability group '%.*ls' is going offline because the corresponding resource in the Windows Server Failover​​ Clustering (WSFC) cluster is no longer online. This is an informational message only. No user action is required.

 

41094

10

AlwaysOn: The local replica of availability group '%.*ls' is restarting because the existing primary replica restarted or the​​ availability group failed over to a new primary replica. This is an informational message only. No user action is required.

 

41095

10

AlwaysOn: Explicitly transitioning the state of the Windows Server Failover Clustering (WSFC) resource that corresponds to availability group '%.*ls' to Failed. The resource state is not consistent with the availability group state in the instance of SQL​​ 

 

41096

10

AlwaysOn: The local replica of availability group '%.*ls' is being removed. The instance of SQL Server failed​​ to validate the integrity of the availability group configuration in the Windows Server Failover Clustering (WSFC) store. ​​ This is expected if th

 

41097

10

AlwaysOn: The local replica of availability group '%.*ls' is going offline. This replica failed to​​ read the persisted configuration because of a version mismatch. This is an informational message only. No user action is required.

 

41098

10

AlwaysOn: The local replica of availability group '%.*ls' is restarting, because it failed to read the persisted configuration. This is an informational message only. No user action is required.

 

41099

10

AlwaysOn: The local replica of availability group '%.*ls' is going offline. This replica failed to read the persisted configuration, and it has exhausted the​​ maximum for restart attempts. This is an informational message only. No user action is required.

 

41100

16

The availability group '%.*ls' and/or its local availability replica does not exist. ​​ Verify that the specified availability group name is correct,​​ and that the local availability replica has joined the availability group, then retry the operation.

 

41101

16

The availability group with Windows Server Failover Clustering resource ID '%.*ls' and/or its local availability replica does not exist. ​​ Verify​​ that the specified availability resource ID is correct, and that the local availability replica has joined the

 

41102

10

Failed to persist configuration data of availability group '%.*ls' in the Windows Server Failover Clustering (WSFC) cluster. ​​ The local availability replica either is not the primary replica or is shutting down.

 

41103

10

Startup of the AlwaysOn Availability Replica Manager has been terminated, because the 'FixQuorum' property of Windows Server Failover Clustering (WSFC) is not​​ present. The prerequisite QFE hotfix, KB 2494036, might not yet be installed on your Windows Ser

 

41104

16

Failover of the availability group '%.*ls' to the local replica failed because the availability group resource did not come online due to a previous​​ error. To identify that error, check the SQL Server error log and the Windows Server Failover Cluster logs

 

41106

16

Cannot create an availability replica for availability group '%.*ls'. ​​ An availability replica of the specified availability group​​ already exists on this instance of SQL Server. ​​ Verify that the specified availability group name is correct and unique, the

 

41107

16

Availability group '%.*ls' failed to create necessary events for the WSFC Lease mechanism. Windows returned error code (%d) when obtaining handles for Lease events. Resolve the windows error and retry the availability group operation.

 

41108

16

An error occurred while removing availability group '%.*ls'. The DROP AVAILABILITY GROUP command removed the availability group​​ configuration from the local metadata. However, the attempt to remove this configuration from the Windows Server Failover Clust

 

41109

17

Could not enqueue a task (SQL OS error: 0x%x) for process actions for the availability group '%.*ls'. ​​ Most likely, the instance of SQL Server is low on resources. Check the SQL Server error log to determine the cause of the failure. Retry the operation l

 

41110

10

AlwaysOn: The availability replica manager is starting. This is an informational message only. No user​​ action is required.

 

41111

10

AlwaysOn: The availability replica manager is waiting for the instance of SQL Server to allow client connections. This is an informational message only. No user action is required.

 

41112

16

A Windows Server Failover​​ Clustering (WSFC) API required by availability groups has not been loaded. AlwaysOn Availability Groups replica manager is not enabled on the local instance SQL Server. If the server instance is running an edition of SQL Server t

 

41113

16

Cannot failover​​ availability group '%.*ls' to this instance of SQL Server because a failover command is already pending on the local replica of this availability group. ​​ Wait for the pending failover command to complete before issuing another command on t

 

41114

16

Cannot create an availability group named '%.*ls' because it already exists in a system table.

 

41115

16

Cannot create the availability group named '%.*ls' because its availability group ID (ID: '%.*ls') already exists in a system table.

 

41116

16

Cannot​​ create an availability group named '%.*ls' with replica ID '%.*ls' because this ID already exists in a system table.

 

41117

16

Cannot map local database ID %d to the availability database ID '%.*ls' within availability group '%.*ls'. This database is​​ already mapped to an availability group.

 

41118

16

Cannot map database ID %d to the availability database ID '%.*ls' within availability group '%.*ls'. Another local database, (ID %d). is already mapped to this availability database.

 

41119

16

Could not​​ find the availability group ID %d in the system table.

 

41121

10

The local availability replica of availability group '%.*ls' cannot accept signal '%s' in its current replica role, '%s', and state (configuration is %s in Windows Server Failover Clustering​​ store, local availability replica has %s joined). ​​ The availabil

 

41122

16

Cannot failover availability group '%.*ls' to this instance of SQL Server. ​​ The local availability replica is already the primary replica of the availability group. ​​ To failover​​ this availability group to another instance of SQL Server, run the failover c

 

41126

16

Operation on the local availability replica of availability group '%.*ls' failed. ​​ The local copy of the availability group configuration does not exist or has not been initialized. ​​ Verify that the availability group exists and that the local copy of the

 

41127

16

Attempt to set database mapping state where the local database id %d is not mapped to any availability group.

 

41128

16

Failed to perform database​​ operation '%s' on database '%.*ls' (ID %d) in availability group '%.*ls'. ​​ The database might be in an incorrect state for the operation. ​​ If the problem persists, you may need to restart the SQL Server instance.

 

41129

16

Failed to schedule or execute​​ database operation '%s' on database '%.*ls' (Database ID: %d) in availability group '%.*ls' (SQL OS error: %d). ​​ The instance of SQL Server may have insufficient resources to carry out the database operation. If the problem p

 

41130

16

Operation '%s' on a​​ database '%.*ls' (Database ID: %d) in availability group '%.*ls' failed with SQL Server error %d (Error details: "%.*ls"). ​​ The operation has been rolled back. ​​ See previous error messages in the SQL Server error log for more details.​​ 

 

41131

10

Failed​​ to bring availability group '%.*ls' online. ​​ The operation timed out. Verify that the local Windows Server Failover Clustering (WSFC) node is online. Then verify that the availability group resource exists in the WSFC cluster. If the problem persis

 

41132

16

Cannot join database '%.*ls' to availability group '%.*ls'. ​​ The specified database does not belong to the availability group. ​​ Verify the names of the database and the availability group, and retry the command specifying the correct names.

 

41133

10

Cannot remove database '%.*ls' from availability group '%.*ls'. ​​ Either the database does not belong to the availability group, or the database has not joined the group. Verify the database and availability group names, and retry the command.

 

41134

16

Cannot bring the availability group '%.*ls' online. ​​ The local instance was not the previous primary replica when the availability group went offline, not all databases are synchronized, and no force failover command was issued on the local availability r

 

41135

10

Startup of AlwaysOn Availability Groups replica manager failed due to SQL Server error %d. ​​ To determine the cause of this error, check the SQL Server error log for the preceding error.

 

41136

16

Failed to join the availability replica to​​ availability group '%.*ls' because the group is not online. ​​ Either bring the availability group online, or drop and recreate it. Then retry the join operation.

 

41137

10

Abandoning a database operation '%ls' on availability database '%.*ls' of availability group '%.*ls'. ​​ The sequence number of local availability replica has changed (Previous sequence number: %u, current sequence number: %u). ​​ This is an informational mes

 

41138

17

Cannot accept AlwaysOn Availability Groups operation operation on​​ database '%.*ls' of availability group '%.*ls'. ​​ The database is currently processing another operation that might change the database state. Retry the operation later. If the condition per

 

41139

10

Failed to set database information for availability group %.*ls. ​​ The local availability replica is not the primary replica, or it is shutting down. ​​ This is an informational message only. No user action is required.

 

41140

16

Availability group '%.*ls' cannot process the ALTER AVAILABILITY GROUP command, because the local availability replica is not the primary replica. ​​ Connect to the server instance that is currently hosting the primary replica of this availability group, an

 

41141

16

Failed to set availability group database information for availability​​ group %.*ls. ​​ The local availability replica is not the primary, or is shutting down. ​​ This is an informational message only. No user action is required.

 

41142

16

The availability replica for availability group '%.*ls' on this instance of SQL Server cannot become the primary replica. One or more databases are not synchronized or have not joined the availability group, or the WSFC cluster was started in Force Quorum

  • Check if *_FailoverCluster_health_XeLogs confirm that the cluster was started in forced_quorum mode (forced_quorum True in availability_replica_automatic_failover_validation event).​​ 

    • When a cluster is started in forced_quorum mode (net.exe start clussvc /forcequorum, OR Failover Cluster Manager>left pane>right-click Windows cluster>choose Force​​ Cluster Start>Confirm by clicking Yes.) on primary replica, the availability group is required to be started by issuing command ALTER AVAILABILITY GROUP <AGNAME> FORCE_FAILOVER_ALLOW_DATA_LOSS;. Because this is the original primary, despite issuing force allow data loss, NO data loss occurs

    • When a cluster is started in forced_quorum mode on secondary replica, first check sys.dm_hadr_database_replica_cluster_states.is_failover_ready on secondary. If this value is 1, and be assured no data loss will occur. If​​ this value is not 1 (for example because it’s asynchronous commit), then can check sys.dm_hadr_database_replica_states.last_commit_time, so as to know estimate start time of data loss. The sys.dm_hadr_database_replica_states.last_hardened_lsn value may be​​ used to consider other secondaries for failover. Then can initiate failover with ALTER AVAILABILITY GROUP <AGNAME> FORCE_FAILOVER_ALLOW_DATA_LOSS; on secondary.

41143

16

Cannot process the operation. ​​ The local replica of availability Group '%.*ls' is in​​ a failed state. ​​ A previous operation to read or update persisted configuration data for the availability group has failed. ​​ To recover from this failure, either restart​​ 

 

41144

16

The local availability replica of availability group '%.*ls' is in a​​ failed state. ​​ The replica failed to read or update the persisted configuration data (SQL Server error: %d). ​​ To recover from this failure, either restart the local Windows Server Failov

 

41145

10

Cannot join database '%.*ls' to availability group '%.*ls'. ​​ The database has already joined the availability group. ​​ This is an informational message. ​​ No user action is required.

 

41146

16

Failed to bring Availability Group '%.*ls' online. ​​ The Windows Server Failover Clustering (WSFC) service may not be​​ running, or it may not be accessible in its current state. ​​ Please verify the local WSFC node is up and then retry the operation.

 

41147

10

AlwaysOn Availability Groups was not started because %ls. ​​ This is an informational message. ​​ No user action is required.

 

41148

16

Cannot add or join database '%.*ls' to availability group '%.*ls'. ​​ The database does not exist on this instance of SQL Server. ​​ Verify the database name and that the database exists on the server instance. ​​ Then retry the operation,​​ specifying the correc

 

41149

16

Operation on the availability group '%.*ls' has been cancelled or terminated, either because of a connection timeout or cancellation by user. ​​ This is an informational message. ​​ No user action is required.

 

41150

16

Failed​​ to take availability group '%.*ls' offline. ​​ The Windows Server Failover Clustering (WSFC) service may not be running, or it may not be accessible in its current state. ​​ Verify the local WSFC node is up and then retry the operation.

 

41151

16

Error​​ accessing the Availability Groups manager. ​​ The local Availability Groups manager has not been initialized. ​​ Wait until the Availability Groups manager is in a state that allows access, and retry the operation.

 

41152

16

Failed to create availability group '%.*ls'. ​​ The operation encountered SQL Server error %d and has been rolled back. ​​ Check the SQL Server error log for more details. ​​ When the cause of the error has been resolved, retry CREATE AVAILABILITY GROUP command

  • Check this document for error number mentioned in %d.

41153

16

Failed to create availability group '%.*ls'. ​​ The operation encountered SQL Server error %d. ​​ An attempt to roll back the operation failed. ​​ Check the SQL Server error log for more details. ​​ Then execute the DROP AVAILABILITY​​ GROUP command to clean up any

 

41154

16

Cannot failover availability group '%.*ls' to this SQL Server instance. ​​ The availability group is still being created. ​​ Verify that the specified availability group name is correct. ​​ Wait for CREATE AVAILABILITY GROUP command to finish, then retry the op

 

41155

16

Cannot failover availability group '%.*ls' to this instance of SQL Server. ​​ The availability group is being dropped. ​​ Verify that the specified availability group name is correct. ​​ The availability group​​ may need to be recreated if the drop operation was​​ 

 

41156

16

Cannot drop availability group '%.*ls' from this instance of SQL Server. ​​ The availability group is either being dropped, or the local availability replica is being removed from the​​ availability group. ​​ Verify that the specified availability group name is

 

41157

16

Cannot remove the local availability replica from availability group '%.*ls' from this instance of SQL Server. ​​ The availability group is either being dropped, or the local availability replica is being disjoined. ​​ Verify that the specified availability g

 

41158

16

Failed to join local availability replica to availability group '%.*ls'. ​​ The operation encountered SQL Server error %d and has been rolled back. ​​ Check the SQL​​ Server error log for more details. ​​ When the cause of the error has been resolved, retry the A

  • Check for SQL/Windows messages around same time.

41159

16

Failed to join local availability replica to availability group '%.*ls'. ​​ The operation encountered SQL Server error %d. ​​ An attempt to rollback the operation failed. ​​ Check SQL Server error log for more details. ​​ Run DROP AVAILABILITY GROUP command to cl

 

41160

16

Failed to designate the local availability replica of availability group '%.*ls' as the​​ primary replica. ​​ The operation encountered SQL Server error %d and has been terminated. ​​ Check the preceding error and the SQL Server error log for more details about

 

41161

16

Failed to validate the Cyclic Redundancy Check (CRC) of the configuration of​​ availability group '%.*ls'. ​​ The operation encountered SQL Server error %d, and the availability group has been taken offline to protect its configuration and the consistency of​​ 

 

41162

16

Failed to validate sequence number of the configuration of​​ availability group '%.*ls'. ​​ The in-memory sequence number does not match the persisted sequence number. ​​ The availability group and/or the local availability replica will be restarted automatical

 

41163

16

An error occurred while waiting for the local availability replica of availability group '%.*ls' to transition to the primary role. ​​ The operation encountered SQL OS error %d and has been terminated. ​​ Verify that the Windows Server Failover Clustering (WS

 

41164

16

An error occurred while waiting for the local availability replica of availability group '%.*ls' to transition to the resolving role. ​​ The operation encountered SQL OS error %d and has been terminated. ​​ Verify that the Windows Server Failover Clustering (

 

41165

16

A timeout error occurred​​ while waiting to access the local availability replica of availability group '%.*ls'. ​​ The availability replica is currently being accessed by another operation. ​​ Wait for the in-progress operation to complete, and then retry the​​ 

 

41166

16

An error occurred while waiting to access the local availability replica of availability group '%.*ls'. ​​ The operation encountered SQL OS error %d, and has been terminated. ​​ Verify that the local availability replica is in the correct state, and then retr

 

41167

16

An​​ error occurred while attempting to access availability replica '%.*ls' in availability group '%.*ls'. ​​ The availability replica is not found in the availability group configuration. ​​ Verify that the availability group and availability replica names are

 

41168

16

An error occurred while attempting to access availability replica with ID '%.*ls' in availability group '%.*ls'. ​​ The availability replica is not found in the availability group configuration. ​​ Verify that the availability group name and availability repl

 

41169

16

An error occurred while attempting to access the availability group database with ID '%.*ls' in availability group '%.*ls'. ​​ The availability database is not found in the availability group configuration. ​​ Verify that the availability group name and avail

 

41170

10

Post-online processing for availability group '%.*ls' has been terminated. ​​ Either post-online processing has already completed, the local availability replica is no longer the primary replica, or the availability group is​​ being dropped. ​​ This is an infor

 

41171

16

Failed to create availability group '%.*ls', because a Windows Server Failover Cluster (WSFC) group with the specified name already exists. ​​ The operation has been rolled back successfully. ​​ To retry creating an​​ availability group, either remove or rename

 

41172

16

An error occurred while dropping availability group '%.*ls' from Windows Server Failover Clustering (WSFC) cluster and from the local metadata. ​​ The operation encountered SQL OS error %d, and has been​​ terminated. ​​ Verify that the specified availability gr

  • In DOS prompt, run “Net helpmsg” on the %d value, and proceed accordingly.

 

41173

16

An error occurred while removing the local availability replica from availability group '%.*ls'. ​​ The operation encountered SQL OS error %d, and has been terminated. ​​ Verify that the specified availability group name is correct, and then retry the command

 

41176

10

Failed to acquire exclusive access to local availability group configuration data (SQL OS error: %d). ​​ If the problem persists, you might need to restart the instance of SQL Server.

 

41177

16

The availability replica of the specified availability group '%.*ls' is being dropped. ​​ Wait for the completion of the drop command and retry the operation later.

 

41178

16

Cannot drop availability group '%.*ls' from this SQL Server instance. ​​ The availability group is currently being created. ​​ Verify that the specified availability group name is correct. ​​ Wait for the current operation to complete, then retry the command if

 

41179

16

Cannot remove the local availability replica from availability group '%.*ls' from this instance of SQL Server. ​​ The availability group is currently being created. ​​ Verify that the specified availability group name is correct. ​​ Wait for​​ the current operati

 

41180

16

Attempt to access non-existent or uninitialized availability group with ID '%.*ls'. ​​ This is usually an internal condition, such as the availability group is being dropped or the local WSFC node has lost quorum. In such cases, and no user action is requir

 

41181

16

The local availability replica of availability group '%.*ls' did not become primary. ​​ A concurrent operation may have changed the state of the availability group in Windows Server Failover Cluster. ​​ Verify that the​​ availability group state in Windows Serv

 

41182

16

Failed to set the local availability replica of availability group '%.*ls' as joined in Windows Server Failover Clustering (WSFC) database. ​​ Either the local availability replica is no longer the primary, or the WSFC service is not accessible. ​​ Verify tha

 

41183

16

Failed to modify availability replica options for availability group '%.*ls'. ​​ Before the availability group configuration could be updated, the operation encountered SQL Server error %d. ​​ The​​ operation has been rolled back. ​​ Refer to the SQL Server error

 

41184

16

Failed to modify availability replica options for availability group '%.*ls'. ​​ The availability group configuration has been updated. ​​ However, the operation encountered SQL Server​​ error %d while applying the new configuration to the local availability re

 

41185

10

Replica option specified in ALTER AVAILABILITY GROUP '%.*ls' MODIFY DDL is same is cached availability group configuration.

 

41186

16

Availability group '%.*ls' cannot​​ process an ALTER AVAILABILITY GROUP command at this time. ​​ The availability group is still being created. ​​ Verify that the specified availability group name is correct. ​​ Wait for CREATE AVAILABILITY GROUP command to finis

 

41187

16

Availability group​​ '%.*ls' cannot process an ALTER AVAILABILITY GROUP command at this time. ​​ The availability group is being dropped. ​​ Verify that the specified availability group name is correct. ​​ The availability group may need to be recreated if it was

 

41188

16

Availability group '%.*ls' failed to process %s-%s command. ​​ The operation encountered SQL Server error %d before the availability group configuration could be updated, and has been rolled back. ​​ Refer to the SQL Server error log for details. ​​ Verify that

 

41189

16

Availability group '%.*ls' failed to process the %s-%s command. ​​ The availability group configuration has been updated. ​​ However, the operation encountered SQL Server error %d while applying the new configuration to the local availability replica, and has

 

41190

16

Availability group '%.*ls' failed to process %s-%s command. ​​ The local availability replica is not in a state that could process the command. ​​ Verify that the availability group is online and that the local availability replica is the primary​​ replica, the

 

41191

16

The local availability replica of availability group '%.*ls' cannot become the primary replica. ​​ The last-known primary availability replica is of a higher version than the local availability replica. ​​ Upgrade the local instance of​​ SQL Server to the same​​ 

 

41192

17

Creating and scheduling a worker task for AlwaysOn Availability Groups failed due to lack of resources (SQL OS error %d).  ​​​​ Processing of new actions might be delayed or stalled until the resource limits are resolved.​​ Reduce the memory or thread count on​​ 

 

41193

10

Cannot join database '%.*ls' to availability group '%.*ls'. ​​ The database is in the process of being removed from the availability group. ​​ When the remove-database operation completes, the database will no longer be joined to the availability group. ​​ Then

 

41194

16

An error occurred while waiting for the local availability replica for availability group '%.*ls' to complet post-online work. ​​ The operation encountered SQL OS error %d and has been terminated. ​​ Verify that the Windows Server Failover Clustering (WSFC) c

 

41195

16

Availability group '%.*ls' failed to process the WSFC lease-renewal command. The local availability replica lease is no longer valid to process the lease renewal command. Availability replica lease expired. This is an informational message only. No user a

 

41196

16

Failed to create availability group '%.*ls', because a Windows Server Failover Cluster (WSFC) group with the specified name already exists. ​​ An attempt to rollback the​​ operation failed. ​​ Check the SQL Server error log for more details. ​​ To manually clean​​ 

 

41199

16

The specified command is invalid because the AlwaysOn Availability Groups feature is not supported by this edition of SQL Server. For information about features supported by the editions of SQL Server, see SQL Server Books Online.

 

41402

16

The WSFC cluster is offline, and this availability group is not available. This issue can be caused by a cluster service issue or by the loss of quorum in the cluster.

 

41403

16

Availability group is offline.

 

41404

16

The availability group is offline, and is unavailable. This issue can be caused by a failure in the server instance that hosts the primary replica or by the WSFC availability group resource going offline.

 

41405

16

Availability group is not ready for automatic failover.

 

41406

16

The availability group is not ready for automatic failover. The primary replica and a secondary replica are configured for automatic failover, however, the secondary replica is​​ not ready for an automatic failover. Possibly the secondary replica is unavail

 

41408

16

In this availability group, at least one secondary replica has a NOT SYNCHRONIZING synchronization state and is not receiving data from the primary replica.

 

41410

16

In this availability group, at least one synchronous replica is not currently synchronized. The replica synchronization state could be either SYNCHONIZING or NOT SYNCHRONIZING.

 

41412

16

In this availability group, at least one availability replica​​ does not currently have the primary or secondary role.

 

41414

16

In this availability group, at least one secondary replica is not connected to the primary replica. The connected state is DISCONNECTED.

 

41424

16

This secondary database is not joined to​​ the availability group. The configuration of this secondary database is incomplete. For information about how to join a secondary database to an availability group, see SQL Server Books Online.

 

41428

16

This secondary replica is not joined to the​​ availability group. For an availability replica to be successfully joined to the availability group, the join state must be Joined Standalone Instance (1) or Joined Failover Cluster (2). For information about ho

 

SQL Server AlwaysOn (SQL Server Configuration Manager)

 

 

 

An error occurred while loading the AlwaysOn High Availability properties [return code: 0x80070005].

 

  • You may receive this error if you don't have the appropriate permission. Try right click ‘SQL Server Configuration Manager’ and select​​ ‘Run as Administrator’.

 

 

The AlwaysOn Availability Groups feature requires the x86(non-WOW) or x64 Enterprise Edition of SQL Server 2012 (or later version) running on Windows Server 2008 (or later version) with WSFC hotfix KB 2494036 installed. This SQL​​ Server edition and/or Windows Server System does not meet one or more of the requirements

  • Verify OS is Windows 2008 or later version.

  • If OS is Windows 2008 or Windows 2008 R2, install Windows hotfix KB 2494036, if not already installed.

 

 

 

 

SQL Server​​ AlwaysOn (New Availability Group wizard errors)

 

 

 

The current WSFC cluster quorum vote configuration is not recommended for this availability group.​​ 

For more information, see the following topic in SQL Server Books Online: http://go.microsoft.com/fwlink/?LinkId=224159.

  • When validating WSFC quorum vote configuration, the AlwaysOn Availability Group Wizard shows a warning if any of the following conditions are true:

    • The cluster node that hosts the primary replica does not have a vote. If this vote is not required for that environment, then this message may be ignored. Powershell command to check vote is “Get-ClusterNode | f1 Name, Nodeweight”.

    • A secondary replica is configured for automatic failover and its cluster node does not have a vote. If this vote is​​ not required for that environment, then this message may be ignored.

    • KB2494036 is not installed on all cluster nodes that host availability replicas. This patch is required to add or remove votes for cluster nodes in multi-site deployments. However, in single-site deployments, it is usually not required and you may safely ignore the warning.

    • While setting up the availability group in the wizard, you configured a replica to asynchronous availability mode. This warning is only reported in SQL 2012 RTM, not in​​ SP1 of SQL 2012 RTM.

    • While setting up the availability group in the wizard, you configured a replica for manual failover mode. This warning is only reported in SQL 2012 RTM, not in SP1 of SQL 2012 RTM.

  • Check the link. Redirects to​​ http://msdn.microsoft.com/en-us/library/hh270280.aspx

 

 

Checking for the​​ database files on the secondary replica​​ resulted in an error. (Microsoft.SqlServer.Management.HadrTasks)

------------------------------

Program Location:

 ​​ ​​​​ at Microsoft.SqlServer.Management.Hadr.TestDatabaseFileExisting.DoWork()

 ​​ ​​​​ at Microsoft.SqlServer.Management.TaskForms.SimpleWorkItem.Run()

===================================

The following database files already exist on the server instance that hosts secondary replica HA-VM3\TEST1: ​​ 

\\path\file.mdf

\\path\file_log.ldf

​​ (Microsoft.SqlServer.Management.HadrTasks)

------------------------------

Program Location:

 ​​ ​​​​ at​​ Microsoft.SqlServer.Management.Hadr.TestDatabaseFileExisting.DoWork()

  • Check if database-file paths are identical on source and on destination servers. If different, manual (rather than using AG wizard) restore may be required with WITH MOVE syntax.

 

 

 

SQL​​ Server AlwaysOn (performance issues)

 

983

14

Unable to access database '%.*ls' because its replica role is RESOLVING which does not allow connections. Try the operation again later.

  • Check SQL ERRORLOGs, event logs, etc. for network, storage related​​ messages etc.

3402

10

The database '%ls' is marked %ls and is in a state that does not allow recovery to be run.

  • If the second %ls is RESTORING, then one option is below.

1) Asked the customer to stop the SQL Server Service.

2) Had the customer rename the​​ actual files on disk with a "_old" suffix so that the sql server would not find them and have the db come up suspect.

3) Asked the customer to start the SQL Server Service.

Once the server came back up we were able to delete the database.

4) Had the​​ customer create a new database with the same name and location as the original database. ​​ Used the output from sysaltfiles to verify this.

5) Had the customer stop the SQL Server Service.

6) Had the customer rename these new database files by appending a "_new" on the end of the file.

7) Had the customer take the old database files with the "_old" suffix and changed them back to the original name.

8) Had the customer start the SQL Server Service.

 

 

Slow synchronization. Waittime for HADR_SYNC_COMMIT grows​​ anywhere from 500ms to 900ms (compared to less than 15-20 ms).

  • If KB2723814 not applied, then try the KB workaround of suspend secondary replica and then resume, so that AlwaysOn knows that availability mode has changed back to synchronous commit.

 

 

SQL​​ Server AlwaysOn errors (other)

 

19471

16

The WSFC cluster could not bring the Network Name resource with DNS name '%ls' online. The DNS name may have been taken or have a conflict with existing name services, or the WSFC cluster service may not be running​​ or may be inaccessible. Use a different

  • Check for associated errors like error 19476.

19476

16

The attempt to create the network name and IP address for the listener failed. The WSFC service may not be running or may be inaccessible in its current state,​​ or the values provided for the network name and IP address may be incorrect. Check the state of the WFSC cluster and validate the network name and IP address with the network administrator.

  • Check for associated errors like error 41009.

41009

16

The Windows Server Failover Clustering (WSFC) resource control API returned error code %d. ​​ The WSFC service may not be running or may not be accessible in its current state, or the specified arguments are invalid. ​​ For information about this error code,

  • Net helpmsg​​ on the %d value, and proceed accordingly.

  • May also be accompanied by Microsoft-Windows-FailoverClustering messages in event log.

  • Check cluster log.

  • If there error is in cluster in Windows Azure, then it may be because Windows Azure currently does not support more than one IP address per VM, which is required by the availability group listener to resolve to the availability group owner node. This is documented in​​ http://msdn.microsoft.com/en-us/library/jj870962.aspx​​ (High Availability and Disaster Recovery for SQL Server in Windows Azure Virtual Machines).

 

41015

16

Failed to obtain the Windows Server Failover Clustering (WSFC) node handle (Error code %d). ​​ The WSFC service may not be running or may not be accessible in its current state, or the specified cluster node name is invalid. ​​ For information about this erro

  • Check if Windows cluster service is running.

  • Net helpmsg on the %d value, and proceed accordingly.

 

 

 

SQL Server AlwaysOn​​ errors (network)

 

 

 

TCP Provider, error: 0 - An operation on a socket could not be performed because the system lacked sufficient buffer space or because a queue was full.

  • NetStat output may show hundreds of entries in TIME_WAIT state leading to​​ buffer/port exhaustion

 

SQL Server AlwaysOn errors (Windows)

 

5057

 

The cluster IP address is already in use.

  • Checked if IP is already in use

  • Check permission for cluster network object CNO.

5942

 

The resource failed to come online due to the failure of one or more provider​​ resources.

  • Depending on the resource type, this generally requires involvement of Windows team.

  • If this is a network name or an IP address resource, then try creating a Client Access Point with same name and IP if possible, and see it comes online. If it does not come online, then this requires investigation by Windows team.

 

 

 

 

SQL Server AlwaysOn errors (cluster log)

 

 

 

[RES] Network Name <SPEPD_Tier2>:​​ Unable to update password for computer account​​ DCR-SPEPD on DC \\ DC-04.xxxx.org, status​​ 5.

[RHS]​​ Online for resource SPEPD_Tier2 failed.

  • Status 5 is Access Denied

  • In Windows AD, check for CNO (cluster network object) permissions.

  • For example, below to grant the "create computer objects" to the cluster virtual host (virtual cluster name) for the OU in​​ Active Directory on "Computers" and "DomainServer" OU that was created. [To be tested]

 

Instructions [to be tested]

1) Log into the Domain Controller machine.

2) Select Start - Administrative Tools - Active Directory users and computer.

--> Active Directory users and computer screen displays.

3) Display-select advanced features.

4) Select the appropriate domain - Computers.

5) Verify that the cluster virtual hosts.

6) Select the appropriate domain - Computers - right click - Properties.

--> Computers Properties screen appears.

7) Security tab-click the Advanced button.

--> Advanced Security for Computers "screen appears.

8) If you have access permissions tab - cluster virtual host registration registration no "Edit" button if you choose the "add" button.

+​​ Case "add": conducted a check in the "computer", click the "object type" button, search for cluster virtual hosts.

--> [Permission entries on Computers] screen is displayed.

9) Target: Select this object and all child objects check the permission "create computer objects" and click the "OK" button.

--> Returns to the advanced security for Computers "screen.

10) Click "OK" button after clicking "the apply" button.

--> Returns to the Computers Properties screen.

11) And click the "OK" button and exit the Active Directory users and computer screen.

 

 

INFO ​​ [RES] Network Name: [NNLIB]​​ NetUserAdd​​ object aglisten11 on DC: \\DC.contoso.com, result:​​ 8557

ERR  ​​​​ [RES] Network Name: [NNLIB]​​ Failed to create Computer Object​​ aglisten11​​ in the Active Directory, error​​ 8557

  • net helpmsg 8557 says "Your computer could not be joined to the domain. You have exceeded the maximum number of computer accounts you are allowed to create in this domain. Contact your system administrator to have this limit reset or increased."

 

 

 

SQL Server AlwaysOn (patching/updates in SQL 2012)

 

AG on Standalone Instance

 

Patching steps:

 

  • Patch the secondary replica (B)

  • Bring the secondary replica online (will be new version)

  • Log the original synchronization configurations for each replica.​​ Change the secondary replica and primary replica to “Synchronous Commit” mode, waiting for the secondary replica (B) to be “synchronized.”

    • This will ensure there is no data loss during failover.

    • You can check dashboard or dmv: dm_hadr_database_replica_states for the status

  • Issue a failover through SSMS or T-SQL to failover the AG to the secondary replica. So now the new primary is B, the new secondary is A

  • Patch the original primary replica (A)

  • Bring the original primary replica (A) online (will be new version )

  • Wait for A to become “Synchronized”​​ 

  • Failover the AG back to A

  • Change each replica’s synchronization mode to the original configurations you logged in step3.

 

 

Caveat list:

  • Before you patch, you can still keep automatic/manual failover setting no change. Just a reminder: if during patching time, primary is down, automatic failover may fail if the secondary hasn’t completed the patching.

 

  • If primary and secondary replicas are in multisubnet, your client may experience a little bit longer time of disconnection or timeout during failover.

  • Please do remember to switch back to your original synchronization mode.

 

 

AG on FCIs (Failover Cluster Instances)

 

FCI on AG deployment Example

  • Primary replica(A): FCI1​​ 

    • Node1 – active

    • Node2 - passive

  • Secondary​​ Replica(B): FCI2​​ 

    • Node3 – active

    • Node4 - passive

​​ 

Choose between one of the following two options:

 

Longer downtime, less steps

(FCI rolling patching).

 

Patching steps (basic):

  • Patch Node4

  • Move FCI2 from Node3 to Node4​​ 

  • Patch Node3

  • Move FCI2 from Node4 to​​ Node3

  • Patch Node2

  • Move FCI1 from Node1 to Node2

  • Patch Node1

  • Move FCI1 from Node2 to Node1

 

Optimized downtime, more steps

(Leverage AG failover patching).

 

​​ 

 

Patching steps (optimized):

  • Patch Node4

  • Move FCI2 from Node3 to Node4

  • Patch Node3

  • Move FCI2 from​​ Node4 to Node3

  • Log the original synchronization configurations for each replica. Change the secondary replica and primary replica to “Synchronous Commit” mode, waiting for the secondary replica (B) to be “synchronized”.

  • Manually failover AG from FCI1 to FCI2 (now the new primary is on FCI2, the new secondary is on FCI1)

  • Patch Node2

  • Move FCI1 from Node1 to Node2

  • Patch Node1

  • Move FCI1 from Node2 to Node1

  • Manually failover AG from FCI2 back to FCI1

  • Change each replica’s synchronization mode to the original configurations you logged in step #5.

 

Caveat list:

** Please use basic patching steps if primary and secondary are on different data centers/subnets **

  • FCI rolling patch guarantees zero data lost

  • Use synchronous commit secondary on a high latency network​​ would impact OLTP performance

  • AG failover cross subnets might cause up to 20 seconds delay on client first connections.

 

 

Knowledge worth sharing...Share on linkedin
Linkedin
Share on facebook
Facebook
Share on google
Google
Share on twitter
Twitter

SQL Server – SQL 2012 Installation Step-By-Step

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Knowledge worth sharing...Share on linkedin
Linkedin
Share on facebook
Facebook
Share on google
Google
Share on twitter
Twitter

SQL Server – Add New Replica in AlwaysOn Availability Group

  • Expand​​ AlwaysOn High Availability​​ > Expand Availability Groups >Expand Availability Groups Name. Right click Availability Replica.

 

 

 

 

  • Click​​ Next.

 

 

  • Click​​ Add Replica…

 

 

 

 

  • Click​​ Connect All…

 

 

 

  • Input the instance name of your new replica. Click​​ Connect.

 

 

 

 

  • Change ‘Readable Secondary’ to​​ Yes. But if you don’t want your secondary replica to be readable, then just accept the default ‘No’.

 

 

 

 

  • Click​​ Endpoints​​ Tab just to verify​​ that the new endpoint has been created.

 

 

 

 

  • Go to​​ Backup Preference​​ tab. I just indicated ‘20’ under Backup Priority for the new Replica. We already have an existing secondary Replica already taking the DB backups.

 

  • Choose​​ Join Only​​ because I already​​ took a full backup of the primary database and restored it to the new replica​​ WITH NORECOVERY​​ option.

 

 

 

 

  • Validation

 

 

 

 

 

 

 

  • Verify Database​​ Synchronization.

 

Knowledge worth sharing...Share on linkedin
Linkedin
Share on facebook
Facebook
Share on google
Google
Share on twitter
Twitter

SQL Server – The Transaction Log for Database is Full Due to Replication (Error 9002 Severity 17 State 6)

We have a 3 node AAG (below is our setup). Our transaction log file has a growth limit of​​ 60GB. I found out that this error occurred because the CDC failed. I could not start CDC and the transaction log is not truncating.

 

 

I checked the log_reuse_wait_desc and it indicated REPLICATION which means that there are committed transactions that haven’t yet been scanned by the transaction replication Log Reader Agent job for the purpose of sending them to the replication distributor or harvesting them for​​ Change Data Capture​​ (which uses the same Agent job as transaction replication). The job could have been disabled, could be broken, or could have had its SQL Agent schedule changed.

 

SELECT​​ name,​​ recovery_model_desc,​​ log_reuse_wait_desc

FROM​​ sys.databases

WHERE​​ name​​ =​​ 'DB_Name'

 

 

 

I also ran the command DBCC OPENTRAN and saw Replicated Transaction​​ Information.

 

 

 

To resolve the issue, I did the following steps.

  • Create a new transaction log file (it is a good thing that our currently t-log file has a growth limit else it could have occupied all the disk space.)

  • I started CDC and it ran​​ successfully.

 

But the t-log was still not truncating. I checked the log_reuse_wait_desc again and it indicated “AVAILABILITY REPLICA” which means that the database mirroring partnership has some latency in it and there are log records on the mirroring principal that haven’t yet been sent to the mirroring mirror (called the send queue). This can happen if​​ the mirror is configured for asynchronous operation, where transactions can commit on the principal before their log records have been sent to the mirror.​​ It can also happen in synchronous mode, if the mirror becomes disconnected or the mirroring session is suspended. The amount of log in the send queue can be equated to the expected amount of data (or work) loss in the event of a crash of the principal.

 

I​​ waited for few minutes and I checked the log_reuse_wait_desc and now I see “NOTHING” which means that SQL Server thinks there is no problem with log truncation.

 

Knowledge worth sharing...Share on linkedin
Linkedin
Share on facebook
Facebook
Share on google
Google
Share on twitter
Twitter