This feature is in preview. This feature is subject to change. To share feedback and/or issues, contact Support.
The ALTER VIRTUAL CLUSTER statement initiates a failover in a physical cluster replication (PCR) job and manages a virtual cluster.
PCR happens between an active primary cluster and a passive standby cluster that accepts updates from the primary cluster. The unit of replication is a virtual cluster, which is part of the underlying infrastructure in the primary and standby clusters. The CockroachDB cluster has:
- The system virtual cluster manages the cluster's control plane and the replication of the cluster's data. Admins connect to the system virtual cluster to configure and manage the underlying CockroachDB cluster, set up PCR, create and manage a virtual cluster, and observe metrics and logs for the CockroachDB cluster and each virtual cluster.
- Each other virtual cluster manages its own data plane. Users connect to a virtual cluster by default, rather than the system virtual cluster. To connect to the system virtual cluster, the connection string must be modified. Virtual clusters contain user data and run application workloads. When PCR is enabled, the non-system virtual cluster on both primary and secondary clusters is named
main.
For more detail, refer to the Physical Cluster Replication Overview.
Required privileges
ALTER VIRTUAL CLUSTER requires one of the following privileges:
- The
adminrole. - The
MANAGEVIRTUALCLUSTERsystem privilege allows the user to run all the relatedVIRTUAL CLUSTERSQL statements for PCR.
Use the GRANT SYSTEM statement:
GRANT SYSTEM MANAGEVIRTUALCLUSTER TO user;
Synopsis
Parameters
| Parameter | Description |
|---|---|
virtual_cluster_spec |
The virtual cluster's name. |
PAUSE REPLICATION |
Pause the replication stream. |
RESUME REPLICATION |
Resume the replication stream. |
COMPLETE REPLICATION TO |
Set the time to complete the replication. Use:
|
SET REPLICATION RETENTION = duration |
Change the duration of the retention window that will control how far in the past you can fail over to. We do not recommend setting RETENTION much higher than the 24-hour default on the standby cluster. Accumulated data from an excessive retention (failover) window could affect queries running on the standby cluster that is active following a failover. |
SET REPLICATION EXPIRATION WINDOW = duration |
Override the default producer job's expiration window of 24 hours. The producer job expiration window determines how long the producer job will continue to run without a heartbeat from the consumer job. For more details, refer to the Technical Overview. |
START REPLICATION OF virtual_cluster_spec ON physical_cluster |
Reset a virtual cluster to the time when the virtual cluster on the promoted standby diverged from it. To reuse as much of the existing data on the original primary cluster as possible, you can run this statement as part of the failback process. This command fails if the virtual cluster was not originally replicated from the original primary cluster. Refer to Options for details on how you can configure a PCR stream initiated as a failback. |
START SERVICE SHARED |
Start a virtual cluster so it is ready to accept SQL connections after failover. |
RENAME TO virtual_cluster_spec |
Rename a virtual cluster. |
STOP SERVICE |
Stop the shared service for a virtual cluster. The virtual cluster's data_state will still be ready so that the service can be restarted. |
GRANT ALL CAPABILITIES |
Grant a virtual cluster all capabilities. |
REVOKE ALL CAPABILITIES |
Revoke all capabilities from a virtual cluster. |
GRANT CAPABILITY virtual_cluster_capability_list |
Specify a capability to grant to a virtual cluster. |
REVOKE CAPABILITY virtual_cluster_capability_list |
Revoke a capability from a virtual cluster. |
Options
You can use the following options with ALTER VIRTUAL CLUSTER {vc} START REPLICATION OF virtual_cluster_spec ON physical_cluster to initiate the failback process.
| Option | Value | Description |
|---|---|---|
EXPIRATION WINDOW |
Duration | Override the default producer job's expiration window of 24 hours. The producer job expiration window determines how long the producer job will continue to run without a heartbeat from the consumer job. For more details, refer to the Technical Overview. |
New in v24.3: READ VIRTUAL CLUSTER |
N/A | Configure the PCR stream to allow reads from the standby cluster. Note: This only allows for reads on the standby's virtual cluster. You cannot perform writes or schema changes to user tables while connected to the standby virtual cluster. For more details, refer to Start the failback process. |
RETENTION |
Duration | Change the duration of the retention window that will control how far in the past you can fail over to. We do not recommend setting RETENTION much higher than the 24-hour default on the standby cluster. Accumulated data from an excessive retention (failover) window could affect queries running on the standby cluster that is active following a failover. |
Examples
Start the failover process
To start the failover process, use COMPLETE REPLICATION and provide the timestamp to restore as of:
ALTER VIRTUAL CLUSTER main COMPLETE REPLICATION TO {failover time specification};
You can use either:
SYSTEM TIMEto specify a timestamp.LATESTto specify the most recent replicated timestamp.
If you started the PCR stream with the READ VIRTUAL CLUSTER option, failing over with SYSTEM TIME will destroy the readonly virtual cluster. If you fail over with LATEST, the readonly virtual cluster will remain on the original standby cluster, but will not update with new writes.
Start the failback process
To fail back to a cluster that was previously the primary cluster, use the ALTER VIRTUAL CLUSTER syntax:
ALTER VIRTUAL CLUSTER {original_primary_vc} START REPLICATION OF {promoted_standby_vc} ON {connection_string_standby};
The original primary virtual cluster may be almost up to date with the promoted standby's virtual cluster. The difference in data between the two virtual clusters will include only the writes that have been applied to the promoted standby after failover from the primary cluster.
New in v24.3:
Use the READ VIRTUAL CLUSTER option with the ALTER VIRTUAL CLUSTER failback syntax to start a PCR stream that also creates a read-only virtual cluster on the standby cluster.
Set a retention window
You can change the retention window to protect data from garbage collection. The retention window controls how far in the past you can fail over to:
ALTER VIRTUAL CLUSTER main SET REPLICATION RETENTION = '24h';
We do not recommend setting RETENTION much higher than the 24-hour default on the standby cluster. Accumulated data from an excessive retention (failover) window could affect queries running on the standby cluster that is active following a failover.
Start a virtual cluster
When a virtual cluster is ready after initiating the failover process, you must start the service so that the virtual cluster is ready to accept SQL connections:
ALTER VIRTUAL CLUSTER main START SERVICE SHARED;
To stop the shared service for a virtual cluster and prevent it from accepting SQL connections:
ALTER VIRTUAL CLUSTER main STOP SERVICE;
See also
- Physical Cluster Replication Overview
CREATE VIRTUAL CLUSTERDROP VIRTUAL CLUSTERSHOW VIRTUAL CLUSTER
Was this helpful?
