PDB Switchover and Failover in a Multitenant Configuration (2023)

The sample commands included in the steps below use the following CDB and PDB names.

• CDB100 (source CDB)
  • Contains PDB001, PDB002, PDB003; PDB001 will be configured for switchover
• CDB100_STBY (source standby CDB)
• CDB200 (destination CDB)
• CDB200_STBY (destination standby CDB)

Step 1: Extract PDB Clusterware managed services on the source database

Determine any application and end user services created for the source PDB that have been added to CRS.

Because there are certain service attributes such as database role not stored in the database, the detail attributes should be retrieved from CRS using SRVCTL CONFIG SERVICE.

1. Retrieve the service names from the primary PDB (PDB001 in our example).

   PRIMARY_HOST $ sqlplus sys@cdb100 as sysdbaSQL> alter session set container=pdb001;SQL> select name from dba_services;

2. For each service name returned, retrieve the configuration including DATABASE_ROLE.

   PRIMARY_HOST $ srvctl config service -db cdb100 -s SERVICE_NAME

Step 2: Create an empty target database

Create an empty CDB (CDB200 in our example) on the same cluster as the source CDB (CDB100) which will be the destination for the PDB.

Allocate resources for this CDB to support the use of the PDB for the duration of the testing.

Step 3: Create a target standby database

Enable Oracle Data Guard on the target CDB to create a standby database (CDB200_STBY).

The standby database must reside on the same cluster as the source standby database.

The configuration should resemble the image below.

Step 4: Migrate the PDB

Migrate the PDB (PDB001) from the source CDB (CDB100) to the destination CDB (CDB200).

1. Start a connection to the source primary database using Oracle Data Guard broker command line (DGMGRL).

   This session should run on a host that contains instances of both the source primary CDB and the destination primary CDB. The session should be started with a sysdba user.

   The broker CLI should be run from the command line on the primary CDB environment and run while connected to the source primary CDB. If you are using a TNS alias to connect to the source primary, it should connect to the source primary instance running on the same host as the broker CLI session.

   The host and environment settings when running the broker CLI must have access to SQL*Net aliases for:

   • Destination primary CDB – This alias should connect to the destination primary instance that is on the same host as the broker CLI session/source primary database instance to ensure the plugin operation can access the PDB unplug manifest file that will be created.

   • Destination standby CDB, this can connect to any instance in the standby environment.

   • Standby site ASM instance, this can connect to any instance in the standby environment.

     PRIMARY_HOST1 $ dgmgrl sys@cdb100_prim_inst1 as sysdba

     This session should run on a host that contains instances of both the source primary CDB and the destination primary CDB and connected to a sysdba user. Use specific host/instance combinations instead of SCAN to ensure connections are made to the desired instances.

2. Run the DGMGRL MIGRATE PLUGGABLE DATABASE command.

   The STANDBY FILES keyword is required.

   See Full Example Commands with Output for examples with complete output and MIGRATE PLUGGABLE DATABASE for additional information about the command line arguments.

   (Video) The importance of Fast-Start Failover in an Oracle Data Guard configuration

   • Sample command example without TDE:

     DGMGRL> MIGRATE PLUGGABLE DATABASE PDB001 TO CONTAINER CDB200 USING ‘/tmp/PDB001.xml’ CONNECT AS sys/password@cdb200_inst1 STANDBY FILES sys/standby_asm_sys_password@standby_asm_inst1 SOURCE STANDBY CDB100_STBY DESTINATION STANDBY CDB200_STBY ;

   • Sample command example with TDE

     DGMGRL> MIGRATE PLUGGABLE DATABASE PDB001 TO CONTAINER CDB200 USING ‘/tmp/pdb001.xml’ CONNECT AS sys/password@cdb200_inst1 SECRET "some_value" KEYSTORE IDENTIFIED BY "destination_TDE_keystore_passwd" STANDBY FILES sys/standby_asm_sys_password@standby_asm_inst1 SOURCE STANDBY cdb100_stby DESTINATION STANDBY cdb200_stby;

When the command is executed, it:

1. Connects to the destination database and ASM instances to ensure credentials and connect strings are correct

2. Performs a variety of prechecks - If any precheck fails, the command stops processing and returns control to the user, an error is returned, and no changes are made on the target CDB

3. Creates a flashback guaranteed restore point in the destination standby CDB - This requires a short stop and start of redo apply

4. Closes the PDB on the source primary

5. Unplugs the PDB on the source primary - If TDE is in use, the keys are included in the manifest file generated as part of the unplug operation

6. Drops the PDB on the source primary database with the KEEP DATAFILES clause, ensuring that the source files are not dropped

7. Waits for the drop PDB redo to be applied to the source standby database - It must wait because the files are still owned by the source standby database until the drop redo is applied

   The command waits a maximum of TIMEOUT minutes (default 10). If the redo hasn't been applied by then the command fails and you must manually complete the process.

8. Manages the ASM aliases for the PDB files at the standby, removing any existing aliases and creating new aliases as needed - If the standby files already exist in the correct location, all aliases for the standby copy of the PDB are removed

9. Plugs in the PDB into the destination primary CDB - If TDE is in use, the keys are imported into the destination primary keystore as part of the plugin

10. Ships and applies redo for the plugin operation to the destination CDB, which uses any created aliases (if necessary) to access the files and incorporate them into the standby database

11. Validates that the standby files are added to the destination standby using redo apply

    (Video) 05 Refreshable PDB Switchover

12. Opens the PDB in the destination primary database

13. Stops redo apply

14. Drops the flashback guaranteed restore point from the destination standby database

15. If TDE is enabled, redo apply remains stopped, if TDE is not enabled, redo apply is restarted

Step 5: Post Migration - Optional TDE Configuration Step and Restart Apply

If TDE is in use, redo apply will have been stopped by the broker MIGRATE PLUGGABLE DATABASE operation on the destination standby (CDB200_STBY) to allow the new TDE keys to be managed. Copy the keystore for the destination primary (CDB200) to the destination standby keystore and start redo apply.

SOURCE_HOST $ scp DESTINATION_PRIMARY_WALLET_LOCATION/*> DESTINATION_HOST:DESTINATION_STANDBY_WALLET_LOCATION/$ dgmgrl sys/password@CDB200 as sysdbaDGMGRL> edit database cdb200_stby set state=’APPLY-ON’;

Step 6: Post Migration - Enable Services

Add any application services for the PDB to Cluster Ready Services (CRS), associating them with the PDB and correct database role in the destination CDB, and remove the corresponding service from the source CDB.

1. For each service on both the primary and standby environments, run the following:

   PRIMARY_HOST $ srvctl add service -db cdb200 -s SERVICE_NAME -pdb pdb001 -role [PRIMARY|PHYSICAL_STANDBY]….STANDBY_HOST $ srvctl add service -db cdb200_stby -s SERVICE_NAME -pdb pdb001 -role [PRIMARY|PHYSICAL_STANDBY]….PRIMARY_HOST $ srvctl remove service -db cdb100 -s SERVICE_NAMESTANDBY_HOST $ srvctl remove service -db cdb100_stby -s SERVICE_NAME

2. Start the required services for the appropriate database role.

   1. Start each PRIMARY role database service

      PRIMARY_HOST $ srvctl start service -db cdb200 -s SERVICE_NAME

   2. Start each PHYSICAL_STANDBY role database service:

      STANDBY_HOST $ srvctl start service -db cdb200_stby -s SERVICE_NAME

Step 7: Perform PDB role transition testing

After completing the migration, you can now perform required Oracle Data Guard role transitions or DR test with the PDB in destination CDB (CDB200). No other PDBs in the source CDB (CDB100) are impacted.

In addition, you continue to maintain the Data Guard benefits for both the source and destination CDB, such as DR readiness, Automatic Block Media Recovery for data corruptions, Fast-Start Failover to bound recovery time, Lost Write detection for logical corruptions, offloading reads to the standby to reduce scale and reduce impact of the primary, and so on. etc.

• Connect to the destination CDB using DGMGRL and perform the switchover.

  $ dgmgrl sys@cdb200 as sysdbaDGMGRL> switchover to CDB200_STBY;

You can continue performing your DR testing for the PDB.

Once the DR testing for the PDB is complete, you can switch back and subsequently migrate the PDB back to the original source CDB.

• Connect to the destination CDB (CDB200) using DGMGRL and perform the switchback operation.

  $ dgmgrl sys@cdb200 as sysdbaDGMGRL> switchover to CDB200;

Step 8: Return the PDB to the original CDB

After migration and role transition testing, switch back to the original configuration for this CDB and migrate the PDB back to the original Data Guard configuration, again automatically maintaining the standby database files. Data Guard broker migration handles any aliases that need to be dropped or created as part of the migration process.

See Full Example Commands with Output for examples with complete output.

• Start a connection to the source primary using Data Guard Broker command line (DGMGRL)

  $ dgmgrl DGMGRL> connect sys/@cdb200_inst1 as sysdba

  • Command without TDE

    DGMGRL> MIGRATE PLUGGABLE DATABASE PDB001 TO CONTAINER CDB100 USING ‘/tmp/PDB001_back.xml’ CONNECT AS sys/password@cdb100_inst1 STANDBY FILES sys/standby_asm_sys_password@standby_asm_inst SOURCE STANDBY CDB200_STBY DESTINATION STANDBY CDB100_STBY ;

  • Command with TDE

    DGMGRL> MIGRATE PLUGGABLE DATABASE PDB001 TO CONTAINER CDB100 USING ‘/tmp/PDB001_back.xml’ CONNECT AS sys/password@cdb100_inst1 STANDBY FILES sys/standby_asm_sys_password@standby_asm_inst SOURCE STANDBY CDB200_STBY DESTINATION STANDBY CDB100_STBY ;

Step 9: Post Migration - Enable Services

Add any application services for the PDB to Cluster Ready Services (CRS), associating them with the PDB and correct database role in the destination CDB (CDB100), and remove the corresponding service from the source CDB (CDB200).

• For each service on both the primary and standby environments, run the following:

  PRIMARY_HOST $ srvctl add service -db cdb100 -s SERVICE_NAME -pdb pdb001 -role [PRIMARY|PHYSICAL_STANDBY]….STANDBY_HOST $ srvctl add service -db cdb100_stby -s SERVICE_NAME -pdb pdb001 -role [PRIMARY|PHYSICAL_STANDBY]….<PRIMARY_HOST>PRIMARY_HOST $ srvctl remove service -db cdb200 -s SERVICE_NAMESTANDBY_HOST $ srvctl remove service -db cdb200_stby -s SERVICE_NAME

• Start the required services for the appropriate database role.

  (Video) Webinar-12- Live Demo on Dataguard Switch Over and Dataguard Fail Over || DR Drill Activity

  1. Start each PRIMARY role database service.

     PRIMARY_HOST $ srvctl start service -db cdb100 -s SERVICE_NAME

  2. Start each PHYSICAL_STANDBY role database service.

     STANDBY_HOST $ srvctl start service -db cdb100_stby -s SERVICE_NAME