How to rollback the patch after failed cutover phase in R12.2

There may be scenario  when  cutover phase failed . It is possible to go back to previous state of cutover(rollback the patch) ,if flashback database is either enabled in the database or we have taken full backup prior to cutover

I would be explaining it with respect to Database Flashback to rollback the patch

I am assuming here we have Flashback enabled in the database.We can confirm that using the command

SQL>select FLASHBACK_ON from v$database;
FLASHBACK_ON
------------
Yes

You can learn more about Flashback database in below links

Flashback Oracle Database
How to Flashback when we have dataguard

It is recommended the Online Patching cutover phase should be scheduled for a time when there are few online transactions and batch processing is minimal. You should confirm that critical concurrent requests are not executing during cutover. You should also consider putting scheduled concurrent requests on hold prior to executing cutover as otherwise cutover phase will wait for the program to complete plus you will lose transaction data in case of flashback

Let us look at the Problem Case

Case 1

You are running an Online Patching cycle:

$ adop phase=prepare
$ adop phase=apply patches=99999999
$ adop phase=finalize
$ adop phase=cutover

Cutover fails, and you need to go back to the state of the system before you ran the cutover phase.

If you had not run the cutover phase, you would have been able to roll back the patch application process by running the adop abort phase. However, this is not possible once cutover has been run.

There are two main parts to rollback the patch:
(1) Database Restore : Flashback database is the fastest method to rollback the database changes and go back to point in time. We can use database restore technique also but that is very time consuming

Flashing Back the Database
a).First, shut down the database, then start it up in mount state:

SQL>shutdown immediate
Database closed.
Database dismounted.
ORACLE instance shut down.
SQL>startup mount
ORACLE instance started.

b).Restore the flashback to the specified time.

SQL>flashback database to time to_data(<time before teh cutover>;
Flashback complete.

c).Start the database in read-only mode:

SQL>alter database open read only;
Database altered.

Check all looks as expected.

d).Shut down the database, start it up in mount state, then open it with the resetlogs option:

SQL>shutdown immediate
Database closed.
Database dismounted.
ORACLE instance shut down.
SQL>startup mount
ORACLE instance started.
Database mounted.
SQL>alter database open resetlogs;
Database altered.

2) Filesystem restore: Depending on when cutover failed, you may also need to restore the application tier file systems

Restoring the File Systems

Whether you need to perform this step is conditional, depending on whether cutover failed before the file systems were switched. You can identify which of these cases applies by referring to the cutover logs in $NE_BASE/EBSapps/log/adop/<current_session_id>/cutover_<timestamp>/ for your current session id.

Case 1 – If the log messages indicate that cutover failed before the file systems were switched, do a clean shutdown of any services that are running. Then restart all the services using the normal startup script,

Case 2 – If the log messages indicate that cutover failed after the file systems were switched, then follow below step to switch the file systems back.

(a) Shut down services started from new run file system

1.Source the environment on the new run file system.
2.From $ADMIN_SCRIPTS_HOME, shut down all the services (using adstpall.sh on UNIX).

(b)In a multi-node environment, repeat the preceding two steps on all nodes, leaving the admin node until after all the slave nodes.

(c) Switch file systems back
On all nodes where file systems have been switched, run the following command to switch the file systems back:

$ perl $AD_TOP/patch/115/bin/txkADOPCutOverPhaseCtrlScript.pl \
-action=ctxupdate \
-contextfile=<full path to new run context file> \
-patchcontextfile=<full path to new patch file system context file> \
-outdir=<full path to out directory>

(d)Start up all services from the old run file system (using adstrtal.sh on UNIX).
(e)In a multi-node environment, repeat the preceding two steps on all nodes, starting with the admin node and then proceeding to the slave nodes

Conclusion

After the restore is complete, you have two basic options for proceeding:
(a) Abort the current patching cycle, if the issue that required you to restore was caused by the patches you were attempting to apply.

Here are steps to Abort an Online Patching Cycle

If a patching cycle is failing and the issue cannot be resolved quickly, it is possible to abort the patching cycle and return to normal runtime operation. The patch edition will be dropped.

You can abandon a patching cycle (without applying any patches) by running the command:
$ adop phase=abort

Aborting a patching cycle will drop the patch edition, but you must then run the cleanup and fs_clone phases before starting a new patching cycle. The cleanup must be a full cleanup.

For example:
$ adop phase=prepare
$ adop phase=apply patches=9999999
$ adop phase=abort
$ adop phase=cleanup cleanup_mode=full
$ adop phase=fs_clone

Optionally, you can combine the abort and cleanup commands as follows:

$ adop phase=abort,cleanup cleanup_mode=full

(b) Identify and fix any other issues in the current patching cycle, and proceed with patching.