I’ve been running RAC for many years, but I have been consistently frustrated at having to lose availability to perform patches/upgrades. Obviously, I have heard a lot of buzz about rolling upgrades but so far, in the versions I have worked on up to and including 10.2.0.3 I have never seen an upgrade that I was going to apply whereby it was possible to perform an upgrade on my RAC cluster without incurring downtime. Perhaps, “rolling upgradeable” is more marketing smoke and mirrors than something actually based on reality?The January Quarterly critical patch update has recently come out, and at first I thought it was going to be possible to apply this doing a rolling upgrade and not incur any downtime, you will probably be surprised finding a dba owning up to applying a CPU, but if it was possible to apply this CPU without downtime then I thought I should give it a shot. So reading the README for the CPUJAN2008 patch, I page down the section regarding patch installation instructions for a RAC environment, and it clearly states that you can patch 1 node at a time. Great, is this a rolling upgradeable CPU?I then look at the post-installation instructions which state the following:
Select one node to execute the post installation steps. Follow the same set of instructions as mentioned in the Section 3.3.3, “Post Installation Instructions for a Non-RAC Environment”.
Users can continue to access the database during the post-installation steps.
So I go to the post-install instructions for the Non-Rac Environment, and sure I can run the catcpu.sql online, and looking at this script, Oracle have obviously thought about availability in a RAC environment:
Rem Check open UPGRADE status; set session attributes
Rem Following call to check_server_instance is commented as it checks if
Rem is started up in UPGRADE mode else quits.
Rem CPU patches needs to be RAC compliant means database should not be started
Rem in UPGRADE mode.
-- EXECUTE dbms_registry.check_server_instance;
But there is something new with this CPU, you must run a sql script called view_recompile_jan2008cpu.sql and here is a problem. You have to run this sql script after the database has been started using startup upgrade. This is bad news for system availability as only sysdba privilege users can access the system. Oracle are aware of this as a problem as they have the following note:
Depending on these considerations and your downtime schedule, you can choose to schedule the recompilation of views independent of the rest of the CPU installation. If you do this, your system will continue to work; however, the CPU installation will not be complete until the view recompilation is completed.
There are two thoughts for me here, why does the documentation state users can access the system during the post-installation phase, when to fully patch the security holes you must have downtime on your RAC cluster. The second point is, that it is no wonder that hardly anyone is applying these CPU’s if they are requiring system unavailability. If you have made a large investment in RAC to improve your availability then to have this compromised by a security patch is not a nice choice.
You should not have to make the choice between availability and security but it seems like for now, you cannot have both.