Performance of Large Oracle RAC Databases on MPP Machines
Unlike a cluster where only a few nodes are interconnected, massively parallel processing (MPP) systems allow a large number of nodes to be interconnected. There are MPP systems with more than a thousand nodes. Each of these nodes is a separate system that has its own CPU, disks, controllers, memory, and internal system buses to form a “loosely coupled” architecture that shares nothing. All of these nodes are connected through a high-speed, high-bandwidth interconnection. Each node has a separate copy of the operating system. In the MPP architecture, Oracle is installed in RAC mode.
Each Oracle instance on each node is responsible for all the resources it contains, has a view of the entire database, and can find out which node has a lock on any part of the database. If you need something that is locked by another node, it will require a ping between instances, where the other instance would have to write all the changes it has made to disk. A typical database query would go against a node; the data would be fetched from the memory or disks of any appropriate node and travel across the interconnect to return the data to the requesting node. Each node controls its own set of disks and can take over another set if a node fails. Therefore, all nodes can be configured to be primary or a combination of primary and secondary. The database files are placed on the primary nodes, while the secondary nodes provide the necessary redundancy to take over and replace the primary nodes, should the latter fail.
The biggest drawback of MPP architectures is not the architecture itself, but the application design flaws that exist in a typical implementation. Most application designers are well versed in deploying applications on SMP machines. However, MPP architectures require a complete paradigm shift in terms of analyzing what data should be placed on which node in order to reduce data sharing and inter-instance ping between nodes. Since each node is very independent in an architecture that does not share anything, any situation that causes nodes to exchange resources on a large scale due to data exchange generates high traffic through the interconnect. This situation is causing severe performance issues in Oracle RAC databases. Oracle RAC database performance depends primarily on incorrect hardware architecture configuration. You must consider the hardware architecture and operating system implementation before processing the Oracle Real Application Cluster configuration. Otherwise, you will have a lot of problems after deployment, and changing hardware configuration requires downtime of your high-availability databases.
