Previous Table of Contents Next


Chapter 33
Working with Very Large Databases

by Meghraj Thakkar

•  Oracle8 and High-End Database Systems
•  Oracle8 Features for Large Databases
•  Tuning for Very Large Databases
•  Operation of a Large Database

Very large databases (VLDBs) are in the range of terabytes and before long will become petabytes. The task of managing and maintaining databases increases at least proportionally, if not exponentially, with their size. With the increase in the data, the tasks such as backups, recovery, data loads, purging, and data reorganization become more and more complex and time consuming. Most of the customers who have large databases installed usually have additional requirements such as high availability, high response time, and high scalability.

Introduction

There are several design issues that need to be considered when creating very large databases in order to take full advantage of the features provided by Oracle8. This will also have a major impact on the management and performance of VLDBs.

This chapter addresses the various issues related to designing and managing VLDBs under Oracle8 or migrating them from Oracle7. We will also discuss the different features provided by Oracle8 from a VLDB perspective. Some of these features are

  Table partitioning
  Index partitioning
  Parallel DML (PDML)
  Bitmap indexes
  Backup and recovery using Recovery Manager (RMAN)
  Object technology
  Data cartridges

Oracle8 and High-End Database Systems

You should be very careful when dealing with high-end applications. These applications tend to test the database limits, and one has to be very creative to make these applications run optimally. High-end applications have one or more of the following characteristics:

  A very large database ranging in size from several gigabytes to terabytes.
  An extremely high number of concurrent users in the tens of thousands.
  A database system that should provide high availability (24×7×52)
  A highly heterogeneous environment that uses a number of components to make the system work
  Systems with heavy transactional loads

Most of the high-end database environments are implemented by using mainframes and non-relational databases, because even though relational database systems are easy to use and manage, they are not able to handle the types of load imposed by high-end systems.

Oracle8 provides several features that allow it to work in an acceptable manner in such high-end environments. These features are discussed in the following sections.

Partitioned Tables and Indexes

Oracle8’s data partitioning features can simplify the demanding tasks involved when dealing with large amounts of data. Table 33.1 shows some problems faced in VLDB systems and how Oracle8 attempts to resolve those problems.

Table 33.1. Problems in VLDB systems and Oracle8’s solutions to them.

Problem Oracle8’s Solution

Performing full table scans on a large table can degrade performance. Oracle8 provides the ability to perform parallel DMLs and query individual partitions. The optimizer is “partition aware” and eliminates searches from partitions that do not apply to a particular SQL query.
High availability is required in a system. Mission-critical applications depend on atable that is not available due to media failure. Oracle8 allows you to recover at the partition-level and minimizes the downtime.
VLDBs are difficult to manage. Oracle8’s partition-level management simplifies a lot of administrative tasks.

Managing a Large Number of Users

Oracle8 and Net8 provide four possible solutions to achieve more concurrent connections per Oracle8 instance:

  MultiThreaded Server (MTS)—MTS pools connections and does not allocate a single thread per connection. As a result, it avoids stack overflow and memory allocation errors that would occur from a dedicated connection per thread. The multithreaded server configuration allows many user threads to share very few server threads. The user threads connect to a dispatcher process that routes client requests to the next available server thread, thereby supporting more users.
  Connection Manager—Concentrates multiple clients into a single multiplexed data connection, even if the clients use different protocols. It is ideal for those users who need to use the applications continuously.
  Connection pooling—Places idle users in a suspended mode and reassigns their physical connections until they become active again. It is ideal for users who need to be logged on all the time, but don’t need to really use the application.
  Orastack—Allows customers to change the amount of the default reserved stack space used by each Oracle thread. Use this utility with caution.

Deferred Transaction Support

In VLDB systems, there are usually several tasks and applications that can benefit from deferring the execution of low-priority transactions. Third-party solutions like Transaction Processing Monitor (TP Monitor) or Message-Oriented Middleware (MOM) have problems because they are not able to protect the deferred transactions against system failure, and there is additional overhead associated with them. Oracle8 provides a new feature—advanced queuing—which is a database-centric solution that protects deferred transactions.

Parallel Query Enhancements

Oracle8 provides several enhancements to parallel query:

  Parallel DML (insert, update, and delete).
  All queries can be run in parallel, including those based on index scans.
  Partition-aware optimizer.
  Intra-partition parallelism.
  Inter-partition parallelism.

Parallel Server Enhancements

Oracle8 provides several enhancements to Parallel Server:

  Improved distributed lock manager.
  Reverse-key indexes for minimizing data block contention.
  Improved application failover provides high availability by transparently migrating connections from a failed node to a working node.


Previous Table of Contents Next
Используются технологии uCoz