What is Oracle Golden-Gate?
Oracle Golden-Gate is Oracle's strategic solution for real time data integration. Golden-Gate software enables mission critical systems to have continuous availability and access to real-time data. It offers a fast and robust solution for replicating transactional data between operational and analytical systems.
As a competitor to Oracle Golden-Gate, data replication products and solutions exist from other software companies and vendors. These are mainly storage replication solutions that provide fast point in time data restoration. The following is a list of the most common solutions available today:
• EMC SRDF and EMC RecoverPoint
• IBM PPRC and Global Mirror (known together as IBM Copy Services)
• Hitachi TrueCopy
• Hewlett-Packard Continuous Access (HP CA)
• Symantec Veritas Volume Replicator (VVR)
• DataCore SANsymphony and SANmelody
• FalconStor Replication and Mirroring
• Compellent Remote Instant Replay
Data replication techniques have improved enormously over the past 10 years and have always been a requirement in nearly every IT project in every industry.
Whether for Disaster Recovery (DR), High Availability (HA), Business Intelligence(BI), or even regulatory reasons, the requirements and expected performance have also increased, making the implementation of efficient and scalable data replication solutions a welcome challenge.
Oracle Golden-Gate evolution
Golden-Gate Software Inc was founded in 1995. Originating in San Francisco, the company was named after the famous Golden Gate Bridge by its founders, Eric Fish and Todd Davidson. The tried and tested product that emerged quickly became very popular within the financial industry. Originally designed for the fault tolerant Tandem computers, the resilient and fast data replication solution was in demand.
The banks initially used Golden-Gate software in their ATM networks for sending transactional data from high street machines to mainframe central computers. The data integrity and guaranteed zero data loss is obviously paramount and plays a key factor.
The key architectural properties of the product are as follows:
• Data is sent in "real time" with sub-second speed.
• Supports heterogeneous environments across different database and hardware types. "Transaction aware" maintaining its read-consistent and referential integrity between source and target systems.
• High performance with low impact; able to move large volumes of data very efficiently while maintaining very low lag times and latency.
• Flexible modular architecture.
• Reliable and extremely resilient to failure and data loss. No single point of failure or dependencies, and easy to recover.
Oracle Corporation acquired Golden-Gate Software in September 2009. Today there are more than 500 customers around the world using Golden-Gate technology for over 4000 solutions, realizing over $100 million in revenue for Oracle.
Oracle Golden-Gate solutions
Oracle Golden-Gate provides five data replication solutions:
1. High Availability
°° Live Standby for an immediate fail-over solution that can later re-synchronize with your primary source.
°° Active-Active solutions for continuous availability and transaction load distribution between two or more active systems.
2. Zero-Downtime Upgrades and Migrations
°° Eliminates downtime for upgrades and migrations.
3. Live Reporting
°° Feeding a reporting database so as not to burden the source production systems with BI users or tools.
4. Operational Business Intelligence (BI)
°° Real-time data feeds to operational data stores or data warehouses, directly or via Extract Transform and Load (ETL) tools.
5. Transactional Data Integration
°° Real-time data feeds to messaging systems for business activity monitoring, business process monitoring, and complex event processing.
°° Uses event-driven architecture and service-oriented architecture (SOA).
The following diagram shows the basic architecture for the various solutions available from Golden-Gate software:
So what makes Golden-Gate different from other data replication products?
The quick answer is the architecture. Golden-Gate can achieve heterogeneous and homogeneous-real-time transactional Change Data Capture and integration by decoupling itself from the database architecture. This in itself provides a performance boost as well as flexibility through its modular components.
A number of system architecture solutions are offered for data replication and synchronization:
• One-to-one (source to target)
• One-to-many (one source to many targets)
• Many to one (hub and spoke)
• Cascading
• Bi-directional (active active)
• Bi-directional (active passive)
No one configuration is better than another. The one you choose is largely dependent on your business requirements.One-to-One
By far the simplest and most common configuration is the "source to target". Here we are performing real-time or batch change data replication between two sites in a unidirectional fashion. This could be, for example, between a primary and standby site for Disaster Recovery (DR) or an OLTP to data warehouse for Business Intelligence (BI) and OLAP.
One-to-One architecture provides a data replication solution that offers the following key benefits:
• Live reporting
• Fastest possible recovery and switchover (when the target is synchronized with the source)
• Backup site that can be used for reporting
• Supports DDL replication
Due to its simplicity, the One-to-One architecture is referred to many times in this book to effectively demonstrate:
• Process configuration
• Data transformation
• Troubleshooting techniques
• Performance tuning tips and tricks
One-to-Many
Another popular GoldenGate configuration is the One-to-Many architecture. This architecture lends itself perfectly to provide two solutions. One data replication feed for reporting and one for backup and DR. The following example helps to illustrate the method.
One-to-Many architecture provides a data replication solution that offers the following key benefits:
• Dedicated site for Live reporting.
• Dedicated site for backup data from source database.
• Fastest possible recovery and switchover, when using a dedicated backup site. It minimizes logical data corruption as the backup database is separate from the read-write OLAP database.
The One-to-Many architecture is very flexible, given that it provides two solutions in one—a reporting and a standby database, both of which can have different table structures.
Many-to-One
The Many-to-One configuration comes into play for peripheral sites updating a central computer system representing a hub and spokes on a wheel. This scenario is common in all industries, from retail outlets taking customer orders to high street bank branches processing customer transactions. Ultimately, the data needs to make it to the central database ASAP and cannot become lost or corrupted. Golden-Gate's architecture lends itself perfectly to this scenario, as seen in the next example. Here we have three spoke sites sending data to the central hub site.
One important point to mention here is Conflict Handling. In a "hub and spoke" configuration, with concurrent updates taking place, data conflicts are highly likely to occur. Should the same database table row or field be updated by more than one source, on the target the conflict must be handled by Golden-Gate to allow either one of the transactions to succeed or to fail all.
Another "hub and spoke" solution includes the One-to-Many configuration. A typical example being the company head office sending data to its branches. Here, conflict handling is less of an issue.
Cascading
The cascading architecture offers data replication at n sites, originating from a single source. As the data flows from the originating source database, parts or all of it are "dropped off" at each site in a cascading fashion until the final target is populated. In the following example, we have one source (Site A) and three targets (Sites B, C and D). Intermediate Sites B and C have both source and target trails, whereas Site A has only a source and Site D only a target trail.
What data to replicate is configured by using Filters in the Golden-Gate parameter files at each target site, making the Cascade architecture one of the most powerful, yet complex configurations. Users at each site input data that can also be replicated to the next site.
Bi-directional (Active-Active)
The following is an example of an active-active configuration, where Site A sends changed data to Site B and vice versa. Again, Conflict Handling is an important consideration. A conflict is likely to occur in a bi-directional environment, where the same row or field is being updated at both sites. When the change is replicated, a conflict occurs. This needs to be resolved by GoldenGate based on the business rules, that is, should data from Site B overwrite Site A, or should both transactions fail?
Bi-directional (active-active) architecture provides a data replication solution that offers the following key benefits:
• High availability
• Transaction load distribution
• Performance scalability
Another key element to include in your configuration is Loop Detection. We do not want data changes going round in a loop, where Site A updates Site B, then Site B updates Site A, and so on.
Do not be put off by the Bi-directional architecture. When configured correctly, this architecture offers the most appropriate solution for global companies and organizations, allowing users in two centers, both sides of the globe to share the same system and data.
The active-active configuration is very different from the active-passive, which we discuss in the following section.
Bi-directional (Active-Passive)
The following is an example of an active-passive configuration, sometimes called "Live Standby", where Site A sends changed data to Site B only. You'll notice that the path from Site B to Site A is "grayed-out", suggesting that the data replication path can be re-enabled at short notice. This means that the GoldenGate processes exist and are configured, but have not been started. Bi-directional (active-passive) architecture provides a data replication solution that offers the following key benefits:
• Both sites have database open read-write
• Fastest possible recovery and switchover
• Reverse direction data replication ready
The active-passive configuration lends itself to being a DR solution, supporting a backup site should processes fail on the production site.
Installing and Preparing Golden-Gate
GoldenGate installation in the order specified as follows:
1. Downloading the software from the Oracle Website.
2. Unpacking the installation zip file.
3. Preparing the source and target systems.
4. Installing the software on source and target systems.
5. Preparing the source database.
6. Configuring the Manager process on the source and target systems.
7. Configuring the Extract process on the source system.
8. Configuring the Data Pump process on the source system.
9. Configuring the Replicate process on the target system.
10. Starting the Extract process.
11. Starting the Data Pump process.
12. Starting the Replicate process.
Oracle Golden-Gate is Oracle's strategic solution for real time data integration. Golden-Gate software enables mission critical systems to have continuous availability and access to real-time data. It offers a fast and robust solution for replicating transactional data between operational and analytical systems.
As a competitor to Oracle Golden-Gate, data replication products and solutions exist from other software companies and vendors. These are mainly storage replication solutions that provide fast point in time data restoration. The following is a list of the most common solutions available today:
• EMC SRDF and EMC RecoverPoint
• IBM PPRC and Global Mirror (known together as IBM Copy Services)
• Hitachi TrueCopy
• Hewlett-Packard Continuous Access (HP CA)
• Symantec Veritas Volume Replicator (VVR)
• DataCore SANsymphony and SANmelody
• FalconStor Replication and Mirroring
• Compellent Remote Instant Replay
Data replication techniques have improved enormously over the past 10 years and have always been a requirement in nearly every IT project in every industry.
Whether for Disaster Recovery (DR), High Availability (HA), Business Intelligence(BI), or even regulatory reasons, the requirements and expected performance have also increased, making the implementation of efficient and scalable data replication solutions a welcome challenge.
Oracle Golden-Gate evolution
Golden-Gate Software Inc was founded in 1995. Originating in San Francisco, the company was named after the famous Golden Gate Bridge by its founders, Eric Fish and Todd Davidson. The tried and tested product that emerged quickly became very popular within the financial industry. Originally designed for the fault tolerant Tandem computers, the resilient and fast data replication solution was in demand.
The banks initially used Golden-Gate software in their ATM networks for sending transactional data from high street machines to mainframe central computers. The data integrity and guaranteed zero data loss is obviously paramount and plays a key factor.
The key architectural properties of the product are as follows:
• Data is sent in "real time" with sub-second speed.
• Supports heterogeneous environments across different database and hardware types. "Transaction aware" maintaining its read-consistent and referential integrity between source and target systems.
• High performance with low impact; able to move large volumes of data very efficiently while maintaining very low lag times and latency.
• Flexible modular architecture.
• Reliable and extremely resilient to failure and data loss. No single point of failure or dependencies, and easy to recover.
Oracle Corporation acquired Golden-Gate Software in September 2009. Today there are more than 500 customers around the world using Golden-Gate technology for over 4000 solutions, realizing over $100 million in revenue for Oracle.
Oracle Golden-Gate solutions
Oracle Golden-Gate provides five data replication solutions:
1. High Availability
°° Live Standby for an immediate fail-over solution that can later re-synchronize with your primary source.
°° Active-Active solutions for continuous availability and transaction load distribution between two or more active systems.
2. Zero-Downtime Upgrades and Migrations
°° Eliminates downtime for upgrades and migrations.
3. Live Reporting
°° Feeding a reporting database so as not to burden the source production systems with BI users or tools.
4. Operational Business Intelligence (BI)
°° Real-time data feeds to operational data stores or data warehouses, directly or via Extract Transform and Load (ETL) tools.
5. Transactional Data Integration
°° Real-time data feeds to messaging systems for business activity monitoring, business process monitoring, and complex event processing.
°° Uses event-driven architecture and service-oriented architecture (SOA).
The following diagram shows the basic architecture for the various solutions available from Golden-Gate software:
Oracle Golden-Gate architecture
So what makes Golden-Gate different from other data replication products?
The quick answer is the architecture. Golden-Gate can achieve heterogeneous and homogeneous-real-time transactional Change Data Capture and integration by decoupling itself from the database architecture. This in itself provides a performance boost as well as flexibility through its modular components.
A number of system architecture solutions are offered for data replication and synchronization:
• One-to-one (source to target)
• One-to-many (one source to many targets)
• Many to one (hub and spoke)
• Cascading
• Bi-directional (active active)
• Bi-directional (active passive)
No one configuration is better than another. The one you choose is largely dependent on your business requirements.One-to-One
By far the simplest and most common configuration is the "source to target". Here we are performing real-time or batch change data replication between two sites in a unidirectional fashion. This could be, for example, between a primary and standby site for Disaster Recovery (DR) or an OLTP to data warehouse for Business Intelligence (BI) and OLAP.
One-to-One architecture provides a data replication solution that offers the following key benefits:
• Live reporting
• Fastest possible recovery and switchover (when the target is synchronized with the source)
• Backup site that can be used for reporting
• Supports DDL replication
Due to its simplicity, the One-to-One architecture is referred to many times in this book to effectively demonstrate:
• Process configuration
• Data transformation
• Troubleshooting techniques
• Performance tuning tips and tricks
One-to-Many
Another popular GoldenGate configuration is the One-to-Many architecture. This architecture lends itself perfectly to provide two solutions. One data replication feed for reporting and one for backup and DR. The following example helps to illustrate the method.
One-to-Many architecture provides a data replication solution that offers the following key benefits:
• Dedicated site for Live reporting.
• Dedicated site for backup data from source database.
• Fastest possible recovery and switchover, when using a dedicated backup site. It minimizes logical data corruption as the backup database is separate from the read-write OLAP database.
The One-to-Many architecture is very flexible, given that it provides two solutions in one—a reporting and a standby database, both of which can have different table structures.
Many-to-One
The Many-to-One configuration comes into play for peripheral sites updating a central computer system representing a hub and spokes on a wheel. This scenario is common in all industries, from retail outlets taking customer orders to high street bank branches processing customer transactions. Ultimately, the data needs to make it to the central database ASAP and cannot become lost or corrupted. Golden-Gate's architecture lends itself perfectly to this scenario, as seen in the next example. Here we have three spoke sites sending data to the central hub site.
One important point to mention here is Conflict Handling. In a "hub and spoke" configuration, with concurrent updates taking place, data conflicts are highly likely to occur. Should the same database table row or field be updated by more than one source, on the target the conflict must be handled by Golden-Gate to allow either one of the transactions to succeed or to fail all.
Another "hub and spoke" solution includes the One-to-Many configuration. A typical example being the company head office sending data to its branches. Here, conflict handling is less of an issue.
The cascading architecture offers data replication at n sites, originating from a single source. As the data flows from the originating source database, parts or all of it are "dropped off" at each site in a cascading fashion until the final target is populated. In the following example, we have one source (Site A) and three targets (Sites B, C and D). Intermediate Sites B and C have both source and target trails, whereas Site A has only a source and Site D only a target trail.
What data to replicate is configured by using Filters in the Golden-Gate parameter files at each target site, making the Cascade architecture one of the most powerful, yet complex configurations. Users at each site input data that can also be replicated to the next site.
Bi-directional (Active-Active)
The following is an example of an active-active configuration, where Site A sends changed data to Site B and vice versa. Again, Conflict Handling is an important consideration. A conflict is likely to occur in a bi-directional environment, where the same row or field is being updated at both sites. When the change is replicated, a conflict occurs. This needs to be resolved by GoldenGate based on the business rules, that is, should data from Site B overwrite Site A, or should both transactions fail?
Bi-directional (active-active) architecture provides a data replication solution that offers the following key benefits:
• High availability
• Transaction load distribution
• Performance scalability
Another key element to include in your configuration is Loop Detection. We do not want data changes going round in a loop, where Site A updates Site B, then Site B updates Site A, and so on.
Do not be put off by the Bi-directional architecture. When configured correctly, this architecture offers the most appropriate solution for global companies and organizations, allowing users in two centers, both sides of the globe to share the same system and data.
The active-active configuration is very different from the active-passive, which we discuss in the following section.
Bi-directional (Active-Passive)
The following is an example of an active-passive configuration, sometimes called "Live Standby", where Site A sends changed data to Site B only. You'll notice that the path from Site B to Site A is "grayed-out", suggesting that the data replication path can be re-enabled at short notice. This means that the GoldenGate processes exist and are configured, but have not been started. Bi-directional (active-passive) architecture provides a data replication solution that offers the following key benefits:
• Both sites have database open read-write
• Fastest possible recovery and switchover
• Reverse direction data replication ready
The active-passive configuration lends itself to being a DR solution, supporting a backup site should processes fail on the production site.
Installing and Preparing Golden-Gate
1. Downloading the software from the Oracle Website.
2. Unpacking the installation zip file.
3. Preparing the source and target systems.
4. Installing the software on source and target systems.
5. Preparing the source database.
6. Configuring the Manager process on the source and target systems.
7. Configuring the Extract process on the source system.
8. Configuring the Data Pump process on the source system.
9. Configuring the Replicate process on the target system.
10. Starting the Extract process.
11. Starting the Data Pump process.
12. Starting the Replicate process.
No comments:
Post a Comment