Oracle GoldenGate Microservices Architecture - 1
Data Replication techniques has gained importance in the market with the need of maintaing production data in other databases like Postgre, Mysql etc.. to avoid heavy license fee charged by top rdbms providers like Oracle, Sql Server .Goldengate is one of the most famous and popular software which can be used for data replication between homogenous and hetrogenous databases.
GoldenGate supports two types of architectures :
Classic Architecture
Microservice Architecture
CLASSIC ARCHITECTURE
It is first architecture supported by the Goldengate when it was introduced 20 years ago. Goldengate Classic architecture drawn many advantages in the market when customers are facing problems when replicating data between heterogenous databases.
Classic Architecture has below components:
Manager - Is the master process running in the Goldengate which takes care of all other slave process like Extract, Datapump, Replicat and Collector. Without Manager, the other Other processes like Extract,Replicat and Datapump will not start in Goldengate
Extract - It takes care of capturing data from redologs of the source database.
Datapump - It picks the trail files generated by extract and transports to the destination server.
Collector - It collects all the trail files tranferred by datapump process and places them in the sequential order in the destination server.
Replicat - It will apply the trail files collected by Collector to the destination database.
Inspite of addressing many data replication issues by Classic Architecture, It has few drawbacks.
Some of them are:
1) Restriction to Goldengate software installation in the database servers.
2) Increase in number of Extract and Replicat processes when more than one database is configured for Goldengate replication at each end which makes administration more difficult.
3) Lack of tools/utilities to monitor the performance of Extract,Pump and Replicat processes.
To overcome the above disadvantages, Oracle has introduced another architecture in Goldengate called as Microservices Architecture, which is built based on RESTful API's.
REST API (Representational State Transfer Application Programming Interface) is an architectural style and a set of principles for designing networked applications. It is commonly used in web development to create scalable and interoperable systems. The core concept of REST is based on the representation of resources and the manipulation of these resources through standard HTTP methods, such as GET, POST, PUT, PATCH, and DELETE. Resources can be anything that can be identified and managed, such as data objects, files, or services.
Key principles of a REST API include:
Stateless : The server does not store any client state. Each request from a client contains all the information needed to understand and process the request.
Client-Server : The client and server are separate entities that communicate over HTTP. The client is responsible for the user interface and user experience, while the server handles the business logic and data storage.
Uniform Interface : The API should have a consistent and standardized set of methods and conventions for accessing and manipulating resources. This includes using standard HTTP methods and status codes, as well as employing self-descriptive messages through the use of hypermedia.
Resource Identification: Each resource within the API is uniquely identified using a URI (Uniform Resource Identifier) or URL (Uniform Resource Locator). Clients can Interact with these resources by making requests to their respective URLs.
State Transfer : The representation of a resource can be transferred between the client and server, allowing clients to manipulate resources by sending representations (such as JSON or XML) in requests and receiving representations in responses.
MICROSERVICES ARCHITECTURE
Microservices Architecure differs Classic Architecture on the basis of components included in it.
The main difference between Classic and Microservices architecture is the interface and the way we interact with them and also seggregation of components based on the services they provide.
The components of Microservices Architecture are :
-Service Manager
-Admin Server
-Distribution Server
-Receiver Server
-Performance Metric Server
1)Service Manager :
The ServiceManager is the watchdog process for the architecture on each server in the replication enviornment. Ideally, you should only have one of these processes running. This process can be configured to run in one of 3 ways. These ways are:
-Manually
-As a daemon
-Integrated with XAG
While the ServiceManager is running, this process will be the main entry point into the Oracle GoldenGate environment. During the configuration process, you will be asked to assign ports for each the servers to run on. The ServiceManager will be the first port you assign. From the HTML5 page of the ServiceManager, you will be able to see all of your deployment homes and associated servers.
In a Microservices Architecture, the service manager would typically be responsible for tasks such as:
Service Discovery : Providing a mechanism for services to register and discover each other dynamically, allowing for service-to-service communication.
Load Balancing : Distributing incoming requests across multiple instances of a service to ensure efficient resource utilization and high availability.
Routing and API Gateway : Handling request routing and acting as an entry point for external clients, often providing additional features like authentication, authorization, and caching.
Health Monitoring : Collecting and monitoring the health and performance metrics of individual services to detect issues and perform automatic scaling or recovery.
Dynamic Configuration Management: Managing and distributing configuration parameters to individual services, allowing for runtime configuration changes without service restarts.
2)Admin Server:
The Administration Server supervises, administers, manages, and monitors processes operating within an Oracle GoldenGate deployment for both active and inactive processes.The Administration Server operates as the central control entity for managing the replication components in your Oracle GoldenGate deployments. You use it to create and manage your local Extract and Replicat processes without having to have access to the server where Oracle GoldenGate is installed. The key feature of the Administration Server is the REST API service Interface that can be accessed from any HTTP or HTTPS client, such as the Microservices Architecture service interfaces or other clients like Perl and Python.
In addition, the Admin Client can be used to make REST API calls to communicate directly with the Administration Server. The Administration Server includes an embedded web application that you can use directly with any web browser and does not require any client software installation.
Use the Administration Server to create and manage:
1)Extract and Replicat processes
Add, alter, and delete
Register and unregister
Start and stop
Review process information, statistics, reports, and status including LAG and checkpoints
Retrieve the report and discard files
2)Configuration (parameter) files
3)Checkpoint, trace, and heartbeat tables
4)Supplemental logging for procedural replication, schema, and tables
5)Tasks both custom and standard, such as auto-restart and purge trails
6)Credential stores
7)Encryption keys (MASTERKEY)
8)Add users and assign their roles
3)Distribution Server:
The DistributionServer is the replacement for the Data Pump Extract. The service performs all the same functionality as the Data Pump Extract with the exception of transformations. Besides providing all the same functionality, you also get a visual representation of where your trail file is being read from and shipped to. It is very clear to see from the overview page of this services. As you dig into the details of the DistributionServer, you can see the statistics on what is being read and written to trail files and adjust TCP/IP items within the distribution path.
4)ReceiverServer:
The ReceiverServer is the replacement for the Collectors. The whole job of the ReceiverServer is to accept transmissions from the DistributionServer and write out the remote trail files. From the overview page of this service, you can clearly see where the information is coming from and what trail it is writing to. Just like the DistributionServer, if you look at the details of this service you can see alot of useful information.
5)Performance Metrics Server:
Finally, we (Oracle) have provide a real-time performance monitoring services with Oracle GoldenGate. Before you get all happy about having a new way to monitor performance, you must have a license for the Oracle Managment Pack for GoldenGate before you can use the GUI or associated metric APIs. If you have that in place, there is so much performance metric information you can retrieve and use in both the GUI and APIs. I would encourage you to take a look.
GoldenGate Source and Destination setup interact via HTTP(UnSecured) and HTTPS(Secured) Network
HTTP (Hypertext Transfer Protocol) and HTTPS (Hypertext Transfer Protocol Secure) are both protocols used for communication over the internet. The main difference between the two lies in the security aspect:
HTTP: HTTP is the standard protocol used for transmitting data between a web server and a web browser. It operates over port 80 and sends data in plain text format. This means that any information transmitted using HTTP, including passwords, credit card details, and other sensitive data, can be intercepted and read by anyone who has access to the network.
HTTPS: HTTPS is the secure version of HTTP. It incorporates encryption to protect the integrity and confidentiality of data transmitted between the web server and the web browser. HTTPS uses SSL/TLS (Secure Sockets Layer/Transport Layer Security) protocols to encrypt the data, making it extremely difficult for anyone to eavesdrop or tamper with the information in transit. HTTPS operates over port 443.
Comments
Post a Comment