From ba1bc0eab242d3b927e44f5bc7dd792958f3d28b Mon Sep 17 00:00:00 2001 From: Josh Earlenbaugh Date: Tue, 14 Jan 2025 10:36:16 -0500 Subject: [PATCH 01/12] Started porting over from bjorked branch --- .../pgd/5.6/overview/basic-architecture.mdx | 74 +++++-------------- 1 file changed, 17 insertions(+), 57 deletions(-) diff --git a/product_docs/docs/pgd/5.6/overview/basic-architecture.mdx b/product_docs/docs/pgd/5.6/overview/basic-architecture.mdx index 415395b1885..0778bc50f67 100644 --- a/product_docs/docs/pgd/5.6/overview/basic-architecture.mdx +++ b/product_docs/docs/pgd/5.6/overview/basic-architecture.mdx @@ -1,64 +1,24 @@ --- -title: "PGD Overview - PGD's basic architecture" -navTitle: Basic architecture -description: An overview of EDB Postgres Distributed's basic architecture, including groups, multiple masters, mesh topology, logical replication, connection management, and high availability. +title: "PGD overview" +description: An overview of EDB Postgres Distributed architecture and performance characteristics deepToC: true redirects: - bdr --- -EDB Postgres Distributed (PGD) provides multi-master replication and data distribution with advanced conflict management, data-loss protection, and [throughput up to 5X faster than native logical replication](https://www.enterprisedb.com/blog/performance-improvements-edb-postgres-distributed). It also enables distributed Postgres clusters with high availability up to five 9s. +
+
+
+Read about why PostgreSQL is better when it’s distributed with EDB Postgres Distributed in Distributed PostgreSQL:The Key to Always On Database Availability +
+
+
+
+PGD Free Trial +
+
+Contact Sales +
+
+
-PGD provides loosely coupled, multimaster logical replication using a mesh topology. This means that you can write to any server and the changes are sent directly, row by row, to all the other servers that are part of the same PGD group. - -By default, PGD uses asynchronous replication, applying changes on the peer nodes only after the local commit. Multiple synchronous replication options are also available. - -## Basic architecture - -### Multiple groups - -A PGD node is a member of at least one *node group*. In the most basic architecture, there's a single node group for the whole PGD cluster. - -### Multiple masters - -Each node (database) participating in a PGD group both receives changes from other members and can be written to directly by the user. - -This is distinct from hot or warm standby, where only one master server accepts writes and all the other nodes are standbys that replicate either from the master or from another standby. - -You don't have to write to all the masters all of the time. A frequent configuration directs writes mostly to just one master called the [write leader](../terminology/#write-leader). - -### Asynchronous, by default - -Changes made on one PGD node aren't replicated to other nodes until they're committed locally. As a result, the data isn't exactly the same on all nodes at any given time. Some nodes have data that hasn't yet arrived at other nodes. PostgreSQL's block-based replication solutions default to asynchronous replication as well. In PGD, there are multiple masters and, as a result, multiple data streams. So data on different nodes might differ even when `synchronous_commit` and `synchronous_standby_names` are used. - -### Mesh topology - -PGD is structured around a mesh network where every node connects to every other node, and all nodes exchange data directly with each other. There's no forwarding of data in PGD except in special circumstances, such as adding and removing nodes. Data can arrive from outside the EDB Postgres Distributed cluster or be sent onward using native PostgreSQL logical replication. - -### Logical replication - -Logical replication is a method of replicating data rows and their changes based on their replication identity (usually a primary key). We use the term *logical* in contrast to *physical* replication, which uses exact block addresses and byte-by-byte replication. Index changes aren't replicated, thereby avoiding write amplification and reducing bandwidth. - -Logical replication starts by copying a snapshot of the data from the source node. Once that's done, later commits are sent to other nodes as they occur in real time. Changes are replicated without executing SQL again, so the exact data written is replicated quickly and accurately. - -Nodes apply data in the order in which commits were made on the source node, ensuring transactional consistency is guaranteed for the changes from any single node. Changes from different nodes are applied independently of other nodes to ensure the rapid replication of changes. - -Replicated data is sent in binary form when it's safe to do so. - - -### Connection management - -[Connection management](../routing) leverages consensus-driven quorum to determine the correct connection endpoint in a semi-exclusive manner to prevent unintended multi-node writes from an application. This approach reduces the potential for data conflicts. The node selected as the correct connection endpoint at any point in time is referred to as the [write leader](../terminology/#write-leader). - -[PGD Proxy](../routing/proxy) is the tool for application connection management provided as part of EDB Postgres Distributed. - -### High availability - -Each master node can be protected by one or more standby nodes, so any node that goes down can be quickly replaced and continue. Each standby node is a logical standby node. -(Postgres physical standbys aren't supported by PGD.) - -Replication continues between currently connected nodes even if one or more nodes are currently unavailable. When the node recovers, replication can restart from where it left off without missing any changes. - -Nodes can run different release levels, negotiating the required protocols to communicate. As a result, EDB Postgres Distributed clusters can use rolling upgrades, even for [major versions](../upgrades/upgrading_major_rolling/) of database software. - -DDL is replicated across nodes by default. If you want, you can control DDL execution to allow rolling application upgrades. From f04de89dadcdbc164fcb49454a5d3ea8ce05aefd Mon Sep 17 00:00:00 2001 From: Josh Earlenbaugh Date: Tue, 14 Jan 2025 10:41:24 -0500 Subject: [PATCH 02/12] Ported in new basic arch section --- .../pgd/5.6/overview/basic-architecture.mdx | 78 +++++++++++++++++++ 1 file changed, 78 insertions(+) diff --git a/product_docs/docs/pgd/5.6/overview/basic-architecture.mdx b/product_docs/docs/pgd/5.6/overview/basic-architecture.mdx index 0778bc50f67..71ea86b6298 100644 --- a/product_docs/docs/pgd/5.6/overview/basic-architecture.mdx +++ b/product_docs/docs/pgd/5.6/overview/basic-architecture.mdx @@ -22,3 +22,81 @@ Read about why PostgreSQL is better when it’s distributed with EDB Postgres Di +## Architectural overview + +EDB Postgres Distributed (PGD) is a distributed database solution that extends PostgreSQL's capabilities, enabling highly available and fault-tolerant database deployments across multiple nodes. +PGD provides data distribution with advanced conflict management, data-loss protection, high availability up to five 9’s, and throughput up to 5X faster than native logical replication. + +PGD is built on a multi-master foundation (Bi-directional replication, or BDR) which is then optimized for performance and availability through PGD Proxy. +PGD proxy ensures lower contention and conflict through the use of a write leader, and for each proxy instance a single endpoint automatically addresses all the data nodes in a group, removing the need for clients to round robin multi-host connection strings. +[Raft](https://en.wikipedia.org/wiki/Raft_(algorithm)) is implemented to help the system make important decisions, like deciding which node is the Raft election leader and which node is the write leader. + +### High-level architecture + +At the highest level, PGD comprises two main components: Bi-Directional Replication (BDR) and PGD-proxy. +BDR is a Postgres extension that enables a multi-master replication mesh between different BDR-enabled Postgres instances/nodes. +[PGD proxy](../routing) sends requests to the write leader—ensuring a lower risk of conflicts between nodes. + +![Diagram showing 3 application nodes, 3 proxy instances, and 3 PGD nodes. Traffic is being directed from each of the proxy instances to the write leader node.](./img/always_on_1x3_updated.png) + +Changes are replicated directly, row-by-row between all nodes. +[Logical replication](../terminology/#logical-replication) in PGD is asynchronous by default, so only eventual consistency is guaranteed (within seconds usually). +However, [commit scope](../durability/commit-scopes) options offer stronger consistency and durability guarantees via [CAMO](/pgd/latest/durability/camo/), [group](../durability/group-commit) and [synchronous](../durability/synchronous_commit) commits. + +The Raft algorithm provides a mechanism for [electing](../routing/raft/04_raft_elections_in_depth/) leaders (both Raft leader and write leader), deciding which nodes should be added or subtracted from the cluster, and generally ensuring that the distributed system remains consistent and fault-tolerant, even in the face of node failures. + +### Architectural elements + +PGD comprises several key architectural elements that work together to provide its distributed database solution: + + - **PGD nodes**: These are individual Postgres instances that store and manage data. They are the basic building blocks of a PGD cluster. + + - **Groups**: PGD nodes are organized into [groups](../node_management/groups_and_subgroups), which enhance manageability and high availability. Each group can contain multiple nodes, allowing for redundancy and failover within the group. Groups facilitate organized replication and data consistency among nodes within the same group and across different groups. Each group has its own write leader. + + - **Replication mechanisms**: PGD's replication mechanisms include Bi-Directional Replication (BDR) for efficient replication across nodes, enabling multi-master replication. BDR supports asynchronous replication by default, but can be configured for varying levels of synchronicity, such as [Group Commit](../durability/group-commit) or [Synchronous Commit](../durability/synchronous_commit), to enhance data durabiltiy and consistency. + + - **Monitoring tools**: To monitor performance, health, and usage with PGD, you can utilize its [built-in command-line interface](../cli) (CLI), which offers several useful commands. For instance, the `pgd show-nodes` command provides a summary of all nodes in the cluster, including their state and status. The `pgd check-health` command checks the health of the cluster, reporting on node accessibility, replication slot health, and other critical metrics. The `pgd show-events` command lists significant events like background worker errors and node membership changes, which helps in tracking the operational status and issues within the cluster. Furthermore, the BDR extension allows for monitoring your cluster using SQL using the [`bdr.monitor`](../security/pgd-predefined-roles/#bdr_monitor) role. + +#### Node types + +All nodes in PGD are effectively data nodes. They vary only in their purpose in the cluster. + + - **[Data nodes](../node_management/node_types/#data-nodes)**: Store and manage data, handle read and write operations, and participate in replication. + +There are then three types of node which, although built like a data node, have a specific purpose. These are: + + - **[Subscriber-only nodes](../node_management/subscriber_only/#subscriber-only-nodes)**: Subscribe to changes from data nodes for read-only purposes, used in reporting or analytics. + + - **[Witness nodes](../node_management/witness_nodes/)**: Participate in concensus proceses without storing data, aiding in achieving quorum and maintaining high availability. + + - **[Logical standby nodes](../node_management/logical_standby_nodes/)**: Act as standby nodes that can be promoted to data nodes if needed, ensuring high availability and disaster recovery. + +#### Node roles + +Data nodes in a group can also take on particular roles to enable particular features. +These roles are transient and can be transferred to any other capable node in the group if needed. +These roles can include: + + - **[Raft leader]()**: Arbitrates and manages consensus between a group's nodes. + + - **[Write leader](../terminology/#write-leader)**: Receives all write operations from PGD Proxy. + +### Architectural Flexibility + +Postgres Distributed (PGD) offers flexible options with how its architecture can be deployed, maintained, and scaled to meet various performance, availability, and compliance needs. + +PGD supports rolling maintenance, including blue/green deployments for both Postgres upgrades and other system or application-level changes. This ensures that the database remains available during routine tasks such as minor or major version upgrades, schema changes, and vacuuming operations. The system seamlessly switches between active database versions, achieving zero-downtime. + +PGD provides automatic failover to ensure high availability. If a node in the cluster becomes unavailable, another node automatically takes over its responsibilities, minimizing downtime. Additionally, PGD includes self-healing capabilities, where nodes that have failed or disconnected can automatically reconnect to the cluster and resume normal operations once the issue is resolved. + +PGD allows for selective replication, enabling users to replicate only a subset of data to specific nodes. This feature can be used to optimize performance by reducing unnecessary data traffic between nodes or to meet regulatory requirements, such as geographical data restrictions. For instance, a healthcare application might only replicate patient data within a specific region to comply with local data privacy laws. + +With commit scopes, PGD also provides configurable durability. Accordingly, durability can be increased from the default asynchronous behavior and tuned using various configurable commit scopes: + +- Synchronous Commit: Works a lot like PostgreSQL’s synchronous_commit option in its underlying operation—requires writing to at least one other node at COMMIT time, but can be tuned to require all nodes. + +- CAMO (Commit at most once): Works by tracking each transaction with a unique ID and using a pair of nodes to confirm the transaction's outcome, ensuring the application knows whether to retry the transaction or not. + +- Group Commit: An experimental commit scope, the goal of which is to protect against data loss in case of single node failures of temporary outages by requiring more than one PGD node to successfully confirm a transaction at COMMIT time. + +- Lag Control: If replication is running outside of set limits (taking too long for another node to be replicated to), a delay is injected into the node that originally received the transaction, slowing things down until other nodes have caught up. \ No newline at end of file From 1ad24afb049b069c4c2db6a36cba60836efa7bf2 Mon Sep 17 00:00:00 2001 From: Josh Earlenbaugh Date: Tue, 14 Jan 2025 11:34:59 -0500 Subject: [PATCH 03/12] Fixed images --- .../images => pgd/5.6/overview/img}/always_on_1x3_updated.png | 0 .../postgres_for_kubernetes/1/images/always_on_1x3_updated.png | 3 +++ 2 files changed, 3 insertions(+) rename product_docs/docs/{postgres_distributed_for_kubernetes/1/images => pgd/5.6/overview/img}/always_on_1x3_updated.png (100%) create mode 100644 product_docs/docs/postgres_for_kubernetes/1/images/always_on_1x3_updated.png diff --git a/product_docs/docs/postgres_distributed_for_kubernetes/1/images/always_on_1x3_updated.png b/product_docs/docs/pgd/5.6/overview/img/always_on_1x3_updated.png similarity index 100% rename from product_docs/docs/postgres_distributed_for_kubernetes/1/images/always_on_1x3_updated.png rename to product_docs/docs/pgd/5.6/overview/img/always_on_1x3_updated.png diff --git a/product_docs/docs/postgres_for_kubernetes/1/images/always_on_1x3_updated.png b/product_docs/docs/postgres_for_kubernetes/1/images/always_on_1x3_updated.png new file mode 100644 index 00000000000..05c7072cd87 --- /dev/null +++ b/product_docs/docs/postgres_for_kubernetes/1/images/always_on_1x3_updated.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ce8008b5cfb11f454d792c48c00f824eda8efe03dcd2e3fb9a32723abc6ab26a +size 12341 From c1227c31f7e421190fd43f784803c6956feec4e0 Mon Sep 17 00:00:00 2001 From: Josh Earlenbaugh Date: Tue, 14 Jan 2025 11:42:54 -0500 Subject: [PATCH 04/12] Small changes. --- .../pgd/5.6/overview/basic-architecture.mdx | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/product_docs/docs/pgd/5.6/overview/basic-architecture.mdx b/product_docs/docs/pgd/5.6/overview/basic-architecture.mdx index 71ea86b6298..1d8cb0d3695 100644 --- a/product_docs/docs/pgd/5.6/overview/basic-architecture.mdx +++ b/product_docs/docs/pgd/5.6/overview/basic-architecture.mdx @@ -1,6 +1,6 @@ --- -title: "PGD overview" -description: An overview of EDB Postgres Distributed architecture and performance characteristics +title: "Basic Architecture" +description: An overview of EDB Postgres Distributed architecture deepToC: true redirects: - bdr @@ -22,7 +22,6 @@ Read about why PostgreSQL is better when it’s distributed with EDB Postgres Di -## Architectural overview EDB Postgres Distributed (PGD) is a distributed database solution that extends PostgreSQL's capabilities, enabling highly available and fault-tolerant database deployments across multiple nodes. PGD provides data distribution with advanced conflict management, data-loss protection, high availability up to five 9’s, and throughput up to 5X faster than native logical replication. @@ -31,13 +30,13 @@ PGD is built on a multi-master foundation (Bi-directional replication, or BDR) w PGD proxy ensures lower contention and conflict through the use of a write leader, and for each proxy instance a single endpoint automatically addresses all the data nodes in a group, removing the need for clients to round robin multi-host connection strings. [Raft](https://en.wikipedia.org/wiki/Raft_(algorithm)) is implemented to help the system make important decisions, like deciding which node is the Raft election leader and which node is the write leader. -### High-level architecture +## High-level architecture At the highest level, PGD comprises two main components: Bi-Directional Replication (BDR) and PGD-proxy. BDR is a Postgres extension that enables a multi-master replication mesh between different BDR-enabled Postgres instances/nodes. [PGD proxy](../routing) sends requests to the write leader—ensuring a lower risk of conflicts between nodes. -![Diagram showing 3 application nodes, 3 proxy instances, and 3 PGD nodes. Traffic is being directed from each of the proxy instances to the write leader node.](./img/always_on_1x3_updated.png) +![Diagram showing 3 application nodes, 3 proxy instances, and 3 PGD nodes. Traffic is being directed from each of the proxy instances to the write leader node.](/docs/product_docs/docs/pgd/5.6/overview/img/always_on_1x3_updated.png) Changes are replicated directly, row-by-row between all nodes. [Logical replication](../terminology/#logical-replication) in PGD is asynchronous by default, so only eventual consistency is guaranteed (within seconds usually). @@ -45,7 +44,7 @@ However, [commit scope](../durability/commit-scopes) options offer stronger cons The Raft algorithm provides a mechanism for [electing](../routing/raft/04_raft_elections_in_depth/) leaders (both Raft leader and write leader), deciding which nodes should be added or subtracted from the cluster, and generally ensuring that the distributed system remains consistent and fault-tolerant, even in the face of node failures. -### Architectural elements +## Architectural elements PGD comprises several key architectural elements that work together to provide its distributed database solution: @@ -57,7 +56,7 @@ PGD comprises several key architectural elements that work together to provide i - **Monitoring tools**: To monitor performance, health, and usage with PGD, you can utilize its [built-in command-line interface](../cli) (CLI), which offers several useful commands. For instance, the `pgd show-nodes` command provides a summary of all nodes in the cluster, including their state and status. The `pgd check-health` command checks the health of the cluster, reporting on node accessibility, replication slot health, and other critical metrics. The `pgd show-events` command lists significant events like background worker errors and node membership changes, which helps in tracking the operational status and issues within the cluster. Furthermore, the BDR extension allows for monitoring your cluster using SQL using the [`bdr.monitor`](../security/pgd-predefined-roles/#bdr_monitor) role. -#### Node types +### Node types All nodes in PGD are effectively data nodes. They vary only in their purpose in the cluster. @@ -71,7 +70,7 @@ There are then three types of node which, although built like a data node, have - **[Logical standby nodes](../node_management/logical_standby_nodes/)**: Act as standby nodes that can be promoted to data nodes if needed, ensuring high availability and disaster recovery. -#### Node roles +### Node roles Data nodes in a group can also take on particular roles to enable particular features. These roles are transient and can be transferred to any other capable node in the group if needed. @@ -81,7 +80,7 @@ These roles can include: - **[Write leader](../terminology/#write-leader)**: Receives all write operations from PGD Proxy. -### Architectural Flexibility +## Architectural Flexibility Postgres Distributed (PGD) offers flexible options with how its architecture can be deployed, maintained, and scaled to meet various performance, availability, and compliance needs. From e631a46368b0c3af0c707a34316dce23e7740e6e Mon Sep 17 00:00:00 2001 From: Josh Earlenbaugh Date: Tue, 14 Jan 2025 11:45:26 -0500 Subject: [PATCH 05/12] More small changes. --- product_docs/docs/pgd/5.6/overview/basic-architecture.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/product_docs/docs/pgd/5.6/overview/basic-architecture.mdx b/product_docs/docs/pgd/5.6/overview/basic-architecture.mdx index 1d8cb0d3695..f6d5cbb5b87 100644 --- a/product_docs/docs/pgd/5.6/overview/basic-architecture.mdx +++ b/product_docs/docs/pgd/5.6/overview/basic-architecture.mdx @@ -1,5 +1,5 @@ --- -title: "Basic Architecture" +title: "Architectural Overview" description: An overview of EDB Postgres Distributed architecture deepToC: true redirects: From 2ddc1c5e281ccde2f6bdff8b5bbc5dea31e4768e Mon Sep 17 00:00:00 2001 From: Josh Earlenbaugh Date: Tue, 14 Jan 2025 11:48:52 -0500 Subject: [PATCH 06/12] Fixed image call --- product_docs/docs/pgd/5.6/overview/basic-architecture.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/product_docs/docs/pgd/5.6/overview/basic-architecture.mdx b/product_docs/docs/pgd/5.6/overview/basic-architecture.mdx index f6d5cbb5b87..f305edb5c6c 100644 --- a/product_docs/docs/pgd/5.6/overview/basic-architecture.mdx +++ b/product_docs/docs/pgd/5.6/overview/basic-architecture.mdx @@ -36,7 +36,7 @@ At the highest level, PGD comprises two main components: Bi-Directional Replicat BDR is a Postgres extension that enables a multi-master replication mesh between different BDR-enabled Postgres instances/nodes. [PGD proxy](../routing) sends requests to the write leader—ensuring a lower risk of conflicts between nodes. -![Diagram showing 3 application nodes, 3 proxy instances, and 3 PGD nodes. Traffic is being directed from each of the proxy instances to the write leader node.](/docs/product_docs/docs/pgd/5.6/overview/img/always_on_1x3_updated.png) +![Diagram showing 3 application nodes, 3 proxy instances, and 3 PGD nodes. Traffic is being directed from each of the proxy instances to the write leader node.](./img/always_on_1x3_updated.png) Changes are replicated directly, row-by-row between all nodes. [Logical replication](../terminology/#logical-replication) in PGD is asynchronous by default, so only eventual consistency is guaranteed (within seconds usually). From 6ca73c0ab5f4a9c9456e780f46b139fe50baa83a Mon Sep 17 00:00:00 2001 From: Josh Earlenbaugh Date: Tue, 14 Jan 2025 11:53:50 -0500 Subject: [PATCH 07/12] added some bold on commit scope callouts. --- product_docs/docs/pgd/5.6/overview/basic-architecture.mdx | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/product_docs/docs/pgd/5.6/overview/basic-architecture.mdx b/product_docs/docs/pgd/5.6/overview/basic-architecture.mdx index f305edb5c6c..b1dc730641a 100644 --- a/product_docs/docs/pgd/5.6/overview/basic-architecture.mdx +++ b/product_docs/docs/pgd/5.6/overview/basic-architecture.mdx @@ -92,10 +92,10 @@ PGD allows for selective replication, enabling users to replicate only a subset With commit scopes, PGD also provides configurable durability. Accordingly, durability can be increased from the default asynchronous behavior and tuned using various configurable commit scopes: -- Synchronous Commit: Works a lot like PostgreSQL’s synchronous_commit option in its underlying operation—requires writing to at least one other node at COMMIT time, but can be tuned to require all nodes. +- **Synchronous Commit**: Works a lot like PostgreSQL’s synchronous_commit option in its underlying operation—requires writing to at least one other node at COMMIT time, but can be tuned to require all nodes. -- CAMO (Commit at most once): Works by tracking each transaction with a unique ID and using a pair of nodes to confirm the transaction's outcome, ensuring the application knows whether to retry the transaction or not. +- **CAMO** (Commit at most once): Works by tracking each transaction with a unique ID and using a pair of nodes to confirm the transaction's outcome, ensuring the application knows whether to retry the transaction or not. -- Group Commit: An experimental commit scope, the goal of which is to protect against data loss in case of single node failures of temporary outages by requiring more than one PGD node to successfully confirm a transaction at COMMIT time. +- **Group Commit**: An experimental commit scope, the goal of which is to protect against data loss in case of single node failures of temporary outages by requiring more than one PGD node to successfully confirm a transaction at COMMIT time. -- Lag Control: If replication is running outside of set limits (taking too long for another node to be replicated to), a delay is injected into the node that originally received the transaction, slowing things down until other nodes have caught up. \ No newline at end of file +- **Lag Control**: If replication is running outside of set limits (taking too long for another node to be replicated to), a delay is injected into the node that originally received the transaction, slowing things down until other nodes have caught up. \ No newline at end of file From 17347ae4606fd01873661290471b0cec81b8f566 Mon Sep 17 00:00:00 2001 From: Josh Earlenbaugh Date: Tue, 14 Jan 2025 12:04:56 -0500 Subject: [PATCH 08/12] Added links to commit scope callouts. --- product_docs/docs/pgd/5.6/overview/basic-architecture.mdx | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/product_docs/docs/pgd/5.6/overview/basic-architecture.mdx b/product_docs/docs/pgd/5.6/overview/basic-architecture.mdx index b1dc730641a..5576b88a4d5 100644 --- a/product_docs/docs/pgd/5.6/overview/basic-architecture.mdx +++ b/product_docs/docs/pgd/5.6/overview/basic-architecture.mdx @@ -92,10 +92,10 @@ PGD allows for selective replication, enabling users to replicate only a subset With commit scopes, PGD also provides configurable durability. Accordingly, durability can be increased from the default asynchronous behavior and tuned using various configurable commit scopes: -- **Synchronous Commit**: Works a lot like PostgreSQL’s synchronous_commit option in its underlying operation—requires writing to at least one other node at COMMIT time, but can be tuned to require all nodes. +- **[Synchronous Commit](../commit-scopes/synchronous_commit.mdx)**: Works a lot like PostgreSQL’s synchronous_commit option in its underlying operation—requires writing to at least one other node at COMMIT time, but can be tuned to require all nodes. -- **CAMO** (Commit at most once): Works by tracking each transaction with a unique ID and using a pair of nodes to confirm the transaction's outcome, ensuring the application knows whether to retry the transaction or not. +- **[CAMO](../commit-scopes/camo.mdx)** (Commit at most once): Works by tracking each transaction with a unique ID and using a pair of nodes to confirm the transaction's outcome, ensuring the application knows whether to retry the transaction or not. -- **Group Commit**: An experimental commit scope, the goal of which is to protect against data loss in case of single node failures of temporary outages by requiring more than one PGD node to successfully confirm a transaction at COMMIT time. +- **[Group Commit](../commit-scopes/group-commit.mdx)**: An experimental commit scope, the goal of which is to protect against data loss in case of single node failures of temporary outages by requiring more than one PGD node to successfully confirm a transaction at COMMIT time. -- **Lag Control**: If replication is running outside of set limits (taking too long for another node to be replicated to), a delay is injected into the node that originally received the transaction, slowing things down until other nodes have caught up. \ No newline at end of file +- **[Lag Control](../commit-scopes/lag-control.mdx)**: If replication is running outside of set limits (taking too long for another node to be replicated to), a delay is injected into the node that originally received the transaction, slowing things down until other nodes have caught up. \ No newline at end of file From 428984232f2ef892db798706fd4691138cc7854d Mon Sep 17 00:00:00 2001 From: Josh Earlenbaugh Date: Tue, 14 Jan 2025 12:59:21 -0500 Subject: [PATCH 09/12] fixed links. --- .../docs/pgd/5.6/overview/basic-architecture.mdx | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/product_docs/docs/pgd/5.6/overview/basic-architecture.mdx b/product_docs/docs/pgd/5.6/overview/basic-architecture.mdx index 5576b88a4d5..b446c399426 100644 --- a/product_docs/docs/pgd/5.6/overview/basic-architecture.mdx +++ b/product_docs/docs/pgd/5.6/overview/basic-architecture.mdx @@ -40,7 +40,7 @@ BDR is a Postgres extension that enables a multi-master replication mesh between Changes are replicated directly, row-by-row between all nodes. [Logical replication](../terminology/#logical-replication) in PGD is asynchronous by default, so only eventual consistency is guaranteed (within seconds usually). -However, [commit scope](../durability/commit-scopes) options offer stronger consistency and durability guarantees via [CAMO](/pgd/latest/durability/camo/), [group](../durability/group-commit) and [synchronous](../durability/synchronous_commit) commits. +However, [commit scope](../commit-scopes/commit-scopes) options offer stronger consistency and durability guarantees via [CAMO](/pgd/latest/commit-scopes/camo/), [group](../commit-scopes/group-commit) and [synchronous](../commit-scopes/synchronous_commit) commits. The Raft algorithm provides a mechanism for [electing](../routing/raft/04_raft_elections_in_depth/) leaders (both Raft leader and write leader), deciding which nodes should be added or subtracted from the cluster, and generally ensuring that the distributed system remains consistent and fault-tolerant, even in the face of node failures. @@ -49,10 +49,10 @@ The Raft algorithm provides a mechanism for [electing](../routing/raft/04_raft_e PGD comprises several key architectural elements that work together to provide its distributed database solution: - **PGD nodes**: These are individual Postgres instances that store and manage data. They are the basic building blocks of a PGD cluster. - + - **Groups**: PGD nodes are organized into [groups](../node_management/groups_and_subgroups), which enhance manageability and high availability. Each group can contain multiple nodes, allowing for redundancy and failover within the group. Groups facilitate organized replication and data consistency among nodes within the same group and across different groups. Each group has its own write leader. - - **Replication mechanisms**: PGD's replication mechanisms include Bi-Directional Replication (BDR) for efficient replication across nodes, enabling multi-master replication. BDR supports asynchronous replication by default, but can be configured for varying levels of synchronicity, such as [Group Commit](../durability/group-commit) or [Synchronous Commit](../durability/synchronous_commit), to enhance data durabiltiy and consistency. + - **Replication mechanisms**: PGD's replication mechanisms include Bi-Directional Replication (BDR) for efficient replication across nodes, enabling multi-master replication. BDR supports asynchronous replication by default, but can be configured for varying levels of synchronicity, such as [Group Commit](../commit-scopes/group-commit) or [Synchronous Commit](../commit-scopes/synchronous_commit), to enhance data durability. - **Monitoring tools**: To monitor performance, health, and usage with PGD, you can utilize its [built-in command-line interface](../cli) (CLI), which offers several useful commands. For instance, the `pgd show-nodes` command provides a summary of all nodes in the cluster, including their state and status. The `pgd check-health` command checks the health of the cluster, reporting on node accessibility, replication slot health, and other critical metrics. The `pgd show-events` command lists significant events like background worker errors and node membership changes, which helps in tracking the operational status and issues within the cluster. Furthermore, the BDR extension allows for monitoring your cluster using SQL using the [`bdr.monitor`](../security/pgd-predefined-roles/#bdr_monitor) role. @@ -60,15 +60,15 @@ PGD comprises several key architectural elements that work together to provide i All nodes in PGD are effectively data nodes. They vary only in their purpose in the cluster. - - **[Data nodes](../node_management/node_types/#data-nodes)**: Store and manage data, handle read and write operations, and participate in replication. + - **[Data nodes](../nodes/#data-nodes)**: Store and manage data, handle read and write operations, and participate in replication. There are then three types of node which, although built like a data node, have a specific purpose. These are: - - **[Subscriber-only nodes](../node_management/subscriber_only/#subscriber-only-nodes)**: Subscribe to changes from data nodes for read-only purposes, used in reporting or analytics. + - **[Subscriber-only nodes](../nodes/subscriber_only/#subscriber-only-nodes)**: Subscribe to changes from data nodes for read-only purposes, used in reporting or analytics. - - **[Witness nodes](../node_management/witness_nodes/)**: Participate in concensus proceses without storing data, aiding in achieving quorum and maintaining high availability. + - **[Witness nodes](../nodes/witness_nodes/)**: Participate in the consensus process without storing data, aiding in achieving quorum and maintaining high availability. - - **[Logical standby nodes](../node_management/logical_standby_nodes/)**: Act as standby nodes that can be promoted to data nodes if needed, ensuring high availability and disaster recovery. + - **[Logical standby nodes](../nodes/logical_standby_nodes/)**: Act as standby nodes that can be promoted to data nodes if needed, ensuring high availability and disaster recovery. ### Node roles @@ -76,7 +76,7 @@ Data nodes in a group can also take on particular roles to enable particular fea These roles are transient and can be transferred to any other capable node in the group if needed. These roles can include: - - **[Raft leader]()**: Arbitrates and manages consensus between a group's nodes. + - **Raft leader**: Arbitrates and manages consensus between a group's nodes. - **[Write leader](../terminology/#write-leader)**: Receives all write operations from PGD Proxy. From 7009b00d23e521666cdc5cba3fa4935a80e5d0b8 Mon Sep 17 00:00:00 2001 From: Josh Earlenbaugh Date: Tue, 14 Jan 2025 14:56:03 -0500 Subject: [PATCH 10/12] small word change. --- product_docs/docs/pgd/5.6/overview/basic-architecture.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/product_docs/docs/pgd/5.6/overview/basic-architecture.mdx b/product_docs/docs/pgd/5.6/overview/basic-architecture.mdx index b446c399426..a3258ef4bc2 100644 --- a/product_docs/docs/pgd/5.6/overview/basic-architecture.mdx +++ b/product_docs/docs/pgd/5.6/overview/basic-architecture.mdx @@ -40,7 +40,7 @@ BDR is a Postgres extension that enables a multi-master replication mesh between Changes are replicated directly, row-by-row between all nodes. [Logical replication](../terminology/#logical-replication) in PGD is asynchronous by default, so only eventual consistency is guaranteed (within seconds usually). -However, [commit scope](../commit-scopes/commit-scopes) options offer stronger consistency and durability guarantees via [CAMO](/pgd/latest/commit-scopes/camo/), [group](../commit-scopes/group-commit) and [synchronous](../commit-scopes/synchronous_commit) commits. +However, [commit scope](../commit-scopes/commit-scopes) options offer immedidate consistency and durability guarantees via [CAMO](/pgd/latest/commit-scopes/camo/), [group](../commit-scopes/group-commit) and [synchronous](../commit-scopes/synchronous_commit) commits. The Raft algorithm provides a mechanism for [electing](../routing/raft/04_raft_elections_in_depth/) leaders (both Raft leader and write leader), deciding which nodes should be added or subtracted from the cluster, and generally ensuring that the distributed system remains consistent and fault-tolerant, even in the face of node failures. From cb1e116576e934f31853789ce51d1bf73f4cc23f Mon Sep 17 00:00:00 2001 From: Josh Earlenbaugh Date: Tue, 14 Jan 2025 15:01:12 -0500 Subject: [PATCH 11/12] Tried fixing index page. --- product_docs/docs/pgd/5.6/overview/index.mdx | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/product_docs/docs/pgd/5.6/overview/index.mdx b/product_docs/docs/pgd/5.6/overview/index.mdx index f0c61349224..fa34df687f8 100644 --- a/product_docs/docs/pgd/5.6/overview/index.mdx +++ b/product_docs/docs/pgd/5.6/overview/index.mdx @@ -5,14 +5,14 @@ indexCards: simple redirects: - bdr navigation: -- basic-architecture +- architectural-overview - architecture-and-performance - compared --- EDB Postgres Distributed (PGD) provides multi-master replication and data distribution with advanced conflict management, data-loss protection, and [throughput up to 5X faster than native logical replication](https://www.enterprisedb.com/blog/performance-improvements-edb-postgres-distributed). It also enables distributed Postgres clusters with high availability up to five 9s. -* [Basic architecture](basic-architecture) +* [Architectural overview](basic-architecture) * [Architectural options and performance](architecture-and-performance) * [Comparison with other replication solutions](compared) From d5949da28808f8f6d844e7434b6fc522d3efb021 Mon Sep 17 00:00:00 2001 From: Josh Earlenbaugh Date: Tue, 14 Jan 2025 15:04:03 -0500 Subject: [PATCH 12/12] Tweaked nav structure to get it to look right. --- product_docs/docs/pgd/5.6/overview/basic-architecture.mdx | 2 +- product_docs/docs/pgd/5.6/overview/index.mdx | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/product_docs/docs/pgd/5.6/overview/basic-architecture.mdx b/product_docs/docs/pgd/5.6/overview/basic-architecture.mdx index a3258ef4bc2..990d7ecb208 100644 --- a/product_docs/docs/pgd/5.6/overview/basic-architecture.mdx +++ b/product_docs/docs/pgd/5.6/overview/basic-architecture.mdx @@ -1,5 +1,5 @@ --- -title: "Architectural Overview" +title: "Architectural overview" description: An overview of EDB Postgres Distributed architecture deepToC: true redirects: diff --git a/product_docs/docs/pgd/5.6/overview/index.mdx b/product_docs/docs/pgd/5.6/overview/index.mdx index fa34df687f8..4da5b89b926 100644 --- a/product_docs/docs/pgd/5.6/overview/index.mdx +++ b/product_docs/docs/pgd/5.6/overview/index.mdx @@ -5,7 +5,7 @@ indexCards: simple redirects: - bdr navigation: -- architectural-overview +- basic-architecture - architecture-and-performance - compared ---