Replication is, simply put, copying data from one place to another. Whether you use PostgreSQL or MySQL for your database system, there are multiple solutions to consider, both built-in and with third-party tools. PostgreSQL includes these capabilities with several built-in replication features such as write-ahead log (WAL) shipping, warm and hot standby, and streaming replication. Some widely-used third-party tools for replication and similar functions include Bucardo (developed by End Point), Slony, Londiste, pgbouncer, and pgpool. MySQL also has built-in replication features for shipping either row data or SQL commands, and can be used in conjunction with other third-party tools.
Each tool has its own strengths and constraints. End Point’s consultants have experience determining the tools that work best in various situations and helping you achieve the right balance between cost, availability, and performance.
Replicas for load distribution
Growing businesses often reach a point where their database systems slow unacceptably under increasingly high demand. Replication allows you to create copies of your main databases, known as slaves or replicas. These replicas can then take up some of the load from reporting jobs and other read-only activities. Bucardo and Slony are popular replication solutions that can support this arrangement. PostgreSQL 9.0 and newer provide another read-only replica option with its “hot standby” feature. End Point can help you determine which of these solutions will be best for you and your business.
Bucardo also offers two-way replication such that two databases can be complete authoritative repositories of all information, updated in near real-time. This can be achieved across multiple datacenters for risk mitigation, lower latency to local services, or to distribute load. We can help you avoid pitfalls and make the most of a multi-master replication system.
There are many ways to back up a database. The most common PostgreSQL backup utility, pg_dump, is simple and easy to use, but can add load to your production system. Some workloads benefit from replicating data to a slave database and exporting that periodically instead. We have experience with other backup solutions such as point-in-time recovery (PITR) and filesystem snapshots, and can create a solution suited to your needs.
Businesses have always collected data; many businesses now want to “mine” or analyze their data for marketing, performance, and process optimization purposes. This usually involves very complex queries, which can be expensive both in time and server processing power. A common approach is to periodically transfer the data from production databases into a “data warehouse” to simplify the queries and ease load on the main database servers, in a process known as “Extraction, Transformation, and Loading”, or ETL. Bucardo allows seamless transformation of data between master and replica, and can handle ETL processes in a single package.
Sometimes businesses or applications need to distribute changes across long distances or delayed over time. In these scenarios latency and disruptions to connectivity can affect replication in unexpected ways. End Point engineers are versed in the options, and can help you build a solution that will handle your more complex needs.
- Postgres statistics and the pain of analyze
- Throw It Away: Suppressing Writes on PostgreSQL Replicas
- Making cross-blogs queries in multi-site WordPress performant
- Postgres connection service file
- Postgres schema differences and views
- pg_healer: repairing Postgres problems automatically
- Executing Custom SQL in Django Migrations
- Disabling Postgres constraints for pg_dump
- WAL-based Estimates For When a Record Was Changed
- Postgres migration speedup with table change analysis
- Bucardo replication workarounds for extremely large Postgres updates
- Postgres concurrent indexes and the curse of IIT
Recent Blog Posts