PostgreSQL Disaster Recovery with Barman

GABRIELE BARTOLINI
PGDay Australia 2013 - Melbourne, 4 February 2013

GABRIELE BARTOLINI

• Co-Founder and Manager of 2ndQuadrant Italia

• Data Architect, Business critical environments

• Data warehousing

• Co-Founder Italian PostgreSQL Users Group

• Co-Founder PostgreSQL Europe

• PostgreSQL Contributor and Advocate

DISCLAIMER
This talk assumes you are familiar with disaster
recovery concepts and PostgreSQL
implementation of Point In Time Recovery

BE AWARE
In 2ndQuadrant, all these concepts usually ﬁt in a
2 day workshop on Disaster Recovery and a 1 day
workshop on Barman alone

OUTLINE

• Business continuity / Disaster recovery for databases

• Disaster recovery with Barman for PostgreSQL

PART I
Business continuity / Disaster recovery for databases

BUSINESS CONTINUITY
activity performed by an organization to ensure that critical
business functions will be available to customers, suppliers,
regulators, and other entities that must have access to those
functions - Wikipedia

INFORMATION TECHNOLOGY

• Business continuity

• High availability

• Disaster recovery

LAW REQUIREMENTS
In Italy, the “Codice dell’Amministrazione Digitale”
deﬁnes business continuity requirements for public
administrations

DISASTER
(touch wood)

system/hardware failures
unintentional errors
natural disaster

REACT TO A DISASTER
Recover systems, data and infrastructures

TOO LATE!
Do not wait for a disaster to happen

PLAN FOR DISASTERS
“Disasters” will happen. Be prepared.

“Plans are worthless, but planning is
everything. There is a very great distinction
because when you are planning for an emergency
you must start with this one thing: the very deﬁnition
of "emergency" is that it is unexpected,
therefore it is not going to happen the way you are
planning.”
- Dwight D. Eisenhower

DATABASE
DISASTER RECOVERY
Let’s just focus on databases!

REQUIREMENTS

• Automated backups • Retention policies

• Notiﬁcations (anomalies) • Data protection

• Frequency of backups • Availability for recovery

POSTGRES BACKUP

• Hot backup • Physical Backup

• MVCC • Full backup (base backup)

• Logical Backup • Differential backup (WAL)

• pg_dump

TRADITIONAL DR
WITH POSTGRESQL
• PostgreSQLprimitives for • Custom scripts
DR are robust and reliable
• Hard to integrate in:
• High level skills
• Backup solutions
• DBA
• Disaster Recovery plans
• Sysadmins
• Hard to test!

EXISTING TOOLS

• Omni-PITR • WAL-E

• WAL centric • EC2 centric, but ...

• WALmgr • good

• WAL centric • came later

• pg-rman

• Server centric

NONE FOR DR
None of them was a pure disaster recovery solution.
We wanted something similar to Oracle’s RMAN.

FILLING A HOLE
The lack of a DR solution is a barrier towards the adoption
of PostgreSQL from Oracle users’ point of view.

DESIDERATA

• Hot, Full, Differential and • Archival and compression
Incremental backups
• WAL segments
• Multiple servers
• Periodical backups
• Remote backup & recovery
• Automation
• Backup catalogues
• Integration
• Retention policies
• Usability

PART II
Disaster recovery with Barman for PostgreSQL

BARMAN

• GNU GPL 3 • Debian/Ubuntu package

• Hosted on Sourceforge.net • Designed, developed,
maintained by 2ndQuadrant
• Python 2.6/2.7

• PostgreSQL 8.4, 9.0, 9.1, 9.2

• PyPI package

• RPM package

Postgres Postgres Postgres

Barman
tape Barman

LAN, hybrid architecture
centralised architecture

Postgres

Continuous archiving SSH commands
(WAL shipping via SSH) SQL commands

Barman

Postgres Barman

WAL secure channel cron

Barman’s
WAL archive

Postgres Barman

Periodical backup (weekly) Full backup - Sat 1, 4AM

Differential backup
Backup catalogue Full backup - Sat 8, 4AM

Full backup - Sat 15, 4AM

Full backup - Sat 22, 4AM

WAL archive

Backup 1 Backup 2

(100MB) (105MB)

WAL WAL WAL WAL WAL WAL WAL WAL WAL WAL

Size: 100MB + 80MB == 260MB Size: 105MB + 80MB = 185MB
160MB 180MB

CONFIGURATION FILE
[barman]
barman_home = /srv/barman
barman_user = barman
log_file = /var/log/barman/barman.log
log_level = NOTICE
compression = gzip

[production]
description = Production PostgreSQL
ssh_command = ssh pg.2ndQuadrant.it
conninfo = host=pg.2ndQuadrant.it user=postgres
compression = bzip2

MULTI-SERVER
CONFIGURATION
[barman]
; General configuration
; …
[server_one]
; Configuration for Server 1
; …
[server_two]
; Configuration for Server 2
; …
[server_X]
; …

MULTIPLE FILES INCLUSION

[barman]
; General configuration
; …
configuration_files_directory = /etc/barman.d

CONVENTION OVER
CONFIGURATION
global/per server options
default directory layout

CONVENTIONAL
DIRECTORIES FOR BARMAN
• barman_home (/srv/barman)

• server directory (/srv/barman/production)

• base directory (/srv/barman/production/base)

• WAL directory (/srv/barman/production/wals)

• incoming directory (/srv/barman/production/incoming)

GLOBAL COMMANDS

• List of managed servers

• barman list-server

• Maintenance operations

• barman cron

SERVER COMMANDS

• Information and • Backup control
diagnostics
• Recovery control
• barman status

• barman check

• barman show-server

• barman list-backup

BACKUP CONTROL

• barman backup

• barman show-backup

• barman list-ﬁles

• standalone, data, wal, full

• barman delete

SHOW BACKUP

• General • WAL

• Servername, Postgres • Number of associated ﬁles
version, status, ...
• disk usage
• Base backup
• Context
• Start/End
time, ﬁrst/last
WAL, disk usage, ... • Previous/Next backup

RECOVERY CONTROL

• Recovery target (full / point in time)

• Local recovery

• barman recover

• Remote recovery

• barman recover --remote-ssh-command

ADVANCED RECOVERY

• Point In Time Recovery

• --target-time = TIME

• --target-xid = XID

• Relocation of tablespaces

• --tablespace NAME:LOCATION [...]

COMMON USE CASES

• Accidental errors recovery

• Disaster recovery

• Sandbox server (BI, staging, ...)

RETENTION POLICIES

• User-deﬁned policy

• Determines how long backups and related WAL ﬁles need to
be retained for recovery procedures (point of recoverability):

• basedon number of backups (REDUNDANCY) or time
(RECOVERY WINDOW)

• Currently implemented only in auto mode

• through ‘barman cron’

RETENTION POLICY
CONFIGURATION
; Base backup retention policy
retention_policy = 'redundancy 3'
retention_policy = 'recovery window of 3 months'

; WAL retention policy
wal_retention_policy = 'base'
; The following is currently not supported
wal_retention_policy =
'recovery window of 14 days'

SPONSOR A FEATURE!

• Sandbox recovery • Replication protocol support
(client / server)
• Export/Import
• Incremental backup
• Logical backup integration
• reduce backup time and
• pg_dump on sandbox size
instances

OUR COMMITMENT

• Keep it open source

• Reinvest money from sale of DR turnkey solutions in R&D

• Support and maintain RPM/Debian packages

• Accept sponsorships for new features development

CSI PIEMONTE

(One of the top 10 ICT companies in Italy for revenue)

“We found in Barman the optimal solution for physical backup
and disaster recovery of PostgreSQL databases. Barman is
robust and easy to use. Its command interface allows an easy
integration with the existing management tools in our
enviroment.”

Sponsors of RPM package and WAL compression

CONCLUSIONS

• Hides complexity of PITR / Keeps unaltered PITR strenghts

• Not invasive

• Fosters migrations from Oracle

• “Standard de facto” for PostgreSQL Disaster Recovery

• Advice: plan for DR (if you have not done it yet)

QUESTIONS?
Gabriele.Bartolini@2ndQuadrant.it
Twitter: _GBartolini_
www.pgbarman.org

LICENSE

This work is licensed under a Creative Commons Attribution-
NonCommercial-ShareAlike 3.0 Unported License

Copyright (c) 2012, 2013 - 2ndQuadrant.it

PostgreSQL Disaster Recovery with Barman

More Related Content

What's hot

Viewers also liked

Similar to PostgreSQL Disaster Recovery with Barman

Recently uploaded

PostgreSQL Disaster Recovery with Barman