anchor.diff:  [[supported-PostgreSQL-topologies]]
   == Supported PostgreSQL topologies
   
  -The PostgreSQL connector can be used both with a standalone PostgreSQL server, or with a cluster of PostgreSQL servers. 
  -
  -As mentioned link:#limitations[in the beginning] however, as of `9.6` logical replication slots are only supported on `master` servers. This means that a replica in a PostgreSQL cluster cannot be configured for logical replication.
  -This also means that the Debezium PostgreSQL Connector can only connect and communicate with the master server. Should this server fail, after the cluster has promoted a new master, the connector has to be restarted and pointed to the new master server. This consequently means that this server need to have the plugin installed and be properly configured.
  +The PostgreSQL connector can be used with a standalone PostgreSQL server or with a cluster of PostgreSQL servers. 
  +
  +As mentioned link:#limitations[in the beginning], PostgreSQL 9.6 only supports logical replication slots on `master` servers. This means that a replica in a PostgreSQL cluster cannot be configured for logical replication, and consequently that the Debezium PostgreSQL Connector can only connect and communicate with the master server. Should this server fail, the connector will stop. When the cluster is repaired, if the original master server is once again promoted to `master`, the connector can simply be restarted. However, if a different PostgreSQL server _with the plugin and proper configuration_ is promoted to `master`, the connector configuration must be changed to point to the new `master` server and then can be restarted.
   
   [[snapshots]]
   === Snapshots
   
  -When a PostgreSQL connector that is configured to follow a PostgreSQL database is first started, it will by default perform an initial _consistent snapshot_ of a database. This is the default mode, since much of the time the PostgreSQL WAL segments no longer contain the complete history of the database.
  -
  -The connector performs the following steps each time it takes a snapshot:
  -
  -1. Starts a transaction with a https://www.postgresql.org/docs/9.6/static/sql-set-transaction.html[SERIALIZABLE, READ ONLY, DEFERRABLE] isolation level to ensure that all subsequent reads within this transaction are done against a single consistent snapshot.
  -2. Obtains a `SHARE UPDATE EXCLUSIVE MODE` lock on each of the monitored tables to ensure that no structural changes can occur to any of the tables while the snapshot is taking place. Note that these locks do not prevent table `INSERTS`, `UPDATES` and `DELETES` from taking place during the operation
  -3. Reads the current position in the server's transaction log.
  -4. Scans all of the database tables and schemas and generates on the appropriate table-specific Kafka topics `READ` events for each row.
  -5. Commits the transaction.
  -6. Record in the connector offsets that the connector successfully completed the snapshot.
  -
  -If the connector fails, is rebalanced, or stops before the snapshot is complete, the connector will begin a new snapshot when it is restarted. Once the connector does complete its initial snapshot, the PostgreSQL connector then continues streaming from the position read during step 3, ensuring that it does not miss any updates. If the connector stops again for any reason, upon restart it will simply continue streaming changes from where it previously left off. However, if the connector remains stopped for long enough, PostgreSQL might purge older WAL segments and the connector's last known position may be lost. In this case, when the connector configured with *initial* snapshot mode (the default) is finally restarted, the PostgreSQL server will no longer have the starting point and the connector will not be able to relay the changes that are not available in the write ahead log.
  -
  -A second snapshot mode allows the connector to perform snapshots *always*. This behaviour tells the connector to always perform a snapshot when it starts up and only after the snapshot is completed to continue streaming changes from step 3 in the above sequence. This mode can be used in cases when it's known that some WAL segments have been deleted and are no longer available, or in case of a cluster failure after a new master has been promoted so that the connector doesn't miss out on any potential changes that could've taken place after the new master had been promoted but before the connector was restarted on the new master.
  -
  -The third snapshot mode ensures the connector *never* performs snapshots. When a new connector is configured this way, if will either continue streaming changes from a previous stored offset or it will start from the point in time when the PostgreSQL logical replication slot was first created on the server.
  -
  -The final snapshot mode - *initial only* - will simply perform a database snapshot and stop streaming any other changes afterwards. If a previous snapshot was in effect but had not completed successfully, the connector will restart the snapshot process each time until this completes successfully.
  +Most PostgreSQL servers are configured to not retain the complete history of the database in the WAL segments, so the PostgreSQL connector would be unable to see the entire history of the database by simply reading the WAL. So, by default the connector will upon first startup perform an initial _consistent snapshot_ of the database. Each snapshot consists of the following steps:
  +
  +1. Start a transaction with a https://www.postgresql.org/docs/9.6/static/sql-set-transaction.html[SERIALIZABLE, READ ONLY, DEFERRABLE] isolation level to ensure that all subsequent reads within this transaction are done against a single consistent version of the data. Any changes to the data due to subsequent `INSERT`, `UPDATE`, and `DELETE` operations by other clients will not be visible to this transaction.
  +2. Obtain a `SHARE UPDATE EXCLUSIVE MODE` lock on each of the monitored tables to ensure that no structural changes can occur to any of the tables while the snapshot is taking place. Note that these locks do not prevent table `INSERTS`, `UPDATES` and `DELETES` from taking place during the operation.
  +3. Read the current position in the server's transaction log.
  +4. Scan all of the database tables and schemas, and generate a `READ` event for each row and write that event to the appropriate table-specific Kafka topic.
  +5. Commit the transaction.
  +6. Record the successful completion of the snapshot in the connector offsets.
  +
  +If the connector fails, is rebalanced, or stops after Step 1 begins but before Step 6 completes, upon restart the connector will begin a new snapshot. Once the connector does complete its initial snapshot, the PostgreSQL connector then continues streaming from the position read during step 3, ensuring that it does not miss any updates. If the connector stops again for any reason, upon restart it will simply continue streaming changes from where it previously left off. However, if the connector remains stopped for long enough, PostgreSQL might purge older WAL segments and the connector's last known position may be lost. In this case, when the connector configured with *initial* snapshot mode (the default) is finally restarted, the PostgreSQL server will no longer have the starting point and the connector will not be able to relay the changes that are not available in the write ahead log.
  +
  +A second snapshot mode allows the connector to perform snapshots *always*. This behavior tells the connector to _always_ perform a snapshot when it starts up, and after the snapshot completes to continue streaming changes from step 3 in the above sequence. This mode can be used in cases when it's known that some WAL segments have been deleted and are no longer available, or in case of a cluster failure after a new master has been promoted so that the connector doesn't miss out on any potential changes that could've taken place after the new master had been promoted but before the connector was restarted on the new master.
  +
  +The third snapshot mode instructs the connector to *never* performs snapshots. When a new connector is configured this way, if will either continue streaming changes from a previous stored offset or it will start from the point in time when the PostgreSQL logical replication slot was first created on the server. Note that this mode is useful only when you know all data of interest is still reflected in the WAL.
  +
  +The final snapshot mode, *initial only*, will perform a database snapshot and then stop before streaming any other changes. If the connector had started but did not complete a snapshot before stopping, the connector will restart the snapshot process and stop once the snapshot completes.
   
   
   [[streaming-changes]]
  
positive.code: Since this is a relatively new feature, it has some limitations which are also reflected by the connector:
  
  . Logical Decoding does not support DDL changes: this means that the connector is unable to report DDL change events back to consumers.
  . Logical Decoding replication slots are only supported on `master` servers: this means that when there is a cluster of PostgreSQL servers, the connector can only run on the active `master` server. It cannot run on `hot` or `warm` standby replicas. If the cluster fails, the connector will need to be restarted on the new `master`, once it has been promoted. Make sure you read more about how the connector behaves link:#when-things-go-wrong[when things go wrong].
  ====
  
  [[setting-up-PostgreSQL]]
  == Setting up PostgreSQL
  
  Before the Debezium PostgreSQL connector can be used to monitor the changes committed on a PostgreSQL server, the server must be set up so that the logical decoding plugin is installed, replication slots are enabled and there is at least a user with sufficient privileges to perform the replication.
  
  The following sections outline in more detail how to set up these features in PostgreSQL.
  
  [[output-plugin]]
  === Installing the logical decoding output plugin
  
  In order to be able to read the changes to the write-ahead-log, a logical decoding output plugin has to be installed prior to the running the PostgreSQL server. This plugin has to be written in C, compiled and installed on the machine which runs the PostgreSQL server. Also, it has to use a number of PostgreSQL specific APIs, as described by the https://www.postgresql.org/docs/9.6/static/logicaldecoding-output-plugin.html[_PostgreSQL documentation_]
  
  Debezium's PostgreSQL connector uses a logical decoding plugin which encodes the changes in https://github.com/google/protobuf[_Protobuf format_]. The requirements for compiling this plugin are described in detail https://github.com/debezium/postgres-decoderbufs/blob/master/README.md[_on the plugin home page_]. 
  
  For simplicity, Debezium also provides a Docker image based on a vanilla PostgreSQL server image on top of which it compiles and installs the plugin. We recommend https://github.com/debezium/docker-images/tree/master/postgres/9.6[_using this image_] as an example for the detailed steps required for the installation.
  
  [WARNING]
  ====
  The plugin has only been installed and tested on _UX_ machines. For Windows and other OSes it may require different installation steps  
  ====
  
  [[server-configuration]]
  ==== Configuring the PostgreSQL server 
  
  Once the link:#output-plugin[plugin] has been installed, the server has to be configured so that it loads the plugin at startup while also allowing replication slots: 
  
  *postgresql.conf*
  [source]
  
negative.code: 
  The PostgreSQL connector will typically spend the vast majority of its time streaming changes from the PostgreSQL server to which it is connected. This mechanism relies on https://www.postgresql.org/docs/9.6/static/protocol-replication.html[_PostgreSQL's replication protocol_] where the client can receive changes from the server as they are committed in the server's transaction log at certain positions (also known as `Log Sequence Numbers` or in short LSNs)
  
  Whenever the server commits a transaction, a separate server process invokes a callback function from the link:#output-plugin[logical decoding plugin]. This function processes the changes from the transaction, converts them to a specific  format (Protobuf in the case of Debezium plugin) and writes them on an output stream which can then be consumed by clients.
  
  The PostgreSQL connector acts as a PostgreSQL client, and when it receives these changes it transforms the events into Debezium _create_, _update_, or _delete_ events that include the LSN position of the event. The PostgreSQL connector forwards these change events to the Kafka Connect framework (running in the same process), which then asynchronously writes them in the same order to the appropriate Kafka topic. Kafka Connect uses the term _offset_ for the source-specific position information that Debezium includes with each event, and Kafka Connect periodically records the most recent offset in another Kafka topic.
  
  When Kafka Connect gracefully shuts down, it stops the connectors, flushes all events to Kafka, and records the last offset received from each connector. Upon restart, Kafka Connect reads the last recorded offset for each connector, and starts the connector from that point. The PostgreSQL connector uses the LSN recorded in each change event as the offset, so that upon restart the connector requests the PostgreSQL server send it the events starting just after that position.
  
  [[topic-names]]
  === Topics names
  