Saturday, March 31, 2018

wal2json 1.0 released

wal2json 1.0 was released! I started to develop wal2json more than 4 years ago. At that time it was just a proof of concept for logical decoding. It was one of the first logical decoding plugins and maybe one of the most popular (because CDC solutions -- such as Kafka -- became popular and it is supported by Amazon RDS since 9.5).

It works on PostgreSQL 9.4+ and in all PostgreSQL supported platforms. Although we don't have packages for it yet, it is easier to install.

It has some parameters to control what it outputs and its behavior. You can choose to include or exclude:
  • transaction id;
  • transaction timestamp;
  • table schema;
  • type names (and also type modifier);
  • type oids;
  • not null information;
  • end of transaction LSN;
  • unchanged TOAST columns.
There is also options to:
  • pretty print output (indentation and formatting);
  • write after each row instead of after each transaction (useful for big transactions because postgres has memory limitations -- 1 GB).
And the last features I implemented:
  • filter tables (don't send rows from those tables);
  • choose tables (send only rows from those tables).
 If you have any questions, suggestions or bugs, open an issue or send a pull request.

Thanks to Daniele Varrazzo, Robert Coup and David Fettter for their contributions.