in Databases

What we did different with our Puppet PgBouncer module

CoverMyMeds recently open sourced our PgBouncer Puppet module! It can be found on Puppet Forge here and on Github here.

I evaluated existing modules and found that none of them hit the sweet spot of working with RHEL/CentOS, allowing configuration of all of the options or variables, and allowing a way to compartmentalize code that builds the config in a way that it can easily be re-used or called from other modules. Many of the modules did not seem to be actively developed either. Due to these shortfalls in existing modules, we decided that we would build our own.

Since other modules did exist, I took the module that seemed to have the most functionality and semi-recent development and used it as a starting point. We were initially going to simply fork their project but it is hosted on Bitbucket and I had plans to make very drastic changes, so we ended up creating our own module.

Highlights of our new module include:

  • Support for RedHat/CentOS systems
  • It added a dependency on the puppetlabs-postgresql module and uses that module for repository management to get the PgBouncer package
  • It added a dependency on the puppetlabs-concat module and uses that module to allow us to break out the various pieces of the configuration files to individualized templates
  • Instead of using individualized puppet parameters for the PgBouncer configuration file it was converted to use a hash structure which means that options in newer versions can be automatically supported from the same module. The hash structure uses a default hash that can be overridden and a standard config hash. The standard config hash overrides the default hash with a deep merge allowing individualized hosts to override a single setting without having to recreate the entire structure
  • The “auth_file” (userlist.txt by default) is broken out to a defined type and uses a hash structure allowing third party modules to call it. In our instance we can use an existing hieradata structure that manages usernames and passwords in a central location for our applications and database servers to call out to PgBouncer so that users are still centrally managed
  • Changed the “auth_file” to use the MD5 mechanism for storing passwords instead of the default plain text method
  • The “databases” list in the pgbouncer.ini file was also broken out to a defined type utilizing a hash structure. This has the same benefit as the auth_file in that it allows us to re-use existing modules and hieradata structures
  • The module has been tested to work on RHEL6 and RHEL7 with version 1.5, 1.6, and the current development build in master of the PgBouncer project
  • Using the newest dev version and no changes to this module I was able to get their new TLS/SSL features working to encrypt traffic! We are testing this in house and looking forward to providing feedback to their project and ultimately the next release
  • I built out a parameter class to deal with differences in operating systems. This should make it easy to manage and expand support to other *nix operating systems

Usage

You can include the covermymeds/puppet-pgbouncer module simply by calling the postgresql::globals file to setup the repo and then calling the PgBouncer module: include ::postgresql::globals
include ::pgbouncer

Gotchas

Since the place I took this from had started with support for Ubuntu, I did my best to keep that functionality, but I did not stand up a testing environment for Ubuntu, so there may be issues. We are completely open to pull requests, or you can create an issue and we’ll work with you to support your request or resolve the problem.

Write a Comment

Comment

  1. You legend.

    I went down almost the same path as you did here with looking at the existing modules then finding landcare’s one on Bitbucket, logged a feature request for hiera support with them only to have Bitbucket crash on me which cheesed me right off.

    Going to look at using your module now – thanks :)