Rust library and daemon for easily starting postgres databases per-test without Docker

Related tags

Database pgtemp
Overview

pgtemp

Coverage Status CI Status crates.io docs.rs

pgtemp is a Rust library and cli tool that allows you to easily create temporary PostgreSQL servers for testing without using Docker.

The pgtemp Rust library allows you to spawn a PostgreSQL server in a temporary directory and get back a full connection URI with the host, port, username, and password.

The pgtemp cli tool allows you to even more simply make temporary connections, and works with any language: Run pgtemp and then use its connection URI when connecting to the database in your tests. pgtemp will then spawn a new postgresql process for each connection it receives and transparently proxy everything over that connection to the temporary database. Note that this means when you make multiple connections in a single test, changes made in one connection will not be visible in the other connections, unless you are using pgtemp's --single mode.

pgtemp supports loading (and dumping, in the library) the database to/from dumpfiles via pg_dump.

Requirements

You must install both the postgresql client and server packages. On Debian/Ubuntu, they are postgresql postgresql-client, on Fedora they are postgresql postgresql-server, and on Arch Linux they are postgresql postgresql-libs. Note also that Debian/Ubuntu install the standard postgres binaries into their own directory, so you must add them to your path. For an Ubuntu GitHub Actions runner, it looks like:

steps:
  - name: Install postgres
    run: sudo apt-get install postgresql postgresql-client
  - name: Update path
    run: find /usr/lib/postgresql/ -type d -name "bin" >> $GITHUB_PATH

To install the CLI tool, you must install it with the --features cli or --all-features options

cargo install pgtemp --features cli

Design

pgtemp is a fairly simple program and there are other existing libraries like testing.postgresql for Python and pgtest for Go that all work the same way:

  • Do some setup, like creating temporary directories and copy files
  • Run the initdb program
  • Start the postgres server
  • Wait until postgres has started

The novel idea (as far as I'm aware, although I also only found out about the above python/go libraries after coming up with the initial library idea) in pgtemp is the CLI/daemon which automatically provides connections to new temporary databases upon each connection.

Examples

CLI

$ cargo install --all-features pgtemp
# username, password, port, and database name are all configurable based on the provided connection URI
$ pgtemp postgresql://localhost:6543/mytestdb 
starting pgtemp server at postgresql://postgres:password@localhost:6543/mytestdb
$ psql postgresql://postgres:password@localhost:6543/mytestdb
psql (16.1)
Type "help" for help.

postgres=#

See examples/ directory for examples:

  • A python example with sqlalchemy and alembic, demonstrating usage with the pgtemp cli's normal and single modes

Library

use pgtemp::PgTempDB;
use sqlx::postgres::PgConnection;
use sqlx::prelude::*;

#[tokio::test]
fn cool_db_test() {
    let db = PgTempDB::async_new().await;
    let mut conn = sqlx::postgres::PgConnection::connect(&db.connection_uri())
        .await
        .expect("failed to connect to temp db");

    // ... do the rest of your test

    // db is shut down and files cleaned up upon drop at the end of the test
}

Examples:

  • A simple diesel example with axum
  • A more complicated "task queue" example using triggers and LISTEN/NOTIFY with sqlx and axum

See the tests/ directory for complete library usage.

You might also like...
Distributed, version controlled, SQL database with cryptographically verifiable storage, queries and results. Think git for postgres.

SDB - SignatureDB Distributed, version controlled, SQL database with cryptographically verifiable storage, queries and results. Think git for postgres

Manage Redshift/Postgres privileges in GitOps style written in Rust

grant-rs An open-source project that aims to manage Postgres/Redshift database roles and privileges in GitOps style, written in Rust. Home | Documenta

cogo rust coroutine database driver (Mysql,Postgres,Sqlite)

cdbc Coroutine Database driver Connectivity.based on cogo High concurrency,based on coroutine No Future'q,Output=*,No async fn, No .await , no Poll*

A Rust application that inserts Discogs data dumps into Postgres

Discogs-load A Rust application that inserts Discogs data dumps into Postgres. Discogs-load uses a simple state machine with the quick-xml Rust librar

Command-line tool to make Rust source code entities from Postgres tables.

pg2rs Command-line tool to make Rust source code entities from Postgres tables. Generates: enums structs which can be then used like mod structs; use

An easy-to-use, zero-downtime schema migration tool for Postgres

Reshape is an easy-to-use, zero-downtime schema migration tool for Postgres. It automatically handles complex migrations that would normally require downtime or manual multi-step changes.

postgres-ical - a PostgreSQL extension that adds features related to parsing RFC-5545 « iCalendar » data from within a PostgreSQL database

postgres-ical - a PostgreSQL extension that adds features related to parsing RFC-5545 « iCalendar » data from within a PostgreSQL database

The simplest implementation of LLM-backed vector search on Postgres.

pg_vectorize under development The simplest implementation of LLM-backed vector search on Postgres. -- initialize an existing table select vectorize.i

Postgres Foreign Data Wrapper for Clerk.com API

Pre-requisites Postgres-15 Rust pgrx Getting Started To run the program locally, clone the repository git clone https://github.com/tembo-io/clerk_fdw.

Owner
Harry Stern
Harry Stern
Engula empowers engineers to build reliable and cost-effective databases.

Engula is a storage engine that empowers engineers to build reliable and cost-effective databases with less effort and more confidence. Engula is in t

Engula 706 Jan 1, 2023
Provides a Rust-based SQLite extension for using Hypercore as the VFS for your databases.

SQLite and Hypercore A Rust library providing SQLite with an virtual file system to enable Hypercore as a means of storage. Contributing The primary r

Jacky Alciné 14 Dec 5, 2022
Optimistic multi-version concurrency control (MVCC) for main memory databases, written in Rust.

MVCC for Rust This is a work-in-progress the Hekaton optimistic multiversion concurrency control library in Rust. The aim of the project is to provide

Pekka Enberg 32 Apr 20, 2023
This project provides a Rust-based solution for migrating MSSQL databases to MySQL.

MSSQL to MySQL Database Migration A Rust project to migrate MSSQL databases to MySQL, including table structures, column data types, and table data ro

Bitalizer 2 Jul 10, 2023
CLI tool to work with Sled key-value databases.

sledtool CLI tool to work with Sled key-value databases. $ sledtool --help Usage: sledtool <dbpath> <command> [<args>] CLI tool to work with Sled da

Vitaly Shukela 27 Sep 26, 2022
Sled - the champagne of beta embedded databases

key value buy a coffee for us to convert into databases documentation chat about databases with us sled - it's all downhill from here!!! An embedded d

Tyler Neely 6.6k Jan 8, 2023
Replibyte - a powerful tool to seed your databases

Seed Your Development Database With Real Data ⚡️ Replibyte is a powerful tool to seed your databases with real data and other cool features ?? Feature

Qovery 3.4k Jan 9, 2023
rust-postgres support library for the r2d2 connection pool

r2d2-postgres Documentation rust-postgres support library for the r2d2 connection pool. Example use std::thread; use r2d2_postgres::{postgres::NoTls,

Steven Fackler 128 Dec 26, 2022
A Pub/Sub library for Rust backed by Postgres

Unisub Unisub is a Pub/Sub library for Rust, using Postgres as the backend. It offers a convenient way to publish and subscribe to messages across dif

Nick Rempel 12 Oct 6, 2023
Making Postgres and Elasticsearch work together like it's 2021

Making Postgres and Elasticsearch work together like it's 2021 Readme ZomboDB brings powerful text-search and analytics features to Postgres by using

ZomboDB 4.2k Jan 2, 2023