Weird connection slowness with PG14 and docker

[pimeys]

Hi,

So, I've been investigating a weird issue with tokio-postgres today, using the official docker images postgres:13 and postgres:14. The benchmark:

use criterion::{black_box, criterion_group, criterion_main, Criterion};
use tokio::runtime::Runtime;
use tokio_postgres::{config::SslMode, Client, Config, NoTls};

async fn connect(port: u16) -> anyhow::Result<Client> {
    let mut config = Config::new();

    config.user("postgres");
    config.password("prisma");
    config.host("127.0.0.1");
    config.port(port);
    config.dbname("postgres");

    let (client, connection) = config.connect(NoTls).await.unwrap();

    tokio::spawn(async move {
        if let Err(e) = connection.await {
            eprintln!("connection error: {}", e);
        }
    });

    Ok(client)
}

fn full_connect_test(rt: &Runtime, port: u16) {
    rt.block_on(async {
        connect(port).await.unwrap();
    })
}

fn _query_test(rt: &Runtime, client: &Client, query: &str) {
    rt.block_on(async {
        client.query(query, &[]).await.unwrap();
    });
}

fn criterion_benchmark(c: &mut Criterion) {
    let rt = Runtime::new().unwrap();

    let pg13 = 5435;
    let pg14 = 5437;

    c.bench_function("pg14 full connect", |b| {
        b.iter(|| full_connect_test(&rt, black_box(pg14)))
    });

    c.bench_function("pg13 full connect", |b| {
        b.iter(|| full_connect_test(&rt, black_box(pg13)))
    });
}

criterion_group!(benches, criterion_benchmark);
criterion_main!(benches);

If running this benchmark over dockerized postgres images, versions 13 and 14, we get:

     Running unittests (target/release/deps/pg-3f2d8fd3a932abaa)
pg14 full connect       time:   [2.4528 ms 2.4639 ms 2.4765 ms]                               
                        change: [+0.2159% +0.8617% +1.5043%] (p = 0.01 < 0.05)
                        Change within noise threshold.
Found 7 outliers among 100 measurements (7.00%)
  4 (4.00%) high mild
  3 (3.00%) high severe

Benchmarking pg13 full connect: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 6.6s, enable flat sampling, or reduce sample count to 60.
pg13 full connect       time:   [1.3142 ms 1.3217 ms 1.3289 ms]                               
                        change: [+1.4325% +2.2729% +3.0495%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 3 outliers among 100 measurements (3.00%)
  2 (2.00%) low mild
  1 (1.00%) high mild

This is +1ms more per connection, which adds up a lot in CI scenarios with thousands of tests opening new connections... Even more interesting here is that without docker, both versions connect in about 0.8 ms. I've tried to take pieces out from tokio-postgres to see where the slow comes from, but in general TCP with or without nagle gets me to the same result.

If this is not the right repo for the issue, would be nice to know if others have witnessed something similar.

[sfackler]

Is there any reason to think this is a problem on the rust-postgres side vs the postgres backend side?

[sfackler]

Postgres 14 did switch to using SCRAM for auth by default which adds several round trips to the login process if relevant.

[pimeys]

Ah, thanks. I'll read about it! No, this is probably not here, but I couldn't find much info yet...

[pimeys]

It was a missing feature in the docker image. Closing.