Skip to content

ConnectionPool crash on keepalive with non-empty request queue #472

Open
@lovetodream

Description

@lovetodream

Describe the issue

The connection pool crashes when a lot of requests occur in a burst

Vapor version

n/a

Operating system and version

macOS 14.4.1

Swift version

Swift Package Manager - Swift 5.10.0-dev

Steps to reproduce

Run the following script:

let config = PostgresClient.Configuration(...)
var logger = Logger(label: "connection-pool-crash")
logger.logLevel = .debug

let client = PostgresClient(
    configuration: config,
    backgroundLogger: logger
)
let task = Task {
    await client.run()
}

func runOnce() async throws {
    let logger = logger
    try await withThrowingDiscardingTaskGroup { group in
        for _ in 0..<20 {
            group.addTask {
                let hello = try await client.withConnection { db in
                    try await db.query("SELECT 'hello'", logger: logger)
                }.collect().first?.decode(String.self)
                print(hello)

                // wait for the connection pool to do ping pong and close
                try await Task.sleep(for: .seconds(idleTimeout + 1))
            }
        }
    }
}

for i in 0..<50 {
    try await runOnce()
    print("Burst done: \(i + 1)/50")
}

task.cancel()

Outcome

A crash happens (typically on the start of the second burst, after running the script for approx. 1.5 min), the crash occurs in PoolStateMachine.connectionKeepAliceTimerTriggered(_:) due to a non-empty request queue. My assumption is, that there's a race condition when the timer triggers the ping and the next batch of requests is "inserted" into the pool.

Additional notes

No response

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions