multi: add database test helpers

[GustavoStingelin]

Change Description

Add sqltest package with RunDatabaseTest function for cross-database integration testing. This utility automatically runs tests against both PostgreSQL and SQLite databases to ensure repository interactions work consistently across different database engines.

This is part of the preparation work for the upcoming SQLization #1015.

features:

• Cross-database compatibility: Tests run against PostgreSQL and SQLite automatically
• Test isolation: Each test gets a fresh, isolated database instance
• Parallel execution: Tests can run safely in parallel with proper isolation
• Performance optimized: PostgreSQL container is shared, deterministic database names enable Go test caching

The main entry point RunDatabaseTest(t *testing.T, testFunc DBTestFunc) takes a test function and executes it against both database types, making it easy to catch database-specific issues early.

Testing

Run integration tests to verify cross-database functionality:

make integration

[GustavoStingelin]

FYI @yyforyongyu

[Roasbeef]

Any reason we aren't just re-using the packages we've created in lnd or tapd for these purposes?

eg:

• https://github.com/lightningnetwork/lnd/blob/master/sqldb/interfaces.go

It's already a sub-module. Has support for handling migrations, has the replacement logic I mentioned to allow for a unified SQL dialect, etc, etc.

I know we're also working to further unify some common operations w.r.t it. cc @guggero

[GustavoStingelin]

TLDR: this PR only sets up the initial test backends. It is not intended to define stable interfaces or include migration logic. Other PR will be opened later, adding more stuff.

This change adds support for Postgres and SQLite as test backends. It does not yet introduce a sqldb interface (see the TODOs), which is why the native *sql.DB is returned directly. It also does not add SQL adaptation helpers or a migration framework at this stage.

The ApplySQL helper is included only to simplify running statements and verifying that backend tests execute correctly.

For integration tests I usually rely on Testcontainers since they make it easy to spin up services like Postgres or even bitcoind later, which helps build a reproducible environment. However, I noticed that LND uses embedded-postgres as a fixture. If needed, I can switch the Postgres backend here to use that instead. Have you guys already considered the trade-offs between testcontainers and embedded-postgres earlier?

[yyforyongyu]

Have you guys already considered the trade-offs between testcontainers and embedded-postgres earlier

Curious about the tradeoffs here

[GustavoStingelin]

IMO, favoring the testcontainers:

embedded-postgres

• Uses pre-built binaries in a complex supply chain:
  • binaries need to be carefully released to multiple systems and architectures, building postgres by the hands of few contributors
  • need to support these binaries and add some wrapping
• Using binaries should be faster, right? but in my tests it takes a little bit longer than spawning the testcontainers

testcontainers

• Relies on Docker/Podman API, which means we have a strong guarantees for cross-platform compatibility and consistent behavior
• Runs in an isolated docker environment
• The docker image is fully customizable to attend our requirements, if needed
• Can be useful to set a pattern to make the integration tests with bitcoind and other nodes
• Has very good Golang tooling and implements an easy way to manage containers and postgres
• More contributors, more standard in industry, more docs and has a website :D

[mohamedawnallah]

Relies on Docker/Podman API, which means we have a strong guarantees for cross-platform compatibility and consistent behavior

I think testcontainers might have potential issues when running in a Docker-in-Docker environment, particularly in CI pipelines, due to its reliance on container runtime engine. This could be especially problematic if we decide to run our workflows on Windows or macOS. So I would give +1 for embedded-postgres

[GustavoStingelin]

The LND unit test command already depends on the Docker socket and pipe runs on Ubuntu. Is the intention here to remove this requirement?

On the other hand, using Docker-in-Docker is a common practice. Here is an example using dind that works well in my environment:

docker run --rm -it \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v $(pwd):/workspace \
  -w /workspace \
  docker:dind \
  sh -c "apk add --no-cache go && go test -v ./internal/sqltest -tags=integration_test"

On Windows, WSL works fine nowadays. For Mac, there are solutions like Colima, and Apple recently launched a native OCI runtime to run containers.

[guggero]

I've documented our intentions with the shared SQL library here in case it's relevant to the discussion (not really about testcontainers vs. embedded-postgres but the re-usability of SQL related code): lightningnetwork/lnd#9762 (comment)

[GustavoStingelin]

and here is some pipeline examples using test containers

[mohamedawnallah]

I've documented our intentions with the shared SQL library here in case it's relevant to the discussion (not really about testcontainers vs. embedded-postgres but the re-usability of SQL related code): lightningnetwork/lnd#9762 (comment)

👍

On Windows, WSL works fine nowadays. For Mac, there are solutions like Colima, and Apple recently launched a native OCI runtime to run containers.

For GitHub hosted runners, container operations not supported on window runners. On mac runners they are not supported as well on GitHub till it enables nested virtualization on M3/macOS-15 runners:
https://developer.apple.com/documentation/virtualization/vzgenericplatformconfiguration/isnestedvirtualizationsupported

This is a common issue as yet. Seeing: https://github.com/marketplace/actions/setup-docker-on-macos

[GustavoStingelin]

So, if Docker runtime is a blocker, I will switch the backend to embedded-Postgres.

[GustavoStingelin]

just removed ApplySQL and pingUntil functions since it's not strong needed in this PR and the testcontainers already implement a good wait strategy.