breaking(sqlite): libsqlite3-sys versioning, feature flags, safety changes

[abonander]

Changes to unreleased code (fold together with related PRs)

• Rename config key common.drivers.sqlite.load-extensions to drivers.sqlite.unsafe-load-extensions
• Added drivers.mysql, drivers.postgres, and support for future external drivers

Breaking Changes

• Changed libsqlite3-sys versioning policy to use version ranges
  • Diesel has been using this successfully for many years: https://github.com/diesel-rs/diesel/blob/cdf7c51c35da7ebc6fb12e71857bfba8487c20cd/diesel/Cargo.toml#L53
• Mark SqliteConnectOptions::extension() and ::extension_with_entrypoint() as unsafe
• Added new non-default features corresponding to conditionally compiled SQLite APIs:
  • sqlite-deserialize enabling SqliteConnection::serialize() and SqliteConnection::deserialize()
  • sqlite-load-extension enabling SqliteConnectOptions::extension() and ::extension_with_entrypoint()
  • sqlite-unlock-notify enables internal use of sqlite3_unlock_notify()
• SqliteValue and SqliteValueRef changes:
  • The sqlite3_value* interface reserves the right to be stateful. Without protection, any call could theoretically invalidate values previously returned, leading to dangling pointers.
  • SqliteValue is now !Sync and SqliteValueRef is !Send to prevent data races from concurrent accesses.
    • Instead, clone or wrap the SqliteValue in Mutex, or convert the SqliteValueRef to an owned value.
  • SqliteValue and any derived SqliteValueRefs now internally track if that value has been used to decode a borrowed &[u8] or &str and errors if it's used to decode any other type.
  • This is not expected to affect the vast majority of usages, which should only decode a single type per SqliteValue/SqliteValueRef.
  • See new docs on SqliteValue for details.
  • Considering filing a RUSTSEC advisory since the previous behavior could theoretically lead to UB in safe code, though it does require some significant contrivances to actually lead to UB.

TODO

• sqlite3_value is not safe to access concurrently
• SqliteValue drop internal Mutex and SqliteValue: Sync, it's largely unnecessary
• list all the issues this fixes

Issues

fixes #3926
fixes #3921
fixes #3905

Superceded PRs

closes #3927
closes #3922
closes #3888
closes #3844

[LecrisUT]

Does the sqlite-deserialize fully conditionalize the compilation of the libsqlite3-sys? Last I've tried to update to 8.6.0 on EPEL9 which does not have that feature, it failed at:

  = note: some arguments are omitted. use `--verbose` to show all linker arguments
  = note: lto-wrapper: warning: using serial compilation of 2 LTRANS jobs
          /usr/bin/ld: /builddir/build/BUILD/atuin-18.6.1/target/rpm/deps/libsqlx_sqlite-f362d1fed65be9de.rlib(sqlx_sqlite-f362d1fed65be9de.sqlx_sqlite.3b75296461245f8c-cgu.0.rcgu.o): in function `sqlx_sqlite::connection::serialize::serialize':
          /usr/share/cargo/registry/sqlx-sqlite-0.8.6/src/connection/serialize.rs:83: undefined reference to `sqlite3_serialize'
          /usr/bin/ld: /builddir/build/BUILD/atuin-18.6.1/target/rpm/deps/libsqlx_sqlite-f362d1fed65be9de.rlib(sqlx_sqlite-f362d1fed65be9de.sqlx_sqlite.3b75296461245f8c-cgu.0.rcgu.o): in function `sqlx_sqlite::connection::serialize::deserialize':
          /usr/share/cargo/registry/sqlx-sqlite-0.8.6/src/connection/serialize.rs:138: undefined reference to `sqlite3_deserialize'
          collect2: error: ld returned 1 exit status
          
  = note: some `extern` functions couldn't be found; some native libraries may need to be installed or have their path specified
  = note: use the `-l` flag to specify native libraries to link
  = note: use the `cargo:rustc-link-lib` directive to specify the native libraries to link with Cargo (see https://doc.rust-lang.org/cargo/reference/build-scripts.html#rustc-link-lib)

warning: `atuin` (bin "atuin") generated 2 warnings
error: could not compile `atuin` (bin "atuin") due to 1 previous error; 2 warnings emitted

I hope 0.9.0 would overcome this issue.