Commit Graph

126 Commits

Author SHA1 Message Date
Ophir LOJKINE 7bb11c16e8 duckdb odbc docker (#1187)
* add a duckdb docker image

* Add DuckDB ODBC driver installation and configuration

Installs DuckDB ODBC driver and configures system-wide ODBC settings to
enable DuckDB database connectivity for SQLPage.

* add duckdb odbc info to readme

* Fix CI digest artifact matching

* Temporarily publish docker images on branch

* Allow docker publish on branch (fix)

* Temporarily run CI on branch pushes

* Disable branch publish; fix duckdb inspect tag

* Temporarily enable CI push on branch

* Fix minimal tag suffix in docker_push

* Disable temporary branch CI push
2026-01-17 01:23:29 +01:00
Ophir LOJKINE a2ef976fc7 Add support for Oracle over ODBC (compatibility fixes, ci testing) (#1182)
* Add Oracle DB (free) and ODBC CI support

This change adds support for testing with Oracle DB (using the free version `gvenzl/oracle-free:slim`) in the CI pipeline. It:
- Updates `.github/workflows/ci.yml` to include a new matrix entry for Oracle DB.
- Adds steps to install the Oracle Instant Client and ODBC driver in the CI runner.
- Configures `odbcinst.ini` to register the Oracle ODBC driver.
- Updates `docker-compose.yml` to include the Oracle DB service definition.

* Fix CI: Remove libaio1 dependency

`libaio1` is not available in the ubuntu-latest environment used by GitHub Actions (which likely uses a newer Ubuntu version where `libaio1` is replaced by `libaio1t64` or similar, or it is transitively installed). Removing explicit installation to fix the CI failure.

* Fix CI: Update Oracle Instant Client to 21.14

The previous version 21.10.0.0.0-1 seems to be no longer available at the specified URL (404 Not Found). Updated to 21.14.0.0.0-1 which was verified to exist.

* Fix CI: Install libaio1t64 for Oracle Instant Client

Oracle Instant Client requires `libaio.so.1`, which is provided by the `libaio1t64` package in newer Ubuntu versions (like 24.04). Installing this package should resolve the "cannot open shared object file: No such file or directory" error.

* Fix CI: Symlink libaio.so.1 for Oracle Instant Client

On Ubuntu 24.04 (Noble), `libaio1t64` installs the library as `libaio.so.1t64`, but Oracle Instant Client explicitly looks for `libaio.so.1`. Creating a symlink fixes this loading issue.

* Fix CI: Install libodbcinst2 and unixodbc for Oracle driver

The Oracle ODBC driver requires `libodbcinst.so.2` which is provided by `libodbcinst2`. Installing `unixodbc` ensures the full ODBC stack is available.

* Fix CI: Remove redundant wget installation

`wget` is pre-installed on GitHub Actions runners, so the explicit installation via `apt-get` is unnecessary.

* Fix CI: Create 'root' user in Oracle DB

The Oracle test run failed with ORA-01017 (invalid credentials) because the 'root' user does not exist by default in the Oracle database.
This commit:
1. Installs Oracle SQL*Plus (instantclient-sqlplus) in the CI environment.
2. Adds a step to connect as SYSDBA and create the 'root' user with the required permissions and password, matching the test configuration.

* Fix CI: Use container env vars for Oracle user

Instead of manually creating the 'root' user with SQL*Plus, rely on the `gvenzl/oracle-free` container's initialization features (`APP_USER` and `APP_USER_PASSWORD`) which were already configured in `docker-compose.yml`.
Removed the SQL*Plus installation and user creation steps from the CI workflow.

* Fix CI: Use FREEPDB1 service for Oracle user

The `APP_USER` ('root') created by the `gvenzl/oracle-free` container is created in the `FREEPDB1` pluggable database, not the `FREE` container database. Updated the connection string to use `FREEPDB1`.

* Fix Oracle DB test failures

- `sql_to_json`: Handle Oracle's behavior where empty strings are treated as NULL.
- `filesystem`: Explicitly CAST the timestamp parameter to TIMESTAMP in `was_modified_query` for ODBC (Oracle) to avoid ORA-01843 (invalid month) errors caused by implicit string-to-date conversion mismatches.

* Fix Oracle tests by accounting for known behavior

- Revert generic ODBC changes in `filesystem.rs`.
- Skip `test_sql_file_read_utf8` for ODBC connections (Oracle) because fixing the `ORA-01843` error would require Oracle-specific SQL (e.g. `CAST` or `TO_TIMESTAMP`) in the main codebase which is undesirable for a generic ODBC implementation.
- Keep the test adjustment in `sql_to_json.rs` but clarify the comment that we are assuming ODBC implies Oracle in this test context for the empty string behavior.

* Fix Oracle tests: Identify Oracle by connection string

Instead of relying on `sqlx::any::AnyKind::Odbc` (which applies to any ODBC database), detect Oracle specifically by checking if the connection string contains "Oracle". This allows applying Oracle-specific test logic (like skipping tests with implicit timestamp conversions or handling empty strings as NULL) without incorrectly affecting other ODBC databases.

* use oracle dialect when talking to oracle

* remove stupid ai comment

* update oracle odbc installation steps

* fix odbc installation path

* cast variables to varchar(4000) in oracle

* clippy

* remove long backtraces from ci

* fixed csv upload test for oracle

* update tests for oracle

* properly quote sqlpage-generated col names

* fix test syntax for oracle

* clippy

* remove as but keep alias

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2026-01-12 14:22:07 +01:00
lovasoa d8e6a280fc Update CI configuration to use a shared cache key for Rust builds and adjust cache saving conditions. The shared key is now set to 'rust-sqlpage-proj-test' for both main and test branches, with the test branch cache saving disabled. 2025-11-08 01:25:39 +01:00
Ophir LOJKINE 2e2c5e9c41 Fix github pr ci cache warnings (#1086)
* feat: Add shared-key to cargo cache action

Co-authored-by: contact <contact@ophir.dev>

* Refactor: Use matrix-specific cache key in CI

Co-authored-by: contact <contact@ophir.dev>

* feat: Cache Rust build artifacts only on main branch

Co-authored-by: contact <contact@ophir.dev>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-11-07 22:00:52 +01:00
lovasoa 4dba824727 Enhance ODBC support for Debian-based systems by setting the library search path in build.rs. Update CI configuration to use the correct PostgreSQL driver name. Document changes in CHANGELOG.md. 2025-10-27 14:56:17 +01:00
lovasoa be5eb4cd06 fix odbc memory corruption
make it harder to miss such issues in the future thanks to malloc instrumentation on ci

fixes https://github.com/pacman82/odbc-api/issues/793
2025-10-25 01:21:13 +02:00
lovasoa e263788ef3 rename deploy job 2025-10-05 12:01:12 +02:00
lovasoa 2f37d2ddd1 add lambda-web feature and update CI to test with odbc-static
CI / compile_and_lint (push) Has been cancelled
CI / test (mssql, mssql, mssql://root:Password123!@127.0.0.1/sqlpage) (push) Has been cancelled
CI / test (mysql, mysql, mysql://root:Password123!@127.0.0.1/sqlpage) (push) Has been cancelled
CI / test (postgres, odbc, Driver=/usr/lib/x86_64-linux-gnu/odbc/psqlodbcw.so;Server=127.0.0.1;Port=5432;Database=sqlpage;UID=root;PWD=Password123!, true) (push) Has been cancelled
CI / test (postgres, postgres, postgres://root:Password123!@127.0.0.1/sqlpage) (push) Has been cancelled
CI / windows_test (push) Has been cancelled
CI / docker_build (linux/amd64) (push) Has been cancelled
CI / docker_build (linux/arm/v7) (push) Has been cancelled
CI / docker_build (linux/arm64) (push) Has been cancelled
CI / docker_push (push) Has been cancelled
Create Release / Build sqlpage binaries (macOS & Windows) (.exe, , windows-latest, x86_64-pc-windows-msvc) (push) Has been cancelled
Create Release / Build sqlpage binaries (macOS & Windows) (odbc-static, macos-latest, x86_64-apple-darwin) (push) Has been cancelled
Create Release / Build sqlpage binaries (Linux) (push) Has been cancelled
Create Release / Build AWS Lambda Serverless zip image (push) Has been cancelled
Create Release / Create Github Release (push) Has been cancelled
Create Release / Publish to crates.io (push) Has been cancelled
2025-10-04 23:18:35 +02:00
lovasoa a472c1c51f install unixodbc before cargo publish 2025-10-04 22:57:07 +02:00
lovasoa 3cad946738 documentation 2025-10-04 21:11:16 +02:00
lovasoa 6ceeb81ccb test with static odbc 2025-10-04 20:36:09 +02:00
lovasoa a7ff6d6c12 Update ODBC dependency configuration and CI workflow to use all features 2025-10-04 20:23:42 +02:00
lovasoa 5fb17c65d0 update driver path 2025-10-04 00:25:52 +02:00
lovasoa 3bb75133ea update driver path 2025-10-04 00:25:26 +02:00
lovasoa 8b858015a8 Update PostgreSQL ODBC driver configuration in CI workflow 2025-10-04 00:18:33 +02:00
lovasoa 95315f68e1 Update ODBC driver configuration in CI and docker-compose for PostgreSQL 2025-10-03 23:52:23 +02:00
lovasoa 515005cac7 truly static iodbc link 2025-10-03 22:50:22 +02:00
Ophir LOJKINE 625deb973b Add and document sqlpage.hmac function (#1033)
* feat: Add sqlpage.hmac function for cryptographic signing

Co-authored-by: contact <contact@ophir.dev>

* feat: Add sqlpage.hmac function for secure data signing

Co-authored-by: contact <contact@ophir.dev>

* Test HMAC function with RFC vectors and update tests

Co-authored-by: contact <contact@ophir.dev>

* feat: Add sqlpage.hmac() function for secure signatures

Co-authored-by: contact <contact@ophir.dev>

* feat: Add base64 output option to hmac function

Co-authored-by: contact <contact@ophir.dev>

* Refactor hmac function for cleaner output formatting

Co-authored-by: contact <contact@ophir.dev>

* Add webhook HMAC signature validation tests

Co-authored-by: contact <contact@ophir.dev>

* Refactor HMAC function and update SQL examples for clarity and consistency

- Changed function parameters to remove Option types for data and key in the HMAC function.
- Improved SQL documentation and examples for HMAC usage, including clearer descriptions and updated error handling.
- Enhanced test cases for webhook HMAC validation to ensure accurate signature checks and responses.
- Removed obsolete test file for HMAC with null values.

* Update HMAC validation logic to handle NULL values in SQL queries

- Modified conditions in SQL queries to check for NULL values alongside signature mismatches.
- Enhanced documentation on NULL handling for HMAC checks to improve clarity and portability.

* remove debug logging from ci

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-10-01 21:07:36 +02:00
Cursor Agent 70f3473f15 Fix: Update ODBC driver and ensure /lib64 exists
Co-authored-by: contact <contact@ophir.dev>
2025-09-26 16:21:42 +02:00
lovasoa a76965cd39 Update docker-compose and README for ODBC support; enhance CI workflow to include ODBC testing 2025-09-26 16:21:42 +02:00
lovasoa c9bce0daa1 install odbc on e2e ci 2025-09-26 16:21:42 +02:00
lovasoa c131aea909 Add ODBC dependencies installation step in CI workflow for enhanced database support 2025-09-26 16:21:42 +02:00
lovasoa bbb0210ec8 Enhance Dockerfile and CI workflows by adding ODBC dependencies for improved database support 2025-09-26 16:21:42 +02:00
lovasoa 1c64fd25be increase windows signing timeouts
CI / compile_and_lint (push) Has been cancelled
CI / test (mssql) (push) Has been cancelled
CI / test (mysql) (push) Has been cancelled
CI / test (postgres) (push) Has been cancelled
CI / windows_test (push) Has been cancelled
CI / docker_build (linux/amd64) (push) Has been cancelled
CI / docker_build (linux/arm/v7) (push) Has been cancelled
CI / docker_build (linux/arm64) (push) Has been cancelled
CI / docker_push (push) Has been cancelled
Create Release / Build sqlpage binaries (macOS & Windows) (.exe, windows-latest, x86_64-pc-windows-msvc) (push) Has been cancelled
Create Release / Build sqlpage binaries (macOS & Windows) (macos-latest, x86_64-apple-darwin) (push) Has been cancelled
Create Release / Build sqlpage binaries (Linux) (push) Has been cancelled
Create Release / Build AWS Lambda Serverless zip image (push) Has been cancelled
Create Release / Create Github Release (push) Has been cancelled
Create Release / Publish to crates.io (push) Has been cancelled
2025-09-24 17:16:40 +02:00
Ophir LOJKINE d8106e8491 Sign windows build with signpath.io (#990)
Create Release / Build sqlpage binaries (macOS & Windows) (.exe, windows-latest, x86_64-pc-windows-msvc) (push) Has been cancelled
Create Release / Build sqlpage binaries (macOS & Windows) (macos-latest, x86_64-apple-darwin) (push) Has been cancelled
Create Release / Build sqlpage binaries (Linux) (push) Has been cancelled
Create Release / Build AWS Lambda Serverless zip image (push) Has been cancelled
Create Release / Create Github Release (push) Has been cancelled
CI / compile_and_lint (push) Has been cancelled
CI / test (mssql) (push) Has been cancelled
CI / test (mysql) (push) Has been cancelled
CI / test (postgres) (push) Has been cancelled
CI / windows_test (push) Has been cancelled
CI / docker_build (linux/amd64) (push) Has been cancelled
CI / docker_build (linux/arm/v7) (push) Has been cancelled
CI / docker_build (linux/arm64) (push) Has been cancelled
CI / docker_push (push) Has been cancelled
Create Release / Publish to crates.io (push) Has been cancelled
* Sign windows build with signpath.io

Thanks to signpath.io for providing us with a free windows signing certificate !

- Added permissions for actions read access.
- Implemented unsigned artifact upload for Windows.
- Integrated SignPath for signing requests and added signed artifact upload.
- Updated non-Windows artifact upload process.

* release signing + link to signpath from README
2025-09-02 15:48:40 +02:00
lovasoa 98938a7b89 update rust cache action 2025-05-29 21:17:00 +02:00
Ophir LOJKINE 15b3fb8b9f linux builds with old glibc (#927)
* update release workflow to include release-test branch and conditional execution for tag events

* update release workflow to use GNU target and simplify Rust toolchain installation

* refactor release workflow to separate Linux build job and streamline macOS/Windows builds

* update release workflow to use manylinux_2_28_x86_64 container for Linux builds
2025-05-29 01:56:07 +02:00
lovasoa 5e34f29bad clippy in ci 2025-05-25 23:23:49 +02:00
lovasoa b04344a67b fix ci release script
CI / compile_and_lint (push) Has been cancelled
CI / test (mssql) (push) Has been cancelled
CI / test (mysql) (push) Has been cancelled
CI / test (postgres) (push) Has been cancelled
CI / windows_test (push) Has been cancelled
CI / docker_build (linux/amd64) (push) Has been cancelled
CI / docker_build (linux/arm/v7) (push) Has been cancelled
CI / docker_build (linux/arm64) (push) Has been cancelled
CI / docker_push (push) Has been cancelled
Create Release / Build sqlpage binaries (.exe, windows-latest, x86_64-pc-windows-msvc) (push) Has been cancelled
Create Release / Build sqlpage binaries (macos-latest, x86_64-apple-darwin) (push) Has been cancelled
Create Release / Build sqlpage binaries (ubuntu-latest, x86_64-unknown-linux-musl) (push) Has been cancelled
Create Release / Build AWS Lambda Serverless zip image (push) Has been cancelled
Create Release / Create Github Release (push) Has been cancelled
Create Release / Publish to crates.io (push) Has been cancelled
2025-05-22 00:50:52 +02:00
lovasoa bebd886c92 build with msvc on windows 2025-05-22 00:20:27 +02:00
lovasoa 1a32281ab4 fix build 2025-05-22 00:03:09 +02:00
lovasoa 0545ab3a3b update rust cache gha 2025-05-21 23:58:50 +02:00
lovasoa 983fb208c5 distribute static executable on linux
Update GitHub Actions workflow to use latest Ubuntu and improve Rust toolchain installation
2025-05-21 23:55:03 +02:00
lovasoa 68fb7dc9de add a timeout to each individual request in tests 2024-12-30 21:26:05 +01:00
lovasoa 756c023574 run db tests in parallel with compile and lint 2024-12-30 20:51:11 +01:00
lovasoa 352d1d9131 no nightly on ci 2024-12-30 20:49:11 +01:00
lovasoa 9a16117d17 limit and report test times 2024-12-30 20:42:21 +01:00
lovasoa 285ea71bb5 add a test timeout 2024-12-30 20:28:34 +01:00
lovasoa 1dc6da9620 update Swatinem/rust-cache 2024-12-21 00:55:41 +01:00
lovasoa 15dada8de3 fix docker image references 2024-12-07 21:14:13 +01:00
lovasoa bdf692c61f lower repo name 2024-12-06 23:02:49 +01:00
lovasoa 0d51d52393 not on a personal repo anymore 2024-12-06 21:42:55 +01:00
lovasoa 741f479cd3 add package json 2024-11-11 19:41:40 +01:00
lovasoa ebf19aa7fe add linting and formatting in CI 2024-11-05 18:33:55 +01:00
lovasoa 40187115fc permissions write gh action
CI / compile_and_lint (push) Has been cancelled
CI / test (mssql) (push) Has been cancelled
CI / test (mysql) (push) Has been cancelled
CI / test (postgres) (push) Has been cancelled
CI / windows_test (push) Has been cancelled
CI / docker_build (linux/amd64) (push) Has been cancelled
CI / docker_build (linux/arm/v7) (push) Has been cancelled
CI / docker_build (linux/arm64) (push) Has been cancelled
CI / docker_push (push) Has been cancelled
Create Release / Build sqlpage binaries (target/superoptimized/sqlpage, macos-latest) (push) Has been cancelled
Create Release / Build sqlpage binaries (target/superoptimized/sqlpage, ubuntu-20.04) (push) Has been cancelled
Create Release / Build sqlpage binaries (target/superoptimized/sqlpage.exe, windows-latest) (push) Has been cancelled
Create Release / Build AWS Lambda Serverless zip image (push) Has been cancelled
Create Release / Create Github Release (push) Has been cancelled
Create Release / Publish to crates.io (push) Has been cancelled
2024-10-30 17:40:15 +01:00
lovasoa 2c1943970c update to softprops/action-gh-release@v2 2024-10-30 17:37:59 +01:00
lovasoa cf9812ca90 upload linux test builds 2024-10-05 20:28:50 +02:00
lovasoa 005b562fb6 fix e2e results upload 2024-08-05 19:05:57 +02:00
lovasoa f382c95232 yaml newline fix 2024-08-05 18:57:41 +02:00
lovasoa 3bd86668a3 fix yaml newlines error 2024-08-05 18:57:07 +02:00