Commit Graph

861 Commits

Author SHA1 Message Date
Ophir LOJKINE bbafdbc4d7 Update OpenTelemetry dependencies 2026-05-29 22:24:53 +02:00
Ophir LOJKINE 40f274e217 Check database filesystem table before preparing queries 2026-05-18 00:09:42 +02:00
Ophir LOJKINE 60be567944 update dependencies 2026-05-16 22:05:49 +02:00
Ophir LOJKINE 4a28658a34 Fix Rust 1.95 CI warnings
Update Rust code for new Clippy lints introduced by the CI toolchain, including Duration unit constructors, loop counter handling, Option checks, and trailing commas.

Stop routine frontend asset downloads from being emitted as Cargo warnings during builds.

Move checkout and rust-cache GitHub Actions to Node 24-compatible releases to avoid runner deprecation warnings.

User-level behavior is unchanged; the only observable change is quieter build logs.
2026-05-16 00:43:04 +02:00
Ophir LOJKINE f6e7abc8b7 Update Rust dependencies
Bump sqlparser from 0.61.0 to 0.62.0, hmac from 0.12 to 0.13, and sha2 from 0.10 to 0.11. Refresh Cargo.lock and remove SQLPage's direct password-hash dependency by using argon2's re-exported password_hash API.

SQLPage parses every .sql file before execution, rewrites SQLPage variables/functions through the sqlparser AST, and serializes the AST back to SQL. The sqlparser bump therefore changes which SQL text reaches the database.

SQLPage-visible sqlparser 0.62 changes for supported dialects:

- PostgreSQL: SQLPage can now parse and pass through LOCK TABLE; PRIMARY KEY/UNIQUE USING INDEX; parenthesized CREATE TABLE ... (LIKE ... INCLUDING/EXCLUDING DEFAULTS); COMMENT ON for additional object types; two-argument TRIM(string, characters); INTERVAL as an unquoted identifier; SETOF return types; ALTER FUNCTION/ALTER AGGREGATE; xml '...' typed strings; COLLATE after compound identifiers; LATERAL function calls WITH ORDINALITY; and pg_hint_plan-style SELECT optimizer hints in leading comments.

- MySQL: SQLPage can now parse and pass through KEY column options in CREATE TABLE and ORDER BY on single-table UPDATE. Incomplete DIV expressions now become SQLPage parser errors instead of panicking the parser.

- SQLite: incomplete REGEXP/MATCH expressions now become SQLPage parser errors instead of panicking the parser; two-argument TRIM(string, characters) is accepted.

- DuckDB: two-argument TRIM(string, characters) is accepted.

- MSSQL: SQLPage can now parse and pass through THROW, WAITFOR, BEGIN/COMMIT/ROLLBACK TRAN, EXEC(@sql) dynamic SQL, OUTPUT clauses on INSERT/UPDATE/DELETE, and dollar-prefixed money constants. Statement-starting keywords such as DECLARE, EXEC, INSERT, UPDATE, DELETE, CREATE, ALTER, RETURN, THROW, RAISERROR, and MERGE are no longer consumed as implicit aliases after SELECT items or table factors, so multi-statement T-SQL batches are less likely to be rejected by SQLPage's pre-parser.

- Oracle: SQLPage can now parse and pass through table aliases on INSERT targets and INSERT INTO (<sub-query>) forms.

- Snowflake: SQLPage can now parse and pass through multi-table INSERT, lambda functions, wildcard EXCLUDE fixes, stage-name/COPY parsing fixes, CHANGES clauses, COPY GRANTS, STORAGE LIFECYCLE POLICY, and text data type modifiers.

- Generic ODBC dialect: SQLPage now accepts the same new GenericDialect syntax as sqlparser 0.62, including UPDATE ... ORDER BY, KEY column options, comma TRIM, CTEs without AS, multi-column SELECT aliases, and xml typed strings. It no longer accepts lambda expressions in GenericDialect; databases detected as Snowflake still use SnowflakeDialect and keep lambda parsing.

SQLPage's special CSV upload handling is unchanged: it still intercepts COPY FROM file targets only. SQLPage's HMAC and password-hash outputs are intended to be unchanged; only imports/API wiring changed for the RustCrypto crates.

Lockfile-only runtime dependency updates include rustls 0.23.37 -> 0.23.40, rustls-pki-types 1.14.0 -> 1.14.1, webpki-roots 1.0.6 -> 1.0.7, aws-lc-rs 1.16.2 -> 1.17.0, tokio 1.51.1 -> 1.52.3, and config 0.15.22 -> 0.15.23. No SQLPage code changed for these; the user-visible surface is the upstream TLS/runtime/config bugfix behavior of those crates.

Validation before amend: cargo fmt --all; cargo clippy --all-targets --all-features -- -D warnings; cargo test.
2026-05-15 22:23:24 +02:00
June 913e0ba888 Fix DuckDB :: casting warning (#1284)
DuckDB natively supports PostgreSQL's :: casting shorthand:
https://duckdb.org/docs/stable/sql/expressions/cast#explicit-casting

Added SupportedDatabase::Duckdb to the allow-list to prevent
unnecessary warning and CAST transformation.

Fixes #1200
2026-05-12 15:11:54 +02:00
Ophir LOJKINE b18e138f34 Fix clippy collapsible-if warnings 2026-04-17 00:20:13 +02:00
Ophir LOJKINE 1e64d7e3ed Fix OpenTelemetry naming in logs and spans 2026-04-16 11:25:31 +02:00
lovasoa 1af44a79d0 fmt 2026-04-13 02:44:47 +02:00
lovasoa 85656c4585 chore: migrate to rust 2024 by fixing implicit borrows and unsafe env modifications 2026-04-13 02:03:54 +02:00
Ophir LOJKINE 320121977c Revert "get_path_segment() and is_path_matching() sqlpage functions (#1266)" (#1274)
This reverts commit a3f91ebea1.
2026-04-11 19:02:07 +02:00
olivierauverlot a3f91ebea1 get_path_segment() and is_path_matching() sqlpage functions (#1266)
* get_path_segment and is_path_matching sqlpage functions

* Fix for an error in example.

* If a segment in the pattern is ''%d'', it will match any non-empty segment that is an integer. If a segment in the pattern is ''%s'', it will match any non-empty segment in the path

---------

Co-authored-by: Olivier Auverlot <olivier.auverlot@icloud.com>
2026-04-11 18:56:13 +02:00
Ophir LOJKINE 0d5373509a new function: sqlpage.regex_match (#1271)
* new function: sqlpage.regex_match

* fix mysql tests

* Update CHANGELOG.md
2026-04-08 21:39:32 +02:00
Ophir LOJKINE 08ae59e696 Suppress startup logs for CLI-only invocations (#1250) 2026-03-19 10:42:29 +01:00
Ophir LOJKINE 3af82f4857 Fix http header only page crashes (#1253)
Return errors early on invalid header values
2026-03-17 16:13:11 +01:00
Ophir LOJKINE 316fa4b5a2 Return BadRequest for non-unicode static paths and add test (#1252)
* Fix non-unicode DB filesystem regression test

* Fix non-unicode DB fs routing test
2026-03-17 13:50:55 +01:00
Ophir LOJKINE db91902d49 Log HTTP responses with correct severity and include status reason (#1251)
* Log non-2xx HTTP responses in request completion log

* fix log message
2026-03-17 11:35:51 +01:00
Ophir LOJKINE e9762528e1 Harden OIDC and fetch debug logging 2026-03-16 17:55:28 +01:00
Ophir LOJKINE 38085d1c9e Merge branch 'main' of github.com:sqlpage/SQLPage 2026-03-16 17:50:05 +01:00
Ophir LOJKINE fda4af3c8a Remove reqwest/tower from default OTLP graph (#1248)
* chore: remove reqwest from default OTLP dependency graph

* fix: run OTLP hyper exporters on tokio async runtime

* fix: propagate OTLP init errors without panic

* fix: surface OTLP exporter failures in default logs

* feat: use awc for OTLP HTTP exports

* refactor: share awc client construction with telemetry
2026-03-16 02:22:37 +01:00
Ophir LOJKINE 8db4f036b8 Remove env_logger and use telemetry logging in tests (#1247)
* Remove env_logger and reuse telemetry logging in tests

* Use test-captured writer for telemetry logs in tests

* Isolate test logging init from OTEL and keep tests quiet
2026-03-15 15:43:39 +01:00
Ophir LOJKINE c026001385 Add support for opentelemetry metrics (#1245)
* feat(telemetry): initialize OpenTelemetry metrics provider

* feat(metrics): implement and integrate HTTP request duration metrics

* feat(metrics): instrument database query durations

* refactor(database): add OTel name mapping helper

* feat(metrics): instrument database connection pool

* feat(telemetry): update OTel collector to receive OTLP metrics

* feat(telemetry): add metrics panels to Grafana dashboard

* refactor(metrics): clean up database instrumentation with helper functions

* fix(metrics): ensure correct db.system.name for ODBC connections by reusing discovery logic

* fix(metrics): remove global OnceLock and pass database type explicitly to pool callbacks

* refactor(metrics): use OpenTelemetry semantic convention constants

* refactor(metrics): eliminate all OTel convention string literals in favor of constants

* fix(telemetry): use standard YAML list notation in OTel collector config to fix parsing error

* fix(telemetry): fix clippy warnings and use latest OTel attribute names

* fix(telemetry): add Default impl for TelemetryMetrics

* Add app-scoped telemetry metrics for HTTP

* Refactor db query metrics recording

* Use semantic convention attribute names

* Track db pool metrics during lifecycle

* Adjust telemetry docker compose

* fix(metrics): configure explicit histogram boundaries and fix database latency measurement

* Enable metrics views for explicit histograms

* Use observable gauge for pool connection count

* Read pool gauge directly from sqlx pool

* telemetry: export OTEL metrics every second in example stack

* Improve Grafana trace list links and readability

* Restore removed DB connection debug logs
2026-03-15 14:17:04 +01:00
Ophir LOJKINE d1e8154222 Add OpenTelemetry distributed tracing support (#1234)
* add OpenTelemetry distributed tracing support

When OTEL_EXPORTER_OTLP_ENDPOINT is set, enables full tracing pipeline
with OTLP export, W3C traceparent propagation, and spans for HTTP
requests, SQL file execution, DB pool acquire, and query execution.
Falls back to env_logger when unset.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix OTel example: tracing init, Dockerfile, Tempo config

- Fix tracing-log bridge initialization order (set subscriber first,
  then LogTracer) to avoid double-set panic
- Add dedicated Dockerfile for example using release profile (avoids
  OOM with superoptimized LTO in Docker)
- Use debian:trixie-slim runtime for glibc compatibility
- Fix nginx image to nginx:otel (official image with OTel module)
- Fix nginx.conf: move otel_trace directives into location block
- Pin Tempo to 2.6.1 (latest has partition ring issues in single-node)
- Fix otel-collector exporter alias (otlp → otlp_grpc)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* propagate trace context to PostgreSQL via application_name

After acquiring a DB connection, set the W3C traceparent as the
PostgreSQL application_name (or MySQL session variable). This makes
trace IDs visible in pg_stat_activity and PostgreSQL logs, enabling
direct correlation between Grafana Tempo traces and database-side
monitoring.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* rewrite OTel example README with setup guides for all major providers

Comprehensive documentation covering:
- Step-by-step quick start for the Docker Compose example
- How OpenTelemetry works (spans, collectors, backends)
- Setup guides for Grafana Tempo, Jaeger, Grafana Cloud, Datadog,
  Honeycomb, New Relic, and Axiom with exact env vars and doc links
- PostgreSQL trace correlation via application_name
- Environment variable reference
- Troubleshooting section

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix todo example: use :title (POST variable) instead of $title

The form submits via POST, so the title field must be referenced with
the : prefix (POST parameter) rather than $ (GET parameter).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* set code.filepath and code.lineno span attributes to user SQL files

OTel span attributes now reference the user's .sql file path and line
number instead of the SQLPage Rust source code. Also improves span
naming, adds JSON log formatting, custom root span builder, and
Grafana dashboard provisioning for the OTel example.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* npm run fix

* use stable OTel semantic convention attribute names

- code.filepath → code.file.path, code.lineno → code.line.number
- db.statement → db.query.text, db.system → db.system.name
- Disable auto code location (.with_location(false)) so spans
  reference user SQL files, not SQLPage Rust source
- Remove redundant sqlpage.file attribute (code.file.path suffices)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* use OTel semantic convention values for db.system.name

Use well-known values from the OpenTelemetry registry instead of raw
DBMS name strings. Cast line numbers to i64 for correct span recording.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* remove json-subscriber dependency

The custom logfmt layer in telemetry.rs replaces it with zero extra
dependencies and precise control over field selection and ordering.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* add Loki + Promtail log aggregation to OTel example

Adds two new services (Loki, Promtail) to scrape SQLPage container logs
and display them in Grafana alongside traces. The home dashboard now
shows a logs panel with trace_id derived fields linking to Tempo.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* add OTel spans for request parsing, rendering, sqlpage functions, and OIDC

Add targeted spans to account for previously untraced time:
- http.parse_request: request/form parsing before SQL execution
- render: template rendering and response streaming
- subprocess: sqlpage.exec() with process.command attribute
- http.client: sqlpage.fetch()/fetch_with_meta() with OTel HTTP client
  semantic conventions (http.request.method, url.full, http.response.status_code)
- sqlpage.file: sqlpage.run_sql() nested file execution
- oidc.callback + http.client: OIDC token exchange

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* add oidc.jwt.verify span for OIDC token verification

This span covers JWT signature verification and claims validation,
which runs on every authenticated request via get_token_claims().

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* add enduser.id attribute to oidc.jwt.verify span

Records the OIDC subject claim (sub) as enduser.id after successful
JWT verification, following OTel semantic conventions.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* add OTel user.* attributes to oidc.jwt.verify span

Record user.id (sub), user.name (preferred_username), user.full_name
(name), and user.email from OIDC claims after JWT verification.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* add sqlpage.file.load span and attributes to http.parse_request

The gap before http.parse_request was the SQL file cache lookup -
now covered by the sqlpage.file.load span with code.file.path.

http.parse_request now records http.request.method and content_type,
which helps identify slow multipart/form-data parsing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix clippy pedantic warnings

* unify log format: logfmt with colors, no OTel noise

Use the custom logfmt layer for both OTel and non-OTel modes instead
of falling back to env_logger. This eliminates the tracing→log bridge
dumping all span fields (user agents, otel.kind, request_id, etc.)
and only shows: ts, level, target, msg, method, path, status,
file, client_ip, and trace_id (when valid).

Adds terminal color support (bold red for errors, green for info,
dim for timestamps/targets). Emits one log line per completed
successful request. Errors are logged once by the error handler.
Suppresses trace_id=000...0 when no real trace context exists.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* use .instrument() instead of .entered() for async spans

Span guards from .entered() do not propagate correctly across await
points. Switch to tracing::Instrument to ensure spans are properly
associated with their async tasks throughout their lifetime.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* preserve multi-line error formatting in terminal log output

When stderr is a terminal and the log message contains newlines
(e.g. SQL syntax errors with source highlighting and arrows),
print the metadata on the first line and the message below with
its original formatting. Machine output (non-terminal) remains
single-line logfmt.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* use root Dockerfile for OTel example, add CARGO_PROFILE build arg

Remove the example's custom Dockerfile and use the main one with a
CARGO_PROFILE=release build arg to avoid OOM from fat LTO in
memory-constrained Docker environments. The build scripts now
read CARGO_PROFILE from the environment, defaulting to
superoptimized for backward compatibility.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* add db.query.parameter and db.response.returned_rows span attributes

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Add official blog post about tracing

* add http.request.body.size and url.query span attributes

Add http.request.body.size to HTTP client spans (fetch, fetch_with_meta)
and to the server-side http.parse_request span (from Content-Length header).
Add url.query to the http.parse_request span.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Replace Promtail with the OpenTelemetry Collector

* Refactor telemetry logging helpers

* Clamp traced fetch body size

* Clamp traced fetch_with_meta body size

* Silence noisy PostgreSQL collector logs

* make startup logs parseable

* update terminal log formats

* Ingest real PostgreSQL logs with trace IDs

* Use raw traceparent for PostgreSQL tracing

* Update opentelemetry example for PostgreSQL query events

* Add nginx logs to opentelemetry example

* Parse nginx error log severity correctly

* Log all span fields when debug logging is enabled

* Rename telemetry example directory

* Fix PostgreSQL Loki log ingestion

* `LOG_LEVEL` is now the primary environment variable for configuring SQLPage's log filter. `RUST_LOG` remains supported as an alias.

* Skip empty trace IDs in logs

* add db errors to otel traces

* add healthcheck

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-14 15:52:32 +01:00
Ophir LOJKINE 99ad2685a0 Add optional mode argument to persist_uploaded_file (#1241)
* Add optional mode argument to persist_uploaded_file

This change adds an optional `mode` argument to the `persist_uploaded_file` function, allowing users to specify the Unix file permissions in octal notation when saving uploaded files.

- Updated `sqlpage.persist_uploaded_file` signature to include `mode`.
- Implemented permission setting logic using `std::os::unix::fs::PermissionsExt` (on Unix platforms).
- Default permission is set to "600" (octal `0o600`).
- Added documentation for the new parameter in `examples/official-site/sqlpage/migrations/39_persist_uploaded_file.sql`, including an explanation of octal notation and a link to Wikipedia.
- Added a unit test `test_set_file_mode` to verify the permission setting logic.

Co-authored-by: lovasoa <552629+lovasoa@users.noreply.github.com>

* Address PR feedback: Add integrated test for persist_uploaded_file mode

- Removed unit test from `functions.rs` and added an integrated test in `tests/uploads/mod.rs`.
- Created `tests/uploads/persist_with_mode.sql` for the integrated test.
- Refactored `set_file_mode` to use `#[cfg(unix)]` and `#[cfg(not(unix))]` on the entire function.
- Replied to PR comments.

Co-authored-by: lovasoa <552629+lovasoa@users.noreply.github.com>

* Fix Windows CI: Normalize paths and improve tests

- Normalized `persist_uploaded_file` return path to use forward slashes for URL compatibility.
- Updated `test_persist_uploaded_file_mode` to handle platform-specific path separators.
- Fixed clippy warning `expect_fun_call` in tests.

Co-authored-by: lovasoa <552629+lovasoa@users.noreply.github.com>

* Address PR feedback: Assert file contents and ignore test uploads

- Added assertion to verify persisted file contents in `test_persist_uploaded_file_mode`.
- Removed accidental test file from git and added `tests_uploads/` to `.gitignore`.
- Replied to PR comments.

Co-authored-by: lovasoa <552629+lovasoa@users.noreply.github.com>

* Address PR feedback: Revert breaking change to return value

- Reverted normalization of `persist_uploaded_file` return value to avoid a breaking change.
- Reverted corresponding test changes that relied on normalized paths.
- Replied to PR comments.

Co-authored-by: lovasoa <552629+lovasoa@users.noreply.github.com>

* Address PR feedback: Query results as JSON in integrated test

- Updated `test_persist_uploaded_file_mode` to directy request and verify JSON results.
- Replied to PR comments.

Co-authored-by: lovasoa <552629+lovasoa@users.noreply.github.com>

* Fix Windows CI: Robust path handling in tests

- Improved integrated test to correctly request JSON results.
- Added platform-specific path normalization when verifying files on disk.
- Ensured `persist_uploaded_file` return value remains OS-specific to avoid breaking changes.

Co-authored-by: lovasoa <552629+lovasoa@users.noreply.github.com>

* Address PR feedback: Delete leftover test file

- Deleted accidental leftover test file `tests_uploads/2026-03-13_15h47m26s_6JaXODDK.txt`.
- Replied to PR comments.

Co-authored-by: lovasoa <552629+lovasoa@users.noreply.github.com>

---------

Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
2026-03-13 19:57:38 +01:00
Ophir LOJKINE f7e4c2bdd7 explain error propagation logic 2026-03-13 11:15:37 +01:00
Ophir LOJKINE 3eb527490e Fix HTTP status mapping for client request errors (#1240) 2026-03-13 11:11:12 +01:00
Ophir LOJKINE c7e5be793b Return 400 for invalid UTF-8 multipart fields (#1239) 2026-03-12 11:57:29 +01:00
Ophir LOJKINE acb1893e96 Cap parallel OIDC login state cookies (#1238)
* Cap parallel OIDC login state cookies

* Cap parallel OIDC login state cookies

* simplify cookie eviction code
2026-03-12 11:41:13 +01:00
Ophir LOJKINE f2e23c09dc update dependencies 2026-03-08 11:48:53 +01:00
Ophir LOJKINE ba06855f89 fmt 2026-03-08 10:57:36 +01:00
Ophir LOJKINE 52c37e0dd5 use tokio::time::Instant for testable time manipulation, simplify tests
Replace std::time::Instant with tokio::time::Instant in OidcSnapshot so
that tokio::time::pause()/advance() controls elapsed time in tests.
Remove force_expire() — tests advance time past MAX_REFRESH_INTERVAL
instead. Simplify slow discovery test from ~70 to ~30 lines.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 10:49:02 +01:00
Ophir LOJKINE d5f7a84999 rethink OIDC state: std::sync::RwLock<Arc<Snapshot>> for lock-free reads
Replace tokio::sync::RwLock<ClientWithTime> with std::sync::RwLock<Arc<OidcSnapshot>>.
The std lock makes it structurally impossible to hold across await points.
Readers clone an Arc (nanoseconds) and use it freely — no lock contention.
Many previously-async functions become synchronous (get_token_claims,
build_auth_url, handle_unauthenticated_request, handle_oidc_logout, etc).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 10:48:27 +01:00
Ophir LOJKINE 817674cded OIDC: non-blocking background refresh and body read timeout
- OIDC provider metadata refreshes now run in a background task via
  spawn_local, never blocking incoming HTTP requests.
- Multiple concurrent refresh triggers are deduplicated via an AtomicBool.
- The write lock on the OIDC client is only held briefly to swap data,
  not during the upstream HTTP call.
- Add a body-read timeout to OIDC HTTP requests to prevent hangs when
  the provider stalls after sending headers.
- Add tests for both scenarios: slow discovery and slow token endpoint.

Fixes https://github.com/sqlpage/SQLPage/issues/1231

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 10:48:02 +01:00
Ophir LOJKINE 431ab87fa5 Add timeout to OIDC HTTP response body read (#1232)
* add timeout to OIDC HTTP response body read

A stalled OIDC provider that sends HTTP headers but never completes the
body would cause response.body().await to hang forever, freezing the
entire SQLPage process. Add a 5-second timeout on the body stream read
using awc's ClientResponse::timeout().

partially Fixes #1231

* simplify test comment

* simplify test: use token_endpoint_delay instead of Notify gate

* simplify test: use tokio time pause + auto-advance instead of select

* use spawn_local + advance instead of select to detect hang

* use Notify sync point instead of yield loop for deterministic test

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* simplify test: replace Notify+yields with sleep+time-advance

Remove the Notify synchronization and yield_now() calls. Instead,
use a small real-time sleep for TCP to complete, then pause+advance
tokio time. Assert the actual response status instead of is_finished().


* fix clippy: remove unnecessary mut on response
2026-03-08 03:55:40 +01:00
Ophir LOJKINE c822b55b2d Standardize statement position formatting and preserve HTTP statuses (#1226) 2026-03-02 00:14:38 +01:00
lovasoa d1804ba402 change sqlpage.link error message format 2026-03-01 21:18:29 +01:00
lovasoa 03e1d65bce sqlpage.link: improve error message 2026-03-01 19:19:57 +01:00
Ophir LOJKINE 17e6e1e712 Improve SQLPage function argument warnings (#1225)
* improve sqlpage function argument warnings with source context

Made-with: Cursor

* Param extraction: Result-based errors, single caller message, no CompoundIdentifier special case

- expr_to_stmt_param returns Result<StmtParam, ExprToParamError>; error carries only line + kind (UnsupportedExpr, UnemulatedFunction, NamedArgs)
- function_args_to_stmt_params logs one formatted message (ctx.format_param_error) then returns Err
- Single unsupported-expr arm; expr_summary() used for description
- Rename ParamWarnContext to ParamExtractContext

Made-with: Cursor

* Surface param extraction error in parse result; add parse_sql error-message tests; remove are_params_extractable

- When func_call_to_param returns StmtParam::Error, store it and have extract_parameters return Err so parse yields ParsedStatement::Error with specialized message
- Add test_parse_sql_unsupported_expr_in_sqlpage_arg and test_parse_sql_unemulated_function_in_sqlpage_arg
- Remove dead are_params_extractable and its unused import

Made-with: Cursor

* Refactor sqlpage function argument error messages to match user expectations

- Overhauled ExprToParamError formatting to construct exact user-friendly descriptions.
- Removed superfluous anyhow::Context prefixes in func_call_to_param.
- Passed source_path properly through validate_function_calls to ensure file line numbers populate the new error template accurately.
- Renamed error test file to match its dynamic error output signature.
- Removed redundant mut mutability warnings on parsing logic loops.

* improve error messages

* Refactor SqlPageFunctionError representation to clean up 'syntax error' wrappers

- Replaced stringly-typed anyhow errors with a strongly typed SqlPageFunctionError.
- Removed source_path threading completely from the parameter extraction phases, conforming to better separation of concerns.
- Appended file path prefix dynamically at the evaluation stage in clone_anyhow_err strictly when downcasting to SqlPageFunctionError.
- Removed the confusing generic 'Caused by: x.sql contains a syntax error...' wrapper from actual function logic errors.

* Remove redundant 'reorganize' hint from error message

* readd deleted test

* split parameter extraction logic into a separate file

* Add to changelog
2026-03-01 09:52:46 +01:00
Ophir LOJKINE 0994b926c6 Sqlpage port variable parsing (#1206)
* Ignore invalid SQLPAGE_PORT values from Kubernetes service environment variables

Co-authored-by: contact <contact@ophir.dev>

* Use custom visitor for port deserialization instead of serde_json::Value

Co-authored-by: contact <contact@ophir.dev>

* Fix clippy warnings about uninlined format args

Co-authored-by: contact <contact@ophir.dev>

* Add robust port deserialization for Kubernetes

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2026-01-29 18:44:42 +01:00
Guillaume Cornu 94f9988fbb Format protected and public paths with site prefix (#1204)
* Format protected and public paths with site prefix

Update protected and public paths to include site prefix.

* Fix formatting of paths in OIDC module

* Validate that paths start with '/' in config

* Refactor path handling for OIDC configuration

* Update OIDC paths configuration details

* code fix+clippy hints+formatting
2026-01-29 15:26:42 +01:00
Ophir LOJKINE dc5da4643b Connection timeouts configuration (#1197)
* Allow disabling database connection timeouts via config

Co-authored-by: contact <contact@ophir.dev>

* Simplify documentation for disabling database timeouts

Co-authored-by: contact <contact@ophir.dev>

* Refactor timeout resolution to AppConfig

Co-authored-by: contact <contact@ophir.dev>

* Remove *_raw fields and use custom deserializer for timeouts

Co-authored-by: contact <contact@ophir.dev>

* Fix formatting in AppConfig

Co-authored-by: contact <contact@ophir.dev>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2026-01-21 12:23:41 +01:00
mtt cb4c1f915e feat(oidc): allow sub domain on localhost (#1188)
Co-authored-by: Your Name <you@example.com>
2026-01-17 16:25:50 +01:00
lovasoa cc55a3d647 v0.42.0
Create Release / Build sqlpage binaries (macOS & Windows) (.exe, , windows-latest, x86_64-pc-windows-msvc) (push) Has been cancelled
Create Release / Build sqlpage binaries (macOS & Windows) (odbc-static, macos-latest, x86_64-apple-darwin) (push) Has been cancelled
Create Release / Build sqlpage binaries (Linux) (push) Has been cancelled
Create Release / Build AWS Lambda Serverless zip image (push) Has been cancelled
Create Release / Create Github Release (push) Has been cancelled
Create Release / Publish to crates.io (push) Has been cancelled
CI / compile_and_lint (push) Has been cancelled
CI / test (mssql, mssql, mssql://root:Password123!@127.0.0.1/sqlpage) (push) Has been cancelled
CI / test (mysql, mysql, mysql://root:Password123!@127.0.0.1/sqlpage) (push) Has been cancelled
CI / test (oracle, oracle, Driver=Oracle 21 ODBC driver;Dbq=//127.0.0.1:1521/FREEPDB1;Uid=root;Pwd=Password123!) (push) Has been cancelled
CI / test (postgres, odbc, Driver=PostgreSQL Unicode;Server=127.0.0.1;Port=5432;Database=sqlpage;UID=root;PWD=Password123!, true) (push) Has been cancelled
CI / test (postgres, postgres, postgres://root:Password123!@127.0.0.1/sqlpage) (push) Has been cancelled
CI / windows_test (push) Has been cancelled
CI / docker_build (linux/amd64, duckdb) (push) Has been cancelled
CI / docker_build (linux/amd64, minimal) (push) Has been cancelled
CI / docker_build (linux/arm/v7, minimal) (push) Has been cancelled
CI / docker_build (linux/arm64, duckdb) (push) Has been cancelled
CI / docker_build (linux/arm64, minimal) (push) Has been cancelled
CI / docker_push (duckdb) (push) Has been cancelled
CI / docker_push (minimal) (push) Has been cancelled
2026-01-17 16:18:57 +01:00
lovasoa ce2b881227 Add cache_stale_duration_ms configuration option
Introduces a new configuration option `cache_stale_duration_ms` which
allows users to control how long files are cached before their freshness
is checked. This provides more fine-grained control over caching
behavior, especially in production environments.
2026-01-17 16:11:53 +01:00
Ophir LOJKINE 73377f4f62 add sqlpage functions to access sqlpage config paths (#1186) 2026-01-12 18:02:35 +01:00
Ophir LOJKINE a2ef976fc7 Add support for Oracle over ODBC (compatibility fixes, ci testing) (#1182)
* Add Oracle DB (free) and ODBC CI support

This change adds support for testing with Oracle DB (using the free version `gvenzl/oracle-free:slim`) in the CI pipeline. It:
- Updates `.github/workflows/ci.yml` to include a new matrix entry for Oracle DB.
- Adds steps to install the Oracle Instant Client and ODBC driver in the CI runner.
- Configures `odbcinst.ini` to register the Oracle ODBC driver.
- Updates `docker-compose.yml` to include the Oracle DB service definition.

* Fix CI: Remove libaio1 dependency

`libaio1` is not available in the ubuntu-latest environment used by GitHub Actions (which likely uses a newer Ubuntu version where `libaio1` is replaced by `libaio1t64` or similar, or it is transitively installed). Removing explicit installation to fix the CI failure.

* Fix CI: Update Oracle Instant Client to 21.14

The previous version 21.10.0.0.0-1 seems to be no longer available at the specified URL (404 Not Found). Updated to 21.14.0.0.0-1 which was verified to exist.

* Fix CI: Install libaio1t64 for Oracle Instant Client

Oracle Instant Client requires `libaio.so.1`, which is provided by the `libaio1t64` package in newer Ubuntu versions (like 24.04). Installing this package should resolve the "cannot open shared object file: No such file or directory" error.

* Fix CI: Symlink libaio.so.1 for Oracle Instant Client

On Ubuntu 24.04 (Noble), `libaio1t64` installs the library as `libaio.so.1t64`, but Oracle Instant Client explicitly looks for `libaio.so.1`. Creating a symlink fixes this loading issue.

* Fix CI: Install libodbcinst2 and unixodbc for Oracle driver

The Oracle ODBC driver requires `libodbcinst.so.2` which is provided by `libodbcinst2`. Installing `unixodbc` ensures the full ODBC stack is available.

* Fix CI: Remove redundant wget installation

`wget` is pre-installed on GitHub Actions runners, so the explicit installation via `apt-get` is unnecessary.

* Fix CI: Create 'root' user in Oracle DB

The Oracle test run failed with ORA-01017 (invalid credentials) because the 'root' user does not exist by default in the Oracle database.
This commit:
1. Installs Oracle SQL*Plus (instantclient-sqlplus) in the CI environment.
2. Adds a step to connect as SYSDBA and create the 'root' user with the required permissions and password, matching the test configuration.

* Fix CI: Use container env vars for Oracle user

Instead of manually creating the 'root' user with SQL*Plus, rely on the `gvenzl/oracle-free` container's initialization features (`APP_USER` and `APP_USER_PASSWORD`) which were already configured in `docker-compose.yml`.
Removed the SQL*Plus installation and user creation steps from the CI workflow.

* Fix CI: Use FREEPDB1 service for Oracle user

The `APP_USER` ('root') created by the `gvenzl/oracle-free` container is created in the `FREEPDB1` pluggable database, not the `FREE` container database. Updated the connection string to use `FREEPDB1`.

* Fix Oracle DB test failures

- `sql_to_json`: Handle Oracle's behavior where empty strings are treated as NULL.
- `filesystem`: Explicitly CAST the timestamp parameter to TIMESTAMP in `was_modified_query` for ODBC (Oracle) to avoid ORA-01843 (invalid month) errors caused by implicit string-to-date conversion mismatches.

* Fix Oracle tests by accounting for known behavior

- Revert generic ODBC changes in `filesystem.rs`.
- Skip `test_sql_file_read_utf8` for ODBC connections (Oracle) because fixing the `ORA-01843` error would require Oracle-specific SQL (e.g. `CAST` or `TO_TIMESTAMP`) in the main codebase which is undesirable for a generic ODBC implementation.
- Keep the test adjustment in `sql_to_json.rs` but clarify the comment that we are assuming ODBC implies Oracle in this test context for the empty string behavior.

* Fix Oracle tests: Identify Oracle by connection string

Instead of relying on `sqlx::any::AnyKind::Odbc` (which applies to any ODBC database), detect Oracle specifically by checking if the connection string contains "Oracle". This allows applying Oracle-specific test logic (like skipping tests with implicit timestamp conversions or handling empty strings as NULL) without incorrectly affecting other ODBC databases.

* use oracle dialect when talking to oracle

* remove stupid ai comment

* update oracle odbc installation steps

* fix odbc installation path

* cast variables to varchar(4000) in oracle

* clippy

* remove long backtraces from ci

* fixed csv upload test for oracle

* update tests for oracle

* properly quote sqlpage-generated col names

* fix test syntax for oracle

* clippy

* remove as but keep alias

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2026-01-12 14:22:07 +01:00
Mukhtar 5cf1882dc3 fix: handle temp file removal race condition in concurrent initializa… (#1185)
* fix: handle temp file removal race condition in concurrent initialization

When multiple SQLPage instances start simultaneously in the same directory,
they can encounter a race condition during initialization. The create_default_database()
function creates a temporary file to test directory writability, then removes it.
If multiple instances try to remove the same file concurrently, some panic with
'No such file or directory'.

This commit replaces the .expect() panic with graceful error handling using
if let Err(). The writability test has already succeeded by the time we try
to remove the file, so whether another instance removed it is irrelevant.

Includes a test that spawns 10 concurrent threads initializing AppConfig
to verify no panics occur.

ref #1183

* cargo fmt

* the error may have another cause

* Remove concurrent initialization test

Removed the test for concurrent initialization. The test did not work

---------

Co-authored-by: lovasoa <contact@ophir.dev>
2026-01-12 12:43:19 +01:00
Ophir LOJKINE 8d106fb677 Oidc site prefix handling (#1179)
* fix(oidc): respect site_prefix in OIDC redirect and logout URLs

This change ensures that when `site_prefix` is configured, the OIDC redirect URI and logout URI include this prefix.
Previously, `site_prefix` was ignored, causing OIDC callbacks to fail when the application was served under a sub-path.

- Added `site_prefix` to `OidcConfig`.
- Updated `make_oidc_client` to prepend `site_prefix` to the redirect URI.
- Updated `handle_request` to match paths with `site_prefix` included.
- Updated `validate_redirect_url` to respect the prefix when verifying redirect targets.
- Added a regression test `test_oidc_with_site_prefix`.

* Refactor: Update dependencies and remove unused crates

This commit updates several dependencies to their latest versions and removes unused crates to streamline the project.

Co-authored-by: contact <contact@ophir.dev>

* removed unused config

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2026-01-08 16:46:10 +01:00
Ophir LOJKINE 72eff8079a Fix invalid redirect url scheme behind reverse proxy (#1178)
This commit refactors the OIDC tests to use a more robust fake OIDC provider and improves the logout URL generation to correctly handle the scheme.

Fixes https://github.com/sqlpage/SQLPage/issues/1174
2026-01-08 15:03:23 +01:00
Ophir Lojkine fb40807ce7 Add DuckDB support including specific syntax parsing 2026-01-05 16:01:55 +01:00