75 Commits

Author SHA1 Message Date
Ophir LOJKINE 8b9d6a0319 Improve CI Docker cache reuse (#1296)
* Improve CI Docker cache reuse

* Ignore git metadata in Docker builds

* Use BuildKit cache mounts for Cargo Docker builds

* Make Docker dependency layer match real build inputs

* Revert "Make Docker dependency layer match real build inputs"

This reverts commit 3a9372150ed30437c0b7872907a3e03ce1c38f5d.

* Revert "Use BuildKit cache mounts for Cargo Docker builds"

This reverts commit dac2c5264bee72fc287877fcfe15e49c319d99dd.

* Test CI incremental Rust caches

* Remove slow Windows port diagnostic

* Narrow Windows incremental cache

* Reuse Linux test artifacts in CI

* Avoid duplicate Linux test execution in CI

* Isolate Docker cache scopes by recipe

* Warm Docker dependencies for lib target

* Warm Docker dependencies with Cargo config

* Stabilize Docker metadata for cache keys

* Use Git context for Docker builds
2026-06-04 00:20:09 +02:00
Ophir LOJKINE 2c89bd04f7 Update remaining dependency pins 2026-05-29 22:24:53 +02:00
Ophir LOJKINE d1e8154222 Add OpenTelemetry distributed tracing support (#1234)
* add OpenTelemetry distributed tracing support

When OTEL_EXPORTER_OTLP_ENDPOINT is set, enables full tracing pipeline
with OTLP export, W3C traceparent propagation, and spans for HTTP
requests, SQL file execution, DB pool acquire, and query execution.
Falls back to env_logger when unset.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix OTel example: tracing init, Dockerfile, Tempo config

- Fix tracing-log bridge initialization order (set subscriber first,
  then LogTracer) to avoid double-set panic
- Add dedicated Dockerfile for example using release profile (avoids
  OOM with superoptimized LTO in Docker)
- Use debian:trixie-slim runtime for glibc compatibility
- Fix nginx image to nginx:otel (official image with OTel module)
- Fix nginx.conf: move otel_trace directives into location block
- Pin Tempo to 2.6.1 (latest has partition ring issues in single-node)
- Fix otel-collector exporter alias (otlp → otlp_grpc)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* propagate trace context to PostgreSQL via application_name

After acquiring a DB connection, set the W3C traceparent as the
PostgreSQL application_name (or MySQL session variable). This makes
trace IDs visible in pg_stat_activity and PostgreSQL logs, enabling
direct correlation between Grafana Tempo traces and database-side
monitoring.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* rewrite OTel example README with setup guides for all major providers

Comprehensive documentation covering:
- Step-by-step quick start for the Docker Compose example
- How OpenTelemetry works (spans, collectors, backends)
- Setup guides for Grafana Tempo, Jaeger, Grafana Cloud, Datadog,
  Honeycomb, New Relic, and Axiom with exact env vars and doc links
- PostgreSQL trace correlation via application_name
- Environment variable reference
- Troubleshooting section

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix todo example: use :title (POST variable) instead of $title

The form submits via POST, so the title field must be referenced with
the : prefix (POST parameter) rather than $ (GET parameter).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* set code.filepath and code.lineno span attributes to user SQL files

OTel span attributes now reference the user's .sql file path and line
number instead of the SQLPage Rust source code. Also improves span
naming, adds JSON log formatting, custom root span builder, and
Grafana dashboard provisioning for the OTel example.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* npm run fix

* use stable OTel semantic convention attribute names

- code.filepath → code.file.path, code.lineno → code.line.number
- db.statement → db.query.text, db.system → db.system.name
- Disable auto code location (.with_location(false)) so spans
  reference user SQL files, not SQLPage Rust source
- Remove redundant sqlpage.file attribute (code.file.path suffices)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* use OTel semantic convention values for db.system.name

Use well-known values from the OpenTelemetry registry instead of raw
DBMS name strings. Cast line numbers to i64 for correct span recording.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* remove json-subscriber dependency

The custom logfmt layer in telemetry.rs replaces it with zero extra
dependencies and precise control over field selection and ordering.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* add Loki + Promtail log aggregation to OTel example

Adds two new services (Loki, Promtail) to scrape SQLPage container logs
and display them in Grafana alongside traces. The home dashboard now
shows a logs panel with trace_id derived fields linking to Tempo.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* add OTel spans for request parsing, rendering, sqlpage functions, and OIDC

Add targeted spans to account for previously untraced time:
- http.parse_request: request/form parsing before SQL execution
- render: template rendering and response streaming
- subprocess: sqlpage.exec() with process.command attribute
- http.client: sqlpage.fetch()/fetch_with_meta() with OTel HTTP client
  semantic conventions (http.request.method, url.full, http.response.status_code)
- sqlpage.file: sqlpage.run_sql() nested file execution
- oidc.callback + http.client: OIDC token exchange

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* add oidc.jwt.verify span for OIDC token verification

This span covers JWT signature verification and claims validation,
which runs on every authenticated request via get_token_claims().

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* add enduser.id attribute to oidc.jwt.verify span

Records the OIDC subject claim (sub) as enduser.id after successful
JWT verification, following OTel semantic conventions.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* add OTel user.* attributes to oidc.jwt.verify span

Record user.id (sub), user.name (preferred_username), user.full_name
(name), and user.email from OIDC claims after JWT verification.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* add sqlpage.file.load span and attributes to http.parse_request

The gap before http.parse_request was the SQL file cache lookup -
now covered by the sqlpage.file.load span with code.file.path.

http.parse_request now records http.request.method and content_type,
which helps identify slow multipart/form-data parsing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix clippy pedantic warnings

* unify log format: logfmt with colors, no OTel noise

Use the custom logfmt layer for both OTel and non-OTel modes instead
of falling back to env_logger. This eliminates the tracing→log bridge
dumping all span fields (user agents, otel.kind, request_id, etc.)
and only shows: ts, level, target, msg, method, path, status,
file, client_ip, and trace_id (when valid).

Adds terminal color support (bold red for errors, green for info,
dim for timestamps/targets). Emits one log line per completed
successful request. Errors are logged once by the error handler.
Suppresses trace_id=000...0 when no real trace context exists.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* use .instrument() instead of .entered() for async spans

Span guards from .entered() do not propagate correctly across await
points. Switch to tracing::Instrument to ensure spans are properly
associated with their async tasks throughout their lifetime.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* preserve multi-line error formatting in terminal log output

When stderr is a terminal and the log message contains newlines
(e.g. SQL syntax errors with source highlighting and arrows),
print the metadata on the first line and the message below with
its original formatting. Machine output (non-terminal) remains
single-line logfmt.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* use root Dockerfile for OTel example, add CARGO_PROFILE build arg

Remove the example's custom Dockerfile and use the main one with a
CARGO_PROFILE=release build arg to avoid OOM from fat LTO in
memory-constrained Docker environments. The build scripts now
read CARGO_PROFILE from the environment, defaulting to
superoptimized for backward compatibility.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* add db.query.parameter and db.response.returned_rows span attributes

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Add official blog post about tracing

* add http.request.body.size and url.query span attributes

Add http.request.body.size to HTTP client spans (fetch, fetch_with_meta)
and to the server-side http.parse_request span (from Content-Length header).
Add url.query to the http.parse_request span.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Replace Promtail with the OpenTelemetry Collector

* Refactor telemetry logging helpers

* Clamp traced fetch body size

* Clamp traced fetch_with_meta body size

* Silence noisy PostgreSQL collector logs

* make startup logs parseable

* update terminal log formats

* Ingest real PostgreSQL logs with trace IDs

* Use raw traceparent for PostgreSQL tracing

* Update opentelemetry example for PostgreSQL query events

* Add nginx logs to opentelemetry example

* Parse nginx error log severity correctly

* Log all span fields when debug logging is enabled

* Rename telemetry example directory

* Fix PostgreSQL Loki log ingestion

* `LOG_LEVEL` is now the primary environment variable for configuring SQLPage's log filter. `RUST_LOG` remains supported as an alias.

* Skip empty trace IDs in logs

* add db errors to otel traces

* add healthcheck

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-14 15:52:32 +01:00
Ophir LOJKINE 7bb11c16e8 duckdb odbc docker (#1187)
* add a duckdb docker image

* Add DuckDB ODBC driver installation and configuration

Installs DuckDB ODBC driver and configures system-wide ODBC settings to
enable DuckDB database connectivity for SQLPage.

* add duckdb odbc info to readme

* Fix CI digest artifact matching

* Temporarily publish docker images on branch

* Allow docker publish on branch (fix)

* Temporarily run CI on branch pushes

* Disable branch publish; fix duckdb inspect tag

* Temporarily enable CI push on branch

* Fix minimal tag suffix in docker_push

* Disable temporary branch CI push
2026-01-17 01:23:29 +01:00
lovasoa 3ada9b496d update rust 2025-11-29 01:27:37 +01:00
Ophir LOJKINE a52b20cc35 remove docker build cruft (#1115)
* remove docker build cruft

- Simplified environment variable sourcing in build scripts by using a single build-env.sh file.
- Updated Dockerfile to streamline the build process and reduce unnecessary comments.
- Enhanced setup-cross-compilation.sh to directly set target and linker variables without temporary files.
- Improved clarity and maintainability of build scripts.

* fix setup-cross-compilation.sh

- Updated the script to use a more concise syntax for exporting environment variables.
- Improved readability by using echo and conditional checks for BINDGEN_EXTRA_CLANG_ARGS.
- This change simplifies the script while maintaining functionality.

* Refactor setup-cross-compilation.sh for improved readability

- Changed the syntax for checking BINDGEN_EXTRA_CLANG_ARGS to use a conditional statement for clarity.
- This enhances the script's maintainability while preserving its functionality.
2025-11-20 00:36:41 +01:00
lovasoa b278764b18 Dockerfile: extract scripts to separate files 2025-10-05 10:41:13 +02:00
lovasoa 656c73f86d Dockerfile: update library paths for aarch64 and arm architectures 2025-10-05 10:22:37 +02:00
lovasoa 05b7a55f48 Dockerfile: statically link odbc 2025-10-05 09:45:54 +02:00
Cursor Agent df242b4fae Refactor: Optimize dependencies and library paths in Dockerfile
Co-authored-by: contact <contact@ophir.dev>
2025-10-02 12:48:50 +00:00
Cursor Agent 35514347a6 Build: Add unixODBC build and cross-compilation support
Co-authored-by: contact <contact@ophir.dev>
2025-09-29 17:25:37 +00:00
Cursor Agent 9511964d92 Update rpath to include sqlpage directory for ODBC drivers
Co-authored-by: contact <contact@ophir.dev>
2025-09-29 15:34:56 +00:00
Cursor Agent 83efc1aae5 Checkpoint before follow-up message
Co-authored-by: contact <contact@ophir.dev>
2025-09-29 15:32:49 +00:00
lovasoa 0ae9072712 fix Dockerfile: enhance cross-compilation setup by adding clang and updating BINDGEN_EXTRA_CLANG_ARGS 2025-09-26 16:21:42 +02:00
lovasoa fbde06e83e fix Dockerfile: streamline library installation and update paths for cross-compilation 2025-09-26 16:21:42 +02:00
lovasoa 4ba0e07f14 fix Dockerfile: update library paths for arm architecture 2025-09-26 16:21:42 +02:00
lovasoa 38bc00e5d0 fix Dockerfile: include additional libraries 2025-09-26 16:21:42 +02:00
lovasoa 7fcd138360 Update Dockerfiles to use Rust 1.90 for improved performance and compatibility 2025-09-26 16:21:42 +02:00
lovasoa 7278c76349 Update Dockerfile to remove unnecessary package installations for improved efficiency 2025-09-26 16:21:42 +02:00
lovasoa 28ab4db1db undo stupid llm 2025-09-26 16:21:42 +02:00
Cursor Agent 70f3473f15 Fix: Update ODBC driver and ensure /lib64 exists
Co-authored-by: contact <contact@ophir.dev>
2025-09-26 16:21:42 +02:00
Ophir LOJKINE 0907679d1f Fix: Add arm64 and armhf architecture support for dependencies (#1025)
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-09-26 16:21:42 +02:00
lovasoa 2f2bbef032 Enhance Dockerfile to include dependency copying for improved runtime environment 2025-09-26 16:21:42 +02:00
lovasoa bbb0210ec8 Enhance Dockerfile and CI workflows by adding ODBC dependencies for improved database support 2025-09-26 16:21:42 +02:00
lovasoa cedfe3dbde remove unused dep 2025-05-21 23:24:09 +02:00
lovasoa 884640b329 fix arm cross compilation 2025-05-21 23:18:56 +02:00
lovasoa b6188b40b0 fix arm cross compilation 2025-05-21 22:59:59 +02:00
lovasoa fb4a15c9a0 ensure cmake is installed 2025-05-21 22:40:47 +02:00
lovasoa 70b57b9d6e upgrade to rust 1.87 2025-05-21 22:39:23 +02:00
lovasoa 90d4b010fd update to rust 1.86 2025-04-21 09:29:47 +02:00
lovasoa 2f9a9ad4d3 udate to rust 1.83 2024-12-04 18:11:51 +01:00
lovasoa 9feeb13326 update rust to rust 1.82 2024-10-30 11:45:03 +01:00
lovasoa 25ce73de79 update to rust 1.81
CI / compile_and_lint (push) Has been cancelled
CI / test (mssql) (push) Has been cancelled
CI / test (mysql) (push) Has been cancelled
CI / test (postgres) (push) Has been cancelled
CI / windows_test (push) Has been cancelled
CI / docker_build (linux/amd64) (push) Has been cancelled
CI / docker_build (linux/arm/v7) (push) Has been cancelled
CI / docker_build (linux/arm64) (push) Has been cancelled
CI / docker_push (push) Has been cancelled
Create Release / Build sqlpage binaries (target/superoptimized/sqlpage, macos-latest) (push) Has been cancelled
Create Release / Build sqlpage binaries (target/superoptimized/sqlpage, ubuntu-20.04) (push) Has been cancelled
Create Release / Build sqlpage binaries (target/superoptimized/sqlpage.exe, windows-latest) (push) Has been cancelled
Create Release / Build AWS Lambda Serverless zip image (push) Has been cancelled
Create Release / Create Github Release (push) Has been cancelled
Create Release / Publish to crates.io (push) Has been cancelled
2024-09-25 21:29:12 +02:00
lovasoa 6c6a119256 update docker to rust 1.80 2024-08-06 21:49:08 +02:00
lovasoa b0b52f33e5 fix dockerfile format warning 2024-07-23 22:28:11 +02:00
lovasoa c1a1a302fd update rust version 2024-07-12 19:46:39 +02:00
lovasoa 1cb8f9455a simplify docker cross-compilation 2024-05-04 15:38:29 +02:00
lovasoa 63626e48f6 fix location of cargo config 2024-05-04 15:14:26 +02:00
lovasoa 5ba6b50782 fix dockerfile 2024-05-04 14:26:43 +02:00
lovasoa 7727ff4a1a update to rust 1.78 2024-05-02 18:45:39 +02:00
lovasoa 2abcb77627 initial oidc example 2024-04-26 01:11:51 +02:00
lovasoa 59f4d8a562 update rust version in docker images 2024-03-30 22:57:44 +01:00
Ophir LOJKINE e3fb2f8acc Introduce SQLPAGE_CONFIGURATION_DIRECTORY (#243)
* Introduce SQLPAGE_CONFIGURATION_DIRECTORY

Makes the configuration directory configurable instead of it being hardcoded to "./sqlpage/"

* logging
2024-02-25 19:23:17 +01:00
lovasoa 79a7c833b0 run as user 1000 by default in docker images
update rust version in docker images to 1.74
2023-11-21 00:42:56 +01:00
lovasoa 5889de7866 Update to rust 1.73 2023-10-10 15:16:18 +02:00
lovasoa 68e83d8e8c docker fix typo in env variable
fixes #98
2023-10-09 13:48:11 +02:00
lovasoa a499d9671c change default working directory to /etc 2023-10-03 13:07:41 +02:00
lovasoa 2f38e8482e Include a default rw sqlite db in the docker image
Fixes https://github.com/lovasoa/SQLpage/issues/94
2023-10-03 12:51:41 +02:00
lovasoa a1501d882d fix redundant docker layer 2023-10-03 03:58:20 +02:00
lovasoa 59412b9e3f fix armv7 build 2023-10-03 03:24:12 +02:00