* improve sqlpage function argument warnings with source context
Made-with: Cursor
* Param extraction: Result-based errors, single caller message, no CompoundIdentifier special case
- expr_to_stmt_param returns Result<StmtParam, ExprToParamError>; error carries only line + kind (UnsupportedExpr, UnemulatedFunction, NamedArgs)
- function_args_to_stmt_params logs one formatted message (ctx.format_param_error) then returns Err
- Single unsupported-expr arm; expr_summary() used for description
- Rename ParamWarnContext to ParamExtractContext
Made-with: Cursor
* Surface param extraction error in parse result; add parse_sql error-message tests; remove are_params_extractable
- When func_call_to_param returns StmtParam::Error, store it and have extract_parameters return Err so parse yields ParsedStatement::Error with specialized message
- Add test_parse_sql_unsupported_expr_in_sqlpage_arg and test_parse_sql_unemulated_function_in_sqlpage_arg
- Remove dead are_params_extractable and its unused import
Made-with: Cursor
* Refactor sqlpage function argument error messages to match user expectations
- Overhauled ExprToParamError formatting to construct exact user-friendly descriptions.
- Removed superfluous anyhow::Context prefixes in func_call_to_param.
- Passed source_path properly through validate_function_calls to ensure file line numbers populate the new error template accurately.
- Renamed error test file to match its dynamic error output signature.
- Removed redundant mut mutability warnings on parsing logic loops.
* improve error messages
* Refactor SqlPageFunctionError representation to clean up 'syntax error' wrappers
- Replaced stringly-typed anyhow errors with a strongly typed SqlPageFunctionError.
- Removed source_path threading completely from the parameter extraction phases, conforming to better separation of concerns.
- Appended file path prefix dynamically at the evaluation stage in clone_anyhow_err strictly when downcasting to SqlPageFunctionError.
- Removed the confusing generic 'Caused by: x.sql contains a syntax error...' wrapper from actual function logic errors.
* Remove redundant 'reorganize' hint from error message
* readd deleted test
* split parameter extraction logic into a separate file
* Add to changelog
* Add Oracle DB (free) and ODBC CI support
This change adds support for testing with Oracle DB (using the free version `gvenzl/oracle-free:slim`) in the CI pipeline. It:
- Updates `.github/workflows/ci.yml` to include a new matrix entry for Oracle DB.
- Adds steps to install the Oracle Instant Client and ODBC driver in the CI runner.
- Configures `odbcinst.ini` to register the Oracle ODBC driver.
- Updates `docker-compose.yml` to include the Oracle DB service definition.
* Fix CI: Remove libaio1 dependency
`libaio1` is not available in the ubuntu-latest environment used by GitHub Actions (which likely uses a newer Ubuntu version where `libaio1` is replaced by `libaio1t64` or similar, or it is transitively installed). Removing explicit installation to fix the CI failure.
* Fix CI: Update Oracle Instant Client to 21.14
The previous version 21.10.0.0.0-1 seems to be no longer available at the specified URL (404 Not Found). Updated to 21.14.0.0.0-1 which was verified to exist.
* Fix CI: Install libaio1t64 for Oracle Instant Client
Oracle Instant Client requires `libaio.so.1`, which is provided by the `libaio1t64` package in newer Ubuntu versions (like 24.04). Installing this package should resolve the "cannot open shared object file: No such file or directory" error.
* Fix CI: Symlink libaio.so.1 for Oracle Instant Client
On Ubuntu 24.04 (Noble), `libaio1t64` installs the library as `libaio.so.1t64`, but Oracle Instant Client explicitly looks for `libaio.so.1`. Creating a symlink fixes this loading issue.
* Fix CI: Install libodbcinst2 and unixodbc for Oracle driver
The Oracle ODBC driver requires `libodbcinst.so.2` which is provided by `libodbcinst2`. Installing `unixodbc` ensures the full ODBC stack is available.
* Fix CI: Remove redundant wget installation
`wget` is pre-installed on GitHub Actions runners, so the explicit installation via `apt-get` is unnecessary.
* Fix CI: Create 'root' user in Oracle DB
The Oracle test run failed with ORA-01017 (invalid credentials) because the 'root' user does not exist by default in the Oracle database.
This commit:
1. Installs Oracle SQL*Plus (instantclient-sqlplus) in the CI environment.
2. Adds a step to connect as SYSDBA and create the 'root' user with the required permissions and password, matching the test configuration.
* Fix CI: Use container env vars for Oracle user
Instead of manually creating the 'root' user with SQL*Plus, rely on the `gvenzl/oracle-free` container's initialization features (`APP_USER` and `APP_USER_PASSWORD`) which were already configured in `docker-compose.yml`.
Removed the SQL*Plus installation and user creation steps from the CI workflow.
* Fix CI: Use FREEPDB1 service for Oracle user
The `APP_USER` ('root') created by the `gvenzl/oracle-free` container is created in the `FREEPDB1` pluggable database, not the `FREE` container database. Updated the connection string to use `FREEPDB1`.
* Fix Oracle DB test failures
- `sql_to_json`: Handle Oracle's behavior where empty strings are treated as NULL.
- `filesystem`: Explicitly CAST the timestamp parameter to TIMESTAMP in `was_modified_query` for ODBC (Oracle) to avoid ORA-01843 (invalid month) errors caused by implicit string-to-date conversion mismatches.
* Fix Oracle tests by accounting for known behavior
- Revert generic ODBC changes in `filesystem.rs`.
- Skip `test_sql_file_read_utf8` for ODBC connections (Oracle) because fixing the `ORA-01843` error would require Oracle-specific SQL (e.g. `CAST` or `TO_TIMESTAMP`) in the main codebase which is undesirable for a generic ODBC implementation.
- Keep the test adjustment in `sql_to_json.rs` but clarify the comment that we are assuming ODBC implies Oracle in this test context for the empty string behavior.
* Fix Oracle tests: Identify Oracle by connection string
Instead of relying on `sqlx::any::AnyKind::Odbc` (which applies to any ODBC database), detect Oracle specifically by checking if the connection string contains "Oracle". This allows applying Oracle-specific test logic (like skipping tests with implicit timestamp conversions or handling empty strings as NULL) without incorrectly affecting other ODBC databases.
* use oracle dialect when talking to oracle
* remove stupid ai comment
* update oracle odbc installation steps
* fix odbc installation path
* cast variables to varchar(4000) in oracle
* clippy
* remove long backtraces from ci
* fixed csv upload test for oracle
* update tests for oracle
* properly quote sqlpage-generated col names
* fix test syntax for oracle
* clippy
* remove as but keep alias
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
The `sqlpage.variables()` function previously allowed duplicate keys
when GET, POST, and SET variables of the same name were present. This
commit ensures that the returned JSON object contains only unique keys,
with precedence given to SET variables, then POST, then GET.
* simplify SQL test files
- Simplified SQL test files by removing unnecessary components and restructuring queries to focus on expected vs actual results.
- Updated the request handling in `run_sql_test` to differentiate between JSON and HTML responses based on test file content.
- Enhanced error handling and assertions for both JSON and HTML responses to improve test reliability and clarity.
- Removed redundant code and improved readability in the test execution flow.
* move more tests to the new expected/actual format
* migrate more tests to the expected/actual format
- Changed references in various test files to point to the new `simple.sql` instead of the outdated `it_works_simple.sql`.
- Removed several obsolete SQL test files that are no longer needed, streamlining the test suite.
- Updated assertions and request paths in the test cases to reflect the new structure and improve clarity.
* Enhance error handling in SQL tests and update SQL syntax
- Added a new function `format_error` to improve error reporting in SQL test assertions, capturing detailed error descriptions and backtraces.
- Updated SQL syntax in `sqrt.sql` to use `INT` instead of `integer` for consistency with SQL standards.
* sqrt test
* feat: Add sqlpage.set_variable function
Co-authored-by: contact <contact@ophir.dev>
* Refactor: Fix set_variable serialization and update tests
Co-authored-by: contact <contact@ophir.dev>
* fix tests: no json_extract on mssql
* Refactor: Update URLParameters handling in set_variable function
- Replaced serde_json::Map with a custom URLParameters struct for better management of URL parameters.
- Introduced methods for handling single and vector values in URLParameters.
- Updated tests to reflect changes in the set_variable function's behavior.
* cargo fmt
* clippy
* retsore set var test
* remove redundant test
* ensure set_variable only takes into account GET variables, not SET
* factor url parameter setting code
* v0.40
* sqlpage.set_variable links to "?" when no parameter is present
- Renamed URLParameters module for clarity and removed the deprecated url_parameter_deserializer.
- Updated the set_variable function to return parameters directly instead of appending to a URL.
- Adjusted related function calls to reflect changes in URL parameter management.
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
* Make URL and POST parameters immutable, separate from SET variables
- URL and POST parameters are now immutable after request initialization
- SET command creates user-defined variables in separate namespace
- Variable lookup: SET variables shadow request parameters
- Added sqlpage.variables('set') to inspect user-defined variables
- Simplified API: most functions now use &RequestInfo instead of &mut
- All tests passing (151 total)
* Restore deprecation warning for SET on POST variable names
* Restore deprecation warnings for $var accessing POST variables
- Warn when both URL and POST have same variable name
- Warn when $var is used for POST-only variable (should use :var)
* Simplify run_sql: always use clone_without_variables
No need to branch on whether variables are provided since we clone in both cases anyway.
* Revert "Simplify run_sql: always use clone_without_variables"
This reverts commit 60f5a05446.
* Fix cross-database test compatibility for immutable variables
Renamed test to run only on SQLite since json_extract() is SQLite-specific.
Other databases (PostgreSQL, MySQL, MSSQL) have different JSON functions.
* Fix test to work across all databases without json_extract
PostgreSQL doesn't have json_extract, so compare the full JSON string instead.
* Document variable system improvements in CHANGELOG
* Make CHANGELOG more explicit about breaking changes with examples
* Fix CHANGELOG: SET overwrites GET parameters, not POST
* Add database-specific examples for accessing original URL parameters
- Enabled arbitrary precision for `serde_json` to handle large numbers without rounding.
- Refactored decimal handling in `sql_to_json.rs` to utilize a new function for converting decimals to JSON format.
Fixes https://github.com/sqlpage/SQLPage/issues/1052
* feat: Add sqlpage.hmac function for cryptographic signing
Co-authored-by: contact <contact@ophir.dev>
* feat: Add sqlpage.hmac function for secure data signing
Co-authored-by: contact <contact@ophir.dev>
* Test HMAC function with RFC vectors and update tests
Co-authored-by: contact <contact@ophir.dev>
* feat: Add sqlpage.hmac() function for secure signatures
Co-authored-by: contact <contact@ophir.dev>
* feat: Add base64 output option to hmac function
Co-authored-by: contact <contact@ophir.dev>
* Refactor hmac function for cleaner output formatting
Co-authored-by: contact <contact@ophir.dev>
* Add webhook HMAC signature validation tests
Co-authored-by: contact <contact@ophir.dev>
* Refactor HMAC function and update SQL examples for clarity and consistency
- Changed function parameters to remove Option types for data and key in the HMAC function.
- Improved SQL documentation and examples for HMAC usage, including clearer descriptions and updated error handling.
- Enhanced test cases for webhook HMAC validation to ensure accurate signature checks and responses.
- Removed obsolete test file for HMAC with null values.
* Update HMAC validation logic to handle NULL values in SQL queries
- Modified conditions in SQL queries to check for NULL values alongside signature mismatches.
- Enhanced documentation on NULL handling for HMAC checks to improve clarity and portability.
* remove debug logging from ci
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
* implemented logger Component
* changed target for logger to "sqlpage::logger" and made the log::Level case insensitive
* changed error message on missing message key, to be more precise
* changed custom method for String to log::Level conversion to builtin method log::Level::from_str()
* switched to utility method get_object_str and inlined constants
* dynamically target based on file and statement
* disabled ci error (large difference in enum variants) for ResponseWithWriter<S>
* added functionality to work in Header context
* Refactor log component to use compact error handling
* Rename comp_str variable to component_name
* Documented log component
* fixxed missing values statement
* fixxed pipeline errors
* very simple test case for logger
---------
Co-authored-by: Lenardt Gerhardts <lenardt.ger+git@gmail.com>
Co-authored-by: lovasoa <contact@ophir.dev>
- Introduce a new download component to facilitate file downloads.
- Implement download handling in the header context, supporting data URLs.
- Add a test for the download functionality to ensure correct behavior.
see https://github.com/sqlpage/SQLPage/discussions/996
- Adds a new parameter `response_encoding` to the fetch function. - This
parameter allows the user to specify a charset for decoding the response
body. - It supports all standard web encodings, and also `hex` and
`base64`. - If no encoding is specified, the response is decoded as
utf-8, or as base64 if it contains binary data.
* use numbered parameters in sqlite
* implement parameter deduplication
* implement parameter re-duplication for mysql
* Improve error messages on invalid sqlpage function calls. The messages now contain actionable advice.
* simplify extract_set_variable
Ophir LOJKINE