Add and document sqlpage.hmac function (#1033)

* feat: Add sqlpage.hmac function for cryptographic signing Co-authored-by: contact <contact@ophir.dev> * feat: Add sqlpage.hmac function for secure data signing Co-authored-by: contact <contact@ophir.dev> * Test HMAC function with RFC vectors and update tests Co-authored-by: contact <contact@ophir.dev> * feat: Add sqlpage.hmac() function for secure signatures Co-authored-by: contact <contact@ophir.dev> * feat: Add base64 output option to hmac function Co-authored-by: contact <contact@ophir.dev> * Refactor hmac function for cleaner output formatting Co-authored-by: contact <contact@ophir.dev> * Add webhook HMAC signature validation tests Co-authored-by: contact <contact@ophir.dev> * Refactor HMAC function and update SQL examples for clarity and consistency - Changed function parameters to remove Option types for data and key in the HMAC function. - Improved SQL documentation and examples for HMAC usage, including clearer descriptions and updated error handling. - Enhanced test cases for webhook HMAC validation to ensure accurate signature checks and responses. - Removed obsolete test file for HMAC with null values. * Update HMAC validation logic to handle NULL values in SQL queries - Modified conditions in SQL queries to check for NULL values alongside signature mismatches. - Enhanced documentation on NULL handling for HMAC checks to improve clarity and portability. * remove debug logging from ci --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-10-01 21:07:36 +02:00
parent b029661cf4
commit 625deb973b
14 changed files with 372 additions and 1 deletions
@@ -84,7 +84,6 @@ jobs:
        env:
          DATABASE_URL: ${{ matrix.db_url }}
          RUST_BACKTRACE: 1
-          RUST_LOG: sqlpage=debug

  windows_test:
    runs-on: windows-latest
@@ -1,5 +1,14 @@
 # CHANGELOG.md

+## v0.38.0
+ - Added a new `sqlpage.hmac()` function for cryptographic HMAC (Hash-based Message Authentication Code) operations.
+   - Create and verify secure signatures for webhooks (Shopify, Stripe, GitHub, etc.)
+   - Generate tamper-proof tokens for API authentication
+   - Secure download links and temporary access codes
+   - Supports SHA-256 (default) and SHA-512 algorithms
+   - Output formats: hexadecimal (default) or base64 (e.g., `sha256-base64`)
+   - See the [function documentation](https://sql-page.com/functions.sql?function=hmac) for detailed examples
+
 ## v0.37.1
 - fixed decoding of UUID values
 - Fixed handling of NULL values in `sqlpage.link`. They were encoded as the string `'null'` instead of being omitted from the link's parameters.
@@ -4277,6 +4277,7 @@ dependencies = [
 "env_logger",
 "futures-util",
 "handlebars",
+ "hmac",
 "include_dir",
 "lambda-web",
 "libflate",
@@ -4292,6 +4293,7 @@ dependencies = [
 "rustls-native-certs",
 "serde",
 "serde_json",
+ "sha2",
 "sqlparser",
 "sqlx-oldapi",
 "tokio",
@@ -61,6 +61,8 @@ actix-web-httpauth = "0.8.0"
 rand = "0.9.0"
 actix-multipart = "0.7.2"
 base64 = "0.22"
+hmac = "0.12"
+sha2 = "0.10"
 rustls-acme = "0.14"
 dotenvy = "0.15.7"
 csv-async = { version = "1.2.6", features = ["tokio"] }
@@ -0,0 +1,137 @@
+-- HMAC function documentation and examples
+INSERT INTO
+    sqlpage_functions (
+        "name",
+        "introduced_in_version",
+        "icon",
+        "description_md"
+    )
+VALUES
+    (
+        'hmac',
+        '0.38.0',
+        'shield-lock',
+        'Creates a unique "signature" for some data using a secret key.
+This signature proves that the data hasn''t been tampered with and comes from someone who knows the secret.
+
+### What is HMAC used for?
+
+[**HMAC**](https://en.wikipedia.org/wiki/HMAC) (Hash-based Message Authentication Code) is commonly used to:
+ - **Verify webhooks**: Use HMAC to ensure only a given external service can call a given endpoint in your application.
+The service signs their request with a secret key, and you verify the signature before processing the data they sent you.
+Used for instance by [Stripe](https://docs.stripe.com/webhooks?verify=verify-manually), and [Shopify](https://shopify.dev/docs/apps/build/webhooks/subscribe/https#step-2-validate-the-origin-of-your-webhook-to-ensure-its-coming-from-shopify).
+ - **Secure API requests**: Prove that an API request comes from an authorized source
+ - **Generate secure tokens**: Create temporary access codes for downloads or password resets
+ - **Protect data**: Ensure data hasn''t been modified during transmission
+
+### How to use it
+
+The `sqlpage.hmac` function takes three inputs:
+1. **Your data** - The text you want to sign (like a message or request body)
+2. **Your secret key** - A password only you know (keep this safe!)
+3. **Algorithm** (optional) - The hash algorithm and output format:
+   - `sha256` (default) - SHA-256 with hexadecimal output
+   - `sha256-base64` - SHA-256 with base64 output
+   - `sha512` - SHA-512 with hexadecimal output
+   - `sha512-base64` - SHA-512 with base64 output
+
+It returns a signature string. If someone changes even one letter in your data, the signature will be completely different.
+
+### Example: Verify a Webhooks signature
+
+When Shopify sends you a webhook (like when someone places an order), it includes a signature. Here''s how to verify it''s really from Shopify.
+This supposes you store the secret key in an [environment variable](https://en.wikipedia.org/wiki/Environment_variable) named `WEBHOOK_SECRET`.
+
+```sql
+SET body = sqlpage.request_body();
+SET secret = sqlpage.environment_variable(''WEBHOOK_SECRET'');
+SET expected_signature = sqlpage.hmac($body, $secret, ''sha256'');
+SET actual_signature =  sqlpage.header(''X-Webhook-Signature'');
+
+-- redirect to an error page and stop execution if the signature does not match
+SELECT
+    ''redirect'' as component,
+    ''/error.sql?err=bad_webhook_signature'' as link
+WHERE $actual_signature != $expected_signature OR $actual_signature IS NULL;
+
+-- If we reach here, the signature is valid - process the order
+INSERT INTO orders (order_data) VALUES ($body);
+
+SELECT ''json'' as component, ''jsonlines'' as type;
+SELECT ''success'' as status;
+```
+
+### Example: Time-limited links
+
+You can create links that will be valid only for a limited time by including a signature in them.
+Let''s say we have a `download.sql` page we want to link to,
+but we don''t want it to be accessible to anyone who can find the link.
+Sign `file_id|expires_at` with a secret. Accept only if not expired and the signature matches.
+
+#### Generate a signed link
+
+```sql
+SET expires_at = datetime(''now'', ''+1 hour'');
+SET token = sqlpage.hmac(
+    $file_id || ''|'' || $expires_at,
+    sqlpage.environment_variable(''DOWNLOAD_SECRET''),
+    ''sha256''
+);
+SELECT ''/download.sql?file_id='' || $file_id || ''&expires_at='' || $expires_at || ''&token='' || $token AS link;
+```
+
+#### Verify the signed link
+
+```sql
+SET expected = sqlpage.hmac(
+    $file_id || ''|'' || $expires_at,
+    sqlpage.environment_variable(''DOWNLOAD_SECRET''),
+    ''sha256''
+);
+SELECT ''redirect'' AS component, ''/error.sql?err=expired'' AS link
+WHERE $expected != $token OR $token IS NULL OR $expires_at < datetime(''now'');
+
+-- serve the file
+```
+
+### Important Security Notes
+
+ - **Keep your secret key safe**: If your secret leaks, anyone can forge signatures and access protected pages
+ - **The signature is case-sensitive**: Even a single wrong letter means the signature won''t match
+ - **NULL handling**: Always use `IS DISTINCT FROM`, not `=` to check for hmac matches.
+   - `SELECT ''redirect'' as component WHERE sqlpage.hmac(...) != $signature` will not redirect if `$signature` is NULL (the signature is absent).
+   - `SELECT ''redirect'' as component WHERE sqlpage.hmac(...) IS DISTINCT FROM $signature` checks for both NULL and non-NULL values (but is not available in all SQL dialects).
+   - `SELECT ''redirect'' as component WHERE sqlpage.hmac(...) != $signature OR $signature IS NULL` is the most portable solution.
+'
+    );
+
+INSERT INTO
+    sqlpage_function_parameters (
+        "function",
+        "index",
+        "name",
+        "description_md",
+        "type"
+    )
+VALUES
+    (
+        'hmac',
+        1,
+        'data',
+        'The input data to compute the HMAC for. Can be any text string. Cannot be NULL.',
+        'TEXT'
+    ),
+    (
+        'hmac',
+        2,
+        'key',
+        'The secret key used to compute the HMAC. Should be kept confidential. Cannot be NULL.',
+        'TEXT'
+    ),
+    (
+        'hmac',
+        3,
+        'algorithm',
+        'The hash algorithm and output format. Optional, defaults to `sha256` (hex output). Supported values: `sha256`, `sha256-base64`, `sha512`, `sha512-base64`. Defaults to `sha256`.',
+        'TEXT'
+    );
@@ -32,6 +32,7 @@ super::function_definition_macro::sqlpage_functions! {
    hash_password(password: Option<String>);
    header((&RequestInfo), name: Cow<str>);
    headers((&RequestInfo));
+    hmac(data: Cow<str>, key: Cow<str>, algorithm: Option<Cow<str>>);

    user_info_token((&RequestInfo));
    link(file: Cow<str>, parameters: Option<Cow<str>>, hash: Option<Cow<str>>);
@@ -738,10 +739,83 @@ async fn headers(request: &RequestInfo) -> String {
    serde_json::to_string(&request.headers).unwrap_or_default()
 }

+/// Computes the HMAC (Hash-based Message Authentication Code) of the input data
+/// using the specified key and hashing algorithm.
+async fn hmac<'a>(
+    data: Cow<'a, str>,
+    key: Cow<'a, str>,
+    algorithm: Option<Cow<'a, str>>,
+) -> anyhow::Result<Option<String>> {
+    use hmac::{Hmac, Mac};
+    use sha2::{Sha256, Sha512};
+
+    let algorithm = algorithm.as_deref().unwrap_or("sha256");
+
+    // Parse algorithm and output format (e.g., "sha256" or "sha256-base64")
+    let (hash_algo, output_format) = if let Some((algo, format)) = algorithm.split_once('-') {
+        (algo, format)
+    } else {
+        (algorithm, "hex")
+    };
+
+    let result = match hash_algo.to_lowercase().as_str() {
+        "sha256" => {
+            let mut mac = Hmac::<Sha256>::new_from_slice(key.as_bytes())
+                .map_err(|e| anyhow!("Invalid HMAC key: {e}"))?;
+            mac.update(data.as_bytes());
+            mac.finalize().into_bytes().to_vec()
+        }
+        "sha512" => {
+            let mut mac = Hmac::<Sha512>::new_from_slice(key.as_bytes())
+                .map_err(|e| anyhow!("Invalid HMAC key: {e}"))?;
+            mac.update(data.as_bytes());
+            mac.finalize().into_bytes().to_vec()
+        }
+        _ => {
+            anyhow::bail!(
+                "Unsupported HMAC algorithm: {hash_algo}. Supported algorithms: sha256, sha512"
+            )
+        }
+    };
+
+    // Convert to requested output format
+    let output = match output_format.to_lowercase().as_str() {
+        "hex" => result.into_iter().fold(String::new(), |mut acc, byte| {
+            write!(&mut acc, "{byte:02x}").unwrap();
+            acc
+        }),
+        "base64" => base64::Engine::encode(&base64::engine::general_purpose::STANDARD, result),
+        _ => {
+            anyhow::bail!(
+                "Unsupported output format: {output_format}. Supported formats: hex, base64"
+            )
+        }
+    };
+
+    Ok(Some(output))
+}
+
 async fn client_ip(request: &RequestInfo) -> Option<String> {
    Some(request.client_ip?.to_string())
 }

+#[tokio::test]
+async fn test_hmac() {
+    // Test vector from RFC 4231 - HMAC-SHA256
+    let result = hmac(
+        Cow::Borrowed("The quick brown fox jumps over the lazy dog"),
+        Cow::Borrowed("key"),
+        Some(Cow::Borrowed("sha256")),
+    )
+    .await
+    .unwrap()
+    .unwrap();
+    assert_eq!(
+        result,
+        "f7bc83f430538424b13298e6aa6fb143ef4d59a14946175997479dbc2d1a3cd8"
+    );
+}
+
 /// Returns the ID token claims as a JSON object.
 async fn user_info_token(request: &RequestInfo) -> anyhow::Result<Option<String>> {
    let Some(claims) = &request.oidc_claims else {
@@ -119,3 +119,5 @@ async fn test_large_form_field_roundtrip() -> actix_web::Result<()> {
    );
    Ok(())
 }
+
+mod webhook_hmac;
@@ -0,0 +1,93 @@
+use actix_web::{http::StatusCode, test};
+use sqlpage::webserver::http::main_handler;
+
+use crate::common::get_request_to;
+
+#[actix_web::test]
+async fn test_webhook_hmac_invalid_signature() -> actix_web::Result<()> {
+    // Set up environment variable for webhook secret
+    std::env::set_var("WEBHOOK_SECRET", "test-secret-key");
+
+    let webhook_body = r#"{"order_id":12345,"total":"99.99"}"#;
+    let invalid_signature = "96a5f6f65c85a2d4d1f3a37813ab2c0b44041bdc17691fbb0884e3eb52b7c54b";
+
+    let req = get_request_to("/tests/webhook_hmac_validation.sql")
+        .await?
+        .insert_header(("content-type", "application/json"))
+        .insert_header(("X-Webhook-Signature", invalid_signature))
+        .set_payload(webhook_body)
+        .to_srv_request();
+
+    let resp = main_handler(req).await?;
+
+    // Should redirect to error page when signature is invalid
+    assert!(
+        resp.status() == StatusCode::FOUND || resp.status() == StatusCode::SEE_OTHER,
+        "Expected redirect (302 or 303) for invalid signature, got: {}",
+        resp.status()
+    );
+
+    let location = resp
+        .headers()
+        .get("location")
+        .expect("Should have Location header")
+        .to_str()
+        .unwrap();
+    assert_eq!(location, "/error.sql?err=bad_webhook_signature");
+    Ok(())
+}
+
+#[actix_web::test]
+async fn test_webhook_hmac_valid_signature() -> actix_web::Result<()> {
+    // Set up environment variable for webhook secret
+    std::env::set_var("WEBHOOK_SECRET", "test-secret-key");
+
+    let webhook_body = r#"{"order_id":12345,"total":"99.99"}"#;
+    let valid_signature = "260b3b5ead84843645588af82d5d2c3fe24c598a950d36c45438c3a5f5bb941c";
+
+    let req = get_request_to("/tests/webhook_hmac_validation.sql")
+        .await?
+        .insert_header(("content-type", "application/json"))
+        .insert_header(("X-Webhook-Signature", valid_signature))
+        .set_payload(webhook_body)
+        .to_srv_request();
+
+    let resp = main_handler(req).await?;
+
+    // Should return success when signature is valid
+    assert_eq!(resp.status(), StatusCode::OK, "200 resp for signed req");
+    assert!(!resp.headers().contains_key("location"), "no redirect");
+
+    assert_eq!(
+        test::read_body_json::<serde_json::Value, _>(resp).await,
+        serde_json::json! ({"msg": "Webhook signature is valid !"})
+    );
+    Ok(())
+}
+
+#[actix_web::test]
+async fn test_webhook_hmac_missing_signature() -> actix_web::Result<()> {
+    // Set up environment variable for webhook secret
+    std::env::set_var("WEBHOOK_SECRET", "test-secret-key");
+
+    let webhook_body = r#"{"order_id":12345,"total":"99.99"}"#;
+
+    // Don't include the X-Webhook-Signature header
+    let req = get_request_to("/tests/webhook_hmac_validation.sql")
+        .await?
+        .insert_header(("content-type", "application/json"))
+        .set_payload(webhook_body)
+        .to_srv_request();
+
+    let resp = main_handler(req).await?;
+
+    let location = resp
+        .headers()
+        .get("location")
+        .expect("Should have Location header")
+        .to_str()
+        .unwrap();
+    assert_eq!(location, "/error.sql?err=bad_webhook_signature");
+
+    Ok(())
+}
@@ -0,0 +1,6 @@
+-- Test HMAC with base64 output format
+-- Redirect if hash doesn't match expected value
+SELECT 'redirect' as component, '/error.sql' as link
+WHERE sqlpage.hmac('The quick brown fox jumps over the lazy dog', 'key', 'sha256-base64') != '97yD9DBThCSxMpjmqm+xQ+9NWaFJRhdZl0edvC0aPNg=';
+
+SELECT 'text' as component, 'It works ! HMAC SHA-256 base64 output is correct' as contents;
@@ -0,0 +1,5 @@
+-- Redirect if default algorithm doesn't match sha256
+SELECT 'redirect' as component, '/error.sql' as link
+WHERE sqlpage.hmac('test data', 'test key') != sqlpage.hmac('test data', 'test key', 'sha256');
+
+SELECT 'text' as component, 'It works ! HMAC default algorithm is SHA-256' as contents;
@@ -0,0 +1,5 @@
+-- Redirect if hash doesn't match expected value
+SELECT 'redirect' as component, '/error.sql' as link
+WHERE sqlpage.hmac('The quick brown fox jumps over the lazy dog', 'key', 'sha256') != 'f7bc83f430538424b13298e6aa6fb143ef4d59a14946175997479dbc2d1a3cd8';
+
+SELECT 'text' as component, 'It works ! HMAC SHA-256 hash is correct' as contents;
@@ -0,0 +1,5 @@
+-- Redirect if hash doesn't match expected value
+SELECT 'redirect' as component, '/error.sql' as link
+WHERE sqlpage.hmac('The quick brown fox jumps over the lazy dog', 'key', 'sha512') != 'b42af09057bac1e2d41708e48a902e09b5ff7f12ab428a4fe86653c73dd248fb82f948a549f7b791a5b41915ee4d1ec3935357e4e2317250d0372afa2ebeeb3a';
+
+SELECT 'text' as component, 'It works ! HMAC SHA-512 hash is correct' as contents;
@@ -0,0 +1,15 @@
+-- Test Shopify webhook HMAC validation with base64 output
+-- Shopify sends webhook body and HMAC signature in X-Shopify-Hmac-SHA256 header (base64 format)
+
+-- Redirect to error if signature doesn't match (proper pattern)
+SELECT 'redirect' as component,
+    '/error.sql?msg=invalid_signature' as link
+WHERE sqlpage.hmac(
+    '{"id":1234567890,"email":"customer@example.com","total_price":"123.45"}',
+    'test-webhook-secret',
+    'sha256-base64'
+) != 'QNyObTlKbMx2qDlPF/ZOZcBqg5OgPg+2oky3zldc0Gw=';
+
+-- If we reach here, signature is valid
+SELECT 'text' as component,
+    'It works ! Shopify webhook signature verified (base64 format)' as contents;
@@ -0,0 +1,17 @@
+-- Webhook HMAC signature validation example
+-- This simulates receiving a webhook with HMAC signature in header
+-- Redirect to error page if signature is invalid
+-- test this with: curl localhost:8080/tests/webhook_hmac_validation.sql -H 'X-Webhook-Signature: 260b3b5ead84843645588af82d5d2c3fe24c598a950d36c45438c3a5f5bb941c' -H 'Content-Type: application/json' --data-raw '{"order_id":12345,"total":"99.99"}' -v
+SET body = sqlpage.request_body();
+SET secret = sqlpage.environment_variable('WEBHOOK_SECRET');
+SET expected_signature = sqlpage.hmac($body, $secret, 'sha256');
+SET actual_signature =  sqlpage.header('X-Webhook-Signature');
+
+SELECT
+    'redirect' as component,
+    '/error.sql?err=bad_webhook_signature' as link
+WHERE $actual_signature != $expected_signature OR $actual_signature IS NULL;
+
+-- If we reach here, signature is valid - return success
+SELECT 'json' as component, 'jsonlines' as type;
+select 'Webhook signature is valid !' as msg;