Files
omnigent-ai--omnigent/tests/server/test_feature_flags.py
T
Zeyi (Rice) Fan b9d53f0a96 feat(server): add deployment-wide release feature flags (#4775)
## Related issue

Follow-up to #4673.

## Summary

- Add a typed, default-off release-feature registry driven by one comma-separated `OMNIGENT_FEATURES` environment variable, with strict validation and lifecycle metadata.
- Gate the web Usage route/navigation and page-only report enrichment while preserving the existing `GET /v1/usage` CLI API.
- Migrate web-driven harness installation to the same immutable startup snapshot and wire rollout configuration across Docker, Kubernetes, Render, Railway, and Databricks.

ELI5: the server reads one list of enabled features when it starts, enforces that same list on backend routes, and tells the web app which controls and pages to show.

```text
OMNIGENT_FEATURES
        |
        v
  FeatureFlags snapshot
     /             \
backend gates    GET /v1/info
                       |
                       v
                 frontend gates
```

## Test Plan

- `uv run pytest tests/server/test_feature_flags.py tests/host/test_local_server.py tests/server/integration/test_utility_endpoints.py tests/server/integration/test_hosts_install_harness.py tests/server/integration/test_hosts_store_credential.py tests/server/routes/test_usage_report.py tests/server/test_openapi_drift.py -q`
- `cd web && pnpm vitest run src/lib/capabilities.test.ts src/lib/harnessSetup.test.ts src/App.test.tsx src/shell/Sidebar.test.tsx`
- `uv run pytest tests/e2e_ui/sessions/test_usage_page_feature.py -q`
- `uv run python scripts/dump_openapi.py --check`
- `pre-commit run --files <changed files>`
- Verified default-off and enabled Usage route/sidebar behavior, strict unknown-feature rejection, legacy CLI usage compatibility, and harness route enforcement.

## Demo

- Default off: the updated visual baselines show the original sidebar without the Usage row.
- Enabled Usage page: https://github.com/user-attachments/assets/8385d4f0-47ad-430f-bf2c-06c35af6c499

## Type of change

- [ ] Bug fix
- [x] Feature
- [x] UI / frontend change
- [ ] Refactor / chore
- [x] Docs
- [ ] Test / CI
- [ ] Breaking change

## Test coverage

- [x] Unit tests added / updated
- [x] Integration tests added / updated
- [x] E2E tests added / updated
- [x] Manual verification completed
- [x] Existing tests cover this change
- [ ] Not applicable

## Coverage notes

Manually reviewed the default-off visual output and verified that the Usage route is absent while the capability is disabled. Targeted backend and frontend tests cover both flag states, capability parsing, startup snapshots, and harness enforcement.

## Changelog

Usage and web-driven harness setup can now be enabled per deployment with `OMNIGENT_FEATURES`.

Signed-off-by: Zeyi (Rice) Fan <zeyi.f@databricks.com>
2026-08-14 10:57:20 -07:00

78 lines
2.5 KiB
Python

"""Tests for deployment-wide release feature resolution."""
from __future__ import annotations
import dataclasses
import pytest
from omnigent.server.feature_flags import (
FEATURE_DEFINITIONS,
FEATURES_ENV_VAR,
Feature,
FeatureFlags,
resolve_feature_flags,
)
def test_features_default_off() -> None:
flags = resolve_feature_flags({})
assert flags.enabled_features == frozenset()
assert flags.frontend_dict() == {
"usage_page": False,
"harness_install": False,
}
def test_resolves_comma_separated_enabled_set() -> None:
flags = resolve_feature_flags({FEATURES_ENV_VAR: " usage_page, harness_install,usage_page "})
assert flags.enabled(Feature.USAGE_PAGE)
assert flags.enabled(Feature.HARNESS_INSTALL)
assert flags.enabled_names() == ("harness_install", "usage_page")
def test_empty_entries_are_ignored() -> None:
assert resolve_feature_flags({FEATURES_ENV_VAR: " , , "}) == FeatureFlags()
def test_removed_harness_install_variable_fails_with_migration_hint() -> None:
with pytest.raises(ValueError, match="OMNIGENT_FEATURES=harness_install"):
resolve_feature_flags({"OMNIGENT_HARNESS_INSTALL_ENABLED": "1"})
def test_removed_harness_install_variable_allows_explicit_off() -> None:
flags = resolve_feature_flags({"OMNIGENT_HARNESS_INSTALL_ENABLED": "0"})
assert not flags.enabled(Feature.HARNESS_INSTALL)
def test_unknown_feature_fails_with_known_names() -> None:
with pytest.raises(ValueError) as exc_info:
resolve_feature_flags({FEATURES_ENV_VAR: "usage-pgae"})
message = str(exc_info.value)
assert "usage-pgae" in message
assert "usage_page" in message
assert "harness_install" in message
def test_snapshot_is_immutable_and_does_not_follow_environment_mutation() -> None:
environ = {FEATURES_ENV_VAR: "usage_page"}
flags = resolve_feature_flags(environ)
environ[FEATURES_ENV_VAR] = "harness_install"
assert flags.enabled(Feature.USAGE_PAGE)
assert not flags.enabled(Feature.HARNESS_INSTALL)
with pytest.raises(dataclasses.FrozenInstanceError):
flags.enabled_features = frozenset() # type: ignore[misc]
def test_release_flags_have_lifecycle_metadata_and_default_off() -> None:
assert {definition.feature for definition in FEATURE_DEFINITIONS} == set(Feature)
for definition in FEATURE_DEFINITIONS:
assert definition.owner
assert definition.review_by_release
assert not FeatureFlags().enabled(definition.feature)