George Weale 0a4a5c6ce2 fix: Port eval identifier validation and persona template rendering to v1 (#6806)

2026-08-19 11:43:47 -07:00

..

simulation

fix: Port eval identifier validation and persona template rendering to v1 (#6806)

2026-08-19 11:43:47 -07:00

__init__.py

chore: Upgrade to headers to 2026

2026-01-20 14:50:09 -08:00

mock_gcs_utils.py

…

test__path_validation.py

fix: Port eval identifier validation and persona template rendering to v1 (#6806)

2026-08-19 11:43:47 -07:00

test_app_details.py

chore: Upgrade to headers to 2026

2026-01-20 14:50:09 -08:00

test_custom_metric_evaluator.py

feat: Remove overall_eval_status calculation from _CustomMetricEvaluator and add threshold to custom metric function expected signature

2026-01-26 11:01:32 -08:00

test_eval_case.py

chore: Upgrade to headers to 2026

2026-01-20 14:50:09 -08:00

test_eval_config.py

chore: Upgrade to headers to 2026

2026-01-20 14:50:09 -08:00

test_evaluation_generator.py

feat: add general support for Gemini Live API in ADK evaluate

2026-05-15 16:14:21 -07:00

test_final_response_match_v1.py

chore: Upgrade to headers to 2026

2026-01-20 14:50:09 -08:00

test_final_response_match_v2.py

chore: Upgrade to headers to 2026

2026-01-20 14:50:09 -08:00

test_gcs_eval_set_results_manager.py

fix: Port eval identifier validation and persona template rendering to v1 (#6806)

2026-08-19 11:43:47 -07:00

test_gcs_eval_sets_manager.py

fix: Port eval identifier validation and persona template rendering to v1 (#6806)

2026-08-19 11:43:47 -07:00

test_hallucinations_v1.py

chore: Upgrade to headers to 2026

2026-01-20 14:50:09 -08:00

test_in_memory_eval_sets_manager.py

chore: Upgrade to headers to 2026

2026-01-20 14:50:09 -08:00

test_llm_as_judge_utils.py

fix(auth): isolate resolved credentials in context to prevent race conditions and data leakage

2026-04-18 19:22:40 -07:00

test_llm_as_judge.py

chore: Upgrade to headers to 2026

2026-01-20 14:50:09 -08:00

test_local_eval_service.py

fix(eval): handle failed inference results without invocations (cherry-pick) (#6565)

2026-08-04 12:10:52 -07:00

test_local_eval_set_results_manager.py

fix: Port eval identifier validation and persona template rendering to v1 (#6806)

2026-08-19 11:43:47 -07:00

test_local_eval_sets_manager.py

fix: Port eval identifier validation and persona template rendering to v1 (#6806)

2026-08-19 11:43:47 -07:00

test_metric_evaluator_registry.py

fix: Scope custom eval metrics per registry and resolve their function path from the eval config (v1) (#6792)

2026-08-19 11:42:46 -07:00

test_multi_turn_task_success_evaluator.py

feat: Add MultiTurn Task success metric

2026-03-16 10:09:05 -07:00

test_multi_turn_tool_use_quality_evaluator.py

feat: Add MultiTurn Task trajectory and tool trajectory metrics

2026-03-16 10:44:28 -07:00

test_multi_turn_trajectory_quality_evaluator.py

feat: Add MultiTurn Task trajectory and tool trajectory metrics

2026-03-16 10:44:28 -07:00

test_request_intercepter_plugin.py

chore: Upgrade to headers to 2026

2026-01-20 14:50:09 -08:00

test_response_evaluator.py

chore: Upgrade to headers to 2026

2026-01-20 14:50:09 -08:00

test_retry_options_utils.py

chore: Upgrade to headers to 2026

2026-01-20 14:50:09 -08:00

test_rubric_based_evaluator.py

chore: Upgrade to headers to 2026

2026-01-20 14:50:09 -08:00

test_rubric_based_final_response_quality_v1.py

chore: Upgrade to headers to 2026

2026-01-20 14:50:09 -08:00

test_rubric_based_tool_use_quality_v1.py

chore: Upgrade to headers to 2026

2026-01-20 14:50:09 -08:00

test_safety_evaluator.py

chore: Upgrade to headers to 2026

2026-01-20 14:50:09 -08:00

test_trajectory_evaluator.py

ci: match the environment with internal ones for pyink/isort/unittest

2026-02-11 13:01:32 -08:00

test_vertex_ai_eval_facade.py

feat: Add MultiTurn Task success metric

2026-03-16 10:09:05 -07:00

test_vertex_ai_scenario_generation_facade.py

feat(eval): Add ADK CLI for eval_set generation

2026-04-07 09:43:35 -07:00