Giskard
AI Evaluation & Testing
Open-source testing platform backed by European Commission grants for LLM red-teaming and automated vulnerability scanning.
Supporting capability — scope dependentSupporting governance capability
Testing, Evaluation & Observability
Testing, evaluation and monitoring capabilities can support several AI Act obligations, but a tool capability should not be treated as proof of legal compliance.
Statutory basisArticle 15 & Article 72
Practical checkpoints
- Prompt injection & jailbreak red-teaming
- Token tracing and latency monitoring
- Model drift and bias alerting
Navigator mappings are a discovery aid. They do not establish that an organisation or system is legally in scope, compliant or certified.
Start with the evidence
These software matches come from the requirement tags recorded in the Navigator catalogue. Review each vendor profile and its source evidence before making a procurement or compliance decision.
Run assessmentMAPPED SOFTWARE
Tools recorded for this requirement
50 source-linked profiles currently mapped to this requirement.
LatticeFlow AI
Technical Documentation
Co-creator of the COMPL-AI framework evaluating foundation models directly against EU statutory mandates.
Promptfoo
AI Security & Guardrails
Open-source developer CLI with automated red-teaming, prompt injection benchmarks, and CI/CD security assertions.
Langfuse
AI Evaluation & Testing
Berlin-based open-source LLM observability platform providing continuous execution logging and trace audits in Frankfurt.
Lakera
AI Security & Guardrails
Swiss AI security company providing real-time inference firewalls against prompt injection, jailbreaks, and PII leaks.
LangSmith
AI Evaluation & Testing
Developer-centric LLM observability platform providing automated tracing, regression evaluation, and runtime prompt logging.
Protect AI
AI Security & Guardrails
Security platform scanning model weights, pipelines, and producing AI Software Bill of Materials (AI-SBOM).
Cisco AI Defense (Robust Intelligence)
AI Security & Guardrails
Enterprise AI security and stress-testing platform acquired by Cisco providing real-time firewalls and automated adversarial validation.
Fiddler AI
AI Evaluation & Testing
Enterprise observability platform providing explainable AI (XAI), fairness metrics, and production drift telemetry.
Arize AI
AI Evaluation & Testing
AI evaluation and observability platform tracking embedding drift, real-time evals, and prompt telemetry.
WhyLabs
AI Evaluation & Testing
Privacy-preserving AI observability platform computing statistical summaries on-device without data leakage.
TruEra
AI Evaluation & Testing
AI quality management suite evaluating explainability, relevance, and hallucination scores across LLMs and predictive ML.
Encord
Technical Documentation
Data development platform providing automated annotation quality control, data lineage, and bias curation.
Kolena
AI Evaluation & Testing
Scenario-driven model evaluation platform verifying model behavior across granular sub-cohorts and edge cases.
Monolith AI
Technical Documentation
No-code AI platform helping physical engineering teams test and validate models used in critical safety products.
Armilla AI
AI Governance & Inventory
AI assurance engine providing automated risk verification and performance warranties for enterprise AI models.
Arthur AI
AI Evaluation & Testing
Model monitoring and governance engine offering real-time prompt risk scoring, hallucination tracking, and bias evaluation.
Deepchecks
AI Evaluation & Testing
Continuous AI validation platform providing automated test suites across data integrity, model behavior, and LLM evaluation.
Patronus AI
AI Evaluation & Testing
Automated LLM evaluation platform specializing in hallucination scoring, copyright exposure detection, and enterprise safety benchmarking.
Galileo
AI Evaluation & Testing
Evaluation and guardrails platform providing automated chain-of-thought metrics, prompt debugging, and production safety scoring.
Braintrust
AI Evaluation & Testing
Enterprise-grade evaluation engine providing fast automated testing, scoring loops, and production AI telemetry.
Tecnotree Sensa AI
AI Evaluation & Testing
Enterprise AI intelligence platform integrating CognitiveScale's trusted AI engines for automated risk, bias, and explainability governance.
Snyk AI Security
AI Security & Guardrails
Developer security platform detecting vulnerabilities in AI pipelines, open-source model packages, and training code.
Securiti
AI Governance & Inventory
DataAI security and governance platform that discovers and catalogs AI models and agents, classifies AI risk, maps data-to-AI relationships, and assesses AI systems against the EU AI Act and other regulations.
Confident AI (DeepEval)
AI Evaluation & Testing
Production evaluation platform built on DeepEval, enabling automated regression tests and hallucination metrics directly in CI/CD.
Iterative (CML / DVC)
Technical Documentation
Open-source developer platform bringing Git version control to datasets, machine learning experiments, and model pipelines.
Qualdo
AI Evaluation & Testing
Continuous ML and data quality monitoring platform providing automated anomaly detection, performance tracking, and confidence scoring.
Aimon AI
AI Evaluation & Testing
Continuous observability and safety API evaluating LLM hallucination, instruction drift, and factual alignment.
DeepKeep
AI Security & Guardrails
AI security and robustness platform running automated white-box and black-box penetration tests across vision and language models.
Enkrypt AI
AI Security & Guardrails
AI security and compliance platform providing red teaming, runtime guardrails, policy enforcement, monitoring and audit-ready evidence.
Unify AI
AI Evaluation & Testing
Dynamic LLM routing infrastructure ensuring high availability, latency optimization, and automated failover compliance.
OpenLLMetry (Traceloop)
AI Evaluation & Testing
Open-source standard and platform for tracing LLM execution spans, token metrics, and runtime errors.
Trustwise
AI Security & Guardrails
Safety framework and proxy for autonomous AI agents, preventing unauthorized tool calls, hallucinations, and policy breaches.
OctoAI Evaluation
AI Evaluation & Testing
Inference optimization platform evaluating compute usage, model efficiency, and token latency across cloud hardware.
Relari AI
AI Evaluation & Testing
Synthetic dataset generation and RAG evaluation platform stress-testing generative pipelines on enterprise edge cases.
ModelScan
AI Security & Guardrails
Open-source serialization vulnerability scanner inspecting PyTorch, Keras, and Pickle model weights for code execution exploits.
Kili Technology
Technical Documentation
French data labeling and quality platform auditing annotation consensus, data lineage, and training dataset integrity.
ClearML
Technical Documentation
Open-source MLOps platform recording full environment configurations, dataset versions, and training telemetry for model reproducibility.
HiddenLayer
AI Security & Guardrails
AI security platform covering AI asset discovery, model scanning, red teaming and runtime protection across the AI lifecycle.
Synthetik Applied Technologies
Technical Documentation
Physics-based modeling and deep learning validation suite verifying AI reliability under extreme physical operating environments.
NVIDIA NeMo Guardrails
AI Security & Guardrails
Open-source Python toolkit for adding programmable guardrails that control or validate inputs and outputs of LLM applications.
Vectara (HHEM Hallucination Engine)
AI Evaluation & Testing
GenAI platform hosting the open Hughes Hallucination Evaluation Model (HHEM) for measuring factual consistency in RAG outputs.
EvalML (Alteryx)
AI Evaluation & Testing
Open-source AutoML library that builds, optimizes, and evaluates machine learning pipelines using domain-specific objective functions.
Fairlearn
AI Evaluation & Testing
Open-source Python framework providing fairness metrics, disparate impact mitigation algorithms, and comparative fairness dashboards.
AI Fairness 360 (AIF360 - IBM)
AI Evaluation & Testing
Open-source toolkit containing over 70 fairness metrics and 10 bias mitigation algorithms developed by IBM Research and Linux Foundation AI.
Alibi Detect (Seldon)
AI Evaluation & Testing
Open-source Python library focusing on outlier, adversarial, and concept drift detection for tabular, image, and text models.
WhyLogs Bias & Fairness Auditing
AI Evaluation & Testing
Open-source data logging standard enabling differential statistical profiling and demographic fairness metric computation directly on-device.
Garak LLM Vulnerability Scanner
AI Security & Guardrails
Open-source generative AI vulnerability scanner probing language models for prompt injection, data leakage, and toxic generation.
Mindgard AI Security
AI Security & Guardrails
Cybersecurity platform dedicated to testing, detecting, and mitigating threats against enterprise AI models and GenAI applications.
Context.ai
AI Evaluation & Testing
Product analytics and user interaction tracking platform for LLM applications, monitoring conversation drop-offs and sentiment drift.