Adversarial Robustness Toolbox (ART) - Python Library for Machine Learning Security - Evasion, Poisoning, Extraction, Inference - Red and Blue Teams

🐢 Open-Source Evaluation & Testing library for LLM Agents

[ACL 2024] An Easy-to-use Knowledge Editing Framework for LLMs.

[EMNLP 2024 Demo] MarkLLM: An Open-Source Toolkit for LLM Watermarking

Babysitter enforces obedience on agentic workforces and enables them to manage extremely complex tasks and workflows through deterministic, hallucination-free self-orchestration

The open-sourced Python toolbox for backdoor attacks and defenses.

[ICML 2024] TrustLLM: Trustworthiness in Large Language Models

Deliver safe & effective language models

🔥🔥🔥[AAAI 2026 Oral] Official Implementation of Robust-R1: Degradation-Aware Reasoning for Robust Visual Understanding

Proof of thought : LLM-based reasoning using Z3 theorem proving with multiple backend support (SMT2 and JSON DSL)

The testing platform for AI teams. Bring engineers, PMs, and domain experts together to generate tests, simulate (adversarial) conversations, and trace every failure to its root cause.

Moonshot - A simple and modular tool to evaluate and red-team any LLM application.

MarkDiffusion: An Open-Source Toolkit for Generative Watermarking of Latent Diffusion Models

[USENIX Security 2025] PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models

The AI Incident Database seeks to identify, define, and catalog artificial intelligence incidents.

🚀 A fast safe reinforcement learning library in PyTorch

[NeurIPS-2023] Annual Conference on Neural Information Processing Systems

A comprehensive toolbox for model inversion attacks and defenses, which is easy to get started.

[NeurIPS'24] "Membership Inference Attacks against Fine-tuned Large Language Models via Self-prompt Calibration"

A toolbox for benchmarking trustworthiness of multimodal large language models (MultiTrust, NeurIPS 2024 Track Datasets and Benchmarks)