garak
An LLM vulnerability scanner that probes a model for hallucination, data leakage, prompt injection, misinformation, toxicity and jailbreaks, in the spirit of nmap or Metasploit but for language models.
garak (Generative AI Red-teaming & Assessment Kit) is a vulnerability scanner for large language models. It probes a model for hallucination, data leakage, prompt injection, misinformation, toxicity generation, jailbreaks and other weaknesses, combining static, dynamic and adaptive probes, in a role its own documentation compares to nmap or the Metasploit Framework for LLMs.
It is a free, command-line tool that supports Hugging Face Hub models, Replicate, the OpenAI chat and completion APIs, AWS Bedrock, LiteLLM, GGUF models through llama.cpp, and most other model endpoints reachable over REST. A target model family and specific model are chosen through command-line options, and a full list of available probes can be listed from the tool itself.
It is written in Python and released under the Apache-2.0 license.