# privacy-filter.cpp

A minimal C++/GGML inference engine for OpenAI's privacy-filter PII/NER model family, detecting personal-data entity spans with exact byte offsets much faster and with far less memory than Hugging Face Transformers.

privacy-filter.cpp is a minimal GGML inference engine for the openai-privacy-filter token-classification model family, detecting personally identifiable information and named entities as spans with exact UTF-8 byte offsets. It uses stock, unpatched upstream GGML, computing the model's YaRN frequency factors at load time.

It ships with pre-converted GGUF versions of the standard, multilingual and Nemotron variants of the model, and a self-contained script converts a Hugging Face checkpoint to GGUF without a llama.cpp dependency.

Benchmarked against the same models in Hugging Face Transformers on matched hardware, it classifies an 8,000-token document about 7.7 times faster on CPU, and on GPU stays flat in memory out to 131,000 tokens where Transformers runs out of memory around 16,000; on a Raspberry Pi 5 with no GPU it classifies more than 1,300 tokens of mixed PII in under 4 seconds.

Website: https://github.com/localai-org/privacy-filter.cpp