zylon-ai/PrivateGPT
Interact with your documents using the power of GPT, 100% privately, no data leaks
PrivateGPT is an open-source, on-premise AI layer by Zylon for building private AI apps on top of local models. Originally a popular tool for chatting with documents offline, its v1.0.0 release turns it into an API layer that connects to any OpenAI-compatible inference server.
Our Review
With more than 57,000 GitHub stars, PrivateGPT was one of the breakout projects of 2023, the tool that showed how to run useful AI over your own documents with nothing leaving your machine. It is actively maintained by Zylon and reached a milestone v1.0.0 in June 2026.
What PrivateGPT does:
- •Private AI API layer provides higher-level building blocks (chat, RAG, ingestion) over local models so you do not rebuild backend primitives for every private-AI app.
- •Bring your own inference server connects to any OpenAI-compatible server (Ollama, llama.cpp, vLLM) via an environment variable rather than running models itself.
- •Document ingestion and RAG ingest and query your own documents so answers stay grounded in your data, fully on-premise.
- •100% private runs entirely in your environment with no data leaving your infrastructure, the project's founding promise.
- •Built-in workbench UI a testing and demo interface at /ui, though the API is the actual product.
- •Production-tested the same engine behind Zylon, an on-premise AI platform used by enterprises.
PrivateGPT ecosystem:
- •Zylon the commercial on-premise AI platform built on PrivateGPT for teams that want a finished product.
- •OpenAI-compatible backends works with Ollama, llama.cpp, vLLM, and anything implementing the standard chat and models endpoints.
- •Documentation and community docs at docs.privategpt.dev plus an active Discord.
Getting started:
Point PrivateGPT at an OpenAI-compatible inference server with the OPENAI_API_BASE variable, then run the API and open the workbench at /ui to test. Full setup and the new v1.0.0 architecture are documented at docs.privategpt.dev.
Limitations:
PrivateGPT does not run models itself, so you also need to set up an inference server like Ollama or vLLM separately. The v1.0.0 release is a significant repositioning from the original document-chat app, so older tutorials and the GitHub description may describe the previous design. It is a developer-focused API layer, not a finished end-user app, and the polished product experience lives in the commercial Zylon platform.
Our Verdict
PrivateGPT is one of the defining projects of the private, on-premise AI movement, and its 2026 shift to an API layer makes it more of a foundation for building private apps than a standalone tool.
For Developers, the appeal is skipping the boilerplate: PrivateGPT gives you chat, ingestion, and RAG primitives over whatever local inference server you already run, so you build private AI products without rewriting the backend each time. Connecting it via OPENAI_API_BASE keeps your model choice open.
For Vibe Builders, the open-source layer is more infrastructure than finished app now, so the original appeal of a simple chat-with-your-docs tool is better served today by the commercial Zylon product or a turnkey local app. The repo is where you go to build, not to click.
Skip it if you want a ready-made desktop app to chat with documents, where a tool like Zylon or a local UI will be simpler, or if you have no separate inference server to connect. For teams that need a private AI backend they control, though, PrivateGPT is a strong open-source base.
Frequently Asked Questions
Is PrivateGPT free and open-source?
Yes, PrivateGPT is free and open-source under the Apache 2.0 license, and it has more than 57,000 GitHub stars as of 2026. It is maintained by Zylon, which also offers a commercial on-premise platform built on top of it. You can self-host the open-source layer at no cost beyond your own infrastructure.
What does PrivateGPT do?
As of its v1.0.0 release, PrivateGPT is an open-source API layer that provides higher-level building blocks like chat, document ingestion, and retrieval-augmented generation over local models. It connects to any OpenAI-compatible inference server, so you can build private AI applications that keep all data on your own infrastructure.
Does PrivateGPT run AI models itself?
No, as of v1.0.0 PrivateGPT does not run models itself. It connects to a separate OpenAI-compatible inference server such as Ollama, llama.cpp, or vLLM through the OPENAI_API_BASE setting. This design lets you choose and manage your own model backend while PrivateGPT handles the application-layer primitives.
What is the difference between PrivateGPT and Zylon?
PrivateGPT is the open-source API layer you self-host and build on, while Zylon is the commercial, finished on-premise AI platform built on top of it by the same team. Choose PrivateGPT when you want to build your own private AI product. Choose Zylon when you want a ready-made enterprise application.
Is PrivateGPT actually private?
Yes, privacy is its founding purpose: PrivateGPT runs entirely in your own environment and connects only to inference servers you control, so no document data leaves your infrastructure. This makes it suited to regulated industries and on-premise deployments. Released in 2023, it remains actively maintained, reaching v1.0.0 in June 2026.
What is PrivateGPT?
PrivateGPT is an open-source, on-premise AI layer by Zylon for building private AI apps on top of local models. Originally a popular tool for chatting with documents offline, its v1.0.0 release turns it into an API layer that connects to any OpenAI-compatible inference server.
How do I install PrivateGPT?
Visit the GitHub repository at https://github.com/zylon-ai/private-gpt for installation instructions.
What license does PrivateGPT use?
PrivateGPT uses the Apache-2.0 license.
What are alternatives to PrivateGPT?
Explore related tools and alternatives on My AI Guide.
Open source & community-verified
Apache-2.0 licensed: free to use in any project, no strings attached. 57,210 developers have starred this, meaning the community has reviewed and trusted it.
Reviewed by My AI Guide for relevance, quality, and active maintenance before listing.
Topics