This content originally appeared on DEV Community and was authored by Aman Yadav

This is a submission for the Open Source AI Challenge with pgai and Ollama

What I Built

I developed a fully local, AI-powered Knowledge Base Management System that enables users to upload documents and interact with them through RAG (Retrieval-Augmented Generation). This system harnesses the capabilities of PostgreSQL’s AI extensions alongside Ollama's local models, creating a privacy-centered solution that operates entirely on your own infrastructure.

Key Features:

Fully Local Operation
- All AI processing occurs directly on your machine, powered by Llama 3.1
- Ensures that no data leaves your infrastructure, providing complete privacy and control.
Intelligent chat interface with RAG capabilities
- Uses Retrieval-Augmented Generation to deliver accurate responses.
- Vector similarity search is driven by pgvector, with efficient querying through pgvectorscale.
Function Calling
- Supports extensible operations for expanded interaction capabilities.
Modern Web Interface
- Features a clean, intuitive UI designed with shadcn/ui.
- Enables real-time chat interactions with integrated file management.

Demo

In this demo, users can select multiple PDFs and receive answers based on the content. Here, the hackathon announcement is used as a source. Additionally, users have the option to manage files within the app.

User also has option to manage files in the app

Below is the chat interface, where users interact with the documents.

GitHub Repository:

yadav-aman / ai-chatbot

Next.js AI Knowledge-base chatbot

An fully local Open-Source AI knowledge made with Next.js and the AI SDK Powered by Ollama and Postgres

Features · Model Providers · Deploy Your Own · Running locally

Features

Next.js App Router
- Advanced routing for seamless navigation and performance
- React Server Components (RSCs) and Server Actions for server-side rendering and increased performance
AI SDK
- Unified API for generating text, structured objects, and tool calls with LLMs
- Hooks for building dynamic chat and generative user interfaces
shadcn/ui
- Styling with Tailwind CSS
- Component primitives from Radix UI for accessibility and flexibility
Data Persistence
- Postgres for saving chat history and user data
- Min.io for efficient file storage
NextAuth.js
- Simple and secure authentication
Ollama for AI model management
- Easily switch between different AI models and providers
pgvector for vector similarity search
- Efficiently store embeddings for similarity search
pgvectorscale for scaling vector similarity search
- Query embeddings for RAG

Model Providers

…

View on GitHub

Tools Used

The project leverages several powerful open-source tools:

AI and Database:

PostgreSQL as the primary database
pgvector for storing vectors
pgvectorscale for similarity search
Ollama running
- Llama 3.1 for chat
- mxbai-embed-large for embeddings

Frontend and Backend:

Next.js 14 with App Router
Vercel AI SDK for chat interfaces
shadcn/ui for component design
Tailwind CSS for styling
NextAuth.js for authentication
Min.io for file storage
Drizzle as ORM

Final Thoughts

Building this project for the Open Source AI Challenge was a fulfilling journey. The most challenging yet rewarding aspect was enabling it to run entirely locally, offering a significant data privacy advantage.

Using pgvector and pgvectorscale made the RAG implementation seamless.
The pgai Vectorizer currently supports only OpenAI embeddings. While I initially aimed to integrate it with Ollama, the absence of a compatible tokenization endpoint made this integration unfeasible. As an alternative, I implemented the embedding logic directly.

Prize categories:

Main Category
Open-source Models from Ollama

Future Improvements

Fine-tune prompts for better tool interaction
Add support for more document formats
Implement multi-model support
Optimize vector search performance
Add batch processing capabilities for large document sets

This content originally appeared on DEV Community and was authored by Aman Yadav

Print Share Comment Cite Upload Translate Updates

APA

Aman Yadav | Sciencx (2024-11-10T19:19:31+00:00) Local AI Knowledge Base with Next.js, Ollama, and PostgreSQL. Retrieved from https://www.scien.cx/2024/11/10/local-ai-knowledge-base-with-next-js-ollama-and-postgresql/

MLA

" » Local AI Knowledge Base with Next.js, Ollama, and PostgreSQL." Aman Yadav | Sciencx - Sunday November 10, 2024, https://www.scien.cx/2024/11/10/local-ai-knowledge-base-with-next-js-ollama-and-postgresql/

HARVARD

Aman Yadav | Sciencx Sunday November 10, 2024 » Local AI Knowledge Base with Next.js, Ollama, and PostgreSQL., viewed ,<https://www.scien.cx/2024/11/10/local-ai-knowledge-base-with-next-js-ollama-and-postgresql/>

VANCOUVER

Aman Yadav | Sciencx - » Local AI Knowledge Base with Next.js, Ollama, and PostgreSQL. [Internet]. [Accessed ]. Available from: https://www.scien.cx/2024/11/10/local-ai-knowledge-base-with-next-js-ollama-and-postgresql/

CHICAGO

" » Local AI Knowledge Base with Next.js, Ollama, and PostgreSQL." Aman Yadav | Sciencx - Accessed . https://www.scien.cx/2024/11/10/local-ai-knowledge-base-with-next-js-ollama-and-postgresql/

IEEE

" » Local AI Knowledge Base with Next.js, Ollama, and PostgreSQL." Aman Yadav | Sciencx [Online]. Available: https://www.scien.cx/2024/11/10/local-ai-knowledge-base-with-next-js-ollama-and-postgresql/. [Accessed: ]

rf:citation

» Local AI Knowledge Base with Next.js, Ollama, and PostgreSQL | Aman Yadav | Sciencx | https://www.scien.cx/2024/11/10/local-ai-knowledge-base-with-next-js-ollama-and-postgresql/ |

Please log in to upload a file.

There are no updates yet.
Click the Upload button above to add an update.

You must be logged in to translate posts. Please log in or register.