This content originally appeared on DEV Community and was authored by Khalil Najjar
Over the past year, I've been working on a chatbot builder called Fastmind. My goal with this project was to learn how to build a fully automated service that could scale to thousands of users without breaking the bank. Given the nature of a chatbot builder, I encountered several interesting challenges, such as exposing an AI model to the web, preventing malicious users from abusing the service (which, in contrast to traditional CRUD apps, could turn out to be insanely expensive), and handling a large number of concurrent users for the chat stream without performance bottlenecks. These challenges were in addition to the more common SaaS issues like billing, user management, and monitoring.
In this post, I'll share the lessons I learned during this long journey and offer my advice on building software as a solo developer (spoiler alert: don't do what I did). By the time you finish reading this, I hope you'll find it useful whether you're building a similar product or just curious about how over-engineered chatbots work behind the scenes.
The Architecture Behind Fastmind
When building a new project, it's important to choose the right tools for the job.
So what is the best tool for this job? Well, you guessed it! Always choose what you know. In my case, I've been working with the JavaScript ecosystem for a while, so I decided to stick with React for the frontend and Hono for the backend. I also used Convex heavily for the database, cron jobs, real-time capabilities, and more, all bundled together in a Turborepo. I'll go into more detail about each part of the architecture in the following sections.
Here is a rough overview of the folder structure:
-
apps
chatbot-builderchat-widgetmarketing-websitehono-api
-
packages
convexuiutils
Frontend
I have three separate frontend applications. This separation was intentional to maintain a clear division between the chatbot builder (dashboard), the chatbot itself (the chat widget embedded on your website), and the marketing website. This separation makes it easier to maintain and scale the applications independently and push updates without affecting other parts of the system. For me, this setup works because it's the same one I use at work, so I feel very comfortable with this architecture. Again, this might not be your case, and that’s perfectly fine. I won’t go into too much detail about the marketing website, as it's not the focus of this post, but it’s a Next.js app hosted on Vercel.
Chatbot Builder
For the main app (chatbot builder), I used Next.js paired with Shadcn for UI components and Tailwind CSS for styling. Since I use Convex for the DB layer, I used their React library to connect to the DB to perform queries and mutations with real-time features and to call serverless functions with what they call "actions." I'll delve deeper into Convex and all of its amazing features in another post if you're interested. For authentication, I used Clerk to avoid dealing with user management and building all the different auth flows, allowing me to focus on product development. This is one of my favorite tools out there—you have to check them out if you're a React developer. This app is hosted on Vercel, which is a great platform for hosting Next.js apps.
Chat Widget
The chat widget is a basic Next.js application that connects to the long-running API to get the chatbot configuration, stream, and send messages. The chat widget is hosted on Cloudflare Workers, a serverless platform that allows you to run JavaScript code at the edge of the network. This is very useful for a chat widget, as it’s closer to the user and can significantly reduce latency and costs, especially if a large number of users start using the chat widget. This could easily be hosted alongside the chatbot builder, and it would be perfectly fine—and cheaper. But I am a bit of a performance freak, so I decided to go with Cloudflare Workers. Also, I've heard horror stories about the costs of apps hosted on Vercel being DDoSed, so I decided to go with Cloudflare Workers to prevent this from happening.
Backend
Long-Running API Server
The backend consists of a long-running Hono server that handles most of the chat widget requests, as explained above. It’s a Bun application that connects to Convex to read and write data to the database. It’s hosted on Railway, a platform that allows you to deploy and scale your applications without worrying about infrastructure. This server could be replaced with Convex and use their serverless functions or even HTTP solution, but I prefer to have a long-running API server for the chat widget to prevent DDoS attacks on my serverless functions. To achieve this, I have a local Redis instance available for the server to rate-limit requests by IP. Then I have another rate-limit layer directly on the DB (Convex) to block multiple requests for an account. Believe it or not, this helps me sleep at night.
Convex
Now the star of the show—most of the backend logic runs in Convex. It is very similar to Firebase or Supabase, but with an even better developer experience, in my opinion. I highly recommend checking them out if you're building a new project, especially in React. Caching, real-time updates, serverless functions, and cron jobs development environment are all handled by Convex. I also use their HTTP actions to handle incoming webhooks from Lemonsqueezy and Clerk.
Why did I choose Convex over Firebase, Supabase, or even a self-hosted solution? I initially started Fastmind using the T3 stack, but found it to be too much work to handle some of the real-time features I needed. I also required a lot of background jobs, for which I tried solutions like Bull, and services like Inngest and Trigger.dev. While these are great services, I wanted everything in one place, so I decided to give Convex a try, and I’m very happy with the results.
Infrastructure
As mentioned above, I use Vercel for the frontend apps, Cloudflare Workers for the chat widget, and Railway for the backend. I also use Sentry for error tracking, Lemonsqueezy for billing as my Merchant of Record so I don't have to worry about taxes, Clerk for authentication, and Convex for the database, cron jobs, and real-time features. The AI models are Command R and Command R+ from Cohere.
Another star of the show is Cloudflare. I use them heavily in Fastmind. In addition to what I’ve mentioned above, I use their AI Gateway service. This is a great product to use when you need to expose an AI model to the web; it handles rate limiting, caching, and DDoS protection for chat requests. All my domains are managed by Cloudflare—they are second to none when it comes to handling DNS and SSL certificates and preventing DDoS attacks. Lastly, their zero-egress pricing for storage is a great deal for me. I use it to store chatbot assets and scripts. Eventually, I plan to allow users to upload PDFs and other files to train their chatbots, as well as conversation logs so they can export them and analyze them later.
My Key Takeaways
It Doesn't Have to Be Perfect
I spent a lot of time trying to make everything perfect, but I realized it's better to have something that works and iterate on it. I waited too long to ship Fastmind, and I was going to postpone it even further. But my competition is already making bank while I was just building, and I must say I don't think Fastmind is missing any features; if anything, it has more. Since realizing this, I adopted a "ship fast, iterate often" mentality, and I’ve been able to launch Fastmind and get users without everything being perfect, and that is okay. I’ve been able to fix bugs and add features as I go, and users are happy. This is the beauty of building a SaaS product—you can always improve it; it’s never finished. So don't worry too much about making everything perfect—just get it out there and start getting feedback. This is something other successful founders have taught me, but being an engineer first, I always focused on the "fun" part of building the product and not on the "hard" part of getting users. I’m still learning this, but I’m getting better at it.
Focus on the User
After I launched, I started to realize what users wanted, and trust me, it's very different from what you think they want. I’ve been able to gather a lot of feedback from users and improve the product based on that feedback. For example, I thought users would want a lot of customization options for the chat widget, but it turns out they just want a simple chatbot that works and does one thing very well. I also thought users would want a lot of AI features, but it turns out they just want a chatbot that can answer their questions accurately. I’ve been able to focus on the features that users want and ignore the ones they don’t care about. This has helped me build a product that users love and are willing to pay for. Onboarding is key—I've seen a lot of users drop off because they don't know how to use the product, so I've been working on improving the onboarding process to make it easier for users to get started. I’ve built the easiest chatbot builder out there; in just 2 minutes you can have a chatbot up and running on your website (yes, I timed it).
The Tech Stack Doesn't Matter
I’ve been able to build Fastmind using the tech stack I know and love, but I’ve seen other successful founders build their products using entirely different tech stacks. What matters is that you build something that works and that users want. Don’t get too caught up in the tech stack—just build something functional and valuable. In the end, if your product is successful, you can always rewrite it in a different tech stack, but by then, you'll have the users, revenue, and motivation to do so.
Don’t Wait Too Long to Launch
I’ve been building Fastmind in the shadows since August 2023. Meanwhile, I have competitors who entered the market with a product similar to mine—not perfect, but they’re making money and collecting feedback while I was just building alone without a clear direction of what my users wanted. I should have launched Fastmind earlier; I would have received insanely valuable feedback from users and been able to improve the product based on that feedback faster. I was afraid of launching because I thought the product wasn't perfect, but I realized that it's better to launch something that works and iterate on it than to wait for it to be perfect.
This content originally appeared on DEV Community and was authored by Khalil Najjar
Khalil Najjar | Sciencx (2024-08-12T19:52:12+00:00) My chatbot builder is over-engineered, and I love it. Retrieved from https://www.scien.cx/2024/08/12/my-chatbot-builder-is-over-engineered-and-i-love-it/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.