This content originally appeared on Tania Rascia | RSS Feed and was authored by hello@taniarascia.com
Not too long ago, I realized I didn't have a really good system for organizing my thoughts, and I wasn't happy with any of the note-taking apps I tried. Evernote was too bloated for me, and required a paid account to sync between multiple devices. SimpleNote was closer to what I wanted, but I found it somewhat ugly and clunky. Keep was too simple, Bear didn't have a web-based version...and so on.
Eventually I thought - why not just make exactly what I want? So I started building an app with the technologies I knew.
That app became TakeNote , which I'm proud to say has been completed, thanks to over 50 open-source contributors who helped along the way. The source is up on GitHub for anyone who wants to see how it was done.
What I Wanted It To Do
A few of my most important wants for the app:
- Plain text notes, no WYSIWYG or rich text editor
- Accessible from a web browser
- Authenticates and syncs with GitHub
- Does not require a database
- Has the look and feel of an IDE like Visual Studio Code, with syntax highlighting, markdown preview, and keyboard shortcuts
- Links to internal notes like a wiki
- Drag and drop
- Easy to back up and import
I'm really happy I saw it through to the end. It took about a full year with some breaks here and there (I started in September, 2019). I learned a lot along the way, much of which became knowledge that I directly used in my job.
Why I Don't Want to Ship It
TakeNote has two versions - the static, client-side only demo version that only saves to local storage, and a self-hosted complete version that runs on an Express server and authenticates and syncs with GitHub. Originally, I had the full, authenticated version hosted and despite not advertising this app at all, over 1,300 people authenticated with it, but now I have the demo version hosted on Netlify at takenote.dev.
Although I spent a lot of time on the project and I'm proud of it, I ultimately decided not to ship it. I spent a long time agonizing about what to do with the project, because I knew I didn't want to maintain it or have real users. There were a few reasons for that:
- I don't want to maintain a server. If I had real users and the app went down, I didn't feel comfortable having that knowledge in the back of mind that if there was a DDoS or DigitalOcean was down or for any other reason the server went down, the app would be down for users.
- I don't want to be responsible for anyone's private data. Although the data was saved in GitHub and not a database of mine, I'm just really not interested in people's private data in any capacity, and a note-taking app is inherently going to contain personal data.
- GitHub OAuth requires full access to all private repositories to get access to any private repository. There is no way around it. The "GitHub App" flow did not pertain to my use case, and all private repos is the only relevant scope for my app. I didn't feel comfortable having these permissions for any reason, and nor do many other people, which is understandable.
- If I were to actually host this, it would have users and likely go over the rate limiting allowed very quickly, rendering the app pretty useless.
- After getting far along in the project, I realized it would be impossible to make a good mobile experience for such an involved app, especially since it's impossible to disable zoom on iOS devices. I originally planned to just make a responsive web app for mobile, but after realizing I would need to make iOS/Android apps for mobile to work, I realized I wasn't willing to double the effort that I had already put in to complete this last step.
What I've ended up with is an excellent desktop web app, that looks and feels exactly like I'd want a note-taking app to feel. The hosted version is perfectly fine for any private, ephemeral notes anyone wants to take that definitely does not connect to any server or database, and imports/exports to JSON.
Personally, I decided to settle on Bear for a notes app for myself, as I found that it had so many of the features I implemented in TakeNote, and I appreciated it. My only issue with it is that it's not available from the web or for Windows (as I now have a Windows PC for gaming/other activities as well and would like to access notes from there).
What I Learned While Writing It
I wrote several articles that were the result of things I learned while writing this app.
Authentication (OAuth)
I wrote Client-side Authentication the Right Way (Cookies vs. Local Storage) , which details how to make a secure authentication set up using HTTP Only, Secure, Same-Site cookies with an Express server and a front end. I also learned the OAuth flow, and how easy it is to set up instead of using some third-party system like Auth0, which deceptively sounds similar to OAuth and can easily trick people into thinking they need to use it.
CI/CD (Docker and Travis)
I wanted to automate deployments and testing for the app, but I had never done it before from scratch. It's easy enough with a static site and something like Netlify or GitHub pages, but it's a whole different beast for a site with a server. I wrote Create a CI/CD Pipeline with Docker, which entailed all that I learned while creating an automation pipeline that would test all my code in PRs, and deploy it to a server behind a load balancer every time a merge was made into the master branch.
State management (Redux)
Before I started this project, I mostly avoided Redux, preferring to use Context in React for projects at work. As a result of setting up Redux Toolkit and Redux sagas for this project, I wrote An Overview and Walkthrough of Redux which builds demo applications of projects using vanilla Redux and Redux Toolkit. After using Redux, I realized I preferred it greatly to using Context. It does add a lot of boilerplate upfront, but the limitations of Context caused issues in previous projects as they grew.
Bundling and configuration (webpack)
I wrote How to Set Up webpack 5 From Scratch based on much of what of what I learned setting up a custom webpack config for the app that would support both an Express-serving-a-frontend app and a static frontend app.
So the moral of the story here is it's always a good idea to reinvent the wheel and build a project because you'll learn something along the way. It's never a waste of time.
How I made it
Here are the decisions and various tech I used to create the app.
Language (TypeScript)
I chose to write TakeNote in TypeScript. I had a little bit of experience here and there with TypeScript, but I wanted to make a complete, full-stack app to see if it lives up to the hype. It did get a bit annoying with various React and web types - or using some third-party dependencies, but I did feel confident that a lot of little bugs were prevented by using the strictest settings.
Front end (React + Redux)
I used React and Redux for the front end. It was either that, or Vue and Vuex, and although I've used Vue in the past for work, I just feel faster and more productive with React. I used simple SCSS for the styles instead of a CSS-in-JS solution, just because it's easier for me and simpler in my opinion.
Back end (Node + Express)
I used Node and Express for the full version of the app. I only needed a few endpoints, for authenticating and syncing. I decided against making a real update on every change, and instead only sync all the data at once on an interval. The server is necessary to obtain an authentication token and securely store it to allow a user to remain logged in, as GitHub does not have a PKCE setup that would allow OAuth with a front end only app.
Data and authentication (GitHub)
I used GitHub to authenticate, instead of creating a database with users, so all of that security could be handled elsewhere. I also used the GitHub API to make commits on every interval with the changes made to the notes.
Primary component
The text editor is built with CodeMirror, which is also used by Chrome and Firefox DevTools, Codepen, GitHub, and basically any other in-browser fiddle environment.
Behind the scenes, it uses webpack for bundling, Jest and Cypress for testing, ESLint, and a few other open-source helper libraries.
GitHub Sync
The part I had the most trouble with in the application was syncing to GitHub. I had the app completed up to that point for a long time before I finally finished it. My original intention was that the app would save all the files exactly like you would organize them in a folder on your computer - all categories would be folders, and all notes would be a .md file within the folder.
I imagined that every time you sync the data, it would be the equivalent of doing git add . && git commit && git push. However, it's not so simple to do that with the GitHub API. I spent a lot of time figuring out how to create and edit multiple files at once, and I realized there was really no way to do it without making an API call for every single file. I know it seems like there should be a multi-file create or update API call, but there's not.
What I finally ended up doing is what you see in the sync.ts file, which does the following:
- Gets a reference to the current head
- Creates a blob for each file (I decided on just creating one file for
notes.jsonand one file forcategories.json, instead of having a file for each note) - Creates a tree path
- Creates a tree
- Creates a commit
- Updates a reference
With these steps, on every sync a new commit is created for any updates that are made since the last sync.
Open Source
Nobody should start to undertake a large project. You start with a small trivial project, and you should never expect it to get large. If you do, you'll just over design and generally think it is more important than it likely is at that stage. Or worse, you might be scared away by the sheer size of the work you envision.
So start small, and think about the details. Don't think about some big picture and fancy design. If it doesn't solve some fairly immediate need, it's almost certainly over-designed. And don't expect people to jump in and help you. That's not how these things work. You need to get something half-way useful first, and then others will say "hey, that almost works for me", and they'll get involved in the project.
— Linus Torvalds
I just started building this for a few months and once it got to a state where it was recognizable a note-taking app, people started making issues and PRs. It even made it to the trending list on GitHub one day, and I also think there were a lot of Hacktoberfest contributions (most positive, a little spam I had to shut down immediately).
Ultimately, there were about 250 pull requests and 200 issues created. I divided the issues - everything got a label of a type and a status. Types were: features, bugs, refactors, testing, questions, documentation, and accessibility. Statuses were: blocked, duplicate, won't do, invalid, can't replicate, in progress, completed, and postponed. Labeling everything kept it all organized and easy to keep track of for me
I tagged and released the big changes, and I connected the app with SonarQube to track any bugs or problems (very few due to TypeScript) and Cypress/Jest with Travis to run tests before merging PRs. I learned a lot about managing and maintaining a coding project while working on this.
I took care of the main behavior notes, the authentication, the GitHub sync, and many contributors fixed bugs, designed the logo, set up a testing framework, and suggested and implemented new features.
Conclusion
Overall, working on TakeNote was a really positive experience for me. It's easy to get excited about a project and never finish it, or get stuck at a road block, but I saw it through to the end making it the fourth big project of mine: Primitive the CSS framework, Laconia the MVC PHP app, a Chip-8 emulator, and now TakeNote.
Ultimately, I decided not to release the app as a real service, but leave it out there as a fully functioning open-source project for people to see and learn from, and use and extend if they want. For myself, I decided to go with Bear for my own personal note-taking, as I realized I did not want to maintain something and spend more time on the maintenance of the app than the actual intended purpose of it.
I hope you check out the demo and play around with it! I encourage you to build a project if you have an idea, because it's fun and you'll learn a lot along the way.
This content originally appeared on Tania Rascia | RSS Feed and was authored by hello@taniarascia.com
hello@taniarascia.com | Sciencx (2020-11-30T00:00:00+00:00) Building TakeNote, a Notes App for Developers With GitHub Sync. Retrieved from https://www.scien.cx/2020/11/30/building-takenote-a-notes-app-for-developers-with-github-sync/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.