This content originally appeared on Bits and Pieces - Medium and was authored by Piyush Dubey
Indexing and caching
You have probably heard much about database indexing and caching where indexes will return the data. Applying indexing to any fetch/search operation will enhance your application performance.
So today we will be focusing on how you can create your indexing and caching mechanism if you have a lot number of data present in runtime to process
Use cases
You might want to know the use case of this approach. In a general way, you can add this use case anywhere in your app that has ample data in runtime and you will need to fetch a few entries based on some repetitive format
- Generating Dashboard filters
- Report analyzers
- Deep history filter
General Approach
First, let's assume you do not want to use the approach so what is the available option you have? Assuming you have a pretty large data source that has tons of records in it and after you have loaded all the data you need to filter most of them to get the desired output
We have a variable called dataSource assume this array will have lots of data and to filter data from the array we have a method called searchInDataSource that will use Array.filter to return data from name key.
The time complexity for this filtration is O(N) where N is the total number of record present. Time complexity is fine for any filtration method but let's say you want to increase your speed for the filter to O(1) this is where indexing comes into a picture
When should we use indexing?
There are a few blockers that come with the indexing. First and most important is the Space complexity.
In the general, approach the space complexity is O(N) where N is the filtered record but in the case of indexing we would need to maintain extra space to hold indexed data that will result in more space complexity but the catch here is that we only need indexing process one time so on load the space complexity is O(M * K) where M is the total number of unique entry per field and K is the total number of index required
Next is to manage indexes on each operation. We have to update the indexes on either of each operation from Create/Edit/Delete
Creating and Searching from indexes
Now, let's move to code and design how the index will look like. For the same example, we are creating a map to store indexes and a method called createIndexes that will generate the indexes for us, we will need to call this method as soon as we receive the data and in our case on load time
Here, we are storing data itself but in an ideal case, you can store the index of the data source and fetch the record in O(1) time complexity
For this example, we have to run createIndexes the method first to generate the method and if we do not have that index we can use the traditional filter
Managing Indexes
Adding new item — While doing any operation including Create/Update/Delete we will need to manage the indexes as well which will increase the time complexity for those operations. Below is an example of adding a new item and also managing indexes.
Best way will be to use Map instead of Array for storing items to avoid duplicate entries
Time complexities
Indexing will give you a good time complexity for reading operations but it will increase time for writing operations so you will need to be careful on which operations are used often
- Read — O(1)
- Create — O(1)
- Update — O(N*K) where N is the total items for a particular index key and K is the total number of indexes
- Delete — O(N*K) where N is the total items for a particular index key and K is the total number of indexes
Conclusion
Indexes and caching the objects are great ways to enhance your response time but it also comes with a cost. It is on the user to choose whether indexing will be good for the app or not. You can find the source code for application-level indexing and I do hope you learn something new today.
Happy Coding!
Build apps with reusable components like Lego
Bit’s open-source tool help 250,000+ devs to build apps with components.
Turn any UI, feature, or page into a reusable component — and share it across your applications. It’s easier to collaborate and build faster.
Split apps into components to make app development easier, and enjoy the best experience for the workflows you want:
→ Micro-Frontends
→ Design System
→ Code-Sharing and reuse
→ Monorepo
Learn more
- How We Build Micro Frontends
- How we Build a Component Design System
- The Bit Blog
- 5 Ways to Build a React Monorepo
- How to Create a Composable React App with Bit
Pros and Cons of Application Level Indexing and Caching was originally published in Bits and Pieces on Medium, where people are continuing the conversation by highlighting and responding to this story.
This content originally appeared on Bits and Pieces - Medium and was authored by Piyush Dubey
Piyush Dubey | Sciencx (2022-10-05T11:00:59+00:00) Pros and Cons of Application Level Indexing and Caching. Retrieved from https://www.scien.cx/2022/10/05/pros-and-cons-of-application-level-indexing-and-caching/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.