This content originally appeared on Bits and Pieces - Medium and was authored by Fernando Doglio
It’s Not Going To Scale — Going From PoC to Production-Ready
The steps you need to take to ensure your product reaches production
So you have your PoC ready and working, you’ve successfully proved to the business that your idea works. “Fantastic!” they say, “let’s deploy it to production next week” they also say, and you suddenly realized they didn’t really understand a single thing you said.
Granted, this doesn’t happen in every situation, sometimes we’re able to correctly communicate the need to refactor and keep working before we can actually deploy an idea into production. But there are situations where the potential is so big that in the minds of a non-technical person the only question that resounds is “why aren’t we selling this now and fixing the code later?”.
As a responsible software developer who cares for the product and the company, it’s your job to explain what needs to happen for a piece of software to be “scalable” and — in some situations — what “scalable” means in this context too.
What do we mean by scalable?
For a piece of software to be usable by the world it needs to handle a certain workload, a certain amount of concurrent users and it should be able to go from 1 user to that upper bound easily. If we need to monitor it and re-launch the service when it crashes because it’s running out of memory, then we’re not doing it right.
And if we’re thinking about having an upper bound that is massive (I’m talking about millions of users per day), then we definitely need to start thinking about optimizing the way we utilize resources and how we can scale out of a single server — given we’d be limited by the physical resources of one computer, no matter how biffy it is.
This is where horizontal scaling comes into play. Instead of adding more memory or a better CPU to our server, we add more servers and share the load between them.
This way you’re not limited by the amount of memory you can add to a single server, instead you can add as many servers as you need, and as long as they’re able to work on the piece of the pie that they receive, then you’re OK.
Granted, this is the way you scale, but it also brings its own set of difficulties.
- Splitting the load between servers. This can be achieved with something like a load balancer, which will centralize the traffic to your service but it’ll “balance” it amongst your cluster of servers. This is great if the interaction with your code is already stateless, but if you’re keeping client/user information in memory, then subsequent requests will reach different servers. And different computers do not share memory with each other, thus each time a new request arrives, it’ll be like a new user started using your software. Of course, there are ways to lock users to a single instance of your servers, but that also diminishes the benefits of the load balancer.
- Servers might still be running out of resources. If your only solution is to duplicate your PoC as many times as possible, you might still be running into the same lack of resources scenario that got you here in the first place. This might be an indication that your monolithic service might be better suited as a set of microservices running independently from each other.
Scaling is not easy, but it is required if you want a piece of software to endure the test of time. If your product is successful, then the number of resources it’ll need will increase, because the number of people using it will increase. If you don’t plan for it, then you won’t be able to cope with the demand and your users will suffer the consequences (i.e bad UX, terrible performance, lack of stability, etc, etc).
Tip: Scale your apps and services with Bit
Compose everything out of independent JS components with Bit. Use it to manage dependencies, test and build in isolation — and most importantly, propagate changes between these decoupled component codebases.
Learn more here:
Welcome to Bit Harmony | Bit Docs
How do you scale a PoC?
This is the key question to ask yourself, because if you really showcased a PoC then you probably wrote that code very fast, with very little attention to quality and the future. Instead, you only focused on the fact that you wanted to prove your idea could be done.
And that is perfectly valid, for a PoC you usually end up cutting some corners and hard-coding some behavior. After all, you want the main idea behind it to stand out, the rest will be thrown away and re-coded.
Why? Look at the scenario we’ve been talking about so far: a single piece of software that now needs to scale. It’s not like you can just say: “let’s put it on an AWS lambda”, or “I’ll deploy it on Azure and it’ll scale”.
Those are all great ideas and fantastic ways to scale an application, but only if the code permits. Which on a single PoC chances are: it won’t.
Let’s take a look at what you’d have to do to start understanding how much work you have ahead.
Understand where the pain points of your system are
The first thing you’ll have to understand before you start splitting up your PoC into multiple services, is what sections of it are using the most resources. Be it memory, I/O or CPU, anything that is resource-intensive will probably require its own server so it can use all of its resources without sharing them with other sections.
You should measure this under stressful situations. In other words, find a load-tester and take your code to the limit. Understand what that limit looks like and why it’s getting there. Is it the number of requests? Or perhaps the number of concurrent requests received? Or maybe they’re just coming one after the other but too close to each other. Whatever the reason, this will give you a hint as to what parts of your application are suffering the most, thus will have to be extracted away and optimized.
Plan your services and their communication protocol
Alright, so now that you understand where the problems are, it’s time for some fun: split things apart.
That means turning each section you identified before into a service on its own. That’s not as easy as it sounds, especially because before, on your monolithic PoC, you probably had something like this:
And that code called all those functions that were part of your local context, but now, after your analysis, you’ve come to the conclusion that both grabALOTOfDataFromTheDataBase and processTheData need to be their own set of services.
And here is where the problem comes and why scaling is not as simple as “I’ll make it run on a lambda”. You were using local memory to juggle the data and parameters from one function to the other. You were pulling data from the database and storing it on the local memory (i.e the bigDataSet variable), and then you were passing that memory to the processTheData function as a parameter. That can’t happen like that anymore because once you turn those functions into external services you’ll have to deal with sending data across the network.
And here is where the communication protocol part comes into play. The way you have your services talk to each other will either make or break your architecture. There are many ways of communicating services, it’ll all depend on what you’re looking to get out of them. Do you need an open channel to be constantly active and sending data back and forth? Or is asynchronous communication the best approach?
Whatever it is, you’ll have to consider the following aspects to decide:
- Latency. How important is it for your service? Some communication protocols are heavier than others. It’s not the same to use REST over HTTP as to have 2 services speak over web sockets.
- Data serialization. Go back to the example I showed above, the bigDataSet variable could be anything, it could be an object of class DataSet that has multiple methods and other non-serializable properties. This data set however will have to travel through the wire all the way to the new service you created. How are you going to do that?
- Data de-serialization. Same as before but in the opposite direction. If you get the results from the new service for grabALOTOfDataFromTheDataBase that service will not be able to send you the data set object through the wire, but your code will need it to be an object. Thus you’ll have to know how to turn that data set into an object.
- Local files. Are you using local files for anything on your PoC? That’s a usual corner that we tend to cut because for the PoC we can deal with a local CSV instead of a full-blown database, but now we can’t. We need an efficient way of loading data and disk I/O is not one. So consider moving all those local heavy files into a proper storage medium.
You can probably think about a couple more problems if you start working on it on your own project. This can be a real headache, and while it’s probably one of the few ways you can reliably scale your PoC into Production-ready status, it’s also a lot of work.
Convince the business to give you more time and resources
The final step before implementation is to get the business’ approval. They wanted to go to production with your PoC, remember? And now, after carefully measuring performance and planning the new architecture and refactor efforts, you have a proposed timeline.
It’s time to sell it. And here is where the fun begins, because unless you’re versed in business talk, you’ll spend 10 minutes going over why your solution is so technically sound and perfect for what they want and nobody is going to listen to you. They might say things like “I don’t get it” or “why can’t we worry about the code looking good afterward?” and while that might make you feel like you’ve been talking to a brick wall this whole time; it’s not their fault, it’s yours.
They’re not dumb, no no, don’t make that mistake. They simply have different goals in mind. While your job is to ensure the solution is technically sound, theirs is to ensure the company makes money, and according to them you have a perfectly good product that you already showcased and now you don’t want to deploy it until you make it “perfect”.
I’ve been in these types of conversations and trust me, I get your frustration.
The first thing you have to understand is that you have to compromise. Yours can’t be a “let’s build this for 6 months and then worry about deploying” kinda plan. That means they don’t see any real benefits for a long time, instead try to plan for a more iterative approach, plan the most critical changes first and deploy your current code with the idea of replacing it with a slightly better version in 2–3 weeks.
Then have a second milestone planned for the next month with an even better code base and architecture. And keep on doing that until you’ve reached the same place you originally wanted to be in.
Granted, that will probably include other activities such as production support, deployment, etc that will delay a bit your timeline, but at least you’ll get a faster approval and the product will be able to evolve.
And you’re done! If you did the above, and you’ve successfully sold the implementation plan to the business, not it’s time to put your money where your mouth is and get coding.
Hopefully you also included things like testing time and a nice padding for emergencies on your timeline estimation. There are always things that can — and thanks to our friend Murphy, will — go wrong from time to time, so make sure you leave some room for them.
Have you ever had to go through this experience before? What did you do? Were you able to sell the idea and implementation plan? Leave your comments below and share your experience!
Learn More
- Independent Components: The Web’s New Building Blocks
- Will Bit Replace npm?
- Are You Going to be Replaced by Code-Generating AI?
It’s Not Going To Scale — Going From PoC to Production-Ready was originally published in Bits and Pieces on Medium, where people are continuing the conversation by highlighting and responding to this story.
This content originally appeared on Bits and Pieces - Medium and was authored by Fernando Doglio
Fernando Doglio | Sciencx (2021-07-21T17:43:27+00:00) It’s Not Going To Scale — Going From PoC to Production-Ready. Retrieved from https://www.scien.cx/2021/07/21/its-not-going-to-scale%e2%80%8a-%e2%80%8agoing-from-poc-to-production-ready/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.