This content originally appeared on Level Up Coding - Medium and was authored by Jake Moore
How are predictions valued in a technology-immersive economy?
Odds are, if you’re reading this post, you already have some exposure to statistics, machine learning, or artificial intelligence. This post isn’t about preparing you to enter the data science workforce. Rather, it’s about understanding how predictions are valued in the global economy. I’ll offer three modeling case studies and a decision tree for evaluating when each is appropriate then discuss the industry at large.
Case studies
Google Translate receives a sequence of English words and returns a sequence of Italian words. This is useful, for example, to English tourists on holiday in Italy. Imagine if every time such a tourist pinged Google with a translation request, human translators at Google typed up the translation and delivered to these tourists. The company would need an army of translators; it wouldn’t make sense, economically. Now, suppose that analysts at Google ran a model and inferred a distribution over candidate translations then used domain knowledge to choose the best, context-dependent translation and delivered it to these tourists. You wouldn’t need an army of translators, but you’d still need an impractical number of analysts on the clock at any given time. What this situation calls for is an automated best guess service at scale. No translators or analysts, no distributions or expert domain knowledge, just a request, a guess, and a delivery mechanism. This is what Machine Learning as a Service — or MLaaS — looks like.
Now, consider a pharma company bringing a new cancer therapy drug to market. Domain expertise shows that a given drug will have profound adverse effects to a user’s health but the drug ought to give the user a fighting chance at overcoming cancer. Here the system elements and their interactions are of chief importance. Statisticians aren’t interested in automating away studies and trials; they need to integrate knowledge from biologists, chemists, and medical doctors to assess if the drug is truly adding a positive benefit and if the prevalence of that benefit truly outweighs the unintended consequences of taking the drug.
Lastly, consider weather forecasts. Small differences in input can trigger massive differences in output. The situation doesn’t call for black-box pattern matching as did Google Translate; it calls for simulations with (at least) semi-interpretable parameters. Domain experts need to be able to dissect what the model thinks and why before selecting from a distribution of possibilities and delivering to viewers. It might have a stronger black-box flavor than pharma trials, but certainly less so than Google Translate.
Some problems require articulation and validation of domain knowledge (pharma). Others require integration of domain knowledge into simulations (weather). And some simply require black-box pattern matching at scale (translation aka MLaaS.) Not every problem is appropriate for an MLaaS solution, however, these solutions tend to be extremely profitable — if viable — and so the market economy heavily suggests pursuing the MLaaS route, whether or not it’s valid. Think of predictions as existing on a spectrum: On one extreme exists statistics where understanding is key; on the other end, MLaaS, where automation is key. If you want to enter the prediction economy, assessing where a hypothetical solution falls on the spectrum is paramount to bringing a solution to market.
Note, with Google Translate, the name of the game is producing high quality predictions at scale; however, the consequence of making a bad prediction is relatively low. At worst, you stand to lose users if predictions are predominantly low quality. In contrast, bad weather predictions regarding hurricane landfall or bringing a dangerous drug to market can end in loss of life. This illuminates the MLaaS viability decision tree:
Decision tree
- How widespread is the desire for a solution?
- How accessible and affordable is the current best solution?
- How easily can a solution be fully automated?
- What’s the consequence of getting a prediction wrong?
If the answers are high, low, easily, and low — this is a textbook MLaaS problem. But if the answers are low, high, hard, and high — a MLaaS solution could endanger users, not be profitable, or both.
So what about middle ground issues? For example, self-driving cars: High, low, hard, high. It’s at the halfway point between yes/no on MLaaS. In other words, there’s a market for such a solution but the costs (economic and societal) are so high that this application isn’t a great opportunity for low-capital market entrants.
Industry at large
So what about AWS, GCP, and other cloud computing platforms?
AWS’s SageMaker is an example of amorphous machine learning — or AML for short. It means the infrastructure to train, productionalize, and delivery predictions as a service is in place to users (engineers, business managers, data scientists, etc.) This infrastructure is heavily biased toward MLaaS applications. In fact, there are no guardrails in place to warn that ‘hey, you’ve actually got an statistical experimentation problem on your hands, not an MLaaS problem.’
With industry powerhouses like Google and Amazon in the AML market, there isn’t room for new entrants to offer amorphous ML. Rather, your variety of ML very much does need a form. This means you need to start with a problem, source the appropriate data, and follow the decision tree before ever bringing a solution to market. In other words, one can’t just get into the business of machine learning. You must enter the business of — well — business, then use machine learning to address a specific problem. If you start with, ‘I’ll offer an MLaaS solution’ before ever identifying a problem that requires an MLaaS solution, as directed by the decision tree, you’re destined for failure.
It’s my belief that we’re presently in an ML/AI bubble. When I’ve socialized this sentiment with peers, the knee-jerk reaction is always, ‘What!? How could you think that? ML isn’t going anywhere!’ Ask yourself this question: Did you start living in a hammock following the 2008 financial crisis?
If the answer is ‘no’, which is almost guaranteed to be the case, you understand: Housing, itself, is not valueless. The housing market was overvalued. Technology did not vanish following the ‘dot-com bubble.’ Amazon and Google were two of the companies to survive the burst. But the technology market was overvalued nonetheless. The companies that didn’t fundamentally understand human-technology interaction did not survive the burst. In its fallout we witnessed one of the largest consolidations of wealth ever witnessed in human history…
The decision tree that I’ve presented above is your compass for navigating the ML/AI bubble. When expectations and reality on the efficacy of ML/AI converge, the companies who don’t understand the value of a prediction will be eliminated. And the ones who survive? They’ll be rewarded handsomely for surviving the fallout of the impending bubble burst.
So what can you do to better position yourself to survive the bubble burst? Paradoxically, you don’t need to become an ML/AI expert. Knowledge for the sake of knowledge never hurts! But the industry is rapidly moving towards auto-ML and low-code ML. Understanding how a random forest works is less important than identifying a problem where a random forest is the perfect solution. This statement will likely incur the wrath, scorn, and ire of my peers. But I’m convinced it’s true. What you really ought to focus at least 50% of your time on is domain knowledge — whether that be finance, marketing, HR, etc. You need to understand the persistent problems, which haven’t yet been solved by an accessible, efficient, accurate, and desirable MLaaS solution. This will help you bring an MLaaS solution to market and survive the bubble burst.
Of course there will always be high compensation for experts who understand the inner-mechanics of ML/AI models. But these experts will find their attention directed by ML-savvy MBAs, whether in a software engineering capacity (training/deploying models) or in a research capacity (architecting truly novel models.) But whether companies survive or die in the impending bubble burst is a task squarely allocated to the ML-savvy MBAs. Don’t discount the market opportunity here. The visionaries who navigate the explosive burst will ride its shockwave into the stratosphere.
The Prediction Economy was originally published in Level Up Coding on Medium, where people are continuing the conversation by highlighting and responding to this story.
This content originally appeared on Level Up Coding - Medium and was authored by Jake Moore
Jake Moore | Sciencx (2022-01-12T22:25:44+00:00) The Prediction Economy. Retrieved from https://www.scien.cx/2022/01/12/the-prediction-economy/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.