This content originally appeared on Level Up Coding - Medium and was authored by Prateek Chhikara
This article elaborates on the Expected Utility, Risk Aversion, and Utility Functions which helps Artificial Intelligence agents to decide action.
This article describes why there was a need to find the best utility function in decision-making. The article uses multiple examples to have a better understanding of the topics.
Decision Theory
Decision theory studies an agent’s rational choices that help advance Artificial Intelligence. It looks at how decisions are made, how multiple choices affect one another, and how decision-makers deal with uncertainties. When we are unsure about what action to take, decision theory selects various actions based on their immediate outcomes’ desirability.
Following are a few of the real-world scenarios of Decision Theory.
- I am looking to purchase an iPhone. Shall I buy the current model or wait for the latest model to arrive? The current iPhone model looks fine, but I will find a new model with extra features for the same price if I wait for a few months.
- Am I going to smoke the next cigarette? One single cigarette is no problem, but if I make the same decision sufficiently many times, it may kill me?
- Shall I bring the umbrella today? The decision depends on something I do not know, namely whether it will rain or not.
Utility Function
The agent’s preferences are captured by a utility function ‘U,’ which returns a numerical value to express the desirability of a state.
Expected Utility (EU)
The expected utility of an action is the average utility value of the outcomes, weighted by the probability that the outcome occurs.
Maximum Expected Utility (MEU)
The principle of maximum expected utility states that a rational agent should select the move that maximizes the agent’s expected utility. In other words, the MEU principle is a prescription for intelligent behavior.
To illustrate, consider the following example. I need to find out whether I need to carry an umbrella? Given that, If it rains and I have no umbrella, my utility is 0 units, while it is 15 units if I have an umbrella. If it does not rain and I have no umbrella, my utility is 18 units, while it is 15 units if I do have an umbrella. The chances of rain on any day is 50%.
In the above example, we calculated the EU for both cases, and according to MEU, the EU of the left part is higher; therefore, I should carry the umbrella.
Utility of Money
Utility theory has its roots in economics, and economics provides one obvious candidate for a utility measure: money (an agent’s total net assets). The almost universal exchangeability of money for all kinds of goods and services implies that money plays an essential role in human utility functions. If all other things are equal, an agent prefers more money to less in all the scenarios. We say that the agent exhibits a monotonic preference for more money.
Utility functions are essential to compare intricate scenarios involving uncertainty and risk. Consider Case 1 shown in Figure 2. Here, there are two lotteries, ‘LA’ and ‘LB,’ with the probability of receiving the reward. Would you prefer $7 million with 0.2 probability or $5 million with 0.25 probability? We need to formalize the decision-making process of an agent in this scenario by allocating a numerical utility to these different outcomes, as shown in Figure 2.
It might be natural to assume that utility should be linearly correlated to payoff. For example, we can pick a utility of 7c for a $7 million reward, 5c for a $5 million reward, and a utility value of 0 for a $0 reward, where c is a constant. By substituting the utility values in the equations for Case 1, we get EU = 1.4c for the left part and EU = 1.25c for the right part. Therefore, a person is more inclined to go for lottery ‘LA’ by following the principle of MEU.
Is this method applicable to all scenarios? To answer that, consider Case 2 as shown in Figure 3. If we follow the method above, we will get 3.2c on ‘LA’ and 2c on ‘LB.’ So, should we go for ‘LA’? Not really… An individual will always prefer to have a $2 million reward with 100% probability instead of $4 million with 80% probability.
To sum up, the utility is not directly proportional to monetary value because the utility for the first million is very high, whereas the utility for an extra million is smaller. In other words, if the utility for $5 million is 10c, then the utility for $10 million will not be 20c; it will be less than that. In a study of actual utility functions, Prof. Grayson found that the utility of money was almost proportional to the logarithm (concave function) of the reward, as shown in Figure 4.
Let’s consider a lottery containing risk. The probability of winning a reward of $1000 is 0.5 (p), and the probability of winning nothing is also 0.5 (1-p). For the point where the $1000 reward probability is 50%, the utility value is lower than the utility of $500. Here, $400 is the certainty equivalent to the lottery. We will be willing to trade this amount to get the money for certain from the lottery. The difference between these two numbers (expected reward and utility of the lottery) is called Insurance or Risk Premium. Therefore, a person is willing to take less money with certainty instead of risking a higher proportion. Studies have shown that most people will accept $400 instead of a gamble that gives $1000 half the time; the certainty equivalent of the lottery is $400, while the Expected Monetary Value is $500. We can say that the utility of being faced with that lottery is less than the utility of being handed the expected monetary value of the lottery as a sure thing. Therefore, U_500 is always greater than EU_500 (0.5 x U_0 + 0.5 x U_1000).
Risk Aversion
For risk-averse agents, the utility of getting the expected value of a gamble for certain is greater than the expected utility of the gamble with that expected value. Risk aversion is the basis for the insurance industry because it means that insurance premiums are positive. People would rather pay a small insurance premium than gamble the price of their house against the chance of a fire. From the insurance company’s point of view, the price of the house is very small compared with the firm’s total reserves. This means that the insurer’s utility curve is approximately linear over such a small region, and the gamble costs the company almost nothing.
Conclusion
This article discussed the decision theory and the utility functions that are used. In addition, the reason behind choosing a non-linear utility function was elaborated with few examples. Finally, the article introduced the utility curve and the risk aversion mechanism in decision theory.
References
- Peter Norvig and Stuart J. Russell. Textbook — Artificial Intelligence: A Modern Approach.
- https://www.youtube.com/watch?v=dDv5PmzpSM0&t=591s
Decision Theory: Expected Utility and Risk Aversion was originally published in Level Up Coding on Medium, where people are continuing the conversation by highlighting and responding to this story.
This content originally appeared on Level Up Coding - Medium and was authored by Prateek Chhikara
Prateek Chhikara | Sciencx (2022-01-24T03:00:06+00:00) Decision Theory: Expected Utility and Risk Aversion. Retrieved from https://www.scien.cx/2022/01/24/decision-theory-expected-utility-and-risk-aversion/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.