The decision rule is a function
that tells us which action to take for every possible
observation. When we make a decision to take an action given an observation, we might not
always be making the best decision, and thus some costs may be incurred due to our decision.
For example, we might want to consider taking out a 100000$ insurance for a business against
fire. The insurance costs 1000$ a year. If we take the decision not to insure it and a fire does
occur, then the cost is 100000$. Of course if we don’t insure and no fire occurs, then we gain
1000$. The key is to find a way to calculate the risk for every possible decision. Naturally, we
may want to take the action that leads to the minimum risk. Each action
had an associated risk
, and
is the loss incurred for deciding
when the true state of nature is
.
To minimize the overall risk, the conditional risk is computed. The definition of conditional risk
is the same for the continuous and discrete case:
In plain English, the risk given our observations for every action is the sum of all losses for that action given all the states, weighted by the probability of occurrence of each state. The action with the minimum risk is then selected.