The
decision rule is a function
that tells us which action to take for every possible observation. When
we make a decision to take an action given an observation, we might not
always be making the best decision, and thus some costs may be incurred
due to our decision. For example, we might want to consider taking out
a 100000$ insurance for a business against fire. The insurance costs
1000$ a year. If we take the decision not to insure it and a fire does
occur, then the cost is 100000$. Of course if we don’t insure
and no fire occurs, then we gain 1000$. The key is to find a way to
calculate the risk for every possible decision. Naturally, we may want
to take the action that leads to the minimum risk. Each action
had an associated risk
, and
is the loss incurred for deciding
when the true state of nature is
. To minimize the overall risk, the conditional risk is computed. The
definition of conditional risk is the same for the continuous and
discrete case:

In plain English, the risk given our observations
for every action is the sum of all losses for that action given all the
states, weighted by the probability of occurrence of each state. The
action with the minimum risk is then selected.