Decision Theory: Daily Checkpoint
A short checkpoint on today's Decision Theory material. The questions are easy if you followed the day and hard otherwise, so they work as a quick self-check on your understanding (and as a light attendance check). You do not need to submit it, and it will not be graded. Answer from memory.
For a dualistic agent, optimization is just pick the action that maximizes expected utility. Why is this harder for an embedded agent?
Embedded agents have no utility function.
For an embedded agent the action is just another fact about the world, so what would happen if I acted differently is not given and counterfactuals must be constructed.
Embedded agents cannot compute expected values.
Embedded agents always have more actions to choose from.
EDT and CDT differ in how they treat the agent's action. Which description is correct?
EDT conditions on the action (treats it as evidence); CDT intervenes on it (the do-operator).
EDT intervenes on the action; CDT conditions on it.
Both condition on the action; they differ only in the prior.
Both intervene on the action; they differ only in the utility function.
In a Bayesian network, the intervention do(X=x) differs from observing X=x because it...
adds new arrows pointing out of X.
conditions on all of X's descendants.
severs the arrows into X (cutting it off from its causes) and then sets X equal to x.
is just another name for conditioning on X equals x.
On Newcomb's problem, how do the two rules come apart?
CDT one-boxes; EDT two-boxes.
CDT two-boxes; EDT one-boxes.
Both one-box.
Both two-box.
The smoking lesion is the case that tells against EDT. Why does EDT give the intuitively wrong verdict (abstain)?
Because smoking directly causes the lesion.
Because the lesion has no effect on anything in the problem.
Because conditioning on smoking raises the probability of the lesion, even though smoking is only correlated with (a symptom of) it rather than a cause of cancer; EDT mistakes that evidence for control.
Because EDT cannot represent the problem at all.
In the sequential-prediction framing, which identification does the deck make?
CDT is the joint predictor over action-and-percept streams; EDT is the chronological semimeasure.
CDT is the chronological semimeasure (the action sits to the right of the double-bar, as an intervention); EDT is the joint predictor that conditions on the action.
Both CDT and EDT correspond to the same predictor.
Neither can be written as a predictor.
Functional decision theory reframes what you are choosing. On FDT, you are choosing...
the action directly, severed from its causes.
a prior over which environment you are in.
the policy of a strictly weaker successor agent.
the output of the fixed decision function you run, scored by the world in which the function gives that output (so predictors, copies, and simulations of it move together).
In the twin prisoner's dilemma (two copies of one decision procedure), how do CDT and FDT differ, and why?
CDT cooperates and FDT defects, because of a causal link between the twins.
Both cooperate, for the same reason.
CDT has both twins defect; FDT has both cooperate, because the twins' actions share a subjunctive (logical) dependence on the same function that CDT ignores.
FDT defects because the twins are physically connected.
In counterfactual mugging, why does a UDT agent pay in the branch where it receives nothing?
Because it is unsure whether it actually lost the coin flip.
Because the paying policy has higher ex-ante expected value (before seeing the coin), and UDT optimizes the policy it would have committed to rather than updating on the observed branch.
Because paying causally increases its own reward.
Because Löb's theorem forces it to pay.
What does UDT 1.1 add over UDT 1.0?
It updates on its observations before acting.
It optimizes over the whole policy (a map from observations to actions) instead of one action at a time, so outputs in different branches can be coordinated.
It throws away the prior.
It replaces logical reasoning with causal intervention.
A central worry about UDT is that it can pay for worlds that don't exist. What is the concern?
Because it optimizes ex-ante expected utility under the prior, a mistaken prior can make it burn real utility in the actual branch for the sake of branches that never occur.
UDT is uncomputable.
UDT always defects in games.
UDT ignores the prior completely.
In the 5-and-10 problem, a proof-searching agent that should obviously take the 10 dollars can end up taking the 5 dollars. What goes wrong?
The agent miscalculates that 10 is greater than 5.
The world's source code is hidden from the agent.
The agent is not permitted to output 10.
A spurious self-referential proof: via Löb's theorem the agent proves something like if A returns 5 then utility is 5, and if A returns 10 then utility is 0, so its search certifies the 5 dollars; counterfactuals over one's own action break down.
Toward a formal optimality criterion, decision problems are written as programs (a world that calls the agent). A problem counts as fair when...
the world hands out rewards at random.
the agent and the world are guaranteed to share the same prior.
the world depends only on the agent's input/output behaviour, not on its internal source code beyond that behaviour.
the world is forbidden from ever simulating the agent.
Submit
Should be Empty: