"We address the online linear optimization problem with bandit feedback. Our contribution is twofold. First, we provide an algorithm (based on exponential weights) with a regret of order $sqrt{d n log N}$ for any finite action set with $N$ actions, under the assumption that the instantaneous loss is bounded by 1. This shaves off an extraneous $sqrt{d}$ factor compared to previous works, and gives a regret bound of order $d sqrt{n log n}$ for any compact set of actions. Without further assumptions on the action set, this last bound is minimax optimal up to a logarithmic factor. Interestingly, our result also shows that the minimax regret for bandit linear optimization with expert advice in $d$ dimension is the same as for the basic $d$-armed bandit with expert advice. Our second contribution is to show how to use the Mirror Descent algorithm to obtain computationally efficient strategies with minimax optimal regret bounds in specific examples. More precisely we study two canonical action sets: the hypercube and the Euclidean ball. In the former case, we obtain the first computationally efficient algorithm with a $d sqrt{n}$ regret, thus improving by a factor $sqrt{d log n}$ over the best known result for a computationally efficient algorithm. In the latter case, our approach gives the first algorithm with a $sqrt{d n log n}$ regret, again shaving off an extraneous $sqrt{d}$ compared to previous works."

"Despite its normative appeal and widespread use, Bayes’ rule has two well-known limitations: first, it does not predict how agents should react to an information to which they assigned probability zero; second, a sizable empirical evidence documents how agents systematically deviate from its prescriptions by overreacting to information to which they assigned a positive but small probability. By replacing Dynamic Consistency with a novel axiom, Dynamic Coherence, we characterize an alternative updating rule that is not subject to these limitations, but at the same time coincides with Bayes’ rule for “normal” events. In particular, we model an agent with a utility function over consequences, a prior over priors ρ, and a threshold. In the first period she chooses the prior that maximizes the prior over priors ρ - a’ la maximum likelihood. As new information is revealed: if the chosen prior assigns to this information a probability above the threshold, she follows Bayes’ rule and updates it. Otherwise, she goes back to her prior over priors ρ, updates it using Bayes’ rule, and then chooses the new prior that maximizes the updated ρ. We also extend our analysis to the case of ambiguity aversion."

"Economists typically assume that preferences are fixed - that people know what they like and how much they like it relative to all other things, and that this rank-ordering is stable over time. But this assumption has never been accepted by any other discipline. Economists are increasingly having difficulty arguing that the assumption is true enough to generate useful predictions and explanations. Indeed, law and economics scholars increasingly acknowledge that preferences are constructed, and that the law itself can help construct preferences. Still, fixed preferences are often treated as a normative ideal: Even if people don't have fixed preferences, they should. Behavioral law and economics scholars offer approaches to deal with this normative shortcoming.

My article argues that preference construction, properly understood, is not normatively undesirable. Having fixed preferences means having a complete and stable rank ordering of what we want that dictates our choices. But we often do not have such an ordering, and rationally so. My article argues instead for an alternative process-based, account of preference construction. Rather than having a complete rank ordering, we have ways of making choices. We construct narratives, using evaluative criteria against a backdrop of wants, desires and inclinations, some of which we rank order and some of which we do not. The evaluative criteria embed a consideration of transaction costs: critically, where a decision is not very consequential, a formulaic decision rule that permits a ready choice among roughly comparable alternatives may serve our purposes better than a more considered alternative-by-alternative assessment. Our wants, desires and inclinations are for both traditional objects of choice and higher order values and desires; they are both previously constructed and constructed and elicited in the choice-making process. My article makes the case for such an account's potential explanatory power, as well as its tractability."

The study of online convex optimization in the bandit setting was initiated by Kleinberg (2004) and Flaxman et al. (2005). Such a setting models a decision maker that has to make decisions in the face of adversarially chosen convex loss functions. Moreover, the only information the decision maker receives are the losses. The identity of the loss functions themselves is not revealed. In this setting, we reduce the gap between the best known lower and upper bounds for the class of smooth convex functions, i.e. convex functions with a Lipschitz continuous gradient. Building upon existing work on self-concordant regularizers and one-point gradient estimation, we give the first algorithm whose expected regret, ignoring constant and logarithmic factors, is O(T^{2/3}).

"We consider multi-armed bandit games with possibly adaptive opponents. We introduce models Theta of constraints based on equivalence classes on the common history (information shared by the player and the opponent) which define two learning scenarios: (1) The opponent is constrained, i.e.~he provides rewards that are stochastic functions of equivalence classes defined by some model theta* in Theta. The regret is measured with respect to (w.r.t.) the best history-dependent strategy. (2) The opponent is arbitrary and we measure the regret w.r.t.~the best strategy among all mappings from classes to actions (i.e.~the best history-class-based strategy) for the best model in Theta. This allows to model opponents (case 1) or strategies (case 2) which handles finite memory, periodicity, standard stochastic bandits and other situations. When Theta={theta}, i.e.~only one model is considered, we derive tractable algorithms achieving a tight regret (at time T) bounded by tilde O(sqrt{TAC}), where C is the number of classes of theta. Now, when many models are available, all known algorithms achieving a nice regret O(sqrt{T}) are unfortunately not tractable and scale poorly with the number of models $|Theta|$. Our contribution here is to provide tractable algorithms with regret bounded by T^{2/3}C^{1/3}log(|Theta|)^{1/2}. "

"Bayesian rationality is the paradigm of rational behavior in neoclassical economics. A rational agent in an economic model is one who maximizes her subjective expected utility and consistently revises her beliefs according to Bayes’s rule. The paper raises the question of how, when and why this characterization of rationality came to be endorsed by mainstream economists. Though no definitive answer is provided, it is argued that the question is far from trivial and of great historiographic importance. The story begins with Abraham Wald’s behaviorist approach to statistics and culminates with Leonard J. Savage’s elaboration of subjective expected utility theory in his 1954 classic The Foundations of Statistics. It is the latter’s acknowledged fiasco to achieve its planned goal, the reinterpretation of traditional inferential techniques along subjectivist and behaviorist lines, which raises the puzzle of how a failed project in statistics could turn into such a tremendous hit in economics. A couple of tentative answers are also offered, involving the role of the consistency requirement in neoclassical analysis and the impact of the postwar transformation of US business schools." --- The guess about business schools at the end seems plausible.

From 2000: "The paper contains some musings about the abstractions introduced by Lucien Le Cam into the asymptotic theory of statistical inference and decision theory. A short, self-contained proof of a key result (existence of randomizations via convergence in distribution of likelihood ratios), and an outline of a proof of a local asymptotic minimax theorem, are presented as an illustration of how Le Cam's approach leads to conceptual simplifications of asymptotic theory."

This is a genuinely brilliant and important paper.  But Becker shows no sign of realizing just how completely he has just undermined the whole normative side of traditional economics!

"Typically, point forecasting methods are compared and assessed by means of an error measure or scoring function, with the absolute error and the squared error being key examples. The individual scores are averaged over forecast cases, to result in a summary measure of the predictive performance, such as the mean absolute error or the mean squared error. I demonstrate that this common practice can lead to grossly misguided inferences, unless the scoring function and the forecasting task are carefully matched...."

Clever, but their key result is a special case of the variational representation of relative entropy. In fact, that might even amount to a new, decision-theoretic interpretation of relative entropy...

Preprint, subsequently published Games and economic Behavior 30 (2000): 44--63

The immune system from the viewpoint of statistical decision theory.  Would be very interesting if it actually integrates the two; haven't read it so I won't presume to guess.

"We propose a broad generalization of standard choice-theoretic welfare economics that encompasses a wide variety of nonstandard behavioral models. Our approach exploits the coherent aspects of choice that those positive models typically attempt to capture. It replaces the standard revealed preference relation with an unambiguous choice relation: roughly, x is (strictly) unambiguously chosen over y (written xP*y) iff y is never chosen when x is available. Under weak assumptions, P* is acyclic and therefore suitable for welfare analysis; it is also the most discerning welfare criterion that never overrules choice. The resulting framework generates natural counterparts for the standard tools of applied welfare economics and is easily applied in the context of specific behavioral theories, with novel implications. Though not universally discerning, it lends itself to principled refinements."

Right: this is the decreasing marginal disutility of bads. (Going from ten people screaming in your ear to nine is much less of an improvement than going from one screamer to zero.)

Obituary by J. Wolfowitz (the father of the idiot). I wonder if my grandparents took any of AW's classes?

"This paper concerns the prescriptive function of decision analysis. Consider an agent who must choose an action yielding welfare that varies with an unknown state of nature. It is often asserted that such an agent should adhere to consistency axioms which imply that behavior can be represented as maximization of expected utility. However, our agent is not concerned the consistency of his behavior across hypothetical choice sets. He only wants to make a reasonable choice from the choice set that he actually faces. Hence, I reason that prescriptions for decision making should respect actuality. That is, they should promote welfare maximization in the choice problem the agent actually faces. Any choice respecting weak and stochastic dominance is rational from the actualist perspective."

This looks like it has to be either really interesting, or numbingly awful.

"We study the problem of testing an expert whose theory has a learnable and predictive parametric representation, as do all standard processes used in Bayesian statistics. We design a test in which the expert is required to submit a date T by which he will have learned enough to deliver sharp predictions about future frequencies. His forecasts are then tested according to a simple hypothesis test. We show that this test passes an expert who knows the data-generating process and cannot be manipulated by an uninformed one. Such a test is not possible if the theory is unrestricted. "

"I study individuals who use statistical models to draw secure or robust inferences from iid data. The main contribution of the paper is a steady-state model in which distinct statistical models are consistent with empirical evidence, even as data increases without bound. Individuals may hold different beliefs and interpret their environment differently even though they know each other's statistical model and base their inferences on identical data. The behavior modeled here is that of rational individuals confronting an environment in which learning is hard, rather than ones beset by cognitive limitations or behavioral biases."

Comment: introducing finitely-additive measures to get ambiguity-in-the-limit seems like a hack. I think much the same effect could be achieved by examining the _rate_ at which ambiguity diminishes under countably additive measures.

Presumably related to his "making decisions in large worlds" paper.

Word.

Aaronsw on Elster's latest book, an unread copy of which glares at me from the shelf as I type.

I love the example for principle #2.

How do you make people look like Bayesian reasoners? By presenting them with explicitly frequentist probabilities, of course. Ought-to-be-classic paper. To be read in conjunction with Cosmides & Tooby.

Only papers, not books.

"we need to look beyond Bayesian decision theory for an answer to the general problem of making rational decisions under uncertainty....assuming that the decision-maker is not able to decide mathematically undecideable propositions."

Interesting thoughts on stochastic risk vs. subjective probability vs. true uncertainty, and its economic (esp. macroeconomic) implications. However makes serious errors about ergodic theory. To think through.

Please stop making revealed preferences arguments when peoples' choices interact. _Thank_ you.

Risk vs. uncertainty again; 1994;might have some interesting thoughts about modeling decision-making under actual uncertainty. Later: stupid mistakes about ergodicity (which is actually NOT relevant to the argument he wants to make). No actual models/theories Smart but vague insights about conventions and institutions as devices for managing uncertainty.

I'd say: you can say "utility" if you (1) use _all_ the von Neumann/Morgenstern axioms and (2) show they hold. Otherwise, it's spurious precision.