"I contrast our own evidence for the hypothesis of organic fossil origins with that available in previous centuries, suggesting that the most powerful contemporary evidence consists in a form of projective support whose distinctive features are not well captured by familiar hypothetico-deductive, abductive, or even more recent and more technically sophisticated (e.g., Bayesian) accounts of scientific confirmation. I suggest that such accounts either misrepresent or ignore something important about the heterogeneous ways in which scientific hypotheses can be supported by evidence, and I go on to suggest that the search for any single such account may be misguided in any case."

"Despite its normative appeal and widespread use, Bayes’ rule has two well-known limitations: first, it does not predict how agents should react to an information to which they assigned probability zero; second, a sizable empirical evidence documents how agents systematically deviate from its prescriptions by overreacting to information to which they assigned a positive but small probability. By replacing Dynamic Consistency with a novel axiom, Dynamic Coherence, we characterize an alternative updating rule that is not subject to these limitations, but at the same time coincides with Bayes’ rule for “normal” events. In particular, we model an agent with a utility function over consequences, a prior over priors ρ, and a threshold. In the first period she chooses the prior that maximizes the prior over priors ρ - a’ la maximum likelihood. As new information is revealed: if the chosen prior assigns to this information a probability above the threshold, she follows Bayes’ rule and updates it. Otherwise, she goes back to her prior over priors ρ, updates it using Bayes’ rule, and then chooses the new prior that maximizes the updated ρ. We also extend our analysis to the case of ambiguity aversion."

DeLong is missing a trick. The rational-expectations dogmatist could simply insist that the true probability of an event like 2008 in 2008 _was_ 0.02%, and we were just unlucky.

"Bayesian rationality is the paradigm of rational behavior in neoclassical economics. A rational agent in an economic model is one who maximizes her subjective expected utility and consistently revises her beliefs according to Bayes’s rule. The paper raises the question of how, when and why this characterization of rationality came to be endorsed by mainstream economists. Though no definitive answer is provided, it is argued that the question is far from trivial and of great historiographic importance. The story begins with Abraham Wald’s behaviorist approach to statistics and culminates with Leonard J. Savage’s elaboration of subjective expected utility theory in his 1954 classic The Foundations of Statistics. It is the latter’s acknowledged fiasco to achieve its planned goal, the reinterpretation of traditional inferential techniques along subjectivist and behaviorist lines, which raises the puzzle of how a failed project in statistics could turn into such a tremendous hit in economics. A couple of tentative answers are also offered, involving the role of the consistency requirement in neoclassical analysis and the impact of the postwar transformation of US business schools." --- The guess about business schools at the end seems plausible.

"the Calibrated Bayesian (CB) approach to statistical inference capitalizes on the strength of Bayesian and frequentist approaches to statistical inference. In the CB approach, inferences under a particular model are Bayesian, but frequentist methods are useful for model development and model checking. In this article the CB approach is outlined. Bayesian methods for missing data are then reviewed from a CB perspective. The basic theory of the Bayesian approach, and the closely related technique of multiple imputation, is described. Then applications of the Bayesian approach to normal models are described, both for monotone and nonmonotone missing data patterns. Sequential Regression Multivariate Imputation and Penalized Spline of Propensity Models are presented as two useful approaches for relaxing distributional assumptions."  Also http://arxiv.org/abs/1108.1917

"Bayesian models of human learning are becoming increasingly popular in cognitive science. We argue that their purported confirmation largely relies on a methodology that depends on premises that are inconsistent with the claim that people are Bayesian about learning and inference. Bayesian models in cognitive science derive their appeal from their normative claim that the modeled inference is in some sense rational. Standard accounts of the rationality of Bayesian inference imply predictions that an agent selects the option that maximizes the posterior expected utility. Experimental confirmation of the models, however, has been claimed because of groups of agents that “probability match” the posterior. Probability matching only constitutes support for the Bayesian claim if additional unobvious and untested (but testable) assumptions are invoked. The alternative strategy of weakening the underlying notion of rationality no longer distinguishes the Bayesian model uniquely."

"It is widely believed that one should not become more confident that _all swans are white and all lions are brave_ simply by observing white swans. Irrelevant conjunction or "tacking" of a theory onto another is often thought problematic for Bayesianism, especially given the ratio measure of confirmation considered here... Using the ratio measure, the irrelevant conjunction is confirmed to the same degree as the relevant conjunct, which... is ideal: the irrelevant conjunct is irrelevant. Because the past's really having been as it now appears to have been is an irrelevant conjunct, present evidence confirms theories about past events only insofar as irrelevant conjunctions are confirmed. Hence the ideal of not confirming irrelevant conjunctions would imply that historical claims are not confirmed. ..."

I like the parts about how it's more important to have a sound epistemology about _revising_ our beliefs (i.e., changing our minds), than about warranting our beliefs at any one time.

"This article calls into question the charge that frequentist testing is susceptible to the base-rate fallacy. It is argued that the apparent similarity between examples like the Harvard Medical School test and frequentist testing is highly misleading. A closer scrutiny reveals that such examples have none of the basic features of a proper frequentist test, such as legitimate data, hypotheses, test statistics, and sampling distributions. Indeed, the relevant error probabilities are replaced with the false positive/negative rates that constitute deductive calculations based on known probabilities among events. As a result, the ampliative dimension of frequentist induction—learning from data about the underlying data-generating mechanism—is missing."

Bwahahahaha!

So-so.  Suspect that most of these results are actually in Claeskens and Hjort's book, but am insufficiently motivated to check.

"Contrary to formal theories of induction, I argue that there are no universal inductive inference schemas. The inductive inferences of science are grounded in matters of fact that hold only in particular domains, so that all inductive inference is local. Some are so localized as to defy familiar characterization. Since inductive inference schemas are underwritten by facts, we can assess and control the inductive risk taken in an induction by investigating the warrant for its underwriting facts. In learning more facts, we extend our inductive reach by supplying more localized inductive inference schemes. Since a material theory no longer separates the factual and schematic parts of an induction, it proves not to be vulnerable to Hume’s problem of the justification of induction."

I like this.

Everything useful in this paper is contained in their Figure 1 and its caption, and even then I think they're incomplete. (In the top left of Figure 1, the "strong support" quadrant, draw another narrow band along the opposite diagonal to the first theory, also going through the small cross of observations: this would be a distinct and incompatible theory which also makes a narrow range of predictions that also match the precisely-measured data.)

"a model credibility index, which is designed to serve as a one-number summary measure of model adequacy. We define the index to be the maximum sample size at which samples from the model and those from the true data generating mechanism are nearly indistinguishable. We use standard notions from hypothesis testing to make this definition precise. We use data subsampling to estimate the index" --- To be blogged, after the paper with Andy is done.

I will be fascinated to see what of this is "Bayesian".

"In likelihood inference we usually assume that the model is fixed and then base inference on the corresponding likelihood function. Often, however, the choice of model is rather arbitrary, and there may be other models which fit the data equally well. We study robustness of likelihood inference over such 'statistically equivalent' models and suggest a simple 'envelope likelihood' to capture this aspect of model uncertainty. Robustness depends critically on how we specify the parameter of interest. Some asymptotic theory is presented, illustrated by three examples."

"We give an overview of statistical models and likelihood, together with two of its variants: penalized and hierarchical likelihood. The Kullback-Leibler divergence is referred to repeatedly in the literature, for defining the misspecification risk of a model and for grounding the likelihood and the likelihood cross-validation, which can be used for choosing weights in penalized likelihood. Families of penalized likelihood and particular sieves estimators are shown to be equivalent. The similarity of these likelihoods with a posteriori distributions in a Bayesian approach is considered."

Interesting, but I suspect the bits about approximating an underlying discrete distribution could be lifted...

Weird. Wonder what Mayo would make of this.

A surprisingly weak paper, along the lines of "hey! did you realize that you can use the prior distribution to penalize things other than not fitting the data?", but I should re-read. Plus: this only makes sense if everyone always had both the old and the new paradigms in the support of their priors. ("Surprising", because Salmon was very good.)

In particular see the bit about "pure Bayesian reasoning" in the rejoinder.

"I hope [my philosophy of statistics is] sufficiently undogmatic not to imply that all those who may think rather differently from me are necessarily stupid. If at times I do seem dogmatic, it is because it is convenient to give my own views as unequivocally as possible."

"We study the problem of testing an expert whose theory has a learnable and predictive parametric representation, as do all standard processes used in Bayesian statistics. We design a test in which the expert is required to submit a date T by which he will have learned enough to deliver sharp predictions about future frequencies. His forecasts are then tested according to a simple hypothesis test. We show that this test passes an expert who knows the data-generating process and cannot be manipulated by an uninformed one. Such a test is not possible if the theory is unrestricted. "