"Exponential families are often used to model data sets with complex dependence. Maximum likelihood estimators (MLE) can be difficult to estimate when the likelihood is expensive to compute. Markov chain Monte Carlo (MCMC) methods based on the MCMC-MLE algorithm in [17] are guaranteed to converge in theory under certain conditions when starting from any value, but in practice such an algorithm may labor to converge when given a poor starting value. We present a simple line search algorithm to find the MLE of a regular exponential family when the MLE exists and is unique. The algorithm can be started from any initial value and avoids the trial and error experimentation associated with calibrating algorithms like stochastic approximation. Unlike many optimization algorithms, this approach utilizes first derivative information only, evaluating neither the likelihood function itself nor derivatives of higher order than first. We show convergence of the algorithm for the case where the gradient can be calculated exactly. When it cannot, it has a particularly convenient form that is easily estimable with MCMC, making the algorithm still useful to a practitioner."

"The hard problem of induction is to argue without begging the question that inductive inference, applied properly in the proper circumstances, is con- ducive to truth. A recent theorem seems to show that the hard problem has a deductive solution. The theorem, provable in , states that a predictive func- tion M exists with the following property: whatever world we live in, M correctly predicts the world’s present state given its previous states at all times apart from a well-ordered subset. On the usual model of time a well-ordered subset is small relative to the set of all times. M’s existence therefore seems to provide a solution to the hard problem.
My paper argues for two conclusions. First, the theorem does not solve the hard problem of induction. More positively though, it solves a version of the problem in which the structure of time is given modulo our choice of set theory."

--- Seems prodigiously strange, on first glance. Ask the people downstairs and up the hall what they think of it?

"In recent years, a wealth of dimension-reduction techniques for data visualization and preprocessing has been established. Nonparametric methods require additional effort for out-of-sample extensions, because they provide only a mapping of a given finite set of points. In this letter, we propose a general view on nonparametric dimension reduction based on the concept of cost functions and properties of the data. Based on this general principle, we transfer nonparametric dimension reduction to explicit mappings of the data manifold such that direct out-of-sample extensions become possible. Furthermore, this concept offers the possibility of investigating the generalization ability of data visualization to new data points. We demonstrate the approach based on a simple global linear mapping, as well as prototype-based local linear mappings. In addition, we can bias the functional form according to given auxiliary information. This leads to explicit supervised visualization mappings with discriminative properties comparable to state-of-the-art approaches."

Tempting to get these and leave them in random places.

"Generalized additive models for location, scale and shape (GAMLSSs) are a popular semiparametric modelling approach that, in contrast with conventional generalized additive models, regress not only the expected mean but also every distribution parameter (e.g. location, scale and shape) to a set of covariates. Current fitting procedures for GAMLSSs are infeasible for high dimensional data set-ups and require variable selection based on (potentially problematic) information criteria. The present work describes a boosting algorithm for high dimensional GAMLSSs that was developed to overcome these limitations. Specifically, the new algorithm was designed to allow the simultaneous estimation of predictor effects and variable selection. The algorithm proposed was applied to Munich rental guide data, which are used by landlords and tenants as a reference for the average rent of a flat depending on its characteristics and spatial features. The net rent predictions that resulted from the high dimensional GAMLSSs were found to be highly competitive and covariate-specific prediction intervals showed a major improvement over classical generalized additive models."

"Assume that one observes the $k$-th, $2k$-th, ...., $nk$-th value of a Markov chain $X_{1,h},...,X_{nk,h}$. That means we assume that a high frequency Markov chain runs in the background on a very fine time grid but that it is only observed on a coarser grid. This asymptotics reflects a set up occurring in the high frequency statistical analysis for financial data where diffusion approximations are used only for coarser time scales. In this paper we show that under appropriate conditions the L$_1$-distance between the joint distribution of the Markov chain and the distribution of the discretized diffusion limit converges to zero. The result implies that the LeCam deficiency distance between the statistical Markov experiment and its diffusion limit converges to zero. This result can be applied to Euler approximations for the joint distribution of diffusions observed at points $Delta, 2 Delta, ,,,, nDelta$. The joint distribution can be approximated by generating Euler approximations at the points $Delta k^{-1}, 2 Delta k^{-1}, ,,,, nDelta$. Our result implies that under our regularity conditions the Euler approximation is consistent for $n to infty$ if $nk^{-2}to 0$."

"We explore the limits of the autoregressive (AR) sieve bootstrap, and show that its applicability extends well beyond the realm of linear time series as has been previously thought. In particular, for appropriate statistics, the AR-sieve bootstrap is valid for stationary processes possessing a general Wold-type autoregressive representation with respect to a white noise; in essence, this includes all stationary, purely nondeterministic processes, whose spectral density is everywhere positive. Our main theorem provides a simple and effective tool in assessing whether the AR-sieve bootstrap is asymptotically valid in any given situation. In effect, the large-sample distribution of the statistic in question must only depend on the first and second order moments of the process; prominent examples include the sample mean and the spectral density. As a counterexample, we show how the AR-sieve bootstrap is not always valid for the sample autocovariance even when the underlying process is linear."

"In some situations, EM algorithm shows slow convergence problems. One possible reason is that standard procedures update the parameters simultaneously. In this paper we focus on finite mixture estimation. In this framework, we propose a component-wise EM, which updates the parameters sequentially. We give an interpretation of this procedure as a proximal point algorithm and use it to prove the convergence. Illustrative numerical experiments show how our algorithm compares to EM and a version of the SAGE algorithm."

Huh, is this related to the way that updating partial response functions simultaneously in a GAM can be trouble, and it's better, as in standard backfitting, to update sequentially?

"The fluctuation relations have received considerable attention since their emergence and development in the 1990s. We present a summary of the main results and suggest ways to interpret this material. Starting with a consideration of the under-determined time evolution of a simple open system, formulated using continuous Markovian stochastic dy- namics, an expression for the entropy generated over a time interval is developed in terms of the probability of observing a trajectory associated with a prescribed driving protocol, and the probability of its time-reverse. This forms the basis for a general theoretical description of non-equilibrium thermodynamic pro- cesses. Having established a connection between entropy production and an inequivalence in probability for forward and time-reversed events, we proceed in the manner of Sekimoto and Seifert, in particular, to derive results in stochastic thermodynamics: a description of the evolution of a system between equilibrium states that ties in with well-established thermodynamic expectations. We derive fluctuation relations, state conditions for their validity, and illustrate their op- eration in some simple cases, thereby providing some introductory insight into the various celebrated symmetry relations that have emerged in this field."

"Multicellularity was one of the most significant innovations in the history of life, but its initial evolution remains poorly understood. Using experimental evolution, we show that key steps in this transition could have occurred quickly. We subjected the unicellular yeast Saccharomyces cerevisiae to an environment in which we expected multicellularity to be adaptive. We observed the rapid evolution of clustering genotypes that display a novel multicellular life history characterized by reproduction via multicellular propagules, a juvenile phase, and determinate growth. The multicellular clusters are uniclonal, minimizing within-cluster genetic conflicts of interest. Simple among-cell division of labor rapidly evolved. Early multicellular strains were composed of physiologically similar cells, but these subsequently evolved higher rates of programmed cell death (apoptosis), an adaptation that increases propagule production. These results show that key aspects of multicellular complexity, a subject of central importance to biology, can readily evolve from unicellular eukaryotes."

Wow, if this holds up.

"This paper presents an alternative method to derive the limiting distribution of residual-based statistics. Our method does not impose an explicit assumption of (asymptotic) smoothness of the statistic of interest with respect to the model's parameters and thus is especially useful in cases where such smoothness is difficult to establish. Instead, we use a locally uniform convergence in distribution condition, which is automatically satisfied by residual-based specification test statistics. To illustrate, we derive the limiting distribution of a new functional form specification test for discrete choice models, as well as a runs-based tests for conditional symmetry in dynamic volatility models." (To-teach tag is tentative.)

"The business cycle is a fundamental yet elusive concept in macroeconomics. In this paper, we consider the problem of measuring the business cycle. First, we argue for the output-gap view that the business cycle corresponds to transitory deviations in economic activity away from a permanent, or trend, level. Then we investigate the extent to which a general model-based approach to estimating trend and cycle for the U.S. economy leads to measures of the business cycle that reflect models versus the data. We find empirical support for a nonlinear time series model that produces a business cycle measure with an asymmetric shape across NBER expansion and recession phases. Specifically, this business cycle measure suggests that recessions are periods of relatively large and negative transitory fluctuations in output. However, several close competitors to the nonlinear model produce business cycle measures of widely differing shapes and magnitudes. Given this model-based uncertainty, we construct a model-averaged measure of the business cycle. This measure also displays an asymmetric shape and is closely related to other measures of economic slack such as the unemployment rate and capacity utilization."
--- Worthy, but at the same time makes me want to lock them in a room with a copy of Li and Racine's _Nonparametric Econometrics_, or even _The Elements of Statistical Learning_, and not let them out until they understand it.

"Instrumental variable (IV) methods are widely used to identify causal effects in models with endogenous explanatory variables. Often the instrument exclusion restriction that underlies the validity of the usual IV inference is suspect; that is, instruments are only plausibly exogenous. We present practical methods for performing inference while relaxing the exclusion restriction. We illustrate the approaches with empirical examples that examine the effect of 401(k) participation on asset accumulation, price elasticity of demand for margarine, and returns to schooling. We find that inference is informative even with a substantial relaxation of the exclusion restriction in two of the three cases."

"We present a new computing environment for authoring mixed natural and computer language documents. In this environment a single hierarchically-organized plain text source file may contain a variety of elements such as code in arbitrary programming languages, raw data, links to external resources, project management data, working notes, and text for publication. Code fragments may be executed in situ with graphical, numerical and textual output captured or linked in the file. Export to LATEX, HTML, LATEX beamer, DocBook and other formats permits working reports, presentations and manuscripts for publication to be generated from the file. In addition, functioning pure code files can be automatically extracted from the file. This environment is implemented as an extension to the Emacs text editor and provides a rich set of features for authoring both prose and code, as well as sophisticated project management capabilities."

"Fukuyama's would-be allies in the business world haven't developed what you could call a coherent plan of attack. They have relatively little to say about the political side of the changes they seek. But they do have energy, optimism, and something of the utopian spirit that's at the heart of most successful social movements.... In the more obviously troubled circumstances of 2012, the business gurus have toned down the technological determinism a little bit. But they're still trying to point the way to a new, better world. You can call this a hopeful sign or a scary one (that Marx stuff didn't work out so well, remember). In any case, it's worth paying attention to."

Occupy Management Consulting is not going to fly, I think.

See if the analysis holds up after tracking down paper and if data is available; if so may make it an assignment (or even an exam?) for uADA.

"The conclusion is straightforward. Although inferring different processing stages (or 'modules') from additive factors in data is a venerable tradition in psychology, and one that remains popular (Sternberg, 2011), it is a mistake. As Henson (2011) points out, there's too much non-linearity in cognitive processing, so that you need additional constraints if you want to make inferences about cognitive modules."

--- I find it astonishing that anyone would ever have been tempted to make this inference at all.

Unpublished (?) draft

"In this paper we address the general question of how social influence determines collective outcomes for large populations of individuals faced with binary decisions. First, we define conditions under which the behavior of individuals making binary decisions can be described in terms of what we call an influence-response function: a one-dimensional function of the (weighted) number of individuals choosing each of the alternatives. And second, we demonstrate that, under the assumptions of global and anonymous interactions, general knowledge of the influence-response functions is sufficient to compute equilibrium, and even non-equilibrium, properties of the collective dynamics. By enabling us to treat in a consistent manner classes of decisions that have previously been analyzed separately, our framework allows us to find similarities between apparently quite different kinds of decision situations, and conversely to identify important differences between decisions that would otherwise appear very similar."

"giant hairless killer-kitten"

How we seem to our friends from afar: "If we’re talking outside the space launch field, then, hell, America innovates every day, in a myriad of fields. The field of metamaterials, for instance — invisibility cloaks, and hiding events from time itself? That’s just today, as I talk to you, and that’s all American. I realize there’s a narrative that America is all done, and doesn’t make stuff any more, and it’s midnight for the American experiment and all that, but that ignores the basic mathematics of a country with three hundred million people in it. For every bunch of dubiously photogenic fetal-alcohol-syndrome cases from New Jersey who get on the TV for ten minutes, there are ten times as many people at MIT inventing the future."

Whittaker introduces spline smoothing in 1922, complete with the Bayesian derivation. Does not use the word "spline", however --- when did that come in?

"Stochastic blockmodels are generative network models where the vertices are separated into discrete groups, and the probability of an edge existing between two vertices is determined solely by their group membership. In this paper, we derive expressions for the entropy of stochastic blockmodel ensembles. We consider several ensemble variants, including the traditional model as well as the newly introduced degree-corrected version [Karrer et al. Phys. Rev. E {bf 83}, 016107 (2011)], which imposes a degree sequence on the vertices, in addition to the block structure. The imposed degree sequence is implemented both as "soft" constraints, where only the expected degrees are imposed, and as "hard" constraints, where they are required to be the same on all samples of the ensemble. We also consider generalizations to multigraphs and directed graphs. We illustrate one of many applications of this measure by directly deriving a log-likelihood function from the entropy expression, and using it to infer latent block structure in observed data. Due to the general nature of the ensembles considered, the method works well for ensembles with intrinsic degree correlations (i.e. with entropic origin) as well as extrinsic degree correlations, which go beyond the block structure."

"We propose methods for inference on the average effect of a treatment on a scalar outcome in the presence of very many controls. Our setting is a partially linear regression model containing the treatment/policy variable and a large number $p$ of controls or series terms, with $p$ that is possibly much larger than the sample size $n$, but where only $s < n$ unknown controls or series terms are needed to approximate the regression function accurately. The latter sparsity condition makes it possible to estimate the entire regression function as well as the average treatment effect by selecting an approximately the right set of controls using Lasso and related methods. We develop estimation and inference methods for the average treatment effect in this setting, proposing a novel "post double selection" method that provides attractive inferential and estimation properties. In our analysis, in order to cover realistic applications, we expressly allow for imperfect selection of the controls and account for the impact of selection errors on estimation and inference. In order to cover typical applications in economics, we employ the selection methods designed to deal with non-Gaussian and heteroscedastic disturbances. We illustrate the use of new methods with numerical simulations and an application to the effect of abortion on crime rates."

"This article is about estimation and inference methods for high dimensional sparse (HDS) regression models in econometrics. High dimensional sparse models arise in situations where many regressors (or series terms) are available and the regression function is well-approximated by a parsimonious, yet unknown set of regressors. The latter condition makes it possible to estimate the entire regression function effectively by searching for approximately the right set of regressors. We discuss methods for identifying this set of regressors and estimating their coefficients based on $ell_1$-penalization and describe key theoretical results. In order to capture realistic practical situations, we expressly allow for imperfect selection of regressors and study the impact of this imperfect selection on estimation and inference results. We focus the main part of the article on the use of HDS models and methods in the instrumental variables model and the partially linear model. We present a set of novel inference results for these models and illustrate their use with applications to returns to schooling and growth regression."

Sweet. (Bayesian estimation seems like overkill here however, especially since predictions are just made from point estimates.)

"Data collection at a massive scale is becoming ubiquitous in a wide variety of settings, from vast offline databases to streaming real-time information. Learning algorithms deployed in such contexts must rely on single-pass inference, where the data history is never revisited. In streaming contexts, learning must also be temporally adaptive to remain up-to-date against unforeseen changes in the data generating mechanism. Although rapidly growing, the online Bayesian inference literature remains challenged by massive data and transient, evolving data streams. Non-parametric modelling techniques can prove particularly ill-suited, as the complexity of the model is allowed to increase with the sample size. In this work, we take steps to overcome these challenges by porting standard streaming techniques, like data discarding and downweighting, into a fully Bayesian framework via the use of informative priors and active learning heuristics. We showcase our methods by augmenting a modern non-parametric modelling framework, dynamic trees, and illustrate its performance on a number of practical examples. The end product is a powerful streaming regression and classification tool, whose performance compares favourably to the state-of-the-art."

"Let $(X_t, Y_t)_{tin T}$ be a discrete or continuous-time Markov process with state space $X times R^d$ where $X$ is an arbitrary measurable set. Its transition semigroup is assumed to be additive with respect to the second component, i.e. $(X_t, Y_t)_{tin T}$ is assumed to be a Markov additive process. In particular, this implies that the first component $(X_t)_{tin T}$ is also a Markov process. Markov random walks or additive functionals of a Markov process are special instances of Markov additive processes. In this paper, the process $(Y_t)_{tin T}$ is shown to satisfy the following classical limit theorems: (a) the central limit theorem, (b) the local limit theorem, (c) the one-dimensional Berry-Esseen theorem, (d) the one-dimensional first-order Edgeworth expansion, provided that we have sup{tin(0,1]cap T : E{pi,0}[|Y_t| ^{alpha}] < 1 with the expected order with respect to the independent case (up to some $varepsilon > 0$ for (c) and (d)). For the statements (b) and (d), a Markov nonlattice condition is also assumed as in the independent case. All the results are derived under the assumption that the Markov process $(X_t)_{tin T}$ has an invariant probability distribution $pi$, is stationary and has the $L^2(pi)$-spectral gap property (that is, $(X_t)tin N}$ is $rho$-mixing in the discrete-time case). The case where $(X_t)_{tin T}$ is non-stationary is briefly discussed. As an application, we derive a Berry-Esseen bound for the M-estimators associated with $rho$-mixing Markov chains."

"Using the renewal approach we prove exponential inequalities for additive functionals and empirical processes of ergodic Markov chains, thus obtaining counterparts of inequalities for sums of independent random variables. The inequalities do not require functions of the chain to be bounded and moreover have all the constants accessible whenever the usual drift condition holds, which is crucial for practical applications e.g. in MCMC algorithms."

"My experiment connects to these ideas because it asked people to make a simple judgement (the colour of the ink), like the experiments supporting an optimal information integration perspective on decision making, but the judgement requested was just marginally more complex because we manipulate both Stroop condition (whether the word and ink matched) and colour strength. If you are a straight-down-the-line optimal information decision theorists then you must believe that evidence about the decision based on the word is combined with evidence about the decision based on the colour to make a single 'amount of evidence' variable which drives the decision. In the paper I call this the 'common metric' hypothesis. The logic is a bit involved (see the paper), but a consequence of this hypothesis is that the size of the effect of the word condition should vary across the colour strength condition, and vice versa. In other words, you should see an interaction. Visually, the lines on the graph of results would be non-parallel." --- You can see where this is heading.

Potential problem of specificity: is this any worse in SF than in any other branch of fiction?

Additive regression models as a general statistical method, complete with a successive-approximation algorithm that's really damn close to modern back-fitting, and a plea for economists to use it. In 1924!

"When using optimal linear prediction to interpolate point observations of a mean square continuous stationary spatial process, one often finds that the interpolant mostly depends on those observations located nearest to the predictand. This phenomenon is called the screening effect. However, there are situations in which a screening effect does not hold in a reasonable asymptotic sense, and theoretical support for the screening effect is limited to some rather specialized settings for the observation locations. This paper explores conditions on the observation locations and the process model under which an asymptotic screening effect holds. A series of examples shows the difficulty in formulating a general result, especially for processes with different degrees of smoothness in different directions, which can naturally occur for spatial-temporal processes. These examples lead to a general conjecture and two special cases of this conjecture are proven. The key condition on the process is that its spectral density should change slowly at high frequencies. Models not satisfying this condition of slow high-frequency change should be used with caution."

"Incidentally, the competition is fierce, among panels that really are being held this year, for the one with the most risibly Davos-like name. "Global Risks 2012: The Safety of Our Safeguards" is a strong contender, as is "The New Context For Leadership." But I have to go with "Managing Complexity With The Santa Fe Institute." It combines a deeply serious tone with a total absence of discernible content; unsubtle institutional branding; and the gravest possible risk that attendees will perish from boredom. This is what you get when the best minds and/or fattest wallets assemble to congratulate one another for being on top."

"Human genetics has been haunted by the mystery of “missing heritability” of common traits. Although studies have discovered >1,200 variants associated with common diseases and traits, these variants typically appear to explain only a minority of the heritability. The proportion of heritability explained by a set of variants is the ratio of (i) the heritability due to these variants (numerator), estimated directly from their observed effects, to (ii) the total heritability (denominator), inferred indirectly from population data. The prevailing view has been that the explanation for missing heritability lies in the numerator—that is, in as-yet undiscovered variants. While many variants surely remain to be found, we show here that a substantial portion of missing heritability could arise from overestimation of the denominator, creating “phantom heritability.” Specifically, (i) estimates of total heritability implicitly assume the trait involves no genetic interactions (epistasis) among loci; (ii) this assumption is not justified, because models with interactions are also consistent with observable data; and (iii) under such models, the total heritability may be much smaller and thus the proportion of heritability explained much larger. For example, 80% of the currently missing heritability for Crohn's disease could be due to genetic interactions, if the disease involves interaction among three pathways. In short, missing heritability need not directly correspond to missing variants, because current estimates of total heritability may be significantly inflated by genetic interactions. Finally, we describe a method for estimating heritability from isolated populations that is not inflated by genetic interactions."
--- I'm not sure about the validity of their slope-based estimator of narrow heritability, I should ask K.R. about that.

Coates contra Paul, III. (The echoes of Lincoln's 2nd inaugural are a nice touch.)

Coates contra Paul, II.

Coates contra Paul, I:

"The fervency for Ron Paul is rooted in the longing for a reedemer, for one who will rise up and cut through the dishonest pablum of horse-races and sloganeering and speak directly to Americans. It is a species of saviorism which hopes to deliver a prophet onto the people, who will be better than the people themselves. 

But every man is a prophet, until he faces a Congress."

"The God that holds you over the pit of hell, much as one holds a spider, or some loathsome insect over the fire, abhors you, and is dreadfully provoked: his wrath towards you burns like fire; he looks upon you as worthy of nothing else, but to be cast into the fire; he is of purer eyes than to bear to have you in his sight; you are ten thousand times more abominable in his eyes, than the most hateful venomous serpent is in ours. You have offended him infinitely more than ever a stubborn rebel did his prince; and yet it is nothing but his hand that holds you from falling into the fire every moment. It is to be ascribed to nothing else, that you did not go to hell the last night; that you was suffered to awake again in this world, after you closed your eyes to sleep. And there is no other reason to be given, why you have not dropped into hell since you arose in the morning, but that God's hand has held you up. There is no other reason to be given why you have not gone to hell, since you have sat here in the house of God, provoking his pure eyes by your sinful wicked manner of attending his solemn worship. Yea, there is nothing else that is to be given as a reason why you do not this very moment drop down into hell."

"Large-scale computer experiments are becoming increasingly important in science. A multi-step procedure is introduced to statisticians for modeling such experiments, which builds an accurate interpolator in multiple steps. In practice, the procedure shows substantial improvements in overall accuracy, but its theoretical properties are not well established. We introduce the terms nominal and numeric error and decompose the overall error of an interpolator into nominal and numeric portions. Bounds on the numeric and nominal error are developed to show theoretically that substantial gains in overall accuracy can be attained with the multi-step approach."

"The frequentist behavior of nonparametric Bayes estimates, more specifically, rates of contraction of the posterior distributions to shrinking Lr-norm neighborhoods, 1 ≤ r ≤ ∞, of the unknown parameter, are studied. A theorem for nonparametric density estimation is proved under general approximation-theoretic assumptions on the prior. The result is applied to a variety of common examples, including Gaussian process, wavelet series, normal mixture and histogram priors. The rates of contraction are minimax-optimal for 1 ≤ r ≤ 2, but deteriorate as r increases beyond 2. In the case of Gaussian nonparametric regression a Gaussian prior is devised for which the posterior contracts at the optimal rate in all Lr-norms, 1 ≤ r ≤ ∞."

"What would an objective article on waterboarding have looked like during the Bush administration? A history of the practice would have to state that the US government considered it torture for several decades and that this status was essentially unquestioned until after September 11th. It would also record that the US had hanged Japanese soldiers for waterboarding US POWs, and perhaps that as Governor of Texas, Bush had imprisoned a sheriff for waterboarding a prisoner. It would describe the experience of being waterboarded as essentially like the experience of being drowned, and record that there were often lingering psychological effects. Against that, such an article would have to report that legal memoranda had argued that waterboarding was not torture and was legal. It might also report that some people responded to the September 11th attacks by arguing that even torture should be justified as a response to terrorism.

"That article would have been objective in even the restrictive sense that the American media uses. Every claim is not only true, but unambiguous and part of the public record.

"And I don’t recall seeing anything like that during the Bush era. Individual claims might appear in an article about the waterboarding debate, but I never remember seeing a single article that would give the  full picture. 

"Journalists are sometimes criticized for treating both sides of any disagreement as equally respectable, even if the facts are squarely on one side. But we can see that it’s not just that–the media won’t even collect and report facts that aren’t in dispute, if the net effect would be to undermine the claims of one side."

"Attention conservation notice: 3200 cranky words on the PhD over-supply "crisis.""

"Longitudinal data refer to the situation where repeated observations are available for each sampled object. Clustered data, where observations are nested in a hierarchical structure within objects (without time necessarily being involved) represent a similar type of situation. Methodologies that take this structure into account allow for the possibilities of systematic differences between objects that are not related to attributes and autocorrelation within objects across time periods. A standard methodology in the statistics literature for this type of data is the mixed effects model, where these differences between objects are represented by so-called “random effects” that are estimated from the data (population-level relationships are termed “fixed effects,” together resulting in a mixed effects model). This paper presents a methodology that combines the structure of mixed effects models for longitudinal and clustered data with the flexibility of tree-based estimation methods. We apply the resulting estimation method, called the RE-EM tree, to pricing in online transactions, showing that the RE-EM tree is less sensitive to parametric assumptions and provides improved predictive power compared to linear models with random effects and regression trees without random effects. We also apply it to a smaller data set examining accident fatalities, and show that the RE-EM tree strongly outperforms a tree without random effects while performing comparably to a linear model with random effects. We also perform extensive simulation experiments to show that the estimator improves predictive performance relative to regression trees without random effects and is comparable or superior to using linear models with random effects in more general situations."

"We analyze classification problems in which data is generated by a two-tiered random process. The class is generated first, then a layer of conditionally independent hidden variables, and finally the observed variables. For sources like this, the Bayes-optimal rule for predicting the class given the values of the observed variables is a two-layer neural network. We show that, if the hidden variables have non-negligible effects on many observed variables, a linear classifier approximates the error rate of the Bayes optimal classifier up to lower order terms. We also show that the hinge loss of a linear classifier is not much more than the Bayes error rate, which implies that an accurate linear classifier can be found efficiently."

Michael preaches the Gospel According to Pearl; and very nicely too. (I would dispute however that DAGs don't give us a handle on mechanisms.)

"The basic mistake that Fish is making is this: he pretends that humanists have no discovery process at all. For Fish, the interpretive act is always fully contained in an encounter with a single piece of evidence. How your “interpretive proposition” got framed in the first place is a matter of no consequence; some readers are just fortunate to have propositions that turn out to be correct. Fish is not alone in this idealized model of interpretation; it’s widespread among humanists."

"We introduce a quantitative measure of the capacity of a small biological network to evolve. We apply our measure to a stochastic description of the experimental setup of Guet et al. (Science 296:1466, 2002), treating chemical inducers as functional inputs to biochemical networks and the expression of a reporter gene as the functional output. We take an information-theoretic approach, allowing the system to set parameters that optimize signal processing ability, thus enumerating each network's highest-fidelity functions. We find that all networks studied are highly evolvable by our measure, meaning that change in function has little dependence on change in parameters. Moreover, we find that each network's functions are connected by paths in the parameter space along which information is not significantly lowered, meaning a network may continuously change its functionality without losing it along the way. This property further underscores the evolvability of the networks."

"Besides familiar and now-commonplace tasks that computers do all the time, what else are they capable of? Stephen Ramsay's intriguing study of computational text analysis examines how computers can be used as "reading machines" to open up entirely new possibilities for literary critics. Computer-based text analysis has been employed for the past several decades as a way of searching, collating, and indexing texts. Despite this, the digital revolution has not penetrated the core activity of literary studies: interpretive analysis of written texts."

"The ever-expanding capacities of computing offer new narrative possibilities for virtual worlds. Yet vast narratives—featuring an ongoing and intricately developed storyline, many characters, and multiple settings—did not originate with, and are not limited to, Massively Multiplayer Online Games. Thomas Mann's Joseph and His Brothers, J. R. R. Tolkien's Lord of the Rings, Marvel's Spiderman, and the complex stories of such television shows as Dr. Who, The Sopranos, and Lost all present vast fictional worlds.

Third Person explores strategies of vast narrative across a variety of media, including video games, television, literature, comic books, tabletop games, and digital art. The contributors—media and television scholars, novelists, comic creators, game designers, and others—investigate such issues as continuity, canonicity, interactivity, fan fiction, technological innovation, and cross-media phenomena.

Chapters examine a range of topics, including storytelling in a multiplayer environment; narrative techniques for a 3,000,000-page novel; continuity (or the impossibility of it) in Doctor Who; managing multiple intertwined narratives in superhero comics; the spatial experience of the Final Fantasy role-playing games; World of Warcraft adventure texts created by designers and fans; and the serial storytelling of The Wire."

"Why isn't there, for every Southern boy fourteen years old, not once but whenever he wants it, an instant when it's still not yet four-thirty on that April morning in 1861, the batteries are in position opposite Fort Sumter, the guns are laid and ready, furled flags are already loosened to break out and Edmund Ruffin himself with his long, stringy white hair and his hat in one hand probably and his sword in the other looking up waiting for Beauregard to give the word and it's all in the balance, it hasn't happened yet, it hasn't even begun yet, it not only hasn't begun yet but there is still time for it not to begin against that position and those circumstances which made more men than Stephens and Toombs look grave yet it's going to begin, we all know that, we have come too far with too much at stake and that moment doesn't need even a fourteen-year-old boy to think This time. Maybe this time with all this much to lose than all this much to gain: Morality and self-respect and progress and development. Maybe this time they won't fire the shot that doomed the South to a century of reactionary backwardness?"

"The problem with a rights-based liberalism is precisely that it is not and never can be the end of history, that it is never secure or stable, that every liberty claimed through toil and protest, no matter how acclaimed and cherished and generative, is one day away from the firing line when some powerful interest decides that some right or practice is inconvenient.

"It doesn’t even matter if the end of a right, a freedom, a possibility will ultimately hurt that powerful interest. The contemporary businesses who have registered a powerful stake in exceptionally restrictive monopolies over intellectual property have themselves been enormous beneficiaries of a conception of the public domain as a fundamental and irreversible right of a free society. No matter: they would now see it ended. Better to kill the future than live in a present where you can only have two Ferraris in the driveway."

"When the “meme” was introduced in 1976, it was as a metaphor intended to illuminate an evolutionary argument. By the late-1980s, however, we see from its use in major US newspapers that this original meaning had become obscured. The meme became a virus of the mind. (In the UK, this occurred slightly later.) It is also now clear that this becoming involved complex sustained interactions between scholars, journalists, and the letter-writing public. We must therefore read the “meme” through lenses provided by its popularization. The results are in turn suggestive of the processes of meaning-construction in scholarly communication more generally."

"Literature professors often encounter and complain about the student who arrives in their classes with a professed ‘love of literature’. We sometimes come to see our job as grimly breaking those blithe spirits on the wheel of the hard labor of criticism and dismissing them from our company when they refuse to come into the quarry and break stone." (Why do I like that sentence? The metaphors are horribly mixed.)

"Many cellular processes are carried out by molecular ‘machines’—assemblies of multiple differentiated proteins that physically interact to execute biological functions1, 2, 3, 4, 5, 6, 7, 8. Despite much speculation, strong evidence of the mechanisms by which these assemblies evolved is lacking. Here we use ancestral gene resurrection9, 10, 11 and manipulative genetic experiments to determine how the complexity of an essential molecular machine—the hexameric transmembrane ring of the eukaryotic V-ATPase proton pump—increased hundreds of millions of years ago. We show that the ring of Fungi, which is composed of three paralogous proteins, evolved from a more ancient two-paralogue complex because of a gene duplication that was followed by loss in each daughter copy of specific interfaces by which it interacts with other ring proteins. These losses were complementary, so both copies became obligate components with restricted spatial roles in the complex. Reintroducing a single historical mutation from each paralogue lineage into the resurrected ancestral proteins is sufficient to recapitulate their asymmetric degeneration and trigger the requirement for the more elaborate three-component ring. Our experiments show that increased complexity in an essential molecular machine evolved because of simple, high-probability evolutionary processes, without the apparent evolution of novel functions. They point to a plausible mechanism for the evolution of complexity in other multi-paralogue protein complexes."

"We introduce a novel mechanism to eliminate endgame effects in repeated prisoner’s dilemma experiments. In the main phase of a supergame our mechanism generates more persistent cooperation than finite horizon or random continuation rules. Moreover, we find evidence for cooperation-enhancing “active/reactive” strategies which concentrate in the initial phase of a supergame as subjects gain experience."

Probably worth reading, despite the endorsement from Ferguson.

"New single-cell recordings show that humans do have mirror neurons, and in more brain regions than previously suspected. Some action-execution neurons were seen to be inhibited during observation, possibly preventing imitation and helping self/other discrimination."

"As Clawback opens, an assassin has begun shooting the country’s worst-performing financiers. A bottom-ranked investment manager; a hedge fund partner down ninety percent; a rotten banker. Someone’s slogan seems to be, “Don’t bail them out, take them out!”

"A coalition of banksters hires a fixer, the sort of contractor whose job description opens with “total deniability” and ends with “unlicensed machine guns.” As bodies fall and markets plunge, he ranges the Greenwich-Midtown axis with an equalizer and an attitude – only to realize he’s become a target himself."

"Nonreductive physicalists have long used multiple realizability to argue for the explanatory “autonomy” of the special sciences. Recently, in the face of the local reduction and disjunctive property responses to multiple realizability, some defenders of nonreductive physicalism have suggested that autonomy can be grounded merely in human cognitive limitations. In this article, I argue that this is mistaken. By distinguishing between two kinds of abstraction I show that the greater explanatory relevance of some special-science predicates (to certain explananda) is both nonanthropocentric and not solely based on considerations of multiple realizability."

"Machamer, Darden, and Craver’s account of the nature and role of mechanisms in the special sciences has been very influential. Unfortunately, a confusing array of ontic, epistemic, and pragmatic distinctions is required to individuate their mechanisms, mechanism schemata, and mechanism sketches. I diagnose this as a conflation of token-level causal relations with type-level relations. I propose instead that a mechanism is an abstraction that relates entity types and activity types on the model of a directed graph. Mechanisms have an ontic status distinct from the causal chains of token entities and token activities that instantiate them."

"Statistical models that include random effects are commonly used to analyze longitudinal and correlated data, often with strong and parametric assumptions about the random effects distribution. There is marked disagreement in the literature as to whether such parametric assumptions are important or innocuous. In the context of generalized linear mixed models used to analyze clustered or longitudinal data, we examine the impact of random effects distribution misspecification on a variety of inferences, including prediction, inference about covariate effects, prediction of random effects and estimation of random effects variances. We describe examples, theoretical calculations and simulations to elucidate situations in which the specification is and is not important. A key conclusion is the large degree of robustness of maximum likelihood for a wide variety of commonly encountered situations."

"When comparing networks (with the same number of nodes) with direct methods, a number of possible distances is already available in literature. Among others, two of the most common families are the set of edit-like distances and the spectral distances. The functions in the former family quantitatively evaluate the differences between two networks in terms of minimum number of edit operations (with possibly different costs) transforming one network into the other, that is, deletion and insertion of links, while spectral measures relies on functions of the eigenvalues of one of the connectivity matrices of the underlying graph.
A noticeable issue affecting edit distance is the fact of being local, i.e. not taking into account the global structure of the networks but only summing the contributions coming from each single link. On the other hand, spectral measures cannot distinguish isomorphic or isospectral graphs. We propose here a possible solution to overcome both issues: combining together an edit and a spectral distance in a product metric we will define textit{glocal}. In what follows we define the two components and the glocal metric itself, with a few examples of applications."

We consider Markov chain with spectral gap in $L^2$ space. Assume that $f$ is a bounded function. Then the probabilities of large deviations of average along trajectory satisfy Hoeffding's-type inequalities. These bounds depend only on the stationary mean, spectral gap and the end-points of support of $f$.

"We study weak convergence of empirical processes of dependent data, indexed by classes of functions. We obtain results that are especially suitable for data arising from dynamical systems and Markov chains, where the Central Limit Theorem for partial sums is commonly derived via the spectral gap technique. Our results apply, e.g. to the empirical process of ergodic torus automorphisms."

"We propose four approaches to estimating the directed information rate between a pair of jointly stationary ergodic processes with the help of universal probability assignments. The four approaches yield estimators with different merits such as nonnegativity and boundedness. We establish consistency of these estimators in various senses and derive near-optimal rates of convergence in the minimax sense under mild conditions. The estimators carry over directly to estimating other information measures of stationary ergodic processes, such as entropy rate and mutual information rate, and provide alternatives to classical approaches in the existing literature. Guided by the theoretical results, we use context tree weighting as the vehicle for the implementations of the proposed estimators. Experiments on synthetic and real data are presented, demonstrating the potential of the proposed schemes in practice and the efficacy of directed information estimation as a tool for detecting and measuring causality and delay."

"Respondent-driven sampling (RDS) is a commonly used substitute for random sampling when studying hidden populations, such as injective drug users or men who have sex with men, for which no sampling frame is known. The method works like a snowball sample but can, given that some assumptions are met, generate unbiased population estimates. One key assumption, not likely to be met, is that the acquaintance network in which the recruitment process takes place is undirected, meaning that all recruiters should have the potential to be recruited by the person they recruit. Here we investigate the potential bias of directedness by simulating RDS on real and artificial network structures. We show that directedness is likely to generate bias that cannot be compensated for unless the sampled individuals know how many potentially may have recruited them (i.e. their indegree), which is unlikely in most situations. We propose three indegree-based estimators for RDS on directed networks, and show that they can be used in the situation when only outdegrees are observed, either together with prior information or in a sensitivity analysis taking uncertainty of indegree properties of the network into account."

"Social networks are rarely observed in full detail. In many situations properties are known for only a sample of the individuals in the network and it is desirable to induce global properties of the full social network from this "egocentric" network data. In the current paper we study a few different types of egocentric data, and show what global network properties are consistent with those egocentric data. Two global network properties are considered: the size of the largest connected component in the network (the giant), and secondly, the possible size of an epidemic outbreak taking place on the network, in which transmission occurs only between network neighbours, and with probability $p$. The main conclusion is that in most cases, egocentric data allow for a large range of possible sizes of the giant and the outbreak. However, there is an upper bound for the latter. For the case that the network is selected uniformly among networks with prescribed egocentric data (satisfying some conditions), the asymptotic size of the giant and the outbreak is characterised."

"In the past several years a wide range of methods for the construction of regression trees and other estimators based on the recursive partitioning of samples have appeared in the statistics literature. Many applications involve data collected through a complex sample design. At present, however, relatively little is known regarding the properties of these methods under complex designs. This article proposes a method for incorporating information about the complex sample design when building a regression tree using a recursive partitioning algorithm. Sufficient conditions are established for asymptotic design L2 consistency of these regression trees as estimators for an arbitrary regression function. The proposed method is illustrated with Occupational Employment Statistics establishment survey data linked to Quarterly Census of Employment and Wage payroll data of the Bureau of Labor Statistics. Performance of the nonparametric estimator is investigated through a simulation study based on this example."

"The popularity of the Web and Internet commerce provides many extremely large datasets from which information can be gleaned by data mining. This book focuses on practical algorithms that have been used to solve key problems in data mining and which can be used on even the largest datasets. It begins with a discussion of the map-reduce framework, an important tool for parallelizing algorithms automatically. The authors explain the tricks of locality-sensitive hashing and stream processing algorithms for mining data that arrives too fast for exhaustive processing. The PageRank idea and related tricks for organizing the Web are covered next. Other chapters cover the problems of finding frequent itemsets and clustering. The final chapters cover two applications: recommendation systems and Web advertising, each vital in e-commerce. Written by two authorities in database and Web technologies, this book is essential reading for students and practitioners alike."

--- What a remarkably hideous cover!

"We describe an adaptive context tree weighting (ACTW) algorithm, as an extension to the standard context tree weighting (CTW) algorithm. Unlike the standard CTW algorithm, which weights all observations equally regardless of the depth, ACTW gives increasing weight to more recent observations, aiming to improve performance in cases where the input sequence is from a non-stationary distribution. Data compression results show ACTW variants improving over CTW on merged files from standard compression benchmark tests while never being significantly worse on any individual file."

"The Motion Picture Association of America, chief sponsor and financier of SOPA and PIPA, addresses Wikipedia, Reddit, and other major sites going dark tomorrow, accusing them of “abuse of power.” “It’s a dangerous and troubling development when the platforms that serve as gateways to information intentionally skew the facts to incite their users in order to further their corporate interests.” In related news, the mutilated body of Irony was found washed up against a pier in the East River. She was pronounced dead at the scene."