Too many good bits to quote just one.

Not every day you read a story like this about a grad school classmate.

"I suspect that the literary equivalent of “code that performs nothing” is authorial “voice.” Literary null-objects are certain kinds of refined word-choices and grammatical stylings that convey the “message” of “I’m the author and you’re the reader.” One can skate along on this blather for quite a while without venturing to state much of anything; but “voice” keeps the channels open, and prevents the crash-state of the reader standing up and throwing the book against the wall."

"When faced with the task of forecasting a dynamic system, practitioners often have available historical data, knowledge of the system, or a combination of both. While intuition dictates that perfect knowledge of the system should in theory yield perfect forecasting, often knowledge of the system is only partially known, known up to parameters, or known incorrectly. In contrast, forecasting using previous data without any process knowledge might result in accurate prediction for simple systems, but will fail for highly nonlinear and chaotic systems. In this paper, the authors demonstrate how even in chaotic systems, forecasting with historical data is preferable to using process knowledge if this knowledge exhibits certain forms of misspecification. Through an extensive simulation study, a range of misspecification and forecasting scenarios are examined with the goal of gaining an improved understanding of the circumstances under which forecasting from historical data is to be preferred over using process knowledge."

"We compute analytically, for large N, the probability P(N+,N) that a N×N Wishart random matrix has N+ eigenvalues exceeding a threshold Nζ, including its large deviation tails. This probability plays a benchmark role when performing the principal component analysis of a large empirical data set. We find that P(N+,N)≈exp⁡[-βN2ψζ(N+/N)], where β is the Dyson index of the ensemble and ψζ(κ) is a rate function that we compute explicitly in the full range 0≤κ≤1 and for any ζ. The rate function ψζ(κ) displays a quadratic behavior modulated by a logarithmic singularity close to its minimum κ⋆(ζ). This is shown to be a consequence of a phase transition in an associated Coulomb gas problem. The variance Δ(N) of the number of relevant components is also shown to grow universally (independent of ζ) as Δ(N)∼(βπ2)-1ln⁡N for large N."

"Wallsend, in North Tyneside, was the end of Hadrian's Wall, constructed by the Romans to defend the border of the Empire against the rebelling tribes beyond. To commemorate this historical connection (which is also reflected in the town's name), since 2003 most of the signs in Wallsend station on the Tyne and Wear Metro have been in both English and Latin. Large photographs taken in Wallsend which have all text changed to Latin are also displayed."

"This paper deals with the factor modeling for high-dimensional time series based on a dimension-reduction viewpoint. Under stationary settings, the inference is simple in the sense that both the number of factors and the factor loadings are estimated in terms of an eigenanalysis for a nonnegative definite matrix, and is therefore applicable when the dimension of time series is on the order of a few thousands. Asymptotic properties of the proposed method are investigated under two settings: (i) the sample size goes to infinity while the dimension of time series is fixed; and (ii) both the sample size and the dimension of time series go to infinity together. In particular, our estimators for zero-eigenvalues enjoy faster convergence (or slower divergence) rates, hence making the estimation for the number of factors easier. In particular, when the sample size and the dimension of time series go to infinity together, the estimators for the eigenvalues are no longer consistent. However, our estimator for the number of the factors, which is based on the ratios of the estimated eigenvalues, still works fine. Furthermore, this estimation shows the so-called “blessing of dimensionality” property in the sense that the performance of the estimation may improve when the dimension of time series increases. A two-step procedure is investigated when the factors are of different degrees of strength. Numerical illustration with both simulated and real data is also reported."

"We provide a limit theory for a general class of kernel smoothed U-statistics that may be used for specification testing in time series regression with nonstationary data. The test framework allows for linear and nonlinear models with endogenous regressors that have autoregressive unit roots or near unit roots. The limit theory for the specification test depends on the self-intersection local time of a Gaussian process. A new weak convergence result is developed for certain partial sums of functions involving nonstationary time series that converges to the intersection local time process. This result is of independent interest and is useful in other applications. Simulations examine the finite sample performance of the test."

"In a regression setup with deterministic design, we study the pure aggregation problem and introduce a natural extension from the Gaussian distribution to distributions in the exponential family. While this extension bears strong connections with generalized linear models, it does not require identifiability of the parameter or even that the model on the systematic component is true. It is shown that this problem can be solved by constrained and/or penalized likelihood maximization and we derive sharp oracle inequalities that hold both in expectation and with high probability. Finally all the bounds are proved to be optimal in a minimax sense."

"We address the choice of the tuning parameter $lambda$ in $ell_1$-penalized M-estimation. Our main concern is models which are highly nonlinear, such as the Gaussian mixture model. The number of parameters $p$ is moreover large, possibly larger than the number of observations $n$. The generic chaining technique of Talagrand[2005] is tailored for this problem. It leads to the choice $lambda asymp sqrt {log p / n}$, as in the standard Lasso procedure (which concerns the linear model and least squares loss)."

"There is an incompatibility between the symmetries of causal structure in relativity theory and the signaling abilities of probabilistic devices with inputs and outputs: while time reversal in relativity will not introduce the ability to signal between spacelike separated regions, this is not the case for probabilistic devices with spacelike separated input-output pairs. We explicitly describe a nonsignaling device which becomes a perfect signaling device under time reversal, where time reversal can be conceptualized as playing backwards a videotape of an agent manipulating the device. This leads to an arrow of time that is identifiable when studying the correlations of events for spacelike separated regions. Somewhat surprisingly, although the time reversal of Popescu-Rohrlich boxes also allows agents to signal, it does not yield a perfect signaling device. Finally, we realize time reversal using postselection, which could to lead experimental implementation."

"Since the 1990s chaotic cat maps are widely used in data encryption, for their very complicated dynamics within a simple model and desired characteristics related to requirements of cryptography. The number of cat map parameters and the map period length after discretization are two major concerns in many applications for security reasons. In this paper, we propose a new family of 36 distinctive 3D cat maps with different spatial configurations taking existing 3D cat maps [1]-[4] as special cases. Our analysis and comparisons show that this new 3D cat maps family has more independent map parameters and much longer averaged period lengths than existing 3D cat maps. The presented cat map family can be extended to higher dimensional cases."

(to_teach tags for clsses which use the cat map as an example)

"But maybe somewhere on Titan there are dry salt pans, old, large, and very flat, like the Bonneville salt pans in Nevada, but composed of something like asphalt. Imagine the drag-racing possibilities..."

"In his book Direct Democracy Worldwide, David Altman moves beyond the classic narratives of Greek city-states and New England town halls to demonstrate that this form of government is pertinent today despite its still relatively modest use at the national level. However, although some forms of direct democracy, particularly citizen initiatives, may enhance a larger representational context, others offer little opportunity for authentic popular voice. Direct democracy here is a tool, rather than a system, a tool that has the potential to be harnessed to refine the limitations of representation. Thus, Altman provides a rich evaluation of the possibilities for such input—a much needed addition to this literature—while initiating a longer term agenda for scholars of democracy.
"More historic understandings of direct democracy have offered a simplistic understanding of its use: Citizens gather in a common place, or through a ballot, and themselves determine the policy that will govern their polity. Yet Altman provokes the reader to consider a much more complex constellation of possibilities in his first chapter. Rather than consider direct democracy as a Weberian ideal type of political order, he effectively offers a vision of this process as a function within a larger representational system."

(etc., etc.)

"Literature is a form of expression whose temporal structure, both in content and style, provides a historical record of the evolution of culture. In this work we take on a quantitative analysis of literary style and conduct the first large-scale temporal stylometric study of literature by using the vast holdings in the Project Gutenberg Digital Library corpus. We find temporal stylistic localization among authors through the analysis of the similarity structure in feature vectors derived from content-free word usage, nonhomogeneous decay rates of stylistic influence, and an accelerating rate of decay of influence among modern authors. Within a given time period we also find evidence for stylistic coherence with a given literary topic, such that writers in different fields adopt different literary styles. This study gives quantitative support to the notion of a literary “style of a time” with a strong trend toward increasingly contemporaneous stylistic influence."

It'll be interesting to see how they handle the bias induced by selective retention.

"Because of advances in methods and theory, archaeology now addresses issues central to debates in the social sciences in a far more sophisticated manner than ever before. Coupled with methodological innovations, multiscalar archaeological studies around the world have produced a wealth of new data that provide a unique perspective on long-term changes in human societies, as they document variation in human behavior and institutions before the modern era. We illustrate these points with three examples: changes in human settlements, the roles of markets and states in deep history, and changes in standards of living. Alternative pathways toward complexity suggest how common processes may operate under contrasting ecologies, populations, and economic integration."

"Many modern statistical applications involve inference for complex stochastic models, where it is easy to simulate from the models, but impossible to calculate likelihoods. Approximate Bayesian computation (ABC) is a method of inference for such models. It replaces calculation of the likelihood by a step which involves simulating artificial data for different parameter values, and comparing summary statistics of the simulated data with summary statistics of the observed data. Here we show how to construct appropriate summary statistics for ABC in a semi-automatic manner. We aim for summary statistics which will enable inference about certain parameters of interest to be as accurate as possible. Theoretical results show that optimal summary statistics are the posterior means of the parameters. Although these cannot be calculated analytically, we use an extra stage of simulation to estimate how the posterior means vary as a function of the data; and we then use these estimates of our summary statistics within ABC. Empirical results show that our approach is a robust method for choosing summary statistics that can result in substantially more accurate ABC analyses than the ad hoc choices of summary statistics that have been proposed in the literature. We also demonstrate advantages over two alternative methods of simulation-based inference."

"Identifying causal effects attributable to network membership is a key challenge in empirical studies of social networks. In this article, we examine the consequences of endogeneity for inferences about the effects of networks on network members’ behavior. Using the House office lottery (in which newly elected members select their office spaces in a randomly chosen order) as an instrumental variable to estimate the causal impact of legislative networks on roll call behavior and cosponsorship decisions in the 105th–112th Houses, we find no evidence that office proximity affects patterns of legislative behavior. These results contrast with decades of congressional scholarship and recent empirical studies. Our analysis demonstrates the importance of accounting for selection processes and omitted variables in estimating the causal impact of networks."

"Recent theory work has found that a special type of spatial partition tree - called a random projection tree - is adaptive to the intrinsic dimension of the data from which it is built. Here we examine this same question, with a combination of theory and experiments, for a broader class of trees that includes k-d trees, dyadic trees, and PCA trees. Our motivation is to get a feel for (i) the kind of intrinsic low dimensional structure that can be empirically verified, (ii) the extent to which a spatial partition can exploit such structure, and (iii) the implications for standard statistical tasks such as regression, vector quantization, and nearest neighbor search."

"How far and how fast does information spread in social media? Researchers have recently examined a number of factors that affect information diffusion in online social networks, including: the novelty of information, users' activity levels, who they pay attention to, and how they respond to friends' recommendations. Using URLs as markers of information, we carry out a detailed study of retweeting, the primary mechanism by which information spreads on the Twitter follower graph. Our empirical study examines how users respond to an incoming stimulus, i.e., a tweet (message) from a friend, and reveals that %retweeting behavior is constrained by a few simple principles. the "principle of least effort" combined with limited attention plays a dominant role in retweeting behavior. Specifically, we observe that users retweet information when it is most visible, such as when it near the top of their Twitter stream. Moreover, our measurements quantify how a user's limited attention is divided among incoming tweets, providing novel evidence that highly connected individuals are less likely to propagate an arbitrary tweet. Our study indicates that the finite ability to process incoming information constrains social contagion, and we conclude that rapid decay of visibility is the primary barrier to information propagation online."

"We develop a likelihood ratio (LR) test procedure for discriminating between a short-memory time series with a change-point (CP) and a long-memory (LM) time series. Under the null hypothesis, the time series consists of two segments of short-memory time series with different means and possibly different covariance functions. The location of the shift in the mean is unknown. Under the alternative, the time series has no shift in mean but rather is LM. The LR statistic is defined as the normalized log-ratio of the Whittle likelihood between the CP model and the LM model, which is asymptotically normally distributed under the null. The LR test provides a parametric alternative to the CUSUM test proposed by Berkes et al. (2006). Moreover, the LR test is more general than the CUSUM test in the sense that it is applicable to changes in other marginal or dependence features other than a change-in-mean. We show its good performance in simulations and apply it to two data examples."

"There's a certain kind of insanity, not a feeling of not being oneself any longer, but a loss of a grip on what it would be to feel, or not feel, oneself at all, which is apt to be brought on at family reunions, especially those at which one encounters distant relatives for the first time. At such gatherings one gets a keen sense of the family "look", and the general family mode of behavior. One sees not only one's features return to one in the features of countless others; one sees, too, one's squabbles with one's siblings (say) reflected in the bickering of great-aunt so-and-so and her husband. There seems to vanish any sense of oneself as an individual, oneself as something substantial, rather than a congeries of free-floating traits that happen to be combined, here, in these proportions. But even this is too much; it's not as if one can say that one is so much of Cousin X in this respect, so much of Great-Aunt Y in this other respect, and so on, for X and Y stand revealed as having no more solidity in themselves than one oneself has (as we know, a family reunion is a system with no positive elements). Nor can one console oneself with the thought that at least one is the only one to have these traits in this precise proportion, because, even if that is true (and how can one be sure?) it is only accidental.
"At its most extreme, someone who suffers from this delusion will not only see himself as just a congeries of shared traits; he will also begin to see in every person one encounters nothing more than a collection of traits previously encountered in oneself or in others ... those others and that self in turn lacking any real substance. Nothing new under the sun, here or there, no potential for surprise; you've seen everything already.
"The name of this delusion is, of course, metempsychosis."

"New measures are proposed for mutual and causal dependence between two time series, based on information theoretical ideas. The measure of mutual dependence is shown to be the sum of the measure of unidirectional causal dependence from the first time series to the second, the measure of unidirectional causal dependence from the second to the first, and the measure of instantaneous causal dependence. The measures are applicable to any kind of time series: continuous, discrete, or categorical."

What I'll be saying when.

"Online learning constitutes a mathematical framework to analyze sequential decision making problems in adversarial environments. The learner repeatedly chooses an action, the environment responds with an outcome, and then the learner receives a reward for the played action. The goal of the learner is to maximize his total reward. However, there are situations in which, in addition to maximizing the cumulative reward, there are some additional constraints/goals on the sequence of decisions that must be satisfied by the learner. For example, in textit{online marketing}, simultaneously maximizing the cumulative reward and the number of buyers to take advantage of word-of-mouth advertising for future marketing seems to be a more ambitious goal than only maximizing cumulative reward. As another example, learning from costly expert advice captures more realistic settings than the original setting in applications such as routing in networks with power constraint. In this paper we study an extension to the online learning where the learner aims to maximize the total reward given that some additional constraints need to be satisfied. We propose Lagrangian exponentially weighted average (textbf{LEWA}) algorithm, an efficient algorithm to solve constrained online learning, which is a primal dual variant of the well known exponentially weighted average algorithm and inspired by the theory of Lagrangian method in constrained optimization. We establish the regret and the violation of the constraint bounds in full information and bandit feedback models."

"The social rank of an animal is distinguished by its behavior relative to others in its community. Although social-status-dependent differences in behavior must arise because of differences in neural function, status-dependent differences in the underlying neural circuitry have only begun to be described. We report that dominant and subordinate crayfish differ in their behavioral orienting response to an unexpected unilateral touch, and that these differences correlate with functional differences in local neural circuits that mediate the responses. The behavioral differences correlate with simultaneously recorded differences in leg depressor muscle EMGs and with differences in the responses of depressor motor neurons recorded in reduced, in vitro preparations from the same animals. The responses of local serotonergic interneurons to unilateral stimuli displayed the same status-dependent differences as the depressor motor neurons. These results indicate that the circuits and their intrinsic serotonergic modulatory components are configured differently according to social status, and that these differences do not depend on a continuous descending signal from higher centers."

"We study the stochastic dynamics of evolutionary games, and focus on the so-called `stochastic slowdown' effect, previously observed in (Altrock et. al, 2010) for simple evolutionary dynamics. Slowdown here refers to the fact that a beneficial mutation may take longer to fixate than a neutral one. More precisely, the fixation time conditioned on the mutant taking over can show a maximum at intermediate selection strength. We show that this phenomenon is present in the prisoner's dilemma, and also discuss counterintuitive slowdown and speedup in coexistence games. In order to establish the microscopic origins of these phenomena, we calculate the average sojourn times. This allows us to identify the transient states which contribute most to the slowdown effect, and enables us to provide an understanding of slowdown in the takeover of a small group of cooperators by defectors: Defection spreads quickly initially, but the final steps to takeover can be delayed significantly. The analysis of coexistence games reveals even more intricate behavior. In small populations, the conditional average fixation time can show multiple extrema as a function of the selection strength, e.g., slowdown, speedup, and slowdown again. We classify two-player games with respect to the possibility to observe non-monotonic behavior of the conditional average fixation time as a function of selection strength."

"Potential outcome counterfactuals represent variation in the outcome of interest after a hypothetical treatment or intervention is performed. Causal graphical models are a concise, intuitive way of representing causal assumptions, including independence constraints among such counterfactuals. Much of modern causal inference is concerned with expressing cause effect relationships of interest in counterfactual form, showing how the resulting counterfactuals can be identified (that is expressed in terms of available data, using domain-specific causal assumptions), and subsequently estimated using statistical methods. In this paper we will use causal graphical models to analyze the identification problem of the so-called emph{path-specific effects}, that is effects of treatment on outcome along certain specified causal paths. Such effects arise in mediation analysis settings where it's important to distinguish direct and indirect effects of treatment. We review existing results on path-specific effects in the fully observable, static treatment setting, and extend them to settings with time-varying treatments, and latent variables."

"Contacts between individuals serve as pathways where infections may propagate. These contact patterns can be represented by network structures. Static structures have been the common modeling paradigm but recent results suggest that temporal structures play different roles to regulate the spread of infections or infection-like dynamics. On temporal networks a vertex is active only at certain moments and inactive otherwise such that a contact is not continuously available. In several empirical networks, the time between two consecutive vertex-activation events typically follows heterogeneous activity (e.g. bursts). In this chapter, we present a simple and intuitive stochastic model of a temporal network and investigate how epidemics co-evolves with the temporal structures, focusing on the growth dynamics of the epidemics. The model assumes no underlying topological structure and is only constrained by the time between two consecutive events of vertex activation. The main observation is that the speed of the infection spread is different in case of heterogeneous and homogeneous temporal patterns but the differences depend on the stage of the epidemics. In comparison to the homogeneous scenario, the power law case results in a faster growth in the beginning but turns out to be slower after a certain time, taking several time steps to reach the whole network."

"The natural gradient allows for more efficient gradient descent by removing dependencies and biases inherent in a function's parameterization. Several papers present the topic thoroughly and precisely. It remains a very difficult idea to get your head around however. The intent of this note is to provide simple intuition for the natural gradient and its use. We review how an ill conditioned parameter space can undermine learning, introduce the natural gradient by analogy to the more widely understood concept of signal whitening, and present tricks and specific prescriptions for applying the natural gradient to learning problems."

Does this ever mention the phrase "Fisher information"?

Home sweet home for the next two weeks. (And yes, I'm bookmarking this so I'll have access to the address.)

"This paper addresses the problem of measurement errors in causal inference and highlights several algebraic and graphical methods for eliminating systematic bias induced by such errors. In particulars, the paper discusses the control of partially observable confounders in parametric and non parametric models and the computational problem of obtaining bias-free effect estimates in such models."

"And the time ... in physics class, when we were doing these basic (very basic) labs on probability, and I had a little handheld pachinko machine? With a bunch of balls, and evenly spaced rods, and stalls at the bottom? And you tilt it down, and all the balls roll to the top, and you tilt it back, and they come cascading down, and hit the rods, and either bounce left or right, and in the end you’ve got this lovely little bell curve of balls at the bottom, because law of averages and such most balls bounce left, then right, then left, or some combination thereof, and end up in the middle? And only a few go left-left-left-left, or right-right-right-right, and end up on either end? —Anyway, it’s my turn, so I tilt it down, then back again, and click-clack-click-clack-click, and wouldn’t you know it, I’ve got an almost perfect reverse bell curve. Towering stacks of balls to the left and right, and almost nothing at all in the middle.
"So I go to the teacher running the show and hold it out to him and say, okay, now what, smart guy? (“If it fails to agree, under novel experiments or with refined measuring techniques, it is not said that one should not be happy.”)
"And the teacher looks at the little handheld pachinko machine, cocks an eyebrow, tilts it down, tilts it back, clack-click-clack-click-clack. Perfect bell curve.
"“There,” he says. “Fixed it for you.”
"—And I can’t for the life of me tell you which of those gestures is the argument with the universe, and which the sermon on the way things ought to be, dammit. —And that might just be my problem."

"The Unheavenly Chorus is the first book to look at the political participation of individual citizens alongside the political advocacy of thousands of organized interests--membership associations such as unions, professional associations, trade associations, and citizens groups, as well as organizations like corporations, hospitals, and universities. Drawing on numerous in-depth surveys of members of the public as well as the largest database of interest organizations ever created--representing more than thirty-five thousand organizations over a twenty-five-year period--this book conclusively demonstrates that American democracy is marred by deeply ingrained and persistent class-based political inequality. The well educated and affluent are active in many ways to make their voices heard, while the less advantaged are not. This book reveals how the political voices of organized interests are even less representative than those of individuals, how political advantage is handed down across generations, how recruitment to political activity perpetuates and exaggerates existing biases, how political voice on the Internet replicates these inequalities--and more."

"So, while the pro-human arguments based on science sound convincing at first, they're ultimately not that great, and fall back on the same sort of vague and airy platitudes as the general "inspiration" argument. And given the gigantic cost multiplier involved in sending humans rather than robots (or in addition to robots), I'd really like to see more than that."

"This paper considers the problem of model selection in a nonparametric additive mixed modeling framework. The fixed effects are modeled nonparametrically using truncated series expansions with B-spline basis. Estimation and selection of such nonparametric fixed effects are simultaneously achieved by using the adaptive group lasso methodology, while the random effects are selected by a traditional backward selection mechanism. To facilitate the automatic selection of model dimension, computable expressions for the degrees of freedom for both the fixed and random effects components are derived, and the Bayesian Information criterion (BIC) is used to select the final model choice. Theoretically it is shown that this BIC model selection method is consistent, while computationally a practical algorithm is developed for solving the optimization problem involved. Simulation results show that the proposed methodology is often capable of selecting the correct significant fixed and random effects components, especially when the sample size and/or signal to noise ratio are not too small. The new method is also applied to two real data sets."

"We study the problem of prediction for evolving graph data. We formulate the problem as the minimization of a convex objective encouraging sparsity and low-rank of the solution, that reflect natural graph properties. The convex formulation allows to obtain oracle inequalities and efficient solvers. We provide empirical results for our algorithm and comparison with competing methods, and point out two open questions related to compressed sensing and algebra of low-rank and sparse matrices."

"The data used by economists can be broadly divided into two categories. First, structured datasets arise when a government agency, trade association, or company can justify the expense of assembling records. The Internet has transformed how economists interact with these datasets by lowering the cost of storing, updating, distributing, finding, and retrieving this information. Second, some economic researchers affirmatively collect data of interest. For researcher-collected data, the Internet opens exceptional possibilities both by increasing the amount of information available for researchers to gather and by lowering researchers' costs of collecting information. In this paper, I explore the Internet's new datasets, present methods for harnessing their wealth, and survey a sampling of the research questions these data help to answer. The first section of this paper discusses "scraping" the Internet for data—that is, collecting data on prices, quantities, and key characteristics that are already available on websites but not yet organized in a form useful for economic research. A second part of the paper considers online experiments, including experiments that the economic researcher observes but does not control (for example, when Amazon or eBay alters site design or bidding rules); and experiments in which a researcher participates in design, including those conducted in partnership with a company or website, and online versions of laboratory experiments. Finally, I discuss certain limits to this type of data collection, including both "terms of use" restrictions on websites and concerns about privacy and confidentiality."

"Neurophysiology is increasingly focused on identifying coincident activity among neurons. Strong inferences about neural computation are made from the results of such studies, so it is important that these results be accurate. However, the preliminary step in the analysis of such data, the assignment of spike waveforms to individual neurons (“spike-sorting”), makes a critical assumption which undermines the analysis: that spikes, and hence neurons, are independent. We show that this assumption guarantees that coincident spiking estimates such as correlation coefficients are biased. We also show how to eliminate this bias. Our solution involves sorting spikes jointly, which contrasts with the current practice of sorting spikes independently of other spikes. This new “ensemble sorting” yields unbiased estimates of coincident spiking, and permits more data to be analyzed with confidence, improving the quality and quantity of neurophysiological inferences. These results should be of interest outside the context of neuronal correlations studies. Indeed, simultaneous recording of many neurons has become the rule rather than the exception in experiments, so it is essential to spike sort correctly if we are to make valid inferences about any properties of, and relationships between, neurons."

"We review predictive techniques from several traditional branches of statistics. Starting with prediction based on the normal model and on the empirical distribution function, we proceed to techniques for various forms of regression and classification. Then, we turn to time series, longitudinal data, and survival analysis. Our focus throughout is on the mechanics of prediction more than on the properties of predictors."

(to_teach tags are tentative.)

Memo to self: bug 'em for materials.

"We develop a novel mechanism for coordinated, distributed multiagent planning. We consider problems stated as a col- lection of single-agent planning problems coupled by com- mon soft constraints on resource consumption. (Resources may be real or fictitious, the latter introduced as a tool for factoring the problem). A key idea is to recast the dis- tributed planning problem as learning in a repeated game between the original agents and a newly introduced group of adversarial agents who influence prices for the resources. The adversarial agents benefit from arbitrage: that is, their incentive is to uncover violations of the resource usage con- straints and, by selfishly adjusting prices, encourage the original agents to avoid plans that cause such violations. If all agents employ no-regret learning algorithms in the course of this repeated interaction, we are able to show that our mechanism can achieve design goals such as social op- timality (efficiency), budget balance, and Nash-equilibrium convergence to within an error which approaches zero as the agents gain experience. In particular, the agents’ average plans converge to a socially optimal solution for the original planning task. We present experiments in a simulated net- work routing domain demonstrating our method’s ability to reliably generate sound plans."

Noah speaks hyperbolically, as usual, but I find myself agreeing more than is comfortable. (Or, to put it more optimistically: if we solve the energy problem, it will be, relative to society's resources, _much cheaper_ to build really powerful accelerators in a generation or three.)

Must get some of those posters.

"Scoring rules assess the quality of probabilistic forecasts, by assigning a numerical score based on the predictive distribution and on the event or value that materializes. A scoring rule is proper if it encourages truthful reporting. It is local of order k if the score depends on the predictive density only through its value and the values of its derivatives of order up to k at the realizing event. Complementing fundamental recent work by Parry, Dawid and Lauritzen, we characterize the local proper scoring rules of order 2 relative to a broad class of Lebesgue densities on the real line, using a different approach. In a data example, we use local and nonlocal proper scoring rules to assess statistically postprocessed ensemble weather forecasts."

"A scoring rule is a loss function measuring the quality of a quoted probability distribution Q for a random variable X, in the light of the realized outcome x of X; it is proper if the expected score, under any distribution P for X, is minimized by quoting Q = P. Using the fact that any differentiable proper scoring rule on a finite sample space is the gradient of a concave homogeneous function, we consider when such a rule can be local in the sense of depending only on the probabilities quoted for points in a nominated neighborhood of x. Under mild conditions, we characterize such a proper local scoring rule in terms of a collection of homogeneous functions on the cliques of an undirected graph on the space . A useful property of such rules is that the quoted distribution Q need only be known up to a scale factor. Examples of the use of such scoring rules include Besag’s pseudo-likelihood and Hyvärinen’s method of ratio matching."

"We investigate proper scoring rules for continuous distributions on the real line. It is known that the log score is the only such rule that depends on the quoted density only through its value at the outcome that materializes. Here we allow further dependence on a finite number m of derivatives of the density at the outcome, and describe a large class of such m-local proper scoring rules: these exist for all even m but no odd m. We further show that for m ≥ 2 all such m-local rules can be computed without knowledge of the normalizing constant of the distribution."

I guessed what the cause must've been, and this was still hilarious. (Also, it's old enough that I might have heard it before.) Still, this is great.

"Given an i.i.d. sample from a distribution $F$ on $mathbb{R}$ with uniformly continuous density $p_0$, purely data-driven estimators are constructed that efficiently estimate $F$ in sup-norm loss and simultaneously estimate $p_0$ at the best possible rate of convergence over H"older balls, also in sup-norm loss. The estimators are obtained by applying a model selection procedure close to Lepski's method with random thresholds to projections of the empirical measure onto spaces spanned by wavelets or $B$-splines. The random thresholds are based on suprema of Rademacher processes indexed by wavelet or spline projection kernels. This requires Bernstein-type analogs of the inequalities in Koltchinskii [Ann. Statist. 34 (2006) 2593-2656] for the deviation of suprema of empirical processes from their Rademacher symmetrizations."

"This paper proposes a new goodness-of-fit test for parametric conditional probability distributions using the nonparametric smoothing methodology. An asymptotic normal distribution is established for the test statistic under the null hypothesis of correct specification of the parametric distribution. The test is shown to have power against local alternatives converging to the null at certain rates. The test can be applied to testing for possible misspecifications in a wide variety of parametric models. A bootstrap procedure is provided for obtaining more accurate critical values for the test. Monte Carlo simulations show that the test has good power against some common alternatives."

This is one of the most beautifully annihilating book reviews I have ever seen, and I say that with deep jealousy.

"Predicting X from Twitter is a popular fad within the Twitter research subculture. It seems both appealing and relatively easy. Among such kind of studies, electoral prediction is maybe the most attractive, and at this moment there is a growing body of literature on such a topic. This is not only an interesting research problem but, above all, it is extremely difficult. However, most of the authors seem to be more interested in claiming positive results than in providing sound and reproducible methods. It is also especially worrisome that many recent papers seem to only acknowledge those studies supporting the idea of Twitter predicting elections, instead of conducting a balanced literature review showing both sides of the matter. After reading many of such papers I have decided to write such a survey myself. Hence, in this paper, every study relevant to the matter of electoral prediction using social media is commented. From this review it can be concluded that the predictive power of Twitter regarding elections has been greatly exaggerated, and that hard research problems still lie ahead."

"Thus I find myself becoming more Paleokeynesian by the hour, as the world keeps hitting me on the head with a brick, pausing after each blow to say: "Now do you understand?!" "

"In sum: It’s awesome that Palmer’s Kickstarter has done so well — but look at what it’s entailed. It’s entailed time, effort, planning and work both backward and forward in time. That currently $439,000 isn’t a windfall for her; it’s a marker of what all that commitment to the work has earned.
"If you’re one of the people looking at her Kickstarter money with stars in your eyes and awesome plans of your own in your head, ask yourself first: Have you put in the time? Earned the credibility? Scoped out the financial balance sheet? Made the commitment to fulfill every single thing you have promised? Palmer has. If you haven’t — on any of this — be aware that your results, shall we say, may vary."

(The earlier part, about the very long process of reputation- and network- building which got Palmer to this point, is quite astute, but too long to excerpt.)

Odd that he doesn't mention slashdot. (Not odd that he doesn't mention Sterling's _Distraction_, with its rival confederations of nomadic biker-gangs, oriented around competing reputation systems.)

"Extracting knowledge and providing insights into complex mechanisms underlying noisy high-dimensional data sets is of utmost importance in many scientific domains. Statistical modeling has become ubiquitous in the analysis of high-dimensional functional data in search of better understanding of cognition mechanisms, in the exploration of large-scale gene regulatory networks in hope of developing drugs for lethal diseases, and in prediction of volatility in stock market in hope of beating the market. Statistical analysis in these high-dimensional data sets is possible only if an estimation procedure exploits hidden structures underlying data.
"This thesis develops flexible estimation procedures with provable theoretical guarantees for uncovering unknown hidden structures underlying data generating process. Of particular interest are procedures that can be used on high-dimensional data sets where the number of samples n much smaller than the ambient dimension p. Learning in high-dimensions is difficult due to the curse of dimensionality, however, the special problem structure makes inference possible. Due to its importance for scientific discovery, we put emphasis on consistent structure recovery throughout the thesis. Particular focus is given to two important problems, semi-parametric estimation of networks and feature selection in multi-task learning."

"So, to recap, 1) Cool paper. 2) Sex between blue-eyed people is not incest. 3) We have no idea when or where this mutation came from, but it is now conceivable that we could ask the question. 4) Embarrassingly bad science reporting spontaneously rises from the grave four years later and tries to eat your brain. "

"We present methods able to predict the presence and strength of conditional and unconditional dependencies (correlations) between two variables Y and Z never jointly measured on the same samples, based on multiple data sets measuring a set of common variables. The algorithms are specializations of prior work on learning causal structures from overlapping variable sets. This problem has also been addressed in the field of statistical matching. The proposed methods are applied to a wide range of domains and are shown to accurately predict the presence of thousands of dependencies. Compared against prototypical statistical matching algorithms and within the scope of our experiments, the proposed algorithms make predictions that are better correlated with the sample estimates of the unknown parameters on test data ; this is particularly the case when the number of commonly measured variables is low.
"The enabling idea behind the methods is to induce one or all causal models that are simultaneously consistent with (fit) all available data sets and prior knowledge and reason with them. This allows constraints stemming from causal assumptions (e.g., Causal Markov Condition, Faithfulness) to propagate. Several methods have been developed based on this idea, for which we propose the unifying name Integrative Causal Analysis (INCA). A contrived example is presented demonstrating the theoretical potential to develop more general methods for co-analyzing heterogeneous data sets. The computational experiments with the novel methods provide evidence that causally-inspired assumptions such as Faithfulness often hold to a good degree of approximation in many real systems and could be exploited for statistical inference. Code, scripts, and data are available at www.mensxmachina.org."

"Asymptotic properties of model selection criteria for high-dimensional regression models are studied where the dimension of covariates is much larger than the sample size. Several sufficient conditions for model selection consistency are provided. Non-Gaussian error distributions are considered and it is shown that the maximal number of covariates for model selection consistency depends on the tail behavior of the error distribution. Also, sufficient conditions for model selection consistency are given when the variance of the noise is neither known nor estimated consistently. Results of simulation studies as well as real data analysis are given to illustrate that finite sample performances of consistent model selection criteria can be quite different."

"We describe an R package named huge which provides easy-to-use functions for estimating high dimensional undirected graphs from data. This package implements recent results in the literature, including Friedman et al. (2007), Liu et al. (2009, 2012) and Liu et al. (2010). Compared with the existing graph estimation package glasso, the huge package provides extra features: (1) instead of using Fortan, it is written in C, which makes the code more portable and easier to modify; (2) besides fitting Gaussian graphical models, it also provides functions for fitting high dimensional semiparametric Gaussian copula models; (3) more functions like data-dependent model selection, data generation and graph visualization; (4) a minor convergence problem of the graphical lasso algorithm is corrected; (5) the package allows the user to apply both lossless and lossy screening rules to scale up large-scale problems, making a tradeoff between computational and statistical efficiency."

"Relational inference denotes the capacity to encode, flexibly retrieve, and integrate multiple memories to combine past experiences to update knowledge and improve decision-making in new situations. Although relational inference is thought to depend on the hippocampus and consciousness, we now show in young, healthy men that it may occur outside consciousness but still recruits the hippocampus. In temporally distinct and unique subliminal episodes, we presented word pairs that either overlapped (“winter–red”, “red–computer”) or not. Effects of unconscious relational inference emerged in reaction times recorded during unconscious encoding and in the outcome of decisions made 1 min later at test, when participants judged the semantic relatedness of two supraliminal words. These words were either episodically related through a common word (“winter–computer” related through “red”) or unrelated. Hippocampal activity increased during the unconscious encoding of overlapping versus nonoverlapping word pairs and during the unconscious retrieval of episodically related versus unrelated words. Furthermore, hippocampal activity during unconscious encoding predicted the outcome of decisions made at test. Hence, unconscious inference may influence decision-making in new situations."

Relations represented spatially?

Well, this is good news for a change.

Going from a p-value of 10^-16 to 0.027 is --- painful. IFrom the lack of a response, I tend to infer that there's no arguing back...
Prediction: the original association will continue to be cited without correction.

Install graphviz, then Rgraphviz, then (?) re-start R. Or at least that worked with the student in office hours. (I swear it's painless on a Mac.)

"Topic models can be seen as a generalization of the clustering problem, in that they posit that observations are generated due to multiple latent factors (e.g. the words in each document are generated as a mixture of several active topics, as opposed to just one). This increased representational power comes at the cost of a more challenging unsupervised learning problem of estimating the topic probability vectors (the distributions over words for each topic), when only the words are observed and the corresponding topics are hidden.
"We provide a simple and efficient learning procedure that is guaranteed to recover the parameters for a wide class of mixture models, including the popular latent Dirichlet allocation (LDA) model. For LDA, the procedure correctly recovers both the topic probability vectors and the prior over the topics, using only trigram statistics (i.e. third order moments, which may be estimated with documents containing just three words). The method, termed Excess Correlation Analysis (ECA), is based on a spectral decomposition of low order moments (third and fourth order) via two singular value decompositions (SVDs). Moreover, the algorithm is scalable since the SVD operations are carried out on k by k matrices, where k is the number of latent factors (e.g. the number of topics), rather than in the d-dimensional observed space (typically d >> k)."

That's a really remarkable claim, and I'd tag it to_be_shot_after_a_fair_trial if it weren't being made by genuinely serious people.

"We introduce the Larger than Life family of two-dimensional two-state cellular automata that generalize certain nearest neighbor outer totalistic cellular automaton rules to large neighborhoods. We describe linear and quadratic rescalings of John Conway's celebrated Game of Life to these large neighborhood cellular automaton rules and present corresponding generalizations of Life's famous gliders and spaceships. We show that, as is becoming well known for nearest neighbor cellular automaton rules, these ``digital creatures'' are ubiquitous for certain parameter values."

(Meta-comment: jeez, guys, how hard is it to re-direct old URLs? Or at least to have a working search box?)

A good summary, except for the very end. I don't think Dewey _does_ ignore deliberation in _Human Nature and Conduct_; everything he has to say about intelligence (and, to some extent, imagination) has to do with this, which is to say, the whole of Part 3 of the book.

"The topic of statistical inference for dynamical systems has been studied extensively across several fields. In this survey we focus on the problem of parameter estimation for non-linear dynamical systems. Our objective is to place results across distinct disciplines in a common setting and highlight opportunities for further research."

Have I really not bookmarked this before?