"A large deviations principle is established for the joint law of the empirical measure and the flow measure of a renewal Markov process on a finite graph. We do not assume any bound on the arrival times, allowing heavy tailed distributions. In particular, the rate functional is in general degenerate (it has a nontrivial set of zeros) and not strictly convex. These features show a behavior highly different from what one may guess with a heuristic Donsker-Varadhan analysis of the problem."

"Stochastic partial differential equations driven by Poisson random measures (PRM) have been proposed as models for many different physical systems, where they are viewed as a refinement of a corresponding noiseless partial differential equations (PDE). A systematic framework for the study of probabilities of deviations of the stochastic PDE from the deterministic PDE is through the theory of large deviations. The goal of this work is to develop the large deviation theory for small Poisson noise perturbations of a general class of deterministic infinite dimensional models. Although the analogous questions for finite dimensional systems have been well studied, there are currently no general results in the infinite dimensional setting. This is in part due to the fact that in this setting solutions may have little spatial regularity, and thus classical approximation methods for large deviation analysis become intractable. The approach taken here, which is based on a variational representation for nonnegative functionals of general PRM, reduces the proof of the large deviation principle to establishing basic qualitative properties for controlled analogues of the underlying stochastic system. As an illustration of the general theory, we consider a particular system that models the spread of a pollutant in a waterway."

"Noether's theorem links the symmetries of a quantum system with its conserved quantities, and is a cornerstone of quantum mechanics. Here we prove a version of Noether's theorem for Markov processes. In quantum mechanics, an observable commutes with the Hamiltonian if and only if its expected value remains constant in time for every state. For Markov processes that no longer holds, but an observable commutes with the Hamiltonian if and only if both its expected value and standard deviation are constant in time for every state."
--- For "Hamiltonian" of a Markov process, read "generator".

"In this paper we re-examine the traditional problem of connecting the internal fluctuations of a system to its response to external forcings and extend the classical theory in order to be able to encompass also nonlinear processes. With this goal, we try to join on the results by Kubo on statistical mechanical systems close to equilibrium, i.e. whose unperturbed state can be described by a canonical ensemble, the theory of dispersion relations, and the response theory recently developed by Ruelle for non-equilibrium systems equipped with an invariant SRB measure. Our derivations highlight the strong link between causality and the possibility of connecting unambiguously fluctuation and response, both at linear and nonlinear level. We first show in a rather general setting how the formalism of the Ruelle response theory can be used to derive in a novel way Kramers-Kronig relations connecting the real and imaginary part of the linear and nonlinear response to external perturbations. We then provide a formal extension at each order of nonlinearity of the fluctuation-dissipation theorem (FDT) for general systems possessing a smooth invariant measure. Finally, we focus on the physically relevant case of systems close to equilibrium, for which we present explicit fluctuation-dissipation relations linking the susceptibility describing the $n^{th}$ order response of the system with the expectation value of suitably defined correlations of $n+1$ observables taken in the equilibrium ensemble. While the FDT has an especially compact structure in the linear case, in the nonlinear case joining the statistical properties of the fluctuations of the system to its response to external perturbations requires linear changes of variables, simple algebraic sums and multiplications, and a multiple convolution integral. These operations, albeit cumbersome, can be easily implemented numerically."

"It is demonstrated how to represent asymptotically mean stationary (AMS) random sources with values in standard spaces as mixtures of ergodic AMS sources. This an extension of the well known decomposition of stationary sources which has facilitated the generalization of prominent source coding theorems to arbitrary, not necessarily ergodic, stationary sources. Asymptotic mean stationarity generalizes the definition of stationarity and covers a much larger variety of real-world examples of random sources of practical interest. It is sketched how to obtain source coding and related theorems for arbitrary, not necessarily ergodic, AMS sources, based on the presented ergodic decomposition."

"Assume that one observes the $k$-th, $2k$-th, ...., $nk$-th value of a Markov chain $X_{1,h},...,X_{nk,h}$. That means we assume that a high frequency Markov chain runs in the background on a very fine time grid but that it is only observed on a coarser grid. This asymptotics reflects a set up occurring in the high frequency statistical analysis for financial data where diffusion approximations are used only for coarser time scales. In this paper we show that under appropriate conditions the L$_1$-distance between the joint distribution of the Markov chain and the distribution of the discretized diffusion limit converges to zero. The result implies that the LeCam deficiency distance between the statistical Markov experiment and its diffusion limit converges to zero. This result can be applied to Euler approximations for the joint distribution of diffusions observed at points $Delta, 2 Delta, ,,,, nDelta$. The joint distribution can be approximated by generating Euler approximations at the points $Delta k^{-1}, 2 Delta k^{-1}, ,,,, nDelta$. Our result implies that under our regularity conditions the Euler approximation is consistent for $n to infty$ if $nk^{-2}to 0$."

"The fluctuation relations have received considerable attention since their emergence and development in the 1990s. We present a summary of the main results and suggest ways to interpret this material. Starting with a consideration of the under-determined time evolution of a simple open system, formulated using continuous Markovian stochastic dy- namics, an expression for the entropy generated over a time interval is developed in terms of the probability of observing a trajectory associated with a prescribed driving protocol, and the probability of its time-reverse. This forms the basis for a general theoretical description of non-equilibrium thermodynamic pro- cesses. Having established a connection between entropy production and an inequivalence in probability for forward and time-reversed events, we proceed in the manner of Sekimoto and Seifert, in particular, to derive results in stochastic thermodynamics: a description of the evolution of a system between equilibrium states that ties in with well-established thermodynamic expectations. We derive fluctuation relations, state conditions for their validity, and illustrate their op- eration in some simple cases, thereby providing some introductory insight into the various celebrated symmetry relations that have emerged in this field."

"The results of Komlós, Major and Tusnády give optimal Wiener approximation of partial sums of i.i.d. random variables and provide an extremely powerful tool in probability and statistical inference. Recently Wu [Ann. Probab. 35 (2007) 2294–2320] obtained Wiener approximation of a class of dependent stationary processes with finite pth moments, 2 < p ≤ 4, with error term o(n1/p(log n)γ), γ > 0, and Liu and Lin [Stochastic Process. Appl. 119 (2009) 249–280] removed the logarithmic factor, reaching the Komlós–Major–Tusnády bound o(n1/p). No similar results exist for p > 4, and in fact, no existing method for dependent approximation yields an a.s. rate better than o(n1/4). In this paper we show that allowing a second Wiener component in the approximation, we can get rates near to o(n1/p) for arbitrary p > 2. This extends the scope of applications of the results essentially, as we illustrate it by proving new limit theorems for increments of stochastic processes and statistical tests for short term (epidemic) changes in stationary processes. Our method works under a general weak dependence condition covering wide classes of linear and nonlinear time series models and classical dynamical systems."

"In this paper, we study the generalization bound for an empirical process of samples independently drawn from an infinitely divisible (ID) distribution, which is termed as the ID empirical process. In particular, based on a martingale method, we develop deviation inequalities for the sequence of random variables of an ID distribution. By applying the obtained deviation inequalities, we then show the generalization bound for ID empirical process based on the annealed Vapnik- Chervonenkis (VC) entropy. Afterward, according to Sauer’s lemma, we get the generalization bound for ID empirical process based on the VC dimension. Finally, by using a resulted result bound, we analyze the asymptotic convergence of ID empirical process and show that the convergence rate of ID empirical process can reach O((frac{Lambda_mathcal{F}(2N)}{N})^{frac{1}{1.3}}) and it is faster than the results of the generic i.i.d. empirical process (Vapnik, 1999) "

"Classical Edgeworth expansions provide asymptotic correction terms to the Central Limit Theorem (CLT) up to an order that depends on the number of moments available. In this paper, we provide subsequent correction terms beyond those given by a standard Edgeworth expansion in the general case of regularly varying distributions with diverging moments (beyond the second). The subsequent terms can be expressed in a simple closed form in terms of certain special functions (Dawson’s integral and parabolic cylinder functions), and there are qualitative differences depending on whether the number of moments available is even, odd, or not an integer, and whether the distributions are symmetric or not. If the increments have an even number of moments, then additional logarithmic corrections must also be incorporated in the expansion parameter. An interesting feature of our correction terms for the CLT is that they become dominant outside the central region and blend naturally with known large-deviation asymptotics when these are applied formally to the spatial scales of the CLT."

"We present a set of high-probability inequalities that control the concentration of weighted averages of multiple (possibly uncountably many) simultaneously evolving and interdependent martingales. We also present a comparison inequality that bounds expectation of a convex function of martingale difference type variables by expectation of the same function of independent Bernoulli variables. This inequality is applied to derive a tighter analog of Hoeffding-Azuma inequality." --- For the record: I hate martingales.

Published version: J. Stat. Phys., 118 (2005): 555--588. Changes seem minor.

"In this paper we study ergodic properties of hidden Markov models with a generalized observation structure. In particular sufficient conditions for the existence of a unique invariant measure for the pair filter-observation are given. Furthermore, necessary and sufficient conditions for the existence of a unique invariant measure of the triple state-observation-filter are provided in terms of asymptotic stability in probability of incorrectly initialized filters. We also study the asymptotic properties of the filter and of the state estimator based on the observations as well as on the knowledge of the initial state. Their connection with minimal and maximal invariant measures is also studied."

When can you write your stochastic process as a recursive transformation of a sequence of IID noise variables?

Cute!

"We describe conditions on non-gradient drift diffusion Fokker-Planck equations for its solutions to converge to equilibrium with a uniform exponential rate in Wasserstein distance. This asymptotic behaviour is related to a functional inequality, which links the distance with its dissipation and ensures a spectral gap in Wasserstein distance. We give practical criteria for this inequality and compare it to classical ones. The key point is to quantify the contribution of the diffusion term to the rate of convergence, which to our knowledge is a novelty."

"Some asymptotic notions for random variables are discussed. In particular, different versions of O and o for sequences of random variables are studied. The results are elementary and more or less well-known, but collected here for future use and easy reference."

Should check how many of them aren't in Project Euclid before paying $$$, I guess. OTOH, perhaps think of it as a contribution to the Fund for Aged Probabilists?

Rigorous results on conditions under which mean field approximations become exact for large ERGMs, and consequently they end up looking like Erdos-Renyi graphs (at some effective density); these come from clever large-deviations arguments.  To over-simplify some pretty theorems, the cases are (1) all interactions between edges are positive, so introducing any positive density of edges at all tends to produce a blow-up towards a nearly complete graph (where edges are independent); or (2) all interactions are extremely weak, and consequently negligible in the large-scale limit.

"Cumulative broadband network traffic is often thought to be well modeled by fractional Brownian motion (FBM). However, some traffic measurements do not show an agreement with the Gaussian marginal distribution assumption. We show that if connection rates are modest relative to heavy tailed connection length distribution tails, then stable Lévy motion is a sensible approximation to cumulative traffic over a time period. If connection rates are large relative to heavy tailed connection length distribution tails, then FBM is the appropriate approximation. The results are framed as limit theorems for a sequence of cumulative input processes whose connection rates are varying in such a way as to remove or induce long range dependence."

"We survey an area of recent development, relating dynamics to theoretical computer science. We discuss the theoretical limits of simulation and computation of interesting quantities in dynamical systems. We will focus on central objects of the theory of dynamics, as invariant measures and invariant sets, showing that even if they can be computed with arbitrary precision in many interesting cases, there exists some cases in which they can not. We also explain how it is possible to compute the speed of convergence of ergodic averages (when the system is known exactly) and how this entails the computation of arbitrarily good approximations of points of the space having typical statistical behaviour (a sort of constructive version of the pointwise ergodic theorem)."

I am not sure whether there shouldn't be a chapter on diagrammatic methods in _Almost None_, because I don't really understand how what I learned about them as a physicist relates to actual probability theory.  I will come back to this one later.

I wonder if this could be used to give a more comprehensible sufficient condition for posterior convergence in the setting of my Bayes-is-a-special-case-of-replicator-dynamics paper?

From a quick skim, looks reasonably teachable.

Very nice. (ETA: Wait, we don't subscribe?!?)

"Various methods of summation for divergent series of real numbers have been generalized to analogous results for sums of i.i.d. random variables. The natural extension of results corresponding to Cesàro summation amounts to proving almost sure convergence of the Cesàro means. In the present paper we extend such results as well as weak laws and results on complete convergence to random fields, more specifically to random variables indexed by ℤ+2, the positive two-dimensional integer lattice points."

"A Central Limit Theorem is proved for linear random fields when sums are taken over finite disjoint union of rectangles. The approach does not rely upon the use of Beveridge Nelson decomposition and the conditions needed are similar to those given by Ibragimov for linear processes. When specializing this result to the case when sums are being taken over rectangles, an analogue of Ibragimov result is obtained with a lot of uniformity."

Garamond, in LaTeX

" We consider dynamical systems for which the spatial extension plays an important role. For these systems, the notions of attractor, ϵ-entropy and topological entropy per unit time and volume have been introduced previously. In this paper we use the notion of Kolmogorov complexity to introduce, for extended dynamical systems, a notion of complexity per unit time and volume which plays the same role as the metric entropy for classical dynamical systems. We introduce this notion as an almost sure limit on orbits of the system. Moreover we prove a kind of variational principle for this complexity."

"The paper focuses on general properties of parametric minimum contrast estimators. The quality of estimation is measured in terms of the rate function related to the contrast, thus allowing to derive exponential risk bounds invariant with respect to the detailed probabilistic structure of the model. This approach works well for small or moderate samples and covers the case of a misspecified parametric model. Another important feature of the presented bounds is that they may be used in the case when the parametric set is unbounded and non-compact. These bounds do not rely on the entropy or covering numbers and can be easily computed. The most important statistical fact resulting from the exponential bonds is a concentration inequality which claims that minimum contrast estimators concentrate with a large probability on the level set of the rate function. In typical situations, every such set is a root-n neighborhood of the parameter of interest."

"We continue our study of the linear response of a nonequilibrium system. This Part II concentrates on models of open and driven inertial dynamics but the structure and the interpretation of the result remain unchanged: the response can be expressed as a sum of two temporal correlations in the unperturbed system, one entropic, the other frenetic. The decomposition arises from the (anti)symmetry under time-reversal on the level of the nonequilibrium action. The response formula involves a statistical averaging over explicitly known observables but, in contrast with the equilibrium situation, they depend on the model dynamics in terms of an excess in dynamical activity. As an example, the Einstein relation between mobility and diffusion constant is modified by a correlation term between the position and the momentum of the particle."

Very clever, though they do not derive explicit rates. --- I wonder if this couldn't be shown as a consequence of general properties of very-weak-Bernoulli processes? The vwB property would seem to imply perfect simulation, but I am not sure about the converse.

"the rate function is exactly that of the averaged equation plus the fluctuating deviation which is a stochastic partial differential equation with small Gaussian perturbation"

Cute test for nonlinear autoregressions.

Vitali-Hahn is used at several points in _Almost None_, so it really needs a nice proof.

"time-average variance constants" = sum of covariances = 1/correlation time (roughly).

"The scale-free distribution of cluster sizes in continuous phase transitions is linked to the law of proportional effect. A numerical study of a two-dimensional Ising model suggests that a cluster size undergoes a multiplicative birth-death process. At the transition the ratio between birth and death rates approaches unity for large clusters, and the resulting steady state shows a power-law behavior. The percolation dynamic, on the other hand, yields a geometric phase transition without ergodicity breaking, where large-scale merging and splitting of clusters dominate the distribution. Instead of short-range birth-death jumps, the percolation transition is characterized by Lévi [sic] flights along the cluster-size axis."

Mine for examples for _Almost None_?

I really wish I had taken this class when I had the chance.

Looks cool.

"For a wide class of continuous-time Markov processes, including all irreducible hypoelliptic diffusions evolving on an open, connected subset of $RL^d$, the following are shown to be equivalent: (i) The process satisfies (a slightly weaker version of) the classical Donsker-Varadhan conditions; (ii) The transition semigroup of the process can be approximated by a finite-state hidden Markov model, in a strong sense in terms of an associated operator norm; (iii) The resolvent kernel of the process is `$v$-separable', that is, it can be approximated arbitrarily well in operator norm by finite-rank kernels."

"We extend the Wiener-Khinchin theorem to non-wide sense stationary (WSS) random processes, i.e. we prove that, under certain assumptions, the power spectral density (PSD) of any random process is equal to the Fourier transform of the time-averaged autocorrelation function"

"In this paper we prove maximal inequalities and study the functional central limit theorem for the partial sums of linear processes generated by dependent innovations. Due to the general weights these processes can exhibit long range dependence and the limiting distribution is a fractional Brownian motion. The proofs are based on new approximations by a linear process with martingale difference innovations."

I'm pretty impressed.

Cute.

not-quite-final PDF draft of the classic book

A good point, to steal next time I teach advanced prob. (N.B., I may be over-trained, but I find the measure-theoretic formalism for probability completely convincing. However, this seems like a good explanation for why our _intuition_ often doesn't work so nicely for probability spaces which aren't also complete metric spaces.)

Pedagogical note. Very slick.