"We compute analytically, for large N, the probability P(N+,N) that a N×N Wishart random matrix has N+ eigenvalues exceeding a threshold Nζ, including its large deviation tails. This probability plays a benchmark role when performing the principal component analysis of a large empirical data set. We find that P(N+,N)≈exp⁡[-βN2ψζ(N+/N)], where β is the Dyson index of the ensemble and ψζ(κ) is a rate function that we compute explicitly in the full range 0≤κ≤1 and for any ζ. The rate function ψζ(κ) displays a quadratic behavior modulated by a logarithmic singularity close to its minimum κ⋆(ζ). This is shown to be a consequence of a phase transition in an associated Coulomb gas problem. The variance Δ(N) of the number of relevant components is also shown to grow universally (independent of ζ) as Δ(N)∼(βπ2)-1ln⁡N for large N."

"New measures are proposed for mutual and causal dependence between two time series, based on information theoretical ideas. The measure of mutual dependence is shown to be the sum of the measure of unidirectional causal dependence from the first time series to the second, the measure of unidirectional causal dependence from the second to the first, and the measure of instantaneous causal dependence. The measures are applicable to any kind of time series: continuous, discrete, or categorical."

"Gibbs measures are the main object of study in equilibrium statistical mechanics, and are used in many other contexts, including dynamical systems and ergodic theory, and spatial statistics. However, in a large number of natural instances one encounters measures that are not of Gibbsian form. We present here a number of examples of such non-Gibbsian measures, and discuss some of the underlying mathematical and physical issues to which they gave rise."

"We consider the evolution of large but finite populations on arbitrary fitness landscapes. We describe the evolutionary process by a Markov, Moran process. We show that to $mathcal O(1/N)$, the time-averaged fitness is lower for the finite population than it is for the infinite population. We also show that fluctuations in the number of individuals for a given genotype can be proportional to a power of the inverse of the mutation rate. Finally, we show that the probability for the system to take a given path through the fitness landscape can be non-monotonic in system size."

"For any stationary $mZ^d$-Gibbs measure that satisfies strong spatial mixing, we obtain sequences of upper and lower approximations that converge to its entropy. In the case, $d=2$, these approximations are efficient in the sense that the approximations are accurate to within $epsilon$ and can be computed in time polynomial in $1/epsilon$."

"The irreversibility of a stationary time series can be quantified using the Kullback-Leibler divergence (KLD) between the probability of observing the series and the probability of observing the time-reversed series. Moreover, this KLD is a tool to estimate entropy production from stationary trajectories since it gives a lower bound to the entropy production of the physical process generating the series. In this paper we introduce analytical and numerical techniques to estimate the KLD between time series generated by several stochastic dynamics with a finite number of states. We examine the accuracy of our estimators for a specific example, a discrete flashing ratchet, and investigate how close the KLD is to the entropy production depending on the number of degrees of freedom of the system that are sampled in the trajectories."

"Without mutation and migration, evolutionary dynamics ultimately leads to the extinction of all but one species. Such fixation processes are well understood and can be characterized analytically with methods from statistical physics. However, many biological arguments focus on stationary distributions in a mutation-selection equilibrium. Here, we address the equilibration time required to reach stationarity in the presence of mutation, this is known as the mixing time in the theory of Markov processes. We show that mixing times in evolutionary games have the opposite behaviour from fixation times when the intensity of selection increases: In coordination games with bistabilities, the fixation time decreases, but the mixing time increases. In coexistence games with metastable states, the fixation time increases, but the mixing time decreases. Our results are based on simulations and the WKB approximation of the master equation."

"About forty years ago it was realized by several researchers that the essential features of certain objects of Probability theory, notably Gaussian processes and limit theorems, may be better understood if they are considered in settings that do not impose structures extraneous to the problems at hand. For instance, in the case of sample continuity and boundedness of Gaussian processes, the essential feature is the metric or pseudometric structure induced on the index set by the covariance structure of the process, regardless of what the index set may be. This point of view ultimately led to the Fernique-Talagrand majorizing measure characterization of sample boundedness and continuity of Gaussian processes, thus solving an important problem posed by Kolmogorov. Similarly, separable Banach spaces provided a minimal setting for the law of large numbers, the central limit theorem and the law of the iterated logarithm, and this led to the elucidation of the minimal (necessary and/or sufficient) geometric properties of the space under which different forms of these theorems hold. However, in light of renewed interest in Empirical processes, a subject that has considerably influenced modern Statistics, one had to deal with a non-separable Banach space, namely $mathcal{L}_{infty}$. With separability discarded, the techniques developed for Gaussian processes and for limit theorems and inequalities in separable Banach spaces, together with combinatorial techniques, led to powerful inequalities and limit theorems for sums of independent bounded processes over general index sets, or, in other words, for general empirical processes."

"We develop explicit, general bounds for the prob- ability that the normalized partial sums of a function of a Markov chain on a general alphabet will exceed the steady-state mean of that function by a given amount. Our bounds combine simple information-theoretic ideas together with techniques from optimization and some fairly elementary tools from analysis. In one direction, we obtain a general bound for the important class of Doeblin chains; this bound is optimal, in the sense that in the special case of independent and identically distributed random variables it essentially reduces to the classical Hoeffding bound. In another direction, motivated by important problems in simulation, we develop a series of bounds in a form which is particularly suited to these problems, and which apply to the more general class of “geometrically ergodic” Markov chains."

"Let be a discrete or continuous-time Markov process with state space where is an arbitrary measurable set. Its transition semigroup is assumed to be additive with respect to the second component, i.e. is assumed to be a Markov additive process. In particular, this implies that the first component is also a Markov process. Markov random walks or additive functionals of a Markov process are special instances of Markov additive processes. In this paper, the process is shown to satisfy the following classical limit theorems:

(a) the central limit theorem,

(b) the local limit theorem,

(c) the one-dimensional Berry–Esseen theorem,

(d) the one-dimensional first-order Edgeworth expansion,

provided that we have with the expected order α with respect to the independent case (up to some ε > 0 for (c) and (d)). For the statements (b) and (d), a Markov nonlattice condition is also assumed as in the independent case. All the results are derived under the assumption that the Markov process has an invariant probability distribution π, is stationary and has the -spectral gap property (that is, (Xt)t∈ℕ is ρ-mixing in the discrete-time case). The case where is non-stationary is briefly discussed. As an application, we derive a Berry–Esseen bound for the M-estimators associated with ρ-mixing Markov chains."

"This paper discusses particle filtering in general hidden Markov models (HMMs) and presents novel theoretical results on the long-term stability of bootstrap-type particle filters. More specifically, we establish that the asymptotic variance of the Monte Carlo estimates produced by the bootstrap filter is uniformly bounded in time. On the contrary to most previous results of this type, which in general presuppose that the state space of the hidden state process is compact (an assumption that is rarely satisfied in practice), our very mild assumptions are satisfied for a large class of HMMs with possibly non-compact state space. In addition, we derive a similar time uniform bound on the asymptotic Lp error. Importantly, our results hold for misspecified models, i.e. we do not at all assume that the data entering into the particle filter originate from the model governing the dynamics of the particles or not even from an HMM."

"This article is concerned with the Axelrod model, a stochastic process which similarly to the voter model includes social influence, but unlike the voter model also accounts for homophily. Each vertex of the network of interactions is characterized by a set of F cultural features, each of which can assume q states. Pairs of adjacent vertices interact at a rate proportional to the number of features they share, which results in the interacting pair having one more cultural feature in common. The Axelrod model has been extensively studied during the past ten years, based on numerical simulations and simple mean-field treatments, while there is a total lack of analytical results for the spatial model itself. Simulation results for the one-dimensional system led physicists to formulate the following conjectures. When the number of features F and the number of states q both equal two, or when the number of features exceeds the number of states, the system converges to a monocultural equilibrium in the sense that the number of cultural domains rescaled by the population size converges to zero as the population goes to infinity. In contrast, when the number of states exceeds the number of features, the system freezes in a highly fragmented configuration in which the ultimate number of cultural domains scales like the population size. In this article, we prove analytically for the one-dimensional system convergence to a monocultural equilibrium in terms of clustering when F = q = 2, as well as fixation to a highly fragmented configuration when the number of states is sufficiently larger than the number of features. Our first result also implies clustering of the one-dimensional constrained voter model."

"Network modeling plays a critical role in identifying statistical regularities and structural principles common to many systems. The large majority of recent modeling approaches are connectivity driven, in the sense that the structural pattern of the network is at the basis of the mechanisms ruling the network formation. Connectivity driven models necessarily provide a time-aggregated representation that may fail to describe the instantaneous and fluctuating dynamics of many networks. We address this challenge by defining the activity potential, a time invariant function characterizing the agents' interactions in real-world networks and constructing an activity driven model capable of encoding the instantaneous time description of the network dynamics. The model provides an explanation of structural features such as the presence of hubs, which simply originate from the heterogeneous activity of agents. Additionally, we find that diffusive processes in highly dynamical networks can be described analytically in terms of the activity potential, allowing a quantitative discussion of the biases induced by the time-aggregated network representation in the analysis of dynamical processes in evolving networks."

"A large deviations principle is established for the joint law of the empirical measure and the flow measure of a renewal Markov process on a finite graph. We do not assume any bound on the arrival times, allowing heavy tailed distributions. In particular, the rate functional is in general degenerate (it has a nontrivial set of zeros) and not strictly convex. These features show a behavior highly different from what one may guess with a heuristic Donsker-Varadhan analysis of the problem."

"The existence of invariant Borel probability measures for random dynamical systems on complete metric spaces is proved under assumptions that the systems have countably many maps and admit finite Markov partitions such that the resulting Markov systems are uniformly continuous and contractive, and satisfy some integrability condition in the infinite case. A one-to-one map between these measures and equilibrium states associated with such systems is established. Some properties of the map and the measures are given."

"This paper studies systems of particles following independent random walks and subject to annihilation, binary branching, coalescence, and deaths. In the case without annihilation, such systems have been studied in our 2005 paper "Branching-coalescing particle systems". The case with annihilation is considerably more difficult, mainly as a consequence of the non-monotonicity of such systems and a more complicated duality. Nevertheless, we show that adding annihilation does not significantly change the long-time behavior of the process and in fact, systems with annihilation can be obtained by thinning systems without annihilation."

"The problem of signal detection using sparse, faint information is closely related to a variety of contemporary statistical problems, including the control of false-discovery rate, and classification using very high-dimensional data. Each problem can be solved by conducting a large number of simultaneous hypothesis tests, the properties of which are readily accessed under the assumption of independence. In this paper we address the case of dependent data, in the context of higher criticism methods for signal detection. Short-range dependence has no first-order impact on performance, but the situation changes dramatically under strong dependence. There, although higher criticism can continue to perform well, it can be bettered using methods based on differences of signal values or on the maximum of the data. The relatively inferior performance of higher criticism in such cases can be explained in terms of the fact that, under strong dependence, the higher criticism statistic behaves as though the data were partitioned into very large blocks, with all but a single representative of each block being eliminated from the dataset."

"Stochastic partial differential equations driven by Poisson random measures (PRM) have been proposed as models for many different physical systems, where they are viewed as a refinement of a corresponding noiseless partial differential equations (PDE). A systematic framework for the study of probabilities of deviations of the stochastic PDE from the deterministic PDE is through the theory of large deviations. The goal of this work is to develop the large deviation theory for small Poisson noise perturbations of a general class of deterministic infinite dimensional models. Although the analogous questions for finite dimensional systems have been well studied, there are currently no general results in the infinite dimensional setting. This is in part due to the fact that in this setting solutions may have little spatial regularity, and thus classical approximation methods for large deviation analysis become intractable. The approach taken here, which is based on a variational representation for nonnegative functionals of general PRM, reduces the proof of the large deviation principle to establishing basic qualitative properties for controlled analogues of the underlying stochastic system. As an illustration of the general theory, we consider a particular system that models the spread of a pollutant in a waterway."

"Noether's theorem links the symmetries of a quantum system with its conserved quantities, and is a cornerstone of quantum mechanics. Here we prove a version of Noether's theorem for Markov processes. In quantum mechanics, an observable commutes with the Hamiltonian if and only if its expected value remains constant in time for every state. For Markov processes that no longer holds, but an observable commutes with the Hamiltonian if and only if both its expected value and standard deviation are constant in time for every state."
--- For "Hamiltonian" of a Markov process, read "generator".

"A Feynman formula is a representation of a solution of an initial (or initial-boundary) value problem for an evolution equation (or, equivalently, a representation of the semigroup resolving the problem) by a limit of $n$-fold iterated integrals of some elementary functions as $ntoinfty$. In this note we obtain some Feynman formulae for a class of semigroups associated with Feller processes. Finite dimensional integrals in the Feynman formulae give approximations for functional integrals in some Feynman--Kac formulae corresponding to the underlying processes. Hence, these Feynman formulae give an effective tool to calculate functional integrals with respect to probability measures generated by these Feller processes and, in particular, to obtain simulations of Feller processes."

"Non-Gaussian concentration estimates are obtained for invariant probability measures of reversible Markov processes. We show that the functional inequalities approach combined with a suitable Lyapunov condition allows us to circumvent the classical Lipschitz assumption of the observables. Our method is general and covers diffusions as well as pure-jump Markov processes on unbounded spaces."

"Sequential Monte Carlo methods which involve sequential importance sampling and resampling are shown to provide a versatile approach to computing probabilities of rare events. By making use of martingale representations of the sequential Monte Carlo estimators, we show how resampling weights can be chosen to yield logarithmically efficient Monte Carlo estimates of large deviation probabilities for multidimensional Markov random walks."

"In this note, we prove a conditionally centered version of the quenched weak invariance principle under the Hannan condition, for stationary processes. In the course, we obtain a (new) construction of the fact that any stationary process may be seen as a functional of a Markov chain."

"It is demonstrated how to represent asymptotically mean stationary (AMS) random sources with values in standard spaces as mixtures of ergodic AMS sources. This an extension of the well known decomposition of stationary sources which has facilitated the generalization of prominent source coding theorems to arbitrary, not necessarily ergodic, stationary sources. Asymptotic mean stationarity generalizes the definition of stationarity and covers a much larger variety of real-world examples of random sources of practical interest. It is sketched how to obtain source coding and related theorems for arbitrary, not necessarily ergodic, AMS sources, based on the presented ergodic decomposition."

"The augmented GARCH model is a unification of numerous extensions of the popular and widely used ARCH process. It was introduced by Duan and besides ordinary (linear) GARCH processes, it contains exponential GARCH, power GARCH, threshold GARCH, asymmetric GARCH, etc. In this paper, we study the probabilistic structure of augmented $mathrm {GARCH}(1,1)$ sequences and the asymptotic distribution of various functionals of the process occurring in problems of statistical inference. Instead of using the Markov structure of the model and implied mixing properties, we utilize independence properties of perturbed GARCH sequences to directly reduce their asymptotic behavior to the case of independent random variables. This method applies for a very large class of functionals and eliminates the fairly restrictive moment and smoothness conditions assumed in the earlier theory. In particular, we derive functional CLTs for powers of the augmented GARCH variables, derive the error rate in the CLT and obtain asymptotic results for their empirical processes under nearly optimal conditions."

"This paper presents some new concentration inequalities for Feynman-Kac particle processes. We analyze different types of stochastic particle models, including particle profile occupation measures, genealogical tree based evolution models, particle free energies, as well as backward Markov chain particle models. We illustrate these results with a series of topics related to computational physics and biology, stochastic optimization, signal processing and Bayesian statistics, and many other probabilistic machine learning algorithms. Special emphasis is given to the stochastic modeling, and to the quantitative performance analysis of a series of advanced Monte Carlo methods, including particle filters, genetic type island models, Markov bridge models, and interacting particle Markov chain Monte Carlo methodologies."

"Two series of binary observations $x_1,x_1,...$ and $y_1,y_2,...$ are presented: at each time $ninN$ we are given $x_n$ and $y_n$. It is assumed that the sequences are generated independently of each other by two B-processes. We are interested in the question of whether the sequences represent a typical realization of two different processes or of the same one. We demonstrate that this is impossible to decide, in the sense that every discrimination procedure is bound to err with non-negligible frequency when presented with sequences from some B-processes. This contrasts earlier positive results on B-processes, in particular those showing that there are consistent $bar d$-distance estimates for this class of processes."

"In this paper, the forgetting of the initial distribution for a non-ergodic Hidden Markov Models (HMM) is studied. A new set of conditions is proposed to establish the forgetting property of the filter, which significantly extends all the existing results. Both a pathwise-type convergence of the total variation distance of the filter started from two different initial distributions, and a convergence in expectation are considered. The results are illustrated using generic models of non-ergodic HMM and extend all the results known so far."

"In this paper, we obtain some uniform laws of large numbers and functional central limit theorems for sequential empirical measure processes indexed by classes of product functions satisfying appropriate Vapnik-Chervonenkis properties."

"It is shown how to compute effective and functional connection matrices (eCMs and fCMs) from anatomical CMs (aCMs) and corresponding strength-of-connection matrices (sCMs) using propagator methods in which neural interactions play the role of scatterings. This analysis demonstrates how network effects dress the bare propagators (the sCMs) to yield effective propagators (the eCMs) that can be used to compute the covariances customarily used to define fCMs. The results incorporate excitatory and inhibitory connections, multiple structures and populations, asymmetries, time delays, and measurement effects. They can also be postprocessed in the same manner as experimental measurements for direct comparison with data and thereby give insights into the role of coarse-graining, thresholding, and other effects in determining the structure of CMs. The spatiotemporal results show how to generalize CMs to include time delays and how natural network modes give rise to long-range coherence at resonant frequencies. The results are demonstrated using tractable analytic cases via neural field theory of cortical and corticothalamic systems. These also demonstrate close connections between the structure of CMs and proximity to critical points of the system, highlight the importance of indirect links between brain regions and raise the possibility of imaging specific levels of indirect connectivity. Aside from the results presented explicitly here, the expression of the connections among aCMs, sCMs, eCMs, and fCMs in terms of propagators opens the way for propagator theory to be further applied to analysis of connectivity."

"Computing smoothing distributions, the distributions of one or more states conditional on past, present, and future observations is a recurring problem when operating on general hidden Markov models. The aim of this paper is to provide a foundation of particle-based approximation of such distributions and to analyze, in a common unifying framework, different schemes producing such approximations. In this setting, general convergence results, including exponential deviation inequalities and central limit theorems, are established. In particular, time uniform bounds on the marginal smoothing error are obtained under appropriate mixing conditions on the transition kernel of the latent chain. In addition, we propose an algorithm approximating the joint smoothing distribution at a cost that grows only linearly with the number of particles."

"Static forecasting of stationary and ergodic time series is considered, i.e., inference of the conditional expectation of the response variable at time zero given the infinite past. It is shown that the mean squared error of a combination of suitably defined localized least squares estimates converges to zero for all distributions where the response variable is square integrable."

"Assume that one observes the $k$-th, $2k$-th, ...., $nk$-th value of a Markov chain $X_{1,h},...,X_{nk,h}$. That means we assume that a high frequency Markov chain runs in the background on a very fine time grid but that it is only observed on a coarser grid. This asymptotics reflects a set up occurring in the high frequency statistical analysis for financial data where diffusion approximations are used only for coarser time scales. In this paper we show that under appropriate conditions the L$_1$-distance between the joint distribution of the Markov chain and the distribution of the discretized diffusion limit converges to zero. The result implies that the LeCam deficiency distance between the statistical Markov experiment and its diffusion limit converges to zero. This result can be applied to Euler approximations for the joint distribution of diffusions observed at points $Delta, 2 Delta, ,,,, nDelta$. The joint distribution can be approximated by generating Euler approximations at the points $Delta k^{-1}, 2 Delta k^{-1}, ,,,, nDelta$. Our result implies that under our regularity conditions the Euler approximation is consistent for $n to infty$ if $nk^{-2}to 0$."

"We explore the limits of the autoregressive (AR) sieve bootstrap, and show that its applicability extends well beyond the realm of linear time series as has been previously thought. In particular, for appropriate statistics, the AR-sieve bootstrap is valid for stationary processes possessing a general Wold-type autoregressive representation with respect to a white noise; in essence, this includes all stationary, purely nondeterministic processes, whose spectral density is everywhere positive. Our main theorem provides a simple and effective tool in assessing whether the AR-sieve bootstrap is asymptotically valid in any given situation. In effect, the large-sample distribution of the statistic in question must only depend on the first and second order moments of the process; prominent examples include the sample mean and the spectral density. As a counterexample, we show how the AR-sieve bootstrap is not always valid for the sample autocovariance even when the underlying process is linear."

"The fluctuation relations have received considerable attention since their emergence and development in the 1990s. We present a summary of the main results and suggest ways to interpret this material. Starting with a consideration of the under-determined time evolution of a simple open system, formulated using continuous Markovian stochastic dy- namics, an expression for the entropy generated over a time interval is developed in terms of the probability of observing a trajectory associated with a prescribed driving protocol, and the probability of its time-reverse. This forms the basis for a general theoretical description of non-equilibrium thermodynamic pro- cesses. Having established a connection between entropy production and an inequivalence in probability for forward and time-reversed events, we proceed in the manner of Sekimoto and Seifert, in particular, to derive results in stochastic thermodynamics: a description of the evolution of a system between equilibrium states that ties in with well-established thermodynamic expectations. We derive fluctuation relations, state conditions for their validity, and illustrate their op- eration in some simple cases, thereby providing some introductory insight into the various celebrated symmetry relations that have emerged in this field."

Sweet. (Bayesian estimation seems like overkill here however, especially since predictions are just made from point estimates.)

"Let $(X_t, Y_t)_{tin T}$ be a discrete or continuous-time Markov process with state space $X times R^d$ where $X$ is an arbitrary measurable set. Its transition semigroup is assumed to be additive with respect to the second component, i.e. $(X_t, Y_t)_{tin T}$ is assumed to be a Markov additive process. In particular, this implies that the first component $(X_t)_{tin T}$ is also a Markov process. Markov random walks or additive functionals of a Markov process are special instances of Markov additive processes. In this paper, the process $(Y_t)_{tin T}$ is shown to satisfy the following classical limit theorems: (a) the central limit theorem, (b) the local limit theorem, (c) the one-dimensional Berry-Esseen theorem, (d) the one-dimensional first-order Edgeworth expansion, provided that we have sup{tin(0,1]cap T : E{pi,0}[|Y_t| ^{alpha}] < 1 with the expected order with respect to the independent case (up to some $varepsilon > 0$ for (c) and (d)). For the statements (b) and (d), a Markov nonlattice condition is also assumed as in the independent case. All the results are derived under the assumption that the Markov process $(X_t)_{tin T}$ has an invariant probability distribution $pi$, is stationary and has the $L^2(pi)$-spectral gap property (that is, $(X_t)tin N}$ is $rho$-mixing in the discrete-time case). The case where $(X_t)_{tin T}$ is non-stationary is briefly discussed. As an application, we derive a Berry-Esseen bound for the M-estimators associated with $rho$-mixing Markov chains."

"Using the renewal approach we prove exponential inequalities for additive functionals and empirical processes of ergodic Markov chains, thus obtaining counterparts of inequalities for sums of independent random variables. The inequalities do not require functions of the chain to be bounded and moreover have all the constants accessible whenever the usual drift condition holds, which is crucial for practical applications e.g. in MCMC algorithms."

We consider Markov chain with spectral gap in $L^2$ space. Assume that $f$ is a bounded function. Then the probabilities of large deviations of average along trajectory satisfy Hoeffding's-type inequalities. These bounds depend only on the stationary mean, spectral gap and the end-points of support of $f$.

"We study weak convergence of empirical processes of dependent data, indexed by classes of functions. We obtain results that are especially suitable for data arising from dynamical systems and Markov chains, where the Central Limit Theorem for partial sums is commonly derived via the spectral gap technique. Our results apply, e.g. to the empirical process of ergodic torus automorphisms."

"We propose four approaches to estimating the directed information rate between a pair of jointly stationary ergodic processes with the help of universal probability assignments. The four approaches yield estimators with different merits such as nonnegativity and boundedness. We establish consistency of these estimators in various senses and derive near-optimal rates of convergence in the minimax sense under mild conditions. The estimators carry over directly to estimating other information measures of stationary ergodic processes, such as entropy rate and mutual information rate, and provide alternatives to classical approaches in the existing literature. Guided by the theoretical results, we use context tree weighting as the vehicle for the implementations of the proposed estimators. Experiments on synthetic and real data are presented, demonstrating the potential of the proposed schemes in practice and the efficacy of directed information estimation as a tool for detecting and measuring causality and delay."

"We describe an adaptive context tree weighting (ACTW) algorithm, as an extension to the standard context tree weighting (CTW) algorithm. Unlike the standard CTW algorithm, which weights all observations equally regardless of the depth, ACTW gives increasing weight to more recent observations, aiming to improve performance in cases where the input sequence is from a non-stationary distribution. Data compression results show ACTW variants improving over CTW on merged files from standard compression benchmark tests while never being significantly worse on any individual file."

In a word: no. In more words, http://bactra.org/weblog/857.html

"We study large deviation properties of systems of weakly interacting particles modeled by Itô stochastic differential equations (SDEs). It is known under certain conditions that the corresponding sequence of empirical measures converges, as the number of particles tends to infinity, to the weak solution of an associated McKean–Vlasov equation. We derive a large deviation principle via the weak convergence approach. The proof, which avoids discretization arguments, is based on a representation theorem, weak convergence and ideas from stochastic optimal control. The method works under rather mild assumptions and also for models described by SDEs not of diffusion type. To illustrate this, we treat the case of SDEs with delay."

"We suggest new recursive formulas to compute the exact value of the Kullback-Leibler distance (KLD) between two general Hidden Markov Trees (HMTs). For homogeneous HMTs with regular topology, such as homogeneous Hidden Markov Models (HMMs), we obtain a closed-form expression for the KLD when no evidence is given. We generalize our recursive formulas to the case of HMMs conditioned on the observable variables. Our proposed formulas are validated through several numerical examples in which we compare the exact KLD value with Monte Carlo estimations."

"The large-deviation method allows to characterize an ergodic counting process in terms of a thermodynamic frame where a free energy function determines the asymptotic non-stationary statistical properties of its fluctuations. Here, we study this formalism through a statistical mechanics approach, i.e., with an auxiliary counting process that maximizes an entropy function associated to the thermodynamic potential. We show that the realizations of this auxiliary process can be obtained after applying a conditional measurement scheme to the original ones, providing is this way an alternative measurement interpretation of the thermodynamic approach. General results are obtained for renewal counting processes, i.e., those where the time intervals between consecutive events are independent and defined by a unique waiting time distribution. The underlying statistical mechanics is controlled by the same waiting time distribution, rescaled by an exponential decay measured by the free energy function. A scale invariance, shift closure, and intermittence phenomena are obtained and interpreted in this context. Similar conclusions apply for non-renewal processes when the memory between successive events is induced by a stochastic waiting time distribution."

"We consider measure-valued processes X_t that solve the following martingale problem: For a given initial measure X_0, and for all smooth, compactly supported test functions phi,
X_t(phi)= X_0 (phi)+ (1/2)int_0^t X_s(Delta phi)ds + thetaint_0^t X_s(phi) ds - int_0^t X_s(L_s phi) ds + M_t(phi). Here L_s(x) is the local time density process associated with X_t, and M_t(phi) is a martingale with quadratic variation [M(phi)]_t=int_0^t X_s(phi^2) ds. Such processes arise as scaling limits of SIR epidemic models. We show that there exist critical values theta_c(d) in (0,infty) for dimensions d=2,3 such that if theta> theta_c(d), then the solution survives forever with positive probability, but if theta< theta_c(d), then the solution dies out in finite time with probability 1. For d=1 we prove that the solution dies out almost surely for all values of theta. We also show that in dimensions d=2,3 the process dies out locally almost surely for any value of theta, that is, for any compact set K, the process X_t (K)=0 eventually."

"Recent advances in single-molecule experiments show that various complex systems display nonergodic behavior. In this paper, we show how to test ergodicity and ergodicity breaking in experimental data. Exploiting the so-called dynamical functional, we introduce a simple test which allows us to verify ergodic properties of a real-life process. The test can be applied to a large family of stationary infinitely divisible processes. We check the performance of the test for various simulated processes and apply it to experimental data describing the motion of mRNA molecules inside live Escherichia coli cells. We show that the data satisfy necessary conditions for mixing and ergodicity. The detailed analysis is presented in the supplementary material."

"We construct non-Gaussian processes that vary continuously in space and time with nonseparable covariance functions. Starting from a general and flexible way of constructing valid nonseparable covariance functions through mixing over separable covariance functions, the resulting models are generalized by allowing for outliers as well as regions with larger variances. We induce this through scale mixing with separate positive-valued processes. Smooth mixing processes are applied to the underlying correlated processes in space and in time, thus leading to regions in space and time of increased spread. An uncorrelated mixing process on the nugget effect accommodates outliers. Posterior and predictive Bayesian inference with these models is implemented through a Markov chain Monte Carlo sampler. An application to temperature data in the Basque country illustrates the potential of this model in the identification of outliers and regions with inflated variance, and shows that this improves the predictive performance."

17 pp. review paper. "Infectious disease remains, despite centuries of work to control and mitigate its effects, a major problem facing humanity. This paper reviews the mathematical modelling of infectious disease epidemics on networks, starting from the simplest Erdos-Renyi random graphs, and building up structure in the form of correlations, heterogeneity and preference, paying particular attention to the links between random graph theory, percolation and dynamical systems representing transmission. Finally, the problems posed by networks with a large number of short closed looks are discussed."

"The results of Komlós, Major and Tusnády give optimal Wiener approximation of partial sums of i.i.d. random variables and provide an extremely powerful tool in probability and statistical inference. Recently Wu [Ann. Probab. 35 (2007) 2294–2320] obtained Wiener approximation of a class of dependent stationary processes with finite pth moments, 2 < p ≤ 4, with error term o(n1/p(log n)γ), γ > 0, and Liu and Lin [Stochastic Process. Appl. 119 (2009) 249–280] removed the logarithmic factor, reaching the Komlós–Major–Tusnády bound o(n1/p). No similar results exist for p > 4, and in fact, no existing method for dependent approximation yields an a.s. rate better than o(n1/4). In this paper we show that allowing a second Wiener component in the approximation, we can get rates near to o(n1/p) for arbitrary p > 2. This extends the scope of applications of the results essentially, as we illustrate it by proving new limit theorems for increments of stochastic processes and statistical tests for short term (epidemic) changes in stationary processes. Our method works under a general weak dependence condition covering wide classes of linear and nonlinear time series models and classical dynamical systems."

To many math symbols to copy the abstract. Shorter: iterating randomly chosen Lipschitz maps can lead to time-averges converging to a heavy-tailed distribution.

"In this paper we present a Bernstein-type tail inequality for the maximum of partial sums of a weakly dependent sequence of random variables that is not necessarily bounded. The class considered includes geometrically and subgeometrically strongly mixing sequences. The result is then used to derive asymptotic moderate deviation results. Applications are given for classes of Markov chains, iterated Lipschitz models and functions of linear processes with absolutely regular innovations." Also: http://arxiv.org/abs/0902.0582

"The present paper has two goals. First to present a natural example of a new class of random fields which are the variable neighborhood random fields. The example we consider is a partially observed nearest neighbor binary Markov random field. The second goal is to establish sufficient conditions ensuring that the variable neighborhoods are almost surely finite. We discuss the relationship between the almost sure finiteness of the interaction neighborhoods and the presence/absence of phase transition of the underlying Markov random field. In the case where the underlying random field has no phase transition we show that the finiteness of neighborhoods depends on a specific relation between the noise level and the Dobrushin coefficient. The case in which there is phase transition is addressed in the frame of the ferromagnetic Ising model. We prove that the existence of infinite interaction neighborhoods depends on the phase. The first result has a probabilistic proof using a Kalikow type decomposition of a Glauber dynamics associated to the field. The second result is proved using cluster expansion."

"Large and moderate deviation probabilities play an important role in many applied areas, such as insurance and risk analysis. This paper studies the exact moderate and large deviation asymptotics in non-logarithmic form for linear processes with independent innovations. The linear processes we analyze are general and therefore they include the long memory case. We give an asymptotic representation for probability of the tail of the normalized sums and specify the zones in which it can be approximated either by a standard normal distribution or by the marginal distribution of the innovation process. The results are then applied to regression estimates, moving averages, fractionally integrated processes, linear processes with regularly varying exponents and functions of linear processes. We also consider the computation of value at risk and expected shortfall, fundamental quantities in risk theory and finance."

"In this paper, we study the generalization bound for an empirical process of samples independently drawn from an infinitely divisible (ID) distribution, which is termed as the ID empirical process. In particular, based on a martingale method, we develop deviation inequalities for the sequence of random variables of an ID distribution. By applying the obtained deviation inequalities, we then show the generalization bound for ID empirical process based on the annealed Vapnik- Chervonenkis (VC) entropy. Afterward, according to Sauer’s lemma, we get the generalization bound for ID empirical process based on the VC dimension. Finally, by using a resulted result bound, we analyze the asymptotic convergence of ID empirical process and show that the convergence rate of ID empirical process can reach O((frac{Lambda_mathcal{F}(2N)}{N})^{frac{1}{1.3}}) and it is faster than the results of the generic i.i.d. empirical process (Vapnik, 1999) "

"We prove that uniqueness of the stationary chain compatible with an attractive regular probability kernel is equivalent to the following two assertions for this chain: (1) it is a finitary coding of an i.i.d. process with discrete state space, (2) the concentration of measure holds at exponential rate. We show in particular that if a stationary chain is uniquely defined by a kernel which is continuous and attractive, then this chain can be sampled using a coupling-from-the-past algorithm. For the original Bramson-Kalikow model we further prove that there exists a unique compatible chain if and only if the chain is a finitary coding of a finite alphabet i.i.d. process. Finally, we obtain some partial results on conditions for phase transition for general chains of infinite order."

Published version: J. Stat. Phys., 118 (2005): 555--588. Changes seem minor.

"In this paper we study ergodic properties of hidden Markov models with a generalized observation structure. In particular sufficient conditions for the existence of a unique invariant measure for the pair filter-observation are given. Furthermore, necessary and sufficient conditions for the existence of a unique invariant measure of the triple state-observation-filter are provided in terms of asymptotic stability in probability of incorrectly initialized filters. We also study the asymptotic properties of the filter and of the state estimator based on the observations as well as on the knowledge of the initial state. Their connection with minimal and maximal invariant measures is also studied."

When can you write your stochastic process as a recursive transformation of a sequence of IID noise variables?

"We explore the limits of the autoregressive (AR) sieve bootstrap, and show that its applicability extends well beyond the realm of linear time series as has been previously thought. In particular, for appropriate statistics, the AR-sieve bootstrap is valid for stationary processes possessing a general Wold-type autoregressive representation with respect to a white noise; in essence, this includes all stationary, purely nondeterministic processes, whose spectral density is everywhere positive. Our main theorem provides a simple and effective tool in assessing whether the AR-sieve bootstrap is asymptotically valid in any given situation. In effect, the large-sample distribution of the statistic in question must only depend on the first and second order moments of the process; prominent examples include the sample mean and the spectral density. As a counterexample, we show how the AR-sieve bootstrap is not always valid for the sample autocovariance even when the underlying process is linear."

Cute!

"A pivotal problem in Bayesian nonparametrics is the construction of prior distributions on the space M(V) of probability measures on a given domain V. In principle, such distributions on the infinite-dimensional space M(V) can be constructed from their finite-dimensional marginals—the most prominent example being the construction of the Dirichlet process from finite-dimensional Dirichlet distributions. This approach is both intuitive and applicable to the construction of arbitrary distributions on M(V), but also hamstrung by a number of technical difficulties. We show how these difficulties can be resolved if the domain V is a Polish topological space, and give a representation theorem directly applicable to the construction of any probability distribution on M(V) whose first moment measure is well-defined. The proof draws on a projective limit theorem of Bochner, and on properties of set functions on Polish spaces to establish countable additivity of the resulting random probabilities."

"We construct an infinitely exchangeable process on the set $cate$ of subsets of the power set of the natural numbers $mathbb{N}$ via a Poisson point process with mean measure $Lambda$ on the power set of $mathbb{N}$. Each $Eincate$ has a least monotone cover in $catf$, the collection of monotone subsets of $cate$, and every monotone subset maps to an undirected graph $Gincatg$, the space of undirected graphs with vertex set $mathbb{N}$. We show a natural mapping $caterightarrowcatfrightarrowcatg$ which induces an infinitely exchangeable measure on the projective system $catg^{rest}$ of graphs $catg$ under permutation and restriction mappings given an infinitely exchangeable family of measures on the projective system $cate^{rest}$ of subsets with permutation and restriction maps. We show potential connections of this process to applications in cluster analysis, machine learning, classification and Bayesian inference."

"We introduce a theoretical framework for performing statistical hypothesis testing simultaneously over a fairly general, possibly uncountably infinite, set of null hypotheses. This extends the standard statistical setting for multiple hypotheses testing, which is restricted to a finite set. This work is motivated by numerous modern applications where the observed signal is modeled by a stochastic process over a continuum. As a measure of type I error, we extend the concept of false discovery rate (FDR) to this setting. The FDR is defined as the average ratio of the measure of two random sets, so that its study presents some challenge and is of some intrinsic mathematical interest. Our main result shows how to use the $p$-value process to control the FDR at a nominal level, either under arbitrary dependence of $p$-values, or under the assumption that the finite dimensional distributions of the $p$-value process have positive correlations of a specific type (weak PRDS). Both cases generalize existing results established in the finite setting, the latter one leading to a less conservative procedure. The interest of this approach is demonstrated in several non-parametric examples: testing the mean/signal in a Gaussian white noise model, testing the intensity of a Poisson process and testing the c.d.f. of i.i.d. random variables. Conceptually, an interesting feature of the setting advocated here is that it focuses directly on the intrinsic hypothesis space associated with a testing model on a random process, without referring to an arbitrary discretization."

"A discrete agent-based model on a periodic lattice of arbitrary dimension is considered. Agents move to nearest-neighbor sites by a motility mechanism accounting for general interactions, which may include volume exclusion. The partial differential equation describing the average occupancy of the agent population is derived systematically. A diffusion equation arises for all types of interactions and is nonlinear except for the simplest interactions. In addition, multiple species of interacting subpopulations give rise to an advection-diffusion equation for each subpopulation. This work extends and generalizes previous specific results, providing a construction method for determining the transport coefficients in terms of a single conditional transition probability, which depends on the occupancy of sites in an influence region. These coefficients characterize the diffusion of agents in a crowded environment in biological and physical processes."

"We prove that bootstrap type Monte Carlo particle filters approximate the optimal nonlinear filter in a time average sense uniformly with respect to the time horizon when the signal is ergodic and the particle system satisfies a tightness property. The latter is satisfied without further assumptions when the signal state space is compact, as well as in the noncompact setting when the signal is geometrically ergodic and the observations satisfy additional regularity assumptions."

"Let H0 denote the class of all real valued i.i.d. processes and H1 all other ergodic real valued stationary processes. In spite of the fact that these classes are not countably tight we give a strongly consistent sequential test for distinguishing between them."

"Let $(U_n(t))_{tinR^d}$ be the empirical process associated to an $R^d$-valued stationary process $(X_i)_{ige 0}$. We give general conditions, which only involve processes $(f(X_i))_{ige 0}$ for a restricted class of functions $f$, under which weak convergence of $(U_n(t))_{tinR^d}$ can be proved. This is particularly useful when dealing with data arising from dynamical systems or functional of Markov chains. This result improves those of [DDV09] and [DD11], where the technique was first introduced, and provides new applications."

"Originally published in 1981" - still relevant?

I think this is the only time I have seen the LIL actually _used_ for anything.

"A new stochastic mode-elimination procedure is introduced for a class of deterministic systems. Under assumptions of ergodicity and mixing, the procedure gives closed-form stochastic models for the slow variables in the limit of infinite separation of timescales. The procedure is applied to the truncated Burgers–Hopf (TBH) system as a test case where the separation of timescale is only approximate. It is shown that the stochastic models reproduce exactly the statistical behaviour of the slow modes in TBH when the fast modes are artificially accelerated to enforce the separation of timescales. It is shown that this operation of acceleration only has a moderate impact on the bulk statistical properties of the slow modes in TBH. As a result, the stochastic models are sound for the original TBH system."