cshalizi + mixing   36

[1204.0608] Mixing times in evolutionary game dynamics
"Without mutation and migration, evolutionary dynamics ultimately leads to the extinction of all but one species. Such fixation processes are well understood and can be characterized analytically with methods from statistical physics. However, many biological arguments focus on stationary distributions in a mutation-selection equilibrium. Here, we address the equilibration time required to reach stationarity in the presence of mutation, this is known as the mixing time in the theory of Markov processes. We show that mixing times in evolutionary games have the opposite behaviour from fixation times when the intensity of selection increases: In coordination games with bistabilities, the fixation time decreases, but the mixing time increases. In coexistence games with metastable states, the fixation time increases, but the mixing time decreases. Our results are based on simulations and the WKB approximation of the master equation."
to:NB  evolutionary_game_theory  markov_models  mixing  re:do-institutions-evolve  stochastic_processes 
6 weeks ago by cshalizi
Relative Entropy and Exponential Deviation Bounds for General Markov Chains
"We develop explicit, general bounds for the prob- ability that the normalized partial sums of a function of a Markov chain on a general alphabet will exceed the steady-state mean of that function by a given amount. Our bounds combine simple information-theoretic ideas together with techniques from optimization and some fairly elementary tools from analysis. In one direction, we obtain a general bound for the important class of Doeblin chains; this bound is optimal, in the sense that in the special case of independent and identically distributed random variables it essentially reduces to the classical Hoeffding bound. In another direction, motivated by important problems in simulation, we develop a series of bounds in a form which is particularly suited to these problems, and which apply to the more general class of “geometrically ergodic” Markov chains."
to:NB  to_read  deviation_bounds  markov_models  stochastic_processes  via:ded-maxim  meyn.sean  kontoyiannis.ioannis  mixing  information_theory 
6 weeks ago by cshalizi
Ferré , Hervé , Ledoux : Limit theorems for stationary Markov processes with L2-spectral gap
"Let be a discrete or continuous-time Markov process with state space where is an arbitrary measurable set. Its transition semigroup is assumed to be additive with respect to the second component, i.e. is assumed to be a Markov additive process. In particular, this implies that the first component is also a Markov process. Markov random walks or additive functionals of a Markov process are special instances of Markov additive processes. In this paper, the process is shown to satisfy the following classical limit theorems:

(a) the central limit theorem,

(b) the local limit theorem,

(c) the one-dimensional Berry–Esseen theorem,

(d) the one-dimensional first-order Edgeworth expansion,

provided that we have with the expected order α with respect to the independent case (up to some ε > 0 for (c) and (d)). For the statements (b) and (d), a Markov nonlattice condition is also assumed as in the independent case. All the results are derived under the assumption that the Markov process has an invariant probability distribution π, is stationary and has the -spectral gap property (that is, (Xt)t∈ℕ is ρ-mixing in the discrete-time case). The case where is non-stationary is briefly discussed. As an application, we derive a Berry–Esseen bound for the M-estimators associated with ρ-mixing Markov chains."
to:NB  stochastic_processes  markov_models  ergodic_theory  mixing  statistical_inference_for_stochastic_processes 
6 weeks ago by cshalizi
[1203.5245] Qualitative robustness of statistical functionals under strong mixing
"A new concept of qualitative robustness for plug-in estimators based on identically distributed possibly {em dependent} observations is introduced, and it is shown that Hampel's theorem for general metrics $d$ still holds. Since Hampel's theorem assumes the UGC property w.r.t. $d$, i.e. convergence in probability of the empirical probability measure to the true marginal distribution w.r.t. $d$ uniformly in the class of all admissible laws on the sample path space, this property is shown for a large class of strongly mixing laws for three different metrics $d$. For real-valued observations the UGC property is established for both the Kolomogorov $phi$-metric and the L'evy $psi$-metric, and for observations in a general locally compact and second countable Hausdorff space the UGC property is established for a certain metric generating the $psi$-weak topology. The key is a new uniform weak LLN for strongly mixing random variables. The latter is of independent interest and relies on Rio's maximal inequality."
to:NB  statistics  mixing  statistical_inference_for_stochastic_processes  learning_theory 
9 weeks ago by cshalizi
[1202.4283] Fast rates in learning with dependent observations
"In this paper we tackle the problem of fast rates in time series forecasting from a statistical learning perspective. In a serie of papers (e.g. Meir 2000, Modha and Masry 1998, Alquier and Wintenberger 2012) it is shown that the main tools used in learning theory with iid observations can be extended to the prediction of time series. The main message of these papers is that, given a family of predictors, we are able to build a new predictor that predicts the series as well as the best predictor in the family, up to a remainder of order $1/sqrt{n}$. It is known that this rate cannot be improved in general. In this paper, we show that in the particular case of the least square loss, and under a strong assumption on the time series (phi-mixing) the remainder is actually of order $1/n$. Thus, the optimal rate for iid variables, see e.g. Tsybakov 2003, and individual sequences, see cite{lugosi} is, for the first time, achieved for uniformly mixing processes. We also show that our method is optimal for aggregating sparse linear combinations of predictors."

--- Assumes observations are in the interval [-B,B] and gets a bound which is O(B^3), and so useless for our purposes.
in_NB  learning_theory  mixing  ergodic_theory  re:your_favorite_dsge_sucks  re:XV_for_mixing  have_read 
february 2012 by cshalizi
[1201.4579] Limit theorems for stationary Markov processes with L2-spectral gap
"Let $(X_t, Y_t)_{tin T}$ be a discrete or continuous-time Markov process with state space $X times R^d$ where $X$ is an arbitrary measurable set. Its transition semigroup is assumed to be additive with respect to the second component, i.e. $(X_t, Y_t)_{tin T}$ is assumed to be a Markov additive process. In particular, this implies that the first component $(X_t)_{tin T}$ is also a Markov process. Markov random walks or additive functionals of a Markov process are special instances of Markov additive processes. In this paper, the process $(Y_t)_{tin T}$ is shown to satisfy the following classical limit theorems: (a) the central limit theorem, (b) the local limit theorem, (c) the one-dimensional Berry-Esseen theorem, (d) the one-dimensional first-order Edgeworth expansion, provided that we have sup{tin(0,1]cap T : E{pi,0}[|Y_t| ^{alpha}] < 1 with the expected order with respect to the independent case (up to some $varepsilon > 0$ for (c) and (d)). For the statements (b) and (d), a Markov nonlattice condition is also assumed as in the independent case. All the results are derived under the assumption that the Markov process $(X_t)_{tin T}$ has an invariant probability distribution $pi$, is stationary and has the $L^2(pi)$-spectral gap property (that is, $(X_t)tin N}$ is $rho$-mixing in the discrete-time case). The case where $(X_t)_{tin T}$ is non-stationary is briefly discussed. As an application, we derive a Berry-Esseen bound for the M-estimators associated with $rho$-mixing Markov chains."
in_NB  markov_models  stochastic_processes  central_limit_theorem  mixing  ergodic_theory 
january 2012 by cshalizi
A Bernstein type inequality and moderate deviations for weakly dependent sequences
"In this paper we present a Bernstein-type tail inequality for the maximum of partial sums of a weakly dependent sequence of random variables that is not necessarily bounded. The class considered includes geometrically and subgeometrically strongly mixing sequences. The result is then used to derive asymptotic moderate deviation results. Applications are given for classes of Markov chains, iterated Lipschitz models and functions of linear processes with absolutely regular innovations." Also: http://arxiv.org/abs/0902.0582
in_NB  to_read  re:XV_for_mixing  re:your_favorite_dsge_sucks  concentration_of_measure  deviation_bounds  mixing  ergodic_theory  stochastic_processes  moderate_deviations 
november 2011 by cshalizi
[1110.2529] The Generalization Ability of Online Algorithms for Dependent Data
"We study the generalization performance of arbitrary online learning algorithms trained on samples coming from a dependent source of data. We show that the generalization error of any stable online algorithm concentrates around its regret--an easily computable statistic of the online performance of the algorithm--when the underlying ergodic process is $beta$- or $phi$-mixing. We show high probability error bounds assuming the loss function is convex, and we also establish sharp convergence rates and deviation bounds for strongly convex losses and several linear prediction problems such as linear and logistic regression, least-squares SVM, and boosting on dependent data. In addition, our results have straightforward applications to stochastic optimization with dependent data, and our analysis requires only martingale convergence arguments; we need not rely on more powerful statistical tools such as empirical process theory."
in_NB  learning_theory  individual_sequence_prediction  ergodic_theory  mixing  re:growing_ensemble_project  re:XV_for_mixing  stability_of_learning  concentration_of_measure  have_read  re:your_favorite_dsge_sucks 
october 2011 by cshalizi
[1107.1794] Some Aspects of Modeling Dependence in Copula-based Markov chains
"Dependence coefficients have been widely studied for Markov processes defined by a set of transition probabilities and an initial distribution. This work clarifies some aspects of the theory of dependence structure of Markov chains generated by copulas... relationship between the notions of geometric ergodicity and geometric {rho}-mixing ... for a large number of well known copulas, such as Clayton, Gumbel or Student, these notions are equivalent. Some of the results published in the last years appear to be redundant if one takes into account this fact. We apply this equivalence to show that any mixture of Clayton, Gumbel or Student copulas generate both geometrically ergodic and geometric {rho}-mixing stationary Markov chains, answering in this way an open question in the literature. We shall also point out that a sufficient condition for {rho}-mixing, used in the literature, actually implies Doeblin recurrence."
markov_models  copulas  ergodic_theory  mixing  statistics  in_NB 
july 2011 by cshalizi
[0711.3924] Moderate deviations for stationary sequences of bounded random variables
"In this paper we derive the moderate deviation principle for stationary sequences of bounded random variables under martingale-type conditions. Applications to functions of $phi$-mixing sequences, contracting Markov chains, expanding maps of the interval, and symmetric random walks on the circle are given."
ergodic_theory  large_deviations  stochastic_processes  mixing  in_NB 
july 2011 by cshalizi
"Frechet differentiability in statistical inference for time series" - Statistical Methods & Applications, Volume 19, Number 4
"It is shown how the method of Fréchet differentiability can simplify the asymptotic derivations in an important range of robust inferential problems for stationary and related time series models. The uniform root-n consistency of the empirical distribution function for the Cramer von Mises norm under a weak mixing condition is indicated. Various regularity conditions naturally implemented and leading to the differentiability are discussed. A simulation study supplementing the theoretical discussion is included."
asymptotics  time_series  statistical_inference_for_stochastic_processes  estimation  statistics  mixing 
october 2010 by cshalizi
[0912.0338] Correlation Decay in Random Decision Networks
We consider a decision network on an undirected graph in which each node corresponds to a decision variable, and each node and edge of the graph is associated with a reward function whose value depends only on the variables of the corresponding nodes. The goal is to construct a decision vector which maximizes the total reward. This decision problem encompasses a variety of models, including maximum-likelihood inference in graphical models (Markov Random Fields), combinatorial optimization on graphs, economic team theory and statistical physics. The network is endowed with a probabilistic structure in which costs are sampled from a distribution. Our aim is to identify sufficient conditions to guarantee average-case polynomiality of the underlying optimization problem. ... we prove that [in some case we can] find near optimal solutions with high probability in a decentralized way ... based on the network exhibiting a correlation decay (long-range independence) property."
collective_cognition  networks  markov_models  via:ded-maxim  mixing  computational_complexity  re:social-networks-as-sensor-networks 
august 2010 by cshalizi
Steif: Consistent estimation of joint distributions for sufficiently mixing random fields
"The joint distribution of a d-dimensional random field restricted to a box of size k can be estimated by looking at a realization in a box of size $n \gg k$ and computing the empirical distribution. This is done by sliding a box of size k around in the box of size n and computing frequencies. We show that when $k = k(n)$ grows as a function of n, then the total variation distance between this empirical distribution and the true distribution goes to 0 a.s. as $n \to \infty$ provided $k(n)^d \leq (\log n^d)/(H + \varepsilon)$ (where H is the entropy of the random field) and providing the random field satisfies a condition called quite weak Bernoulli with exponential rate. ... Marton and Shields have proved such results in one dimension and this paper is an attempt to extend their results to some extent to higher dimensions."
statistics  information_theory  random_fields  estimation  density_estimation  entropy  mixing  to_read  statistical_inference_for_stochastic_processes 
may 2010 by cshalizi
An Outline of Ergodic Theory - Cambridge University Press
"This informal introduction provides a fresh perspective on isomorphism theory, which is the branch of ergodic theory that explores the conditions under which two measure preserving systems are essentially equivalent. It contains a primer in basic measure theory, proofs of fundamental ergodic theorems, and material on entropy, martingales, Bernoulli processes, and various varieties of mixing. Original proofs of classic theorems - including the Shannon–McMillan–Breiman theorem, the Krieger finite generator theorem, and the Ornstein isomorphism theorem - are presented by degrees, together with helpful hints that encourage the reader to develop the proofs on their own. Hundreds of exercises and open problems are also included"
books:noted  ergodic_theory  mixing  dynamical_systems  stochastic_processes  coveted 
april 2010 by cshalizi
A Road to Randomness in Physical Systems
library has this but in off-site storage; request.
"There are many ways of introducing the concept of probability in classical, deterministic physics. This volume is concerned with one approach, known as 'the method of arbitrary functions', which was first considered by Poincare. ... proceeds by associating some uncertainty to our knowledge of both the initial conditions and the values of the physical constants that characterize the evolution of a physical system. By modeling this uncertainty by a probability density distribution ... analyze how the state of the system evolves through time. ... examples as diverse as bouncing balls, simple and coupled harmonic oscillators, integrable systems (such as spinning tops), planetary motion, and billiards. ... study the speed of convergence for solutions in order to determine the practical relevance of the method of arbitrary functions for specific examples. ... new results on convergence, and tractable upper bounds are derived"
probability  dynamical_systems  chaos  foundations_of_statistics  ergodic_theory  statistical_mechanics  mixing  books:noted  poincare  classical_mechanics  coveted 
april 2010 by cshalizi
[1004.1602] Hilbertian decorrelations
Isn't this just \rho-mixing? But the tensorization results may be useful.
hilbert_space  statistical_mechanics  probability  ising_model  mixing 
april 2010 by cshalizi
Large Deviations, Fluctuations and Shrinking Intervals
"This paper concerns the statistical properties of hyperbolic diffeomorphisms. We obtain a large deviation result with respect to slowly shrinking intervals for a large class of Hölder continuous functions. In case of time reversal symmetry, we obtain a corresponding version of the Fluctuation Theorem."
large_deviations  dynamical_systems  ergodic_theory  mixing  in_NB  statistical_mechanics  non-equilibrium 
june 2009 by cshalizi
Uniform Convergence Rates for Kernel Estimation with Dependent Data
"This paper presents a set of rate of uniform consistency results for kernel estimators of density functions and regressions functions. We generalize the existing literature by allowing for stationary strong mixing multivariate data with infinite support, and kernels with unbounded support, and general bandwidth sequences. These results are useful for semiparametric estimation based on a first stage nonparametric estimator."
kernel_estimators  mixing  statistical_inference_for_stochastic_processes  statistics  density_estimation  regression  hansen.bruce 
june 2009 by cshalizi
[0811.1888] Central Limit Theorem and the Bootstrap for U-Statistics of Strongly Mixing Data
"The asymptotic normality of U-statistics has so far been proved for iid data and under various mixing conditions such as absolute regularity, but not for strong mixing. We use a coupling technique introduced in 1983 by Bradley to prove a new generalized covariance inequality similar to Yoshihara's. It follows from the Hoeffding-decomposition and this inequality that U-statistics of strongly mixing observations converge to a normal limit if the kernel of the U-statistic fulfills some moment and continuity conditions.
The validity of the bootstrap for U-statistics has until now only been established in the case of iid data (see Bickel and Freedman). For mixing data, Politis and Romano proposed the circular block bootstrap, which leads to a consistent estimation of the sample mean's distribution. We extend these results to U-statistics of weakly dependent data and prove a CLT for the circular block bootstrap version of U-statistics under absolute regularity and strong mixing. We also calculate a rate of convergence for the bootstrap variance estimator of a U-statistic and give some simulation results."
central_limit_theorem  statistics  bootstrap  mixing  ergodic_theory  stochastic_processes 
june 2009 by cshalizi
[0906.0791] Instability statistics and mixing rates
"We claim that looking at probability distributions of emph{finite time} largest Lyapunov exponents, and more precisely studying their large deviation properties, yields an extremely powerful technique to get quantitative estimates of polynomial decay rates of time correlations and Poincar'e recurrences in the -quite delicate- case of dynamical systems with weak chaotic properties."
dynamical_systems  large_deviations  poincare_recurrence  mixing  ergodic_theory  in_NB  to_read  re:XV_for_mixing 
june 2009 by cshalizi

related tags

asymptotics  books:noted  bootstrap  central_limit_theorem  chains_with_complete_connections  chaos  classical_mechanics  collective_cognition  computational_complexity  concentration_of_measure  convergence_of_stochastic_processes  copulas  coveted  debowski.lukasz  density_estimation  deviation_bounds  dynamical_systems  empirical_processes  entropy  ergodic_theory  estimation  evolutionary_game_theory  foundations_of_statistics  hansen.bruce  have_read  hilbert_space  histograms  individual_sequence_prediction  information_theory  interview  in_NB  ising_model  kernel_estimators  kith_and_kin  kontorovich.aryeh  kontoyiannis.ioannis  large_deviations  learning_theory  lives_of_the_scientists  markov_models  martingales  mathematics  meyn.sean  mixing  model_selection  moderate_deviations  networks  non-equilibrium  path_dependence  poincare  poincare_recurrence  probability  random_fields  re:almost_none  re:AoS_project  re:do-institutions-evolve  re:growing_ensemble_project  re:social-networks-as-sensor-networks  re:XV_for_mixing  re:your_favorite_dsge_sucks  recurrence_times  regression  rosenblatt.murray  self-promotion  stability_of_learning  statistical_inference_for_stochastic_processes  statistical_mechanics  statistics  stochastic_processes  time_series  to:NB  to_read  via:ded-maxim  via:slaniel 

Copy this bookmark:



description:


tags: