"The topic of statistical inference for dynamical systems has been studied extensively across several fields. In this survey we focus on the problem of parameter estimation for non-linear dynamical systems. Our objective is to place results across distinct disciplines in a common setting and highlight opportunities for further research."

"We will show under minimal conditions on differentiability and dependence that the central limit theorem for quantiles holds and that the block bootstrap is weakly consistent. Under slightly stronger conditions, the bootstrap is strongly consistent. Without the differentiability condition, quantiles might have a non-normal asymptotic distribution and the bootstrap might fail."

"Let be a discrete or continuous-time Markov process with state space where is an arbitrary measurable set. Its transition semigroup is assumed to be additive with respect to the second component, i.e. is assumed to be a Markov additive process. In particular, this implies that the first component is also a Markov process. Markov random walks or additive functionals of a Markov process are special instances of Markov additive processes. In this paper, the process is shown to satisfy the following classical limit theorems:

(a) the central limit theorem,

(b) the local limit theorem,

(c) the one-dimensional Berry–Esseen theorem,

(d) the one-dimensional first-order Edgeworth expansion,

provided that we have with the expected order α with respect to the independent case (up to some ε > 0 for (c) and (d)). For the statements (b) and (d), a Markov nonlattice condition is also assumed as in the independent case. All the results are derived under the assumption that the Markov process has an invariant probability distribution π, is stationary and has the -spectral gap property (that is, (Xt)t∈ℕ is ρ-mixing in the discrete-time case). The case where is non-stationary is briefly discussed. As an application, we derive a Berry–Esseen bound for the M-estimators associated with ρ-mixing Markov chains."

"The multiple change-point problem is considered in the most general setting, where the only assumption made on the time-series distributions generating the data is that they are stationary ergodic. No modeling, independence or parametric assumptions are made. While the need for such a general setting is dictated by real applications, the problem of change-point estimation becomes a difficult unsupervised learning problem. In this work a novel algorithm for solving this problem is proposed, and it is shown to be asymptotically consistent under the general assumptions considered."

"New goodness-of-fit tests for Markovian models in time series analysis are developed which are based on the difference between a fully nonparametric estimate of the one-step transition distribution function of the observed process and that of the model class postulated under the null hypothesis. The model specification under the null allows for Markovian models, the transition mechanisms of which depend on an unknown vector of parameters and an unspecified distribution of i.i.d. innovations. Asymptotic properties of the test statistic are derived and the critical values of the test are found using appropriate bootstrap schemes. General properties of the bootstrap for Markovian processes are derived. A new central limit theorem for triangular arrays of weakly dependent random variables is obtained. For the proof of stochastic equicontinuity of multidimensional empirical processes, we use a simple approach based on an anisotropic tiling of the space. The finite-sample behavior of the proposed test is illustrated by some numerical examples and a real-data application is given."

"A new concept of qualitative robustness for plug-in estimators based on identically distributed possibly {em dependent} observations is introduced, and it is shown that Hampel's theorem for general metrics $d$ still holds. Since Hampel's theorem assumes the UGC property w.r.t. $d$, i.e. convergence in probability of the empirical probability measure to the true marginal distribution w.r.t. $d$ uniformly in the class of all admissible laws on the sample path space, this property is shown for a large class of strongly mixing laws for three different metrics $d$. For real-valued observations the UGC property is established for both the Kolomogorov $phi$-metric and the L'evy $psi$-metric, and for observations in a general locally compact and second countable Hausdorff space the UGC property is established for a certain metric generating the $psi$-weak topology. The key is a new uniform weak LLN for strongly mixing random variables. The latter is of independent interest and relies on Rio's maximal inequality."

"We derive a new representation for U- and V-statistics. Using this representation the asymptotic distribution of U- and V-statistics can be derived by a direct application of the Continuous Mapping Theorem. That novel approach not only encompasses most of the results on the asymptotic distribution known in literature, but also allows for the first time a unifying treatment of non-degenerate and degenerate U- and V-statistics. Moreover, it yields a new and powerful tool to derive the asymptotic distribution of very general U- and V-statistics based on long-memory sequences. This will be exemplified by several astonishing examples. In particular, we shall present an example where weak convergence of the U or V-statistic occurs at the rate a_n^3 when a_n is the rate of weak convergence of the empirical process. We also introduce the notion of asymptotic (non-) degeneracy which often appears in the presence of long-memory sequences."

"Hawaiian monk seals (Monachus schauinslandi) are endemic to the Hawaiian Islands and are the most endangered species of marine mammal that lives entirely within the jurisdiction of the United States. The species numbers around 1300 and has been declining owing, among other things, to poor juvenile survival which is evidently related to poor foraging success. Consequently, data have been collected recently on the foraging habitats, movements, and behaviors of monk seals throughout the Northwestern and main Hawaiian Islands. Our work here is directed to exploring a data set located in a relatively shallow offshore submerged bank (Penguin Bank) in our search of a model for a seal's journey. The work ends by fitting a stochastic differential equation (SDE) that mimics some aspects of the behavior of seals by working with location data collected for one seal. The SDE is found by developing a time varying potential function with two points of attraction. The times of location are irregularly spaced and not close together geographically, leading to some difficulties of interpretation. Synthetic plots generated using the model are employed to assess its reasonableness spatially and temporally. One aspect is that the animal stays mainly southwest of Molokai. The work led to the estimation of the lengths and locations of the seal's foraging trips."

"Two series of binary observations $x_1,x_1,...$ and $y_1,y_2,...$ are presented: at each time $ninN$ we are given $x_n$ and $y_n$. It is assumed that the sequences are generated independently of each other by two B-processes. We are interested in the question of whether the sequences represent a typical realization of two different processes or of the same one. We demonstrate that this is impossible to decide, in the sense that every discrimination procedure is bound to err with non-negligible frequency when presented with sequences from some B-processes. This contrasts earlier positive results on B-processes, in particular those showing that there are consistent $bar d$-distance estimates for this class of processes."

"Assume that $(X_t)_{tinZ}$ is a real valued time series admitting a common marginal density $f$ with respect to Lebesgue's measure. Donoho {it et al.} (1996) propose a near-minimax method based on thresholding wavelets to estimate $f$ on a compact set in an independent and identically distributed setting. The aim of the present work is to extend these results to general weak dependent contexts. Weak dependence assumptions are expressed as decreasing bounds of covariance terms and are detailed for different examples. The threshold levels in estimators $widehat f_n$ depend on weak dependence properties of the sequence $(X_t)_{tinZ}$ through the constant. If these properties are unknown, we propose cross-validation procedures to get new estimators. These procedures are illustrated via simulations of dynamical systems and non causal infinite moving averages. We also discuss the efficiency of our estimators with respect to the decrease of covariances bounds."

"Static forecasting of stationary and ergodic time series is considered, i.e., inference of the conditional expectation of the response variable at time zero given the infinite past. It is shown that the mean squared error of a combination of suitably defined localized least squares estimates converges to zero for all distributions where the response variable is square integrable."

Sweet. (Bayesian estimation seems like overkill here however, especially since predictions are just made from point estimates.)

"We study robust estimators of the mean of a probability measure $P$, called robust empirical mean estimators. This elementary construction is then used to revisit a problem of aggregation and a problem of estimator selection, extending these methods to not necessarily bounded collections of previous estimators.
We consider then the problem of robust $M$-estimation. We propose a slightly more complicated construction to handle this problem and, as examples of applications, we apply our general approach to least-squares density estimation, to density estimation with K"ullback loss and to a non-Gaussian, unbounded, random design and heteroscedastic regression problem.
Finally, we show that our strategy can be used when the data are only assumed to be mixing."

"A scheme is developed for estimating state-dependent drift and diffusion coefficients in a stochastic differential equation from time-series data. The scheme does not require to specify parametric forms for the drift and diffusion coefficients in advance. In order to perform the nonparametric estimation, a maximum likelihood method is combined with a concept based on a kernel density estimation. In order to deal with discrete observation or sparsity of the time-series data, a local linearization method is employed, which enables a fast estimation."

"Recent advances in single-molecule experiments show that various complex systems display nonergodic behavior. In this paper, we show how to test ergodicity and ergodicity breaking in experimental data. Exploiting the so-called dynamical functional, we introduce a simple test which allows us to verify ergodic properties of a real-life process. The test can be applied to a large family of stationary infinitely divisible processes. We check the performance of the test for various simulated processes and apply it to experimental data describing the motion of mRNA molecules inside live Escherichia coli cells. We show that the data satisfy necessary conditions for mixing and ergodicity. The detailed analysis is presented in the supplementary material."

"We present an efficient method for estimating variables and parameters of a given system of ordinary differential equations by adapting the model output to an observed time series from the (physical) process described by the model. The proposed method is based on (unconstrained) nonlinear optimization exploiting the particular structure of the relevant cost function. To illustrate the features and performance of the method, simulations are presented using chaotic time series generated by the Colpitts oscillator, the three-dimensional Hindmarsh-Rose neuron model, and a nine-dimensional extended Rössler system." --- Sounds like Hooker & Ramsay.

"Birth-death processes (BDPs) are continuous-time Markov chains that track the number of "particles" in a system over time. While widely used in population biology, genetics and ecology, statistical inference of the instantaneous particle birth and death rates remains largely limited to restrictive linear BDPs in which per-particle birth and death rates are constant. Researchers often observe the number of particles at discrete times, necessitating data augmentation procedures such as expectation-maximization (EM) to find maximum likelihood estimates. The E-step in the EM algorithm is available in closed-form for some linear BDPs, but otherwise previous work has resorted to approximation or simulation. Remarkably, the E-step conditional expectations can also be expressed as convolutions of computable transition probabilities for any general BDP with arbitrary rates. This important observation, along with a convenient continued fraction representation of the Laplace transforms of the transition probabilities, allows novel and efficient computation of the conditional expectations for all BDPs, eliminating the need for approximation or costly simulation. We use this insight to derive EM algorithms that yield maximum likelihood estimation for general BDPs characterized by various rate models, including generalized linear models. We show that our Laplace convolution technique outperforms competing methods when available and demonstrate a technique to accelerate EM algorithm convergence. Finally, we validate our approach using synthetic data and then apply our methods to estimation of mutation parameters in microsatellite evolution."

"A $d$-dimensional nonparametric additive regression model with dependent observations is considered. Using the marginal integration and the methods of wavelets, we develop a new adaptive estimator for a component of the additive regression function. Its asymptotic properties are investigated via the minimax approach under the $mathbb{L}_2$ risk over Besov balls. We prove that it attains a sharp rate of convergence, close to the one obtained in the one-dimensional case. In particular, it is both independent of $d$ and slightly deteriorated by the dependence of the observations."

"The growing availability of network data and of scientific interest in distributed systems has led to the rapid development of statistical models of network structure. Typically, however, these are models for the entire network, while the data consists only of a sampled sub-network. Parameters for the whole network, which is what is of interest, are estimated by applying the model to the sub-network. This assumes that the model is consistent under sampling, or, in terms of the theory of stochastic processes, that it defines a projective family. Focussing on the popular class of exponential random graph models (ERGMs), we show that this apparently trivial condition is in fact violated by many popular and scientifically appealing models, and that satisfying it drastically limits ERGM's expressive power. These results are actually special cases of more general ones about exponential families of dependent random variables, which we also prove. Using such results, we offer easily checked conditions for the consistency of maximum likelihood estimation in ERGMs, and discuss some possible constructive responses."

"We study the problem of parameter estimation for a univariate discretely observed ergodic diffusion process given as a solution to a stochastic differential equation. The estimation procedure we propose consists of two steps. In the first step, which is referred to as a smoothing step, we smooth the data and construct a nonparametric estimator of the invariant density of the process. In the second step, which is referred to as a matching step, we exploit a characterisation of the invariant density as a solution of a certain ordinary differential equation, replace the invariant density in this equation by its nonparametric estimator from the smoothing step in order to arrive at an intuitively appealing criterion function, and next define our estimator of the parameter of interest as a minimiser of this criterion function. In many interesting examples such an estimator will be computationally less intense than the more conventional estimators obtained through approximation of the likelihood function associated with the observations. Our main result shows that our estimator is $sqrt{n}$-consistent under suitable conditions. We also discuss a way of improving its asymptotic performance through a one-step Newton-Raphson type procedure."

Sounds delightful (and Banff is beautiful).

"In this article, the problem of semi-parametric inference on the parameters of a multidimensional Lévy process Lt with independent components based on the low-frequency observations of the corresponding time-changed Lévy process L_T_t, where T_t is a nonnegative, nondecreasing real-valued process independent of Lt, is studied. We show that this problem is closely related to the problem of composite function estimation that has recently gotten much attention in statistical literature. Under suitable identifiability conditions, we propose a consistent estimate for the Lévy density of Lt and derive the uniform as well as the pointwise convergence rates of the estimate proposed. Moreover, we prove that the rates obtained are optimal in a minimax sense over suitable classes of time-changed Lévy models. Finally, we present a simulation study showing the performance of our estimation algorithm in the case of time-changed Normal Inverse Gaussian (NIG) Lévy processes."

"A new negative result for nonparametric distribution estimation of binary ergodic processes is shown. The problem of estimation of distribution with any degree of accuracy is studied. Then it is shown that for any countable class of estimators there is a zero-entropy binary ergodic process that is inconsistent with the class of estimators. Our result is different from other negative results for universal forecasting scheme of ergodic processes."

"Let H0 denote the class of all real valued i.i.d. processes and H1 all other ergodic real valued stationary processes. In spite of the fact that these classes are not countably tight we give a strongly consistent sequential test for distinguishing between them."

"Let $(Y_k)$ be a stationary sequence on a probability space taking values in a standard Borel space. Consider the associated maximum likelihood estimator with respect to a parametrized family of Hidden Markov models such that the law of the observations $(Y_k)$ is not assumed to be described by any of the Hidden Markov models of this family. In this paper we investigate the consistency of this estimator in such mispecified models under mild assumptions."

"The Expectation Maximization (EM) algorithm is a versatile tool for model parameter estimation in latent data models. When processing large data sets or data stream however, EM becomes intractable since it requires the whole data set to be available at each iteration of the algorithm. In this contribution, a new generic online EM algorithm for model parameter inference in general Hidden Markov Model is proposed. This new algorithm updates the parameter estimate after a block of observations is processed (online). The convergence of this new algorithm is established, and the rate of convergence is studied showing the impact of the block size. An averaging procedure is also proposed to improve the rate of convergence. Finally, practical illustrations are presented as well as extensions to some online stochastic EM when Sequential Monte Carlo methods have to be used in combination, in order to make the E-step tractable."

"Mechanistic dynamic models of biochemical networks such as Ordinary Differential Equations (ODEs) contain unknown parameters like the reaction rate constants and the initial concentrations of the compounds. The large number of parameters as well as their nonlinear impact on the model responses hamper the determination of confidence regions for parameter estimates. At the same time, classical approaches translating the uncertainty of the parameters into confidence intervals for model predictions are hardly feasible. In this article it is shown that a so-called prediction profile likelihood yields reliable confidence intervals for model predictions, despite arbitrarily complex and high-dimensional shapes of the confidence regions for the estimated parameters. Prediction confidence intervals of the dynamic states allow a data-based observability analysis. The approach renders the issue of sampling a high-dimensional parameter space into evaluating one-dimensional prediction spaces."

"An estimation method is proposed for a wide variety of discrete time stochastic processes that have an intractable likelihood function but are otherwise conveniently specified by an integral transform such as the characteristic function, the Laplace transform or the probability generating function..." Not sure how often I'll have such a specification, but OK...

I'm 99% sure I read the preprint version.

"It is shown how the method of Fréchet differentiability can simplify the asymptotic derivations in an important range of robust inferential problems for stationary and related time series models. The uniform root-n consistency of the empirical distribution function for the Cramer von Mises norm under a weak mixing condition is indicated. Various regularity conditions naturally implemented and leading to the differentiability are discussed. A simulation study supplementing the theoretical discussion is included."

"This paper develops some theoretical results about the asymptotic behaviour of the empirical likelihood and the empirical profile likelihood statistics, which originate from fairly general estimating functions. The results accommodate, within a unified framework, various situations potentially occurring in a wide range of applications. For this reason, they are potentially useful in several contexts, such as, for example, in inference for dependent data. We provide examples showing that known findings in literature about the asymptotic behaviour of some empirical likelihood statistics in time series models can be derived as particular cases of our results."

"A modified concept of sufficiency, relevant in connection with statistical analysis of stochastic processes, is defined and its basic properties investigated. A method of prediction that applies when the probability structure is partly unknown is introduced and the method is shown to possess certain important invariance properties. The concept of an extreme model is defined and its probabilistic and statistical properties discussed. Existence of maximum likelihood estimators and predictors is established under weak regularity assumptions. For technical convenience, only discrete-valued stochastic processes are considered throughout the paper."

"Hsu et al.(2009) recently proposed an efficient, accurate spectral learning algorithm for Hidden Markov Models (HMMs). In this paper we relax their assumptions and prove a tighter finite-sample error bound for the case of Reduced-Rank HMMs, i.e., HMMs with low-rank transition matrices. Since rank-k RR-HMMs are a larger class of models than k-state HMMs while being equally efficient to work with, this relaxation greatly increases the learning algorithm's scope. In addition, we generalize the algorithm and bounds to models where multiple observations are needed to disambiguate state, and to models that emit multivariate real-valued observations. Finally we prove consistency for learning Predictive State Representations, an even larger class of models. Experiments on synthetic data and a toy video, as well as on difficult robot vision data, yield accurate models that compare favorably with alternatives in simulation quality and prediction accuracy."

"Levy processes play an important role in the stochastic process theory. However, since samples are non-i.i.d., statistical learning results based on the i.i.d. scenarios cannot be utilized to study the risk bounds for Levy processes. In this paper, we present risk bounds for non-i.i.d. samples drawn from Levy processes in the PAC-learning framework. In particular, by using a concentration inequality for infinitely divisible distributions, we first prove that the function of risk error is Lipschitz continuous with a high probability, and then by using a specific concentration inequality for Levy processes, we obtain the risk bounds for non-i.i.d. samples drawn from Levy processes without Gaussian components. Based on the resulted risk bounds, we analyze the factors that affect the convergence of the risk bounds and then prove the convergence."

Sounds like a crude form of indirect inference.

"Norbert Wiener really was that smart" dept.: long-term weather forecasting on the basis of deterministic dynamical models impossible because of limited precision observations and self-amplifying processes; but ergodic theory to the rescue for statistical forecasting; reconstruction of dynamical systems from sufficiently long trajectories (up to the ergodic component); linearization of nonlinear problems by projection into a higher-dimensional space; probably more, I'm not done reading it yet.

Growing-order Markov approximations to the non-Markov ergodic process.  Assumes all predictive probabilities are > 0, and a sort of continuity property for the predictive distribution.  This rules out many interesting processes (e.g., even process) --- could proofs be modified?

" kernel estimation of marginal densities and regression functions of stationary processes. It is shown that for a wide class of time series, with proper centering and scaling, the maximum deviations of kernel density and regression estimates are asymptotically Gumbel. Our results substantially generalize earlier ones which were obtained under independence or beta mixing assumptions. The asymptotic results can be applied to assess patterns of marginal densities or regression functions via the construction of simultaneous confidence bands for which one can perform goodness-of-fit tests"

Sequel to their recent Annals of Probability paper, where they use the same trick to get convergence for function classes in terms of the gap (a.k.a. "fat shattering") dimension.

The trick here is basically as follows: Every exchangeable sequence is a mixture of IID sequences (de Finetti), and which mixture component a realization is in, is an invariant So one demands only that the function f be well-estimated according to the component the sample is in, rather than over all of the mixture. If one could condition on the mixture component, one would lose no precision/accuracy compared to the IID situation; conditioning on (X_1, ... X_n) leaves some uncertainty about which component we're in, hence the cost. All of this should, I'd think, carry over straightforwardly to stationary mixing sequences, since (von Neumann) every stationary sequence is a mixture of ergodic sequences, and ergodic components are invariants of the motion. Perhaps a short paper?

"The joint distribution of a d-dimensional random field restricted to a box of size k can be estimated by looking at a realization in a box of size $n \gg k$ and computing the empirical distribution. This is done by sliding a box of size k around in the box of size n and computing frequencies. We show that when $k = k(n)$ grows as a function of n, then the total variation distance between this empirical distribution and the true distribution goes to 0 a.s. as $n \to \infty$ provided $k(n)^d \leq (\log n^d)/(H + \varepsilon)$ (where H is the entropy of the random field) and providing the random field satisfies a condition called quite weak Bernoulli with exponential rate. ... Marton and Shields have proved such results in one dimension and this paper is an attempt to extend their results to some extent to higher dimensions."

"We present the results of a simulation study into the properties of 12 different estimators of the Hurst parameter, $H$, or the fractional integration parameter, $d$, in long memory time series. We compare and contrast their performance on simulated Fractional Gaussian Noises and fractionally integrated series with lengths between 100 and 10,000 data points and $H$ values between 0.55 and 0.90 or $d$ values between 0.05 and 0.40. We apply all 12 estimators to the Campito Mountain data and estimate the accuracy of their estimates using the Beran goodness of fit test for long memory time series."

Huzzah!

"We study the nonparametric covariance estimation of a stationary Gaussian field X observed on a regular lattice. In the time series setting, some procedures ... achieve optimal model selection among autoregressive models. ... no such equivalent results of adaptivity in a spatial setting. By considering collections of Gaussian Markov random fields (GMRF) as approximation sets for the distribution of X, we introduce a novel model selection procedure for spatial fields. For all neighborhoods m in a given collection , this procedure first amounts to computing a covariance estimator of X within the GMRFs of neighborhood m. Then it selects a neighborhood ̂m by applying a penalization strategy. The so-defined method satisfies a nonasymptotic oracle-type inequality. If X is a GMRF, the procedure is also minimax adaptive to the sparsity of its neighborhood. More generally, the procedure is adaptive to the rate of approximation of the true distribution by GMRFs with growing neighborhoods."

"A framework of generalized linear point process models (glppm) much akin to glm for regression is developed where the intensity depends upon a linear predictor process through a known function. In the general framework the parameter space is a Banach space. Of particular interest is when the intensity depends on the history of the point process itself and possibly additional processes through a linear filter, and where the filter is parametrized by functions in a Sobolev space. We show two main results. First we show that for a special class of models the penalized maximum likelihood estimate is in a finite dimensional subspace of the parameter space -- if it exists. In practice we can find the estimate using a finite dimensional glppm framework. Second, for the general class of models we develop a descent algorithm in the Sobolev space. We conclude the paper by a discussion of additive model specifications."

"Robust nonparametric estimators for regression and autoregression are proposed for $\varphi$- and $\alpha$-mixing processes. Two families of $M$-type robust equivariant estimators are considered: (i) estimators based on kernel methods and (ii) estimators based on $k$-nearest neighbor kernel methods. Strong consistency of both families is proved under mild conditions. For the first class the result is true under no assumptions whatsoever on the distribution of the observations."

"In this paper we find nonasymptotic exponential upper bounds for the deviation in the ergodic theorem for families of homogeneous Markov processes. We find some sufficient conditions for geometric ergodicity uniformly over a parametric family. We apply this property to the nonasymptotic nonparametric estimation problem for ergodic diffusion processes."

"A new negative result for nonparametric estimation of ergodic processes is shown. In this paper we restrict the class of estimators to computable functions and study estimation of distribution of ergodic processes with any given accuracy. Then we show a single ergodic process that is inconsistent with all computable estimators." --- But how is say the Ornstein-Weiss procedure incomputable? Must read carefully.

for 462 and for STACS?

Exchangeable sequences, rather than anything interesting, but worth looking at for ideas?  ETA: Eh.

"A time-varying empirical spectral process indexed by classes of functions is defined for locally stationary time series. We derive weak convergence in a function space, and prove a maximal exponential inequality and a Glivenko--Cantelli-type convergence result. The results use conditions based on the metric entropy of the index class. In contrast to related earlier work, no Gaussian assumption is made. As applications, quasi-likelihood estimation, goodness-of-fit testing and inference under model misspecification are discussed. In an extended application, uniform rates of convergence are derived for local Whittle estimates of the parameter curves of locally stationary time series models."

" A sequence $x_1,...,x_n,...$ of discrete-valued observations is generated according to some unknown [measure] $\mu$. After observing each outcome, ... give the conditional probabilities of the next observation. ... $\mu$ [is in] an arbitrary but known class $C$ of stochastic process measures. We [want] predictors ... whose conditional probabilities converge (in some sense) to the [true] conditional probabilities if any $\mu\in C$ is chosen to generate the sequence. ... [C]haracteriz[e] the families $C$ for which such predictors exist ... a specific and simple form in which to look for a solution. ... if any predictor works, then there exists a Bayesian predictor, whose prior is discrete, and which works too. .... sufficient and necessary conditions for the existence of a predictor, in terms of topological characterizations of the family $C$, as well as in terms of local behaviour of the measures in $C$, which in some cases lead to procedures for constructing such predictors."

FHTAGN!

Today's seminar. Very cool.

Yay, me! (6.5 years from project inception to publication. This is faster than self-organization stuff, which took 8 years.)

"We propose an algorithm to estimate the common density $s$ of a stationary process $X_1,...,X_n$. We suppose that the process is either $\beta$ or $\tau$-mixing. We provide a model selection procedure based on a generalization of Mallows' $C_p$ and we prove oracle inequalities for the selected estimator under a few prior assumptions on the collection of models and on the mixing coefficients. We prove that our estimator is adaptive over a class of Besov spaces, namely, we prove that it achieves the same rates of convergence as in the i.i.d framework."

"time-average variance constants" = sum of covariances = 1/correlation time (roughly).

"We investigate the existence of bounded-memory consistent estimators of various statistical functionals. This question is resolved in the negative in a rather strong sense. We propose various bounded-memory approximations, using techniques from automata theory and stochastic processes. Some questions of potential interest are raised for future work."

In other words: you need an unbounded memory even to estimate the bias of coin-flips, even if you know that there are only just two possible biases. This is pretty sweet (in a thoroughly negative way). --- A lot of work is done here by the strength of the notion of consistency employed (see e.g. the counter-examples with finite but small limiting error probabilities), but this is the standard one!

"This paper presents a set of rate of uniform consistency results for kernel estimators of density functions and regressions functions. We generalize the existing literature by allowing for stationary strong mixing multivariate data with infinite support, and kernels with unbounded support, and general bandwidth sequences. These results are useful for semiparametric estimation based on a first stage nonparametric estimator."

"I hope [my philosophy of statistics is] sufficiently undogmatic not to imply that all those who may think rather differently from me are necessarily stupid. If at times I do seem dogmatic, it is because it is convenient to give my own views as unequivocally as possible."

"Given a discrete-valued sample X_1... X_n we wish to test whether it was generated by a process belonging to a family H_0, or it was generated by a process outside H_0. All process distributions are assumed stationary ergodic, and no further probabilistic or parametric assumptions are made. We require the Type I error of the test to be uniformly bounded, while the probability of Type II error has to tend to zero as the sample size increases. For this notion of consistency we provide necessary and sufficient conditions on the family H_0 for the existence of a consistent test. "