UPenn STAT928 notes

"In this paper we develop a new class of double generalized linear models, introducing a random effect component in the link function describing the linear predictor related to the precision parameter. This is a useful procedure to take into account extra variability and also to make the model more robust. The Bayesian paradigm is adopted to make inference in this class of models. Samples of the joint posterior distribution are draw using standard MCMC procedures. Finally, we illustrate this algorithm by considering simulated and real data sets."

"Online learning constitutes a mathematical framework to analyze sequential decision making problems in adversarial environments. The learner repeatedly chooses an action, the environment responds with an outcome, and then the learner receives a reward for the played action. The goal of the learner is to maximize his total reward. However, there are situations in which, in addition to maximizing the cumulative reward, there are some additional constraints/goals on the sequence of decisions that must be satisfied by the learner. For example, in textit{online marketing}, simultaneously maximizing the cumulative reward and the number of buyers to take advantage of word-of-mouth advertising for future marketing seems to be a more ambitious goal than only maximizing cumulative reward. As another example, learning from costly expert advice captures more realistic settings than the original setting in applications such as routing in networks with power constraint. In this paper we study an extension to the online learning where the learner aims to maximize the total reward given that some additional constraints need to be satisfied. We propose Lagrangian exponentially weighted average (textbf{LEWA}) algorithm, an efficient algorithm to solve constrained online learning, which is a primal dual variant of the well known exponentially weighted average algorithm and inspired by the theory of Lagrangian method in constrained optimization. We establish the regret and the violation of the constraint bounds in full information and bandit feedback models."

"Generalized linear models have become a. standard class of models for data analysts. However in some applications, heterogeneity in samples is too great to be explained by the simple variance function implicit in such models. Utilizing a. two parameter exponential family which is overdispersed relative to a specified one parameter exponential family enables the creation of classes of overdispersed generalized linear models (OGLM’s) which are analytically attractive."

Over the last decade, however, empirical work on selection in insurance markets has gained considerable momentum, and a fairly extensive (and still growing) empirical literature on the topic has emerged. This research has found that adverse selection exists in some insurance markets but not in others. It has also uncovered examples of markets that exhibit “advantageous selection”—a phenomenon not considered by the original theory, and one that has different consequences for equilibrium insurance allocation and optimal public policy than the classical case of adverse selection. Researchers have also taken steps toward estimating the welfare consequences of detected selection and of potential public policy interventions."

"This paper gives an extensive documentation of applications of finite-dimensional nonlinear complementarity problems in engineering and equilibrium modeling. For most applications, we describe the problem briefly, state the defining equations of the model, and give functional
expressions for the complementarity formulations. The goal of this documentation is threefold: (i) to summarize the essential applications of the nonlinear complementarity problem known to date, (ii) to provide a basis for the continued research on the nonlinear complementarity problem, and (iii) to supply a broad collection of realistic complementarity problems for use in algorithmic experimentation and other studies.

The original equilibrium model of Irving Fisher

"We study competitive economy equilibrium computation. We show that, for the first time, the equilibrium sets of the following two markets: 1. A mixed Fisher and Arrow-Debreu market with homogeneous and log-concave utility func-tions; 2. The Fisher and Arrow-Debreu markets with several classes of concave non-homogeneous utility functions; are convex or log-convex. Furthermore, an equilibrium can be computed as convex optimization by an interior-point algorithm in polynomial time."

NB: The difference between Fisher and Arrow-Debreu equilibria

"In competitive asset markets, consumers make intertemporal choices in an uncertain environment. Their attitudes toward risk, production opportunities, and the nature of trades that they can enter into determine equilibrium quantities and the prices of assets that are traded. The intertemporal choice problem of a consumer in an uncertain environment yields restrictions for the behavior of individual consumption over time as well as determining the form of the asset pricing function used to price random payoffs."

"This paper discusses the motivations and principles regarding learning algorithms for deep architectures, in particular those exploiting as building blocks unsupervised learning of single-layer models such as Restricted Boltzmann Machines, used to construct deeper models such as Deep Belief Networks."

"Legendre transformation is at the heart of the duality principle of flat information geometries. Let us explain intuitively this transformation using geometric reasoning. (We shall
skip proofs and concentrate on the essence of the transformation instead."

"Machine Learning competitions such as the Netflix Prize have proven reasonably successful as a method of "crowdsourcing" prediction tasks. But these competitions have a number of weaknesses, particularly in the incentive structure they create for the participants. We propose a new approach, called a Crowdsourced Learning Mechanism, in which participants collaboratively "learn" a hypothesis for a given prediction task. The approach draws heavily from the concept of a prediction market, where traders bet on the likelihood of a future event. In our framework, the mechanism continues to publish the current hypothesis, and participants can modify this hypothesis by wagering on an update. The critical incentive property is that a participant will profit an amount that scales according to how much her update improves performance on a released test set. "

R.A Fisher's classical paper outlining the maximum likelihood principle and the notions of sufficiency, efficiency and consistency.

"Since Edward Leamer's memorable 1983 paper, "Let's Take the Con out of Econometrics," empirical microeconomics has experienced a credibility revolution. While Leamer's suggested remedy, sensitivity analysis, has played a role in this, we argue that the primary engine driving improvement has been a focus on the quality of empirical research designs. The advantages of a good research design are perhaps most easily apparent in research using random assignment. We begin with an overview of Leamer's 1983 critique and his proposed remedies. We then turn to the key factors we see contributing to improved empirical work, including the availability of more and better data, along with advances in theoretical econometric understanding, but especially the fact that research design has moved front and center in much of empirical micro. We offer a brief digression into macroeconomics and industrial organization, where progress -- by our lights -- is less dramatic, although there is work in both fields that we find encouraging. Finally, we discuss the view that the design pendulum has swung too far. Critics of design-driven studies argue that in pursuit of clean and credible research designs, researchers seek good answers instead of good questions. We briefly respond to this concern, which worries us little."

IV / natural experiment proponents revisiting Leamer's critique.

"Online learning is a well established learning paradigm which has both theoretical and practical appeals. The goal of online learning is to make a sequence of accurate predictions given knowledge of the correct answer to previous prediction tasks and possibly additional available information. Online learning has been studied in several research fields including game theory, information theory, and machine learning. It also became of great interest to practitioners due the recent emergence of large scale applications such as online advertisement placement and online web ranking. In this survey we provide a modern overview of online learning. Our goal is to give the reader a sense of some of the interesting ideas and in particular to underscore the centrality of convexity in deriving efficient online learning algorithms. We do not mean to be comprehensive but rather to give a high-level, rigorous yet easy to follow, survey."

http://www.cs.huji.ac.il/~shais/papers/OLsurvey.pdf

"Data structures play a central role in modern computer science. You interact with data structures even more often than with algorithms (think Google, your mail server, and even your network routers). In addition, data structures are essential building blocks in obtaining efficient algorithms. This course covers major results and current directions of research in data structures"

"If the log likelihood is approximately quadratic with constant Hessian, then the maximum likelihood estimator (MLE) is approximately normally distributed. No other assumptions are required. We do not need independent and identically distributed data. We do not need the law of large numbers (LLN) or the central limit theorem (CLT). We do not need sample size going to infinity or anything going to infinity.

The theory presented here is a combination of Le Cam style involving local asymptotic normality (LAN) and local asymptotic mixed normality (LAMN) and Cramér style involving derivatives and Fisher information. The main tool is convergence in law of the log likelihood function and its derivatives considered as random elements of a Polish space of continuous functions with the metric of uniform convergence on compact sets. We obtain results for both one-step-Newton estimators and Newton-iterated-to-convergence estimators."

What to do when "experiments" can not be controlled (not much, but indoctrination helps) - predates current IV methodology?

Something not quite right about this - a distribution over the models would in principle capture all the relevant information for the planning subproblem - no need for joint optimization.

"This paper discusses some subtle, and largely overlooked, differences between conceptual and mathematical optimization goals in statistics, and illustrates them by examples."

Discrete time optimal control / approximate dynamic programming / LP formulation of the discrete time and space control problem

A concise introduction to discrete time "asset pricing". Doesn't make clear distinctions between marginal/equivalence pricing in "incomplete markets" or present "risk neutral probabilities" as dual variables

History of the adoption of the Kalman filter in aero/astro

Additive regression models from 1924, together with an algorithm which looks even more labour intensive than Whittaker graduation!

Whittaker introduces 1D smoothing in 1922, complete with the Bayesian derivation. There is an earlier German paper with a similar model.