The following introduction is to allow viewers to understand the basic concepts and practical implementation of neural nets towards a financial time series. I will not go too deep into detail about the mathematics behind the neural net at the moment. My goal is to get you to understand practical details about how to actually implement a neural net using simple tools and models. We will start with a simple model to understand a basic time series. The time series waveform is a simple sine wave with the period set to 30 days. It is implemented in excel as a source file to be processed in any Machine Learning capable software.

Netz - Clojure Neural Network Library.

"Autoencoders are unsupervised machine learning circuits whose learning goal is to minimize a distortion measure between inputs and outputs. Linear autoencoders can be defined over any field and only real-valued linear autoencoder have been studied so far. Here we study complex-valued linear autoencoders where the components of the training vectors and adjustable matrices are defined over the complex field with the $L_2$ norm. We provide simpler and more general proofs that unify the real-valued and complex-valued cases, showing that in both cases the landscape of the error function is invariant under certain groups of transformations. The landscape has no local minima, a family of global minima associated with Principal Component Analysis, and many families of saddle points associated with orthogonal projections onto sub-space spanned by sub-optimal subsets of eigenvectors of the covariance matrix. The theory yields several iterative, convergent, learning algorithms, a clear understanding of the generalization properties of the trained autoencoders, and can equally be applied to the hetero-associative case when external targets are provided. Partial results on deep architecture as well as the differential geometry of autoencoders are also presented. The general framework described here is useful to classify autoencoders and identify general common properties that ought to be investigated for each class, illuminating some of the connections between information theory, unsupervised learning, clustering, Hebbian learning, and auto encoders."

Best Practices for Convolutional Neural Networks
Applied to Visual Document Analysis

LeCun 2008, Gradient-Based Learning Applied to Document Recognition

"With the aim of producing a stable human-like bipedal gait, a five-link planar walking mechanism is coupled with a central pattern generator (CPG) neural network, consisting of units based on Matsuoka's half-center oscillator model with a firm basis in neurophysiology. As a minimalistic approach to bipedal walking, this type of walking mechanism contains only four actuators, and is lacking feet and ankles. The mechanism is simulated with accurate physics, allowing realistic fitness evaluations for the creation of CPG controllers through evolutionary computation. The oscillatory parameters, internal connectivity structure, and external feedback pathways of the networks are determined through genetic algorithms (GA) optimization. The evolved CPG networks are transferred to a hardware implementation of the mechanism, to test their performance under real-world dynamics. Results confirm that the biologically inspired CPG model is very well suited for controlling legged locomotion, since a diverse manifestation of CPG networks (with and without external feedback) have been observed to succeed during the course of GA evaluations. Observations also imply that while the CPG mechanism is inherently able to sustain a stable gait, the utilization of feedback pathways makes the gait more human-like and is needed to provide a means to adapt to irregularities in the environment."

"In this paper we present a methodological framework that meets novel requirements emerging from upcoming types of accelerated and highly configurable neuromorphic hardware systems. We describe in detail a device with 45 million programmable and dynamic synapses that is currently under development, and we sketch the conceptual challenges that arise from taking this platform into operation. More specifically, we aim at the establishment of this neuromorphic system as a flexible and neuroscientifically valuable modeling tool that can be used by non-hardware-experts. We consider various functional aspects to be crucial for this purpose, and we introduce a consistent workflow with detailed descriptions of all involved modules that implement the suggested steps: The integration of the hardware interface into the simulator-independent model description language PyNN; a fully automated translation between the PyNN domain and appropriate hardware configurations; an executable specification of the future neuromorphic system that can be seamlessly integrated into this biology-to-hardware mapping process as a test bench for all software layers and possible hardware design modifications; an evaluation scheme that deploys models from a dedicated benchmark library, compares the results generated by virtual or prototype hardware devices with reference software simulations and analyzes the differences. The integration of these components into one hardware-software workflow provides an ecosystem for ongoing preparative studies that support the hardware design process and represents the basis for the maturity of the model-to-hardware mapping software. The functionality and flexibility of the latter is proven with a variety of experimental results."

Jonas Sjöberg et. al.
1995-06-21

The network created by these researchers was an able student and readily learned to recognise a variety of words in grammatical contexts. The next thing was to manipulate the parameters of the network in ways that are thought to model what may be happening to biological neuronal networks in schizophrenia.

There are two major hypotheses that were modelled: the first is that networks in schizophrenia are “over-pruned”. This fits with a lot of observations, including neuroimaging data showing reduced connectivity in the brains of people suffering with schizophrenia. It also fits with the age of onset of the florid expression of this disorder, which is usually in the late teens to early twenties. This corresponds to a period of brain maturation characterised by an intense burst of pruning of synapses – the connections between neurons.

This directory (ftp://ftp.sas.com/pub/neural/) contains the Neural
Network FAQ, several papers on neural networks, and various other
more-or-less related things.