SEARCH · PubMed Health

Results for “recurrent neural network”

Explore indexed PubMed citations for clinical trials, systematic reviews and public health research. Read source abstracts and follow each citation to its original PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4Linked to original sources

PubMedDate not supplied

Motor primitive and sequence self-organization in a hierarchical recurrent neural network.

This study describes how complex goal-directed behavior can be obtained through adaptation processes in a hierarchically organized recurrent neural network using a genetic algorithm (GA). Our experiments, using a simulated Khepera robot, showed that different types of dynamic structures self-organize in the lower and higher levels of the network for the purpose of achieving complex navigation tasks. The parametric bifurcation structures that appear in the lower level explain the mechanism of how behavior primitives are switched in a top-down way. In the higher level, a topologically ordered mapping of initial cell activation states to motor primitive sequences self-organizes by utilizing the initial sensitivity characteristics of non-linear dynamical systems. The biological plausibility of the model's essential principles is discussed.

Algorithms↗

PubMed1999

Discontinuities in recurrent neural networks.

This article studies the computational power of various discontinuous real computational models that are based on the classical analog recurrent neural network (ARNN). This ARNN consists of finite number of neurons; each neuron computes a polynomial net function and a sigmoid-like continuous activation function. We introduce arithmetic networks as ARNN augmented with a few simple discontinuous (e.g., threshold or zero test) neurons. We argue that even with weights restricted to polynomial time computable reals, arithmetic networks are able to compute arbitrarily complex recursive functions. We identify many types of neural networks that are at least as powerful as arithmetic nets, some of which are not in fact discontinuous, but they boost other arithmetic operations in the net function (e.g., neurons that can use divisions and polynomial net functions inside sigmoid-like continuous activation functions). These arithmetic networks are equivalent to the Blum-Shub-Smale model, when the latter is restricted to a bounded number of registers. With respect to implementation on digital computers, we show that arithmetic networks with rational weights can be simulated with exponential precision, but even with polynomial-time computable real weights, arithmetic networks are not subject to any fixed precision bounds. This is in contrast with the ARNN that are known to demand precision that is linear in the computation time. When nontrivial periodic functions (e.g., fractional part, sine, tangent) are added to arithmetic networks, the resulting networks are computationally equivalent to a massively parallel machine. Thus, these highly discontinuous networks can solve the presumably intractable class of PSPACE-complete problems in polynomial time.

Algorithms↗

PubMed1996

Stable encoding of large finite-state automata in recurrent neural networks with sigmoid discriminants.

We propose an algorithm for encoding deterministic finite-state automata (DFAs) in second-order recurrent neural networks with sigmoidal discriminant function and we prove that the languages accepted by the constructed network and the DFA are identical. The desired finite-state network dynamics is achieved by programming a small subset of all weights. A worst case analysis reveals a relationship between the weight strength and the maximum allowed network size, which guarantees finite-state behavior of the constructed network. We illustrate the method by encoding random DFAs with 10, 100, and 1000 states. While the theory predicts that the weight strength scales with the DFA size, we find empirically the weight strength to be almost constant for all the random DFAs. These results can be explained by noting that the generated DFAs represent average cases. We empirically demonstrate the existence of extreme DFAs for which the weight strength scales with DFA size.

Algorithms↗

PubMed2005

Global exponential stability of generalized recurrent neural networks with discrete and distributed delays.

This paper is concerned with analysis problem for the global exponential stability of a class of recurrent neural networks (RNNs) with mixed discrete and distributed delays. We first prove the existence and uniqueness of the equilibrium point under mild conditions, assuming neither differentiability nor strict monotonicity for the activation function. Then, by employing a new Lyapunov-Krasovskii functional, a linear matrix inequality (LMI) approach is developed to establish sufficient conditions for the RNNs to be globally exponentially stable. Therefore, the global exponential stability of the delayed RNNs can be easily checked by utilizing the numerically efficient Matlab LMI toolbox, and no tuning of parameters is required. A simulation example is exploited to show the usefulness of the derived LMI-based stability conditions.

Artificial Intelligence↗

PubMed2005

Design and analysis of a general recurrent neural network model for time-varying matrix inversion.

Following the idea of using first-order time derivatives, this paper presents a general recurrent neural network (RNN) model for online inversion of time-varying matrices. Different kinds of activation functions are investigated to guarantee the global exponential convergence of the neural model to the exact inverse of a given time-varying matrix. The robustness of the proposed neural model is also studied with respect to different activation functions and various implementation errors. Simulation results, including the application to kinematic control of redundant manipulators, substantiate the theoretical analysis and demonstrate the efficacy of the neural model on time-varying matrix inversion, especially when using a power-sigmoid activation function.

Algorithms↗

PubMed2002

On unique representations of certain dynamical systems produced by continuous-time recurrent neural networks.

This article extends previous mathematical studies on elucidating the redundancy for describing functions by feedforward neural networks (FNNs) to the elucidation of redundancy for describing dynamical systems (DSs) by continuous-time recurrent neural networks (RNNs). In order to approximate a DS on R(n) using an RNN with n visible units, an n-dimensional affine neural dynamical system (A-NDS) can be used as the DS actually produced by the above RNN under an affine map from its visible state-space R(n) to its hidden state-space. Therefore, we consider the problem of clarifying the redundancy for describing A-NDSs by RNNs and affine maps. We clarify to what extent a pair of an RNN and an affine map is uniquely determined by its corresponding A-NDS and also give a nonredundant sufficient search set for the DS approximation problem based on A-NDS.

Journal Article↗

PubMed2003

A dynamic recurrent neural network for multiple muscles electromyographic mapping to elevation angles of the lower limb in human locomotion.

This paper describes the use of a dynamic recurrent neural network (DRNN) for simulating lower limb coordination in human locomotion. The method is based on mapping between the electromyographic signals (EMG) from six muscles and the elevation angles of the three main lower limb segments (thigh, shank and foot). The DRNN is a fully connected network of 35 hidden units taking into account the temporal relationships history between EMG and lower limb kinematics. Each EMG signal is sent to all 35 units, which converge to three outputs. Each output neurone provides the kinematics of one lower limb segment. The training is supervised, involving learning rule adaptations of synaptic weights and time constant of each unit. Kinematics of the locomotor movements were recorded and analysed using the opto-electronic ELITE system. Comparative analysis of the learning performance with different types of output (position, velocity and acceleration) showed that for common gait mapping velocity data should be used as output, as it is the best compromise between asymptotic error curve, rapid convergence and avoidance of bifurcation. Reproducibility of the identification process and biological plausibility were high, indicating that the DRNN may be used for understanding functional relationships between multiple EMG and locomotion. The DRNN might also be of benefit for prosthetic control.

Adult↗

PubMed2006

Global exponential stability of recurrent neural networks with time-varying delays in the presence of strong external stimuli.

This paper presents new theoretical results on the global exponential stability of recurrent neural networks with bounded activation functions and bounded time-varying delays in the presence of strong external stimuli. It is shown that the Cohen-Grossberg neural network is globally exponentially stable, if the absolute value of the input vector exceeds a criterion. As special cases, the Hopfield neural network and the cellular neural network are examined in detail. In addition, it is shown that criteria herein, if partially satisfied, can still be used in combination with existing stability conditions. Simulation results are also discussed in two illustrative examples.

Animals↗

PubMed2002

Improving the prediction of protein secondary structure in three and eight classes using recurrent neural networks and profiles.

Secondary structure predictions are increasingly becoming the workhorse for several methods aiming at predicting protein structure and function. Here we use ensembles of bidirectional recurrent neural network architectures, PSI-BLAST-derived profiles, and a large nonredundant training set to derive two new predictors: (a) the second version of the SSpro program for secondary structure classification into three categories and (b) the first version of the SSpro8 program for secondary structure classification into the eight classes produced by the DSSP program. We describe the results of three different test sets on which SSpro achieved a sustained performance of about 78% correct prediction. We report confusion matrices, compare PSI-BLAST to BLAST-derived profiles, and assess the corresponding performance improvements. SSpro and SSpro8 are implemented as web servers, available together with other structural feature predictors at: http://promoter.ics.uci.edu/BRNN-PRED/.

Algorithms↗

PubMedDate not supplied

Chaos control and synchronization, with input saturation, via recurrent neural networks.

This paper deals with the adaptive tracking problem of non-linear systems in presence of unknown parameters, unmodelled dynamics and input saturation. A high order recurrent neural network is used in order to identify the unknown system and a learning law is obtained using the Lyapunov methodology. Then a stabilizing control law for the reference tracking error dynamics is developed using the Lyapunov methodology and the Sontag control law. Tracking error boundedness is established as a function of a design parameter. The new approach is illustrated by examples of complex dynamical systems: chaos control and synchronization.

Neural Networks, Computer↗

PubMed2000

A recurrent neural network for solving linear projection equations.

Linear projection equations arise in many optimization problems and have important applications in science and engineering. In this paper, we present a recurrent neural network for solving linear projection equations in real time. The proposed neural network has two layers and is amenable to parallel implementation with simple hardware. In the theoretical aspect, we prove that the proposed neural network can converge globally to the solution set of the problem when the matrix involved in the problem is positive semidefinite and can converge exponentially to a unique solution when the matrix is positive definite. In addition, we analyze the stability of the related dynamic system in detail. As an application, we show that the proposed neural network can be used directly to solve linear and convex quadratic programming problems and linear complementary problems with positive semidefinite matrices. The validity and transient behavior of the neural network are demonstrated by using three numerical examples.

Linear Models↗

PubMed2003

Multistability analysis for recurrent neural networks with unsaturating piecewise linear transfer functions.

Multistability is a property necessary in neural networks in order to enable certain applications (e.g., decision making), where monostable networks can be computationally restrictive. This article focuses on the analysis of multistability for a class of recurrent neural networks with unsaturating piecewise linear transfer functions. It deals fully with the three basic properties of a multistable network: boundedness, global attractivity, and complete convergence. This article makes the following contributions: conditions based on local inhibition are derived that guarantee boundedness of some multistable networks, conditions are established for global attractivity, bounds on global attractive sets are obtained, complete convergence conditions for the network are developed using novel energy-like functions, and simulation examples are employed to illustrate the theory thus developed.

Linear Models↗

PubMed2004

Real-time computation at the edge of chaos in recurrent neural networks.

Depending on the connectivity, recurrent networks of simple computational units can show very different types of dynamics, ranging from totally ordered to chaotic. We analyze how the type of dynamics (ordered or chaotic) exhibited by randomly connected networks of threshold gates driven by a time-varying input signal depends on the parameters describing the distribution of the connectivity matrix. In particular, we calculate the critical boundary in parameter space where the transition from ordered to chaotic dynamics takes place. Employing a recently developed framework for analyzing real-time computations, we show that only near the critical boundary can such networks perform complex computations on time series. Hence, this result strongly supports conjectures that dynamical systems that are capable of doing complex computational tasks should operate near the edge of chaos, that is, the transition from ordered to chaotic dynamics.

Animals↗

PubMed2001

Improved prediction of the number of residue contacts in proteins by recurrent neural networks.

Knowing the number of residue contacts in a protein is crucial for deriving constraints useful in modeling protein folding, protein structure, and/or scoring remote homology searches. Here we use an ensemble of bi-directional recurrent neural network architectures and evolutionary information to improve the state-of-the-art in contact prediction using a large corpus of curated data. The ensemble is used to discriminate between two different states of residue contacts, characterized by a contact number higher or lower than the average value of the residue distribution. The ensemble achieves performances ranging from 70.1% to 73.1% depending on the radius adopted to discriminate contacts (6Ato 12A). These performances represent gains of 15% to 20% over the base line statistical predictors always assigning an aminoacid to the most numerous state, 3% to 7% better than any previous method. Combination of different radius predictors further improves the performance. SERVER: http://promoter.ics.uci.edu/BRNN-PRED/.

Amino Acid Sequence↗

PubMed1998

Extended Kalman Filter-Based Pruning Method for Recurrent Neural Networks.

Pruning is one of the effective techniques for improving the generalization error of neural networks. Existing pruning techniques are derived mainly from the viewpoint of energy minimization, which is commonly used in gradient-based learning methods. In recurrent networks, extended Kalman filter (EKF)-based training has been shown to be superior to gradient-based learning methods in terms of speed. This article explains a pruning procedure for recurrent neural networks using EKF training. The sensitivity of a posterior probability is used as a measure of the importance of a weight instead of error sensitivity since posterior probability density is readily obtained from this training method. The pruning procedure is tested using three problems: (1) the prediction of a simple linear time series, (2) the identification of a nonlinear system, and (3) the prediction of an exchange-rate time series. Simulation results demonstrate that the proposed pruning method is able to reduce the number of parameters and improve the generalization ability of a recurrent network.

Journal Article↗

PubMed2003

On-line identification and reconstruction of finite automata with generalized recurrent neural networks.

In this paper finite automata are treated as general discrete dynamical systems from the viewpoint of systems theory. The unconditional on-line identification of an unknown finite automaton is the problem considered. A generalized architecture of recurrent neural networks with a corresponding on-line learning scheme is proposed as a solution to the problem. An on-line rule-extraction algorithm is further introduced. The architecture presented, the on-line learning scheme and the on-line rule-extraction method are tested on different, strongly connected automata, ranging from a very simple example with two states only to a more interesting and complex one with 64 states; the results of both training and extraction processes are very promising.

Algorithms↗

PubMed2003

Dual extended Kalman filtering in recurrent neural networks(1).

In the classical deterministic Elman model, the estimation of parameters must be very accurate. Otherwise, the system performance is very poor. To improve the system performance, we can use a Kalman filtering algorithm to guide the operation of a trained recurrent neural network (RNN). In this case, during training, we need to estimate the state of hidden layer, as well as the weights of the RNN. This paper discusses how to use the dual extended Kalman filtering (DEKF) for this dual estimation and how to use our proposing DEKF for removing some unimportant weights from a trained RNN. In our approach, one Kalman algorithm is used for estimating the state of the hidden layer, and one recursive least square (RLS) algorithm is used for estimating the weights. After training, we use the error covariance matrix of the RLS algorithm to remove unimportant weights. Simulation showed that our approach is an effective joint-learning-pruning method for RNNs under the online operation.

Algorithms↗

PubMed2000

Multi-step-ahead prediction using dynamic recurrent neural networks.

A method for the development of empirical predictive models for complex processes is presented. The models are capable of performing accurate multi-step-ahead (MS) predictions, while maintaining acceptable single-step-ahead (SS) prediction accuracy. Such predictors find applications in model predictive controllers and in fault diagnosis systems. The proposed method makes use of dynamic recurrent neural networks in the form of a nonlinear infinite impulse response (IIR) filter. A learning algorithm is presented, which is based on a dynamic gradient descent approach. The effectiveness of the method for accurate MS prediction is tested on an artificial problem and on a complex, open-loop unstable process. Comparative results are presented with polynomial Nonlinear AutoRegressive with eXogeneous (NARX) predictors, and with recurrent networks trained using teacher forcing. Validation studies indicate that excellent generalization is obtained for the range of operational dynamics studied. The research demonstrates that the proposed network architecture and the associated learning algorithm are quite effective in modeling the dynamics of complex processes and performing accurate MS predictions.

Algorithms↗