Lexical Development from the Perspectives of Artificial Neural Network Models and Dynamical Systems Theory

Word learning is an essential part of initial language acquisition. A controversial phenomenon related to the growth of vocabulary is a vocal spike and is often present in the rapid growth of productive vocabulary in early childhood languages. In spite of the fact that the first generation of a word started slowly, we believe that children will experience a transition to a later phase of faster vocabulary growth in a few months (Goldfield & Reznick, 1990). There are several theories trying to explain this phenomenon.

Let's start with a language model based on recurrent neural network (RNN). Yoshua Bengio proposes the use of artificial neural network based statistical modeling to calculate the probability of a series of words appearing. This approach has proven to be successful; however, feedforward neural networks do not allow to receive variable length sequences as inputs to limit model power. RNN is, of course, the next step in statistical language modeling, since variable-length sequences can be used as inputs and outputs. The RNN architecture is shown below.

An artificial neural network is a simulated mathematical model inspired by a biological neural network. An artificial neural network can simulate and process a nonlinear relationship between input and output. The adaptive weight between artificial neurons is adjusted by a learning algorithm that reads observational data with the purpose of increasing the output. Artificial neural networks are very powerful. However, even though some neuron mathematics is simple, the whole network becomes complicated. Because this ANN is considered a "black box" algorithm. Care must be taken when selecting ANN as a tool for problem solving. It is impossible to cancel the decision making process of the system in the future.

We need a sufficiently complex and accurate neuron model. Traditional artificial neural network models such as multilayer perceptron network models are considered to be inadequate. A dynamic spike neural network model is required. This reflects that neurons are released only when the membrane potential reaches a certain level. The model should include differential equations describing the relationship between delay, nonlinear functions, and electrical and physical parameters such as current, voltage, membrane state (ion channel state), neuroregulator, etc.