Neural Networks and the Latest Trends

Introduction and Recent Trends of Neural Networks: Traditionally the term neural network has been used to refer to a network or circuit of biological neurons. Modern usage of this term generally refers to an artificial neural network. Digital computer technology has rapidly evolved and is said to perform many tasks very quickly and accurately, but can not do simple operations such as reading, handwritten notes, recognizing faces and voices. Advanced systems are difficult to identify.

Let's start with a bit of history. Deep learning is the latest cool words of the neural network that has been running since the 1960's. If you do not know what the neural network is, do not worry. I will explain later in this article. Around 2006, a smart man such as Jeffrey Sinton published the paper. This paper has an interesting implementation of a neural network. In 2012, two Hinton students won twice the competition (ILSVRC) of the nearest competitor. This shows the worldwide that Hinton's work can solve a very interesting problem.

In the 1980's, Hinton is now a neural network, a simplified model of neurons, and an expert in synaptic networks in our brain. However, at that time, the neural network was convinced that it was a dead end of artificial intelligence research. Perceptron, the earliest neural network developed in the 1960s, was welcomed as a first step towards human intelligence, a book called Marvin Minsky and Seymour Papert by Massachusetts Institute of Technology called Perceptrons in 1969 . Mathematically proved that this type of network can only be expressed. Most basic functions These networks have only two neuron layers, one input layer and one output layer. A network with more layers between the input neuron and the output neuron can theoretically solve various problems but since no one knows how to train them, It is useless. In addition to a few people like Hinton, Perceptrons made most people abandon the neural network altogether.

The convolution neural network has a different architecture from the conventional neural network. Conventional neural networks convert inputs by passing input through a series of hidden layers. Each layer consists of a set of neurons, and each neuron is fully connected to all neurons in that layer. Finally, there is finally a fully connected layer (output layer) that represents the prediction. The convolution neural network is slightly different. First of all, these layers are organized in three dimensions: width, height, depth. In addition, neurons in one layer are not connected to all neurons in the next layer, but are connected to a small portion of them. Finally, the final output is reduced to a single probability score vector organized along the depth dimension.