Deep Neural Networks: What They Are and How They Work

Deep neural networks are a concept that constitutes the main technological architecture used in Deep Learning models. These structures cannot be understood without understanding the general idea of ​​artificial neural networks, fundamental for Artificial Intelligence.

Neural networks are used for a thousand things: recognizing license plates, songs, faces, voices, or even the fruits in our kitchen. They are a particularly useful technology and, although they have only recently become practical, they are going to constitute the future of humanity.

Next We are going to look in depth at the idea of ​​artificial neural networks and deep ones understanding how they work, how they are trained and how the interactions between the different neurons that constitute them occur.

What are deep neural networks and what characterizes them?

Deep neural networks are one of the most important technological architectures used in Deep Learning These particular artificial networks have had dizzying growth in recent years because they constitute a fundamental aspect when it comes to recognizing all types of patterns. Artificial Intelligence exists thanks to the functioning of these particular networks that, in essence, are a replica of how our brains work, although in a technological and mathematical way.

Before we delve deeper into what deep neural networks are, we first need to understand how artificial neural networks work in general and what they are for. lNeural networks are a branch of “Machine Learning” that have had an enormous impact in recent years helping programmers and computer scientists generate things like chatbots that, when we talk to them, make us think we are talking to real human beings.

Artificial neural networks have also been used with cars that drive automatically, mobile applications that recognize our face and transform it into whatever we want, and many more functions. Its applicability is very extensive, serving as the basis of modern Artificial Intelligence and having endless beneficial uses for our daily lives.

Artificial neural networks

Let’s imagine that we are in our kitchen and we decide to look for an orange, a very simple task We know how to identify an orange very easily and we also know how to differentiate it from other fruits that we find in the kitchen, such as bananas, apples and pears. As? Because in our brain we have very assimilated what the typical properties of an orange are: its size, its shape, the color it has, what it smells like… These are all parameters that we use to find an orange.

It is a simple task for humans, but… can a computer do it too? The answer is yes. In principle, it would be enough to define those same parameters and assign a value to a node or something that we could call an “artificial neuron.” We would tell that neuron what oranges are like, indicating their size, weight, shape, color or any other parameter that we attribute to this fruit. Having this information, it is expected that the neuron will know how to identify an orange when one is presented to it.

If we have chosen the parameters well, it will be easy for you to differentiate between oranges and things that are not orange simply by taking those characteristics into account. When presented with an image of any fruit, that neuron will look for the characteristics associated with the orange and decide whether to include it in the “orange” category or the “other fruits” category. In statistical terms, it would be finding a region in a parameter graph that corresponds to what you are looking for, a region that would encompass all the pieces of fruit that share the size, shape, color, weight and aroma of oranges.

At first this all sounds very easy to code, and in fact it is. It works very well to differentiate an orange from a banana or an apple, since they have different colors and shapes. However, what if we introduced you to a grapefruit? And a very large tangerine? They are fruits that can perfectly be confused with an orange. Will the artificial neuron be able to differentiate between oranges and grapefruits on its own? The answer is no, and in fact they are probably thought to be the same thing.

The problem with using only one layer of artificial neurons, or in other words, only using simple first neurons, is that generate very imprecise decision boundaries when presented with something that has many characteristics in common with what it should know how to recognize, but in reality it is not. If we present something that looks like an orange, such as a grapefruit, even if it is not that fruit, it will identify it as such.

These decision boundaries, if represented in the form of a graph, will always be linear. Using a single artificial neuron, that is, a single node that has specific parameters integrated but cannot learn beyond them, very diffuse decision boundaries will be obtained. Its main limitation is that it uses two statistical methods, specifically multiclass regression and logistic regression, which means that when in doubt it includes something that is not what we expected it to identify.

If we were to divide all fruits into “oranges” and “non-oranges”, using only one neuron it is clear that bananas, pears, apples, watermelons and any fruit that does not correspond in size, color, shape, aroma And with the oranges I would put them in the “non-orange” category. However, I would put grapefruits and tangerines in the “orange” category, doing poorly the job for which they were designed.

And when we talk about oranges and grapefruits we could very well talk about dogs and wolves, hens and chickens, books and notebooks… All of these situations are cases in which a simple series of “ifs…” would not be enough (“if… “) to clearly discern between one and the other. A more complex, non-linear system is necessary that is more precise when it comes to differentiating between different elements. Something to keep in mind that there may be differences between things that are similar This is where neural networks come in.

More layers, more similar to the human brain

Artificial neural networks, as their name indicates, are artificial computational models inspired by the neural networks of the human brain, networks that in fact imitate the functioning of this biological organ. This system is inspired by neural functioning and its main application is the recognition of patterns of all kinds: facial identification, voice recognition, fingerprint recognition, handwriting, license plates… Pattern recognition can be used for almost everything

As there are different neurons, the parameters applied are various and a higher degree of precision is obtained. These neural networks are systems that allow us to separate elements into categories when the difference may be subtle, separating them in a non-linear way, something that would be impossible to do otherwise.

With a single node, with a single neuron, what is done when managing information is a multiclass regression. By adding more neurons, as each one of them has its own non-linear activation function which, translated into simpler language, means that they have decision boundaries that are more precise, being graphically represented in a curve and taking into account more characteristics when it comes to differentiating between “oranges” and “non-oranges”, to continue with that example.

The curvature of these decision boundaries will directly depend on how many layers of neurons we add to our neural network. These layers of neurons, which make the system become more complex and have greater precision, are, in effect, deep neural networks. In principle, the more layers we have of deep neural networks, the more precision and similarity the program will have compared to the human brain.

In short, neural networks are nothing more than an intelligent system that allows us to make more precise decisions, very similar to how human beings do Human beings are based on experience, learning from our environment. For example, returning to the case of orange and grapefruit, if we have never seen one, we will perfectly confuse it for an orange. When we have become familiar with it, it will be then that we will know how to identify it and differentiate it from oranges.

The first thing we do is give some parameters to the neural networks so that they know what we want them to learn to identify. Then comes the learning or training phase, so that it is increasingly more precise and progressively has a smaller margin of error. This is the moment when we would present our neural network with an orange and other fruits. In the training phase you will be given cases in which they are orange and cases in which they are not orange, looking to see if you got your answer right and telling you the correct answer.

We will try to ensure that the attempts are numerous and as close to reality as possible In this way we are helping the neural network to operate when real cases arrive and it knows how to discriminate appropriately, in the same way that a human being would do in real life. If the training has been adequate, having chosen good recognition parameters and they have been well classified, the neural network will have a very high pattern recognition success rate.

What are they like and how do they work exactly?

Now that we have seen the general idea of ​​what neural networks are and we are going to understand more in depth what they are like and how these emulators of the neurons of the human brain work and where deep neural networks play a role in this entire process.

Let’s imagine that we have the following neural network: we have three layers of artificial neurons. Let’s say that the first layer has 4 neurons or nodes, the second 3 and the last has only 2. All this is an example of an artificial neural network, quite simple to understand.

The first layer is the one that receives the data, that is, the information that may come in the form of sound, image, aromas, electrical impulses… This first layer is the input layer, and is responsible for receiving all the data in order to later send it to the following layers. During the training of our neural network, this will be the layer that we will first work with, giving it data that we will use to see how well it is able to make predictions or identify the information that is given to it.

The second layer of our hypothetical model is the hidden layer, which sits well in the middle of the first and last layers, as if our neural network were a sandwich. In this example we only have one hidden layer, but there could well be as many as we would like. We could talk about 50, 100, 1000 or even 50,000 layers. In essence, these hidden layers are the part of the neural network that we would call a deep neural network. The greater the depth, the more complex the neural network is.

Finally we have the third layer of our example which is the output layer. This layer, as its name indicates, It is responsible for receiving information from the previous layers, making a decision and giving us a response or result

In the neural network, each artificial neuron is connected to all the following ones. In our example, where we have mentioned that we have three layers of 4, 3 and 2 neurons, the 4 of the input layer are connected with the 3 of the hidden layer, and the 3 of the hidden layer with the 2 of the output layer. , giving us a total of 18 connections.

All these neurons are connected to those of the next layer, sending information in the input->hidden->output direction If there were more hidden layers we would talk about a greater number of connections, sending information from hidden layer to hidden layer until reaching the output layer. The output layer, once it has received the information, will give us a result based on the information it has received and its way of processing it.

When we are training our algorithm, that is, our neural network, this process that we just explained will be done many times. We are going to deliver some data to the network, we are going to see what the result gives us and we are going to analyze it and compare it with what we expected the result to give us. If there is a big difference between what was expected and what was obtained, it means that there is a high margin of error and, therefore, it is necessary to make a few modifications.

How do artificial neurons work?

Now we are going to understand the individual functioning of the neurons that work within a neural network. The neuron receives an input of information from the previous neuron. Let’s say that this neuron receives three inputs of information, each of them coming from the three neurons of the previous layer. In turn, this neuron generates outputs, in this case let’s say it is only connected to a neuron from the next layer.

Each connection that this neuron has with the three neurons of the previous layer brings a value “x”, which is the value that the previous neuron is sending us; and it also has a value “w”, which is the weight that this connection has. Weight is a value which helps us give greater importance to one connection over others. In short, each connection with the previous neurons has a value “x” and a value “w”, which are multiplied (x·w).

We will also have a value called “bias” or bias represented with “b” which is the error number which encourages certain neurons to activate more easily than others Additionally, we have an activation function within the neuron, which is what makes its degree of classification of different items (e.g., oranges) non-linear. By itself, each neuron has different parameters to take into account, which causes the entire system, that is, the neural network, to classify in a non-linear way.

How does the neuron know whether to activate or not? That is, when do you know if you have to send information to the next layer? Well, this decision is governed by the following equation:

This formula means that the sum of all the weights “w” multiplied by all the values ​​of “x” that the neuron is receiving from the previous layer must be made. Added to this, the bias “b” is added.

The result of this equation is sent to an activation function, which is simply a function that tells us that, if the result of this equation is greater than a certain number, the neuron will send a signal to the next layer and, if it is less, then it will not send it. So, this is how an artificial neuron decides whether or not to send the information to the neurons of the next layer through an output that we will call “y”, an output that, in turn, is the input “x” of the next neuron.

And how do you train an entire network?

The first thing to do is deliver data to the first layer, as we have mentioned previously. This layer will send information to the following layers, which are the hidden layers or the deep neural network. The neurons in these layers will activate or not depending on the information received. Finally, the output layer will give us a result, which we will compare with the value we were expecting to see if the neural network has learned correctly what it has to do.

If he did not learn well then we will perform another interaction, i.e. we will present information again and see how the neural network behaves Depending on the results obtained, the “b” values, that is, the bias of each neuron, and the “w” values, that is, the weight of each connection with each neuron, will be adjusted to reduce the error. To know how big this error is, we are going to use another equation, which is the following:

This equation is the mean square error. We are going to add y(x), which is the value that our network gave us in the interaction minus “a”, which is the value we were expecting it to give us, squared. Finally, we are going to multiply this sum by 1/2n, with that “n” being the number of interactions that we have trained our neural network.

For example, let’s imagine that we have the following values

The first column “y(x)” represents what our network has given us in each of the four interactions we have done testing it. The values ​​that we have obtained, as can be seen, do not correspond to those in the second column “a”, which are the desired values ​​for each of the interactions tested. The last column represents the error of each interaction.

Applying the previously mentioned formula and using this data here, taking into account that in this case n = 4 (4 interactions) gives us a value of 3.87, which is the mean square error that our neural network has at the moment. Knowing the error, what we have to do now is, as we have mentioned before, change the bias and weights of each of the neurons and their interactions with the intention that in this way the error is reduced.

At this point, engineers and computer scientists apply an algorithm called gradient descent with which they can obtain values ​​to test and modify the bias and weight of each artificial neuron so that, in this way, an increasingly lower error is obtained, getting closer to the prediction or desired result. It is a matter of testing and the more interactions that are made, the more training there will be and the more the network will learn.

Once the neural network is properly trained, it will give us precise and reliable predictions and identifications. At this point we will have a network that will have a defined weight value in each of its neurons, with a controlled bias and with a decision-making capacity that will make the system work.