Neural Networks
a simple neural network
Structure
A neural network is comprised of layers, split into input layer, output layer and (usually multiple) hidden layers. Each layer is made up of neurons.
Neuron
neuron structure
each neuron has as input the outputs of all neurons of the previous layers, weights and a bias. it calculates the output in a two step process. first it calculates the weighted sum of it's weights and inputs, and adds the bias. it passes the result into an activation function.
Forward pass
A NN's output is calculated by going through layers (\(\(l_1^n\)\) ) and computing the activations of that layer:\(\(a^{[l]} = g^{[l]}(w^{[l]} a^{[l-1]} + b^{[l]})\)\) with \(\(a^{[0]}\)\) being the input layer \(\(X\)\). the output of the NN is then the output of the last layer \(\(\hat{y} = a^{[n]}\)\)