Data Science Notes

Artificial neural networks

  • Human brain has approximately 86 billion neuros which are connected to each other via synapses. Neurons receive signals from other neurons via their dendrites and transmit their output through their axons.
  • A neuron only fires if the total signal received exceeds a certain threshold value.
  • “Neural pathways are strengthened every time that they are used” -Donald hebb. If two neurons fire together, their connection is enhanced.
  • The strength is referred to as the weight. We are effectively training our neurons and changing the weights between them.
  • The learning comes down to adjusting the weights between the neurons and that is also the model that artificial neural network based on.
  • With deep learning, neural network will do the job of feature selection. So deep learning removes the whole feature selection process.
  • Activation function: It determines whether a neuron would activate or not. Examples are sigmoid, tanh or relu functions.
  • Each node in the input layer represents a feature.
  • The goal of a neural network is to discover the optimal combination of features. Since all neurons are connected together, it gets to try out every single combination.
  • If pixels of an image is given as an input, the first hidden layer will start to generate features (or detecting simple patterns) like lines, edges, textures. The second hidden layer would use the features that the first layer outputs it. Here, the second layer is no longer confronted with pixels but with features generated by the first layer. Again the second hidden layer will try to combine lines and edges into something like shapes like rectangles, circles or shadows or something. Again the third hidden layer would get these shapes and generates its own features like eyes or tail or legs. This process continues till the final layer. In general, we don’t know exactly what features the neural network generates.
  • Rule of 10: we probably need 10 times more the amount of data as we have parameters that we need to estimate.
  • Solve simple problems with simple methods.
  • find pre trained models on zoo “https://modelzoo.co/"

Tensor flow

  • Tensor flow uses a graph to organize it’s calculations.