Neurons and artificial neurons

- [Instructor] So how would you describe what you see? After a moment, most of you would probably say a table with handwritten digits. Would you be able to write a program to identify the different digits? What makes this such a difficult task? I'll give you an example. If you look at the last column and the second last row from the bottom, the six doesn't have a hole in the middle. Now, I'm used to seeing sixes with holes in them. And then if you look at the second last column, all of the sixes have a hole in the middle. There doesn't seem to be a lot of consistency in the way the digits have been written, which makes this a difficult task. Now, you and I have no problem recognizing these digits as a six, but it would be difficult to try and code all of the different possible nuances to recognize numbers. How have we learned how to recognize digits? Well, we've just seem several different numbers at kindergarten, at school, all around us, and learned to distinguish between them. I remember when I was putting some slides together for another presentation, my two year old son walked in and he pointed at the screen and he said, "Cat." Now, this was pretty amazing because he's only seen three cats in his entire life and maybe about 10 in his books, yet he was able to recognize a cat. So what has really kicked off image recognition using neural networks is that we now have thousands of images with labels that we can use to train models, and we can then distinguish between different images. Now, our brains are really special and have been created in an amazing way. My two year old hasn't seen thousands of pictures of a cat, he hasn't been to kindergarten, and yet he was able to recognize one. You see, each of the different sections of our brain has a different function. The frontal lobe is for problem solving. The temporal lobe is used for visual processing. And the cerebellum controls things like language and voluntary movement. The neuron is the basic working unit of the brain, a specialized cell designed to transmit information to other nerve cells, muscles, or gland cells. Most neurons have a cell body, an axon, and dendrites. Neurons are then excitable. This means that they produce electrical events called action potentials which are also known as nerve impulses or spikes. Nerve impulses are the basic currency of the brain. They allow neurons to communicate with each other, for computations to be performed, and for information to be processed. Neurons communicate by sending out an electrical signal, and they start a chain reaction when they're stimulated by signals. Every neuron on the path takes up the signal and passes it to the next neuron. The dendrites pick up the impulse and send the message to the axon, which then delivers it to the next neuron. Then the process begins again with another neuron. Finally, once the message hits its target like a muscle or gland cell, the neurotransmitter is stimulated and causes action. All of this happens in about seven milliseconds. So when I want to lift my index finger up in the air in the middle of this video, an electrical impulse from the cerebellum starts a chain reaction, the output of some neurons being the input of others and passing this electrical signal across millions of neurons. The message finally hits my muscles in my hand and I can lift my finger in the air. That's absolutely amazing. So let's talk about artificial neurons. In 1957, Rosenblatt, who was in Cornell, started working on the first artificial neural network, the Perceptron, because he was fascinated how the eye of a fly operates. It was found that most of the operation that tells a fly whether it should flee is done in the eye. The retina of a fly contains several light sensors arranged as a matrix. The sensor outputs are connected to a set of processing elements who recognize particular patterns. The outputs of these processing elements go to a threshold logic unit which is then excited or fires based on a certain level and type of input. This then determines whether a fly flees or stays put. So the Perceptron or artificial neuron that Rosenblatt created computes a weighted sum of the inputs. So we have weight W one multiplied by input X one with W two multiplied by input X two and so on, and then we add a bias b. The weighted sum is then fed into an activation function to determine whether it is triggered or not. This is basically a binary classifier because what we are seeing is that if it exceeds a certain threshold, return a one, otherwise return a zero. There needs to be a way for our network to learn the values of b and W and quantify any error. We'll be looking at that in our next video on gradient descent.