The first programming assignment of Andrew’s second course “Improving Deep Neural Networks: Hyperparameter tuning, Regularization and Optimization” was to build a basic neural network to identify a function that would separate a scatter of red and blue dots based on their X, Y coordinates. For a human this is simple, the red dots formed a circle inside the black dots with a bit of random scatter but for an algorithm in Excel, well, before Andrew’s course it was not so easy.

The assignment required the Neural Network to find a boundary between red and black dots. This diagram shows the neural nets final output as shaded orange & blue areas.I had struggled to visualise what was happening inside the neural network and desperately wanted to see something in ‘my’ language, Excel. I searched the net for examples and came up with nothing other than some single neuron examples and one rough MNIST digit classifier. This didn’t make sense to me as I would have thought that Excel was an ideal teaching medium; lots of native speakers and a 2D layout ideal for exploring the dimensions of the inner hidden layers.

Could Excel expose the mystery of Neural Networks?

So, one sunny afternoon at home in India, I set off on a mission. The first step was to grab Andrew’s Data – nightmare, I couldn’t lift it out of Python, I was illiterate. After 2 futile hours, I did the obvious and constructed it myself. This took 5 minutes and opened the possibility of interesting patterns, say a doughnut or a letter. With the data in hand I began construction under the assumption that at some point I’d encounter insurmountable barriers on the way, but to my delight, there weren’t any. After a couple of hours, and with some luck, I’d built it and was ready to start the iterations and learn the function. I hit the macro button, watched for a while, however, nothing happened, I restarted a few times, checked the code then headed off for a coffee. Now the joy of a neural network is that it programs itself which in Excel takes luck and time. I came back a couple of hours later. Wow! I had iterating patterns and a learning curve that was headed in the right direction. The slow speed of my code was, to some extent, a plus and allowed me to see the function develop and ultimately segregate the dots. With the spreadsheet working and through the process of building it, the mystery of basic neural networks and back-propagation was finally clear and exposed.

The learning curve drops off at 400 iterations and by 1000 the neural network as learnt and represented the hidden function.

The first model was very slow but Andrew was covering momentum and regularisation that week in his course so I plugged the first in and attempted the second, regularisation, by making the random scatter change each iteration, this was probably more like data augmentation but it was all new to me. My thinking was that the base function behind the data was three circles and re-running the scatter on each iteration would help the model learn this underlying function rather than the training set – I got a huge increase in speed and some fascinating results on the boundary diagram.

During the learning process (left to right) the boundary function shows weird patterns as it closes in on the red dots.

The Neural Net architecture and equations were more or less straight out of Andrew’s course but placed into an Excel sheet. The layout shown below is very much orientated towards an Excel instantiation, it allowed me to update the model’s Weights and Momentum in a single line of VBA for each iteration namely:  Parameters (T) = Parameters (T+1) .value. I subsequently built a fully recursive version with no VBA that made use of Excels inbuilt iteration functionality but this is less satisfying to run.

You can view a video of the model in operation here: https://www.youtube.com/watch?v=mIpJu-I13cc

This is the basic layout of the on-sheet Excel. It required approximately 30 separate excel formula

The boundary plot i.e. the orange and blue pixels that cover the X plot and represent the boundary function discovered by the neural network required a second neural net without back propagation (the learning bit) that looks as follows:

The Neural network used for the boundary function was even simpler with only 8 separate formulae.

I subsequently recorded a construction video which runs for over an hour but gives a lot more detail on the Excel needed to get this up and running. You can view the video here: https://www.youtube.com/watch?v=suZhX6N5LAk&t=3042s

You can download the Excel file here: Download Excel

These videos and associated material if released are available to you under a Creative Commons Attribution-NonCommercial 4.0 International Licence the details of which can be found here: https://creativecommons.org/licenses/by-nc/4.0/legalcode

The post Neural Networks in Excel – Finding Andrew Ng’s Hidden Circle appeared first on NN in XL.