Building Convolutional Neural Networks in Excel - NN in XL
21460
post-template-default,single,single-post,postid-21460,single-format-standard,ajax_fade,page_not_loaded,,select-child-theme-ver-1.0.0,select-theme-ver-4.4,menu-animation-underline,wpb-js-composer js-comp-ver-5.4.7,vc_responsive

Building Convolutional Neural Networks in Excel

This blog relates to work I’ve done in Excel to build a handwritten digit classifier; basically, a spreadsheet that can read handwriting up to human levels of accuracy. This required a convolutional neural network – the engine behind just about all machine learning related to images. I’m unaware of anyone else who has done this in Excel so please let me know if you come across others.

I have been deeply involved with financial analysis and mathematical models for most of my career but began re-tooling with machine learning over the last two years.  I’m currently working through Andrew Ng’s brilliant Deep Learning course on Coursera and I’ve reached Course 4, Week 3. Throughout the course, I’ve been building out the neural net architectures he describes in Excel.  Excel is not “yet” the right medium to build convolutional neural nets for real-world applications. However, I know Excel and find it easier to construct these on a spreadsheet rather than in a new language. I certainly hope this state won’t last long.

Building in Excel may be slow, but it makes neural nets very transparent, in that you are visually confronted with the layers, neurons and their associated parameters and related calculations. Excel gives a less abstract view of a neural net than vectorised Python code and it helped me immensely in developing an understanding of these fantastic new tools.

Excel lets you see inside the machine but for me, an even more useful element was the bugs and miss wiring that I introduced. Neural networks are very robust to bugs, in fact, they often continue to learn but fail in odd and interesting ways. Figuring out the failure based on the learning behaviour really forced me to think about the maths and the structure. I would also argue that the speed of Excel gives you time to think as the failures manifest themselves. For a deeper understanding of Neural Nets go to Andrew Ng. My intuitions are still weak but certainly improving.

The classic beginners exercise for deep learning is to build an MNIST digit classifier. This relates to Yann LeCun’s data set of 60,000 handwritten digits (0 to 9) with an associated 10,000-digit test set. Yann has made this data available to all and there are plenty of higher level language examples.

My initial attempts at classifiers using plain vanilla fully connected neural networks are on my YouTube channel below but Mike Pallister was faster off the mark than me and is also worth a look.

Single Digit Classification https://www.youtube.com/watch?v=4P5r0tT7Hsc&t=33s

Batch Digit Classification https://www.youtube.com/watch?v=bJcv9vi4Gqg&t=6s

However, this blog relates to last week’s modelling of a convolutional neural net; again with ADAM optimisation and Batch capability. After several silly but informative mistakes, I completed the model below last night and would welcome your thoughts.

The model has very few layers, neurons and parameters, its tiny in comparison to real-world examples. By way of a full description, there are two convolutional layers with max pooling taking the images of the handwritten digits from 28h x 28w pixels to 24h x 24w x 4c (4 channels) and 12h x 12w x 4c after max pooling. Layer 2 condensed these to 8h x 8w x 8c and then 4h x 4w x 8c after max pooling. The final two fully connected layers had 15 and 10 neurons respectively. Giving a grand total of 936 convolutional parameters and 2,095 in the fully connected layers. This doesn’t sound like a lot of space to capture the vagaries of human handwriting, but it does. On its first 100k iterations I’m seeing 98.75% accuracy on the training data and 98% on the test data.  I would love to know what human accuracy levels on this data are but from my experience, it’s not much more than this.

I’d certainly welcome any questions and queries you may have and will be releasing some of the base models on GitHub. Any expressions of interest prior to this would also be welcome.

These videos and associated material if released are available to you under a Creative Commons Attribution-NonCommercial 4.0 International Licence the details of which can be found here: https://creativecommons.org/licenses/by-nc/4.0/legalcode

14 Comments
  • Jacques Hugo

    August 17, 2018 at 10:35 pm Reply

    Richard, I love this! I have been following NN since 1990 – as a Chemical engineer. I’d like to know if there is a way I can download your excel file? If not I do understand.
    Regards

    Jacques Hugo pjacqueshugo@gamil.com
    Philadelphia USA

  • Gurpreet Singh Kainth

    September 29, 2018 at 11:24 am Reply

    That’s really great Richard. I have been looking for something like that on YouTube for quite some time. Other than Mike Pallister’s video, I couldn’t find much of this great level.
    I believe MS EXCEL may rethink into bringing out a new version of Excel with some predefined library functions which makes programing easier for AI ML in excel.

    Thanks and regards
    Gurpreet Singh
    gpskainth@gmail.com

    • Richard Maddison

      January 1, 2019 at 9:29 pm Reply

      Thanks Gurpreet!

  • Flemming Mathiesen

    November 12, 2018 at 1:40 am Reply

    Hi Richard
    I’m complete new in Neutral Network, but seeing your videos makes me interesting in learning more.!
    Can I ask for a copy of your Excel files and if you have a homepage to go for getting the basic principles.
    Regards
    Flemming,
    Denmark

    • Richard Maddison

      January 1, 2019 at 9:05 pm Reply

      Hi Flemming,
      Thank you for your interest. I’ll also be adding a download page to this site with other models as I clear any remaining bugs. I aim to do this in the next couple of weeks.
      Best wishes,
      Richard

  • mimeid

    December 21, 2018 at 3:38 am Reply

    Hi !!!!! It s so amazing !!!!! I never see that !!!!! can I download this file or not !!!
    grandeloterieus@yahoo.fr

    Tks a lot

    • Richard Maddison

      January 1, 2019 at 8:58 pm Reply

      Thank you for the encouragement. I’m going to add a download section to the site in the next couple of week and will have a bug free version of the Convolutional Neural Net associated with this post for you very soon.
      Richard

    • Richard Maddison

      January 1, 2019 at 9:23 pm Reply

      Hi
      Thanks for the encouragement. I’ll be adding a download page to this site with other models as I clear the remaining bugs. I have a lot more on my YouTube channel https://www.youtube.com/channel/UC44Q4IXVrU6qUtezNU9X_Uw?view_as=subscriber all relating to Neural Networks in Excel. I’ll also be posting what I think is the first Capsule Net NMIST digit classifier in Excel this week – I just managed to fix this yesterday but now have the task of writing about it.
      Best wishes,
      Richard

  • Stephan

    March 13, 2020 at 1:46 am Reply

    Hello Rich,

    this is great, I would like to download it to understand CNN once and for all, can you provide a link ?,
    would be Wonderful. Thanks in Advance.

    cheers

  • Stephan

    March 13, 2020 at 2:13 am Reply

    Hello Rich,

    either I order a book or I pay for your Excel, I can also pay for your Excel :-), does your CNN include
    backpropagation, if yes, I am really inetersted, tell me your opinion, cheers

  • Stephan

    March 14, 2020 at 1:03 am Reply

    Hello Rich,

    can I buy your CNN Excel sheet?.
    I really want to know how it works, especially backpropagation, I don‘t mind if there are Minor bugs inside, thanks in advance, cheers Stephan

  • Mohammed H. Khayyat

    June 5, 2020 at 6:04 pm Reply

    Hello Richard,
    I’m really impressed with your work and I’m trying to learn by reverse engineering so I’d appriciate if you can send me the file to my email:
    Mohammed.h.khayyat@gmail.com

    Thanks!

    • Richard Maddison

      November 16, 2020 at 8:49 am Reply

      Sending it now.

  • Kévin VANCAPPEL

    November 3, 2020 at 1:58 pm Reply

    Hello Richard,

    thank you so much, it’s really interesting.

    Did you add the “download section” on your website please ? I don’t find the excel files associated to this CNN.

    Thank you again, it’s amazing.

Post a Reply to Richard Maddison Cancel Reply