Skip to main content

EnVision

Search This Blog

Computer vision has no paradigm to follow

November 10, 2011

Get link
Facebook
Twitter
Pinterest
Google+
Email

It really is like walking in the dark, with some flashes of brilliance from other fields.

Get link
Facebook
Twitter
Pinterest
Google+
Email

Comments

Post a Comment

Popular Posts

The Black Magic of Deep Learning - Tips and Tricks for the practitioner

I've been using Deep Learning and Deep Belief Networks since 2013.
I was involved in a green field project and I was in charge of deciding the core Machine Learning algorithms to be used in a computer vision platform.

Nothing worked good enough and if it did it wouldn't generalize, required fiddling all the time and when introduced to similar datasets it wouldn't converge. I was lost. I then caught wind from Academia, the new hype of Deep Learning was here and it would solve everything.

I was skeptical, so I read the papers, the books and the notes. I then went and put to work everything I learned. Suprisingly, it was no hype, Deep Learning works and it works well. However it is such a new concept (even though the foundations were laid in the 70's) that a lot of anecdotal tricks and tips started coming out on how to make the most of it (Alex Krizhevsky covered a lot of them and in some ways pre-discovered batch normalization).

Anyway to sum, these are my tricks (that…

Classic Machine Learning Literature

I'm often asked by software engineers on what to read to get into the Machine Learning world
For that purpose I've compiled a list of Machine Learning and Applied Mathematics books that I've used to gain a deeper understanding.

Machine Learning We start of with the classic but very dated "Machine Learning" by Tom M. Mitchell.
This was the first one I read on the subject. Low on math, high on intuition, it is a descent introductory book. You can easily implement most of the algorithms described and get a fair understanding of what's going on. First couple of years in the business you may use it as basic reference but after that you will need the math heavy books.

Pattern Classification We continue with my personal favorite "Pattern Classification" 2nd edition by Richard O. Duda, Peter E. Hart, David G. Stock. This impressive book is heavy on applied math, low on proofs and very readable. It is better used by beginners as well as experienced machine…

If you don't define it then how can you understand it ?

A model's representational capacity is its ability to fit a wide variety of functions. Models with low capacity may struggle to fit the training set (high training error). Models with high capacity can overfit by memorizing properties of the training set that do not server them well on the test set.

Overfitting is the situation where a learning algorithm achieves low training error but high test error. Overfitting is sign of poor generalization.

Simpler models (smaller hypothesis space and smaller capacity) are more likely to generalize (small gap between training and test error) however complex models are more likely to achieve low training error.

In practise the learning algorithm may not be able to find the best model among the model's hypothesis space. This additional limitations such as the imperfection of the optimization algorithm mean that the learning's algorithm effective capacity may be less than the representational capacity of the model family.

Statistical lea…

Powered by Blogger