I wrote a blog/tutorial here, describing Adagrad, RMSProp, Adam, SC-Adagrad and SC-RMSProp in simple terms, so that it is easy to understand the gist of the algorithms. So, you created a deep network ...
A big part of AI and Deep Learning these days is the tuning/optimizing of the algorithms for speed and accuracy. Much of today’s deep learning algorithms involve the use of the gradient descent ...