Brief Overview of G-Ramp - Simple AR Model

Note:- Since G-Ramps platform is under development, you will find missing steps or missing parts of already mentioned steps. That is, you will find some arguments which are either not proved mathematically or not supported by experimentation. With time, there should be a lesser and lesser number of such arguments. If there are such arguments, then I should at least fix them temporarily by providing a good reference for each such argument.

Between any two adjacent steps, either it will be explicitly mention that there are missing steps between these two steps. Or there may not be such mention of missing steps. Similarly, there may or may not be explicit mention of missing parts of any already mentioned steps. Text in square brackets ‘[]’ is used to mention about missing steps or about missing parts of already mentioned steps.

A step title below starts with “LS” meaning logical step. Steps should be numbered like “LS#1 …”. But putting numbers during development creates difficult because it is not straightforward how to increment the numbers when steps are added in between. So, until a g-ramp is complete and contain all the steps, steps are not numbered.

Brief Overview


[Steps about making SGD convergence faster. And fixing manual tuning of learning rate. Which includes steps about momentum, nesterov etc. to Adam]


Note:- To be honest, I haven’t read much about autoregressive model. I have put autoregressive model in this ramp because it is very intuitive to understand and also very easy to implement. Therefore, it works as a good but non-trivial (because NN is involved) baseline to compare with VAE. Being intuitive does not mean that I should be able to logically derive it easily. Not sure if I how derived autoregressive model in the best way possible. I have to go through its resources to get better inspiration. I will update this ramp when I feel that there is a better way to derive it.