G-Ramps - Curiosity Driven Natural Mastery of AI
A platform that helps you naturally become a "non-PhD" AI researcher, where curiosity is the only prerequisite.
Note:- Whatever motivation I mention to propose new teaching style may be not totally correct but definitely most of the parts are correct. I will change it in later version, if I find the need for it. But I am very much confident and excited about the new teaching style. It will be useful to broad audience regardless of what motivation inspired it.
Also, this blog is in draft version currently and you will find grammar mistakes, which I will be fixing in later versions.
Introduction
There are many mountains with different level of difficulty to climb. Mount everest is the tallest mountain but it is not the difficult mountain to climb. Apparently, K2 is the most difficult mountain. Difficulty of climbling a mountain is due to multiple factors includng altitude, weather, avalanches, route complexity e.g. steeper route, and available supports like guides, fixed ropes, oxygen etc.
Let’s consider an analogy of climbing mountains where we are assuming that all people have great mountaineer inside them. In the analogy with mountain and mountaineering, we are comparing mountains to any topic or subject which can be learned. Successfully climbing a mountain means deeply understand or mastering a topic. AI algorithms are like difficult mountains in the sense that efforts to master them is many fold times the other “not so difficult” topics. We all are intelligent just like a great mountaineer inside all of us. Students who are just “curious” to master AI are just like those people who want to summit difficult mountains and want to be confident enough, so to be able to summit any difficult mountain.
All can summit any difficult mountain successfully only if they unveil great mountaineer inside them by realising it. There is exponential distribution. As one moves towards greater amount of realisation, number of people with that amount of realisation exponentially decreases. There are exponentially more people who have not realised it much than people who have realised it to greater amount. Out of all curious people, very few realises fully that they have a great mountaineer inside them. Rest of the people only partially realises it to different extent but do not fully realise it.
To climb any difficult mountains, one may or may not first train themselves either formally via courses or informally by first climbing easier mountains. One may or may not have to first put required efforts in training before they can climb any difficult mountain. The very few ones who fully realises, they can directly climb any difficult mountain successfully without going through any training. They can learn mountaineering skill while directly climbing on difficult mountains. Once they learn skills, then they will be able to summit any difficult mountain relatively easy. But they will learn in unstructured manner, so it will take some time before they can learn all mountaineering skills. So, they will go through proper training so that they can learn skills in a structured manner and hence in less time. Majority of the people who do not realises fully have to go through training first. So that they can fully realise after learning mountaineering skills. Full realisation is necessary before one can attempt to climb any difficult mountain because it is a matter of life and death. And one do not want to be in doubt while climbing difficult mountain that whether they can successfully summit or not. Doubt may interfere with their mental state and may let them take more time than needed to climb the mountain, which in turn make their physical health more worse.
The one who fully realises, they will be able to go through training very naturally, which is nothing but they are able to put required efforts for training effortlessly.
The ones who do not fully realises won’t be able to put efforts effortlessly, i.e. they won’t train naturally. Because they are not sure whether they will be able to put all required efforts. Because they are only curious about climbing difficult mountains and they are not sure that training will unveil great mountaineer inside them and hence let them climb any difficult mountains. While training, one is away from difficult mountains and its condition. There will be a doubt factor that whether they will be able to satisfy their curiosity of climbing any difficult mountain or not. And because of this, there curiosity may also drop and which will create further doubt that whether they will be able to put required efforts for training or not. Because of all this doubt, one will constantly think why to put efforts, if it is not fruitful or does not satisfy curiosity. Doubt will not let you put required efforts effortlessly. It will try you to stop putting efforts. Training is like you want to move forward but against some force. Different people will have different level of doubt factor. More the doubt factor, less efforts can be put effortlessly. And therefore they will put different percentages of required efforts effortlessly depending on their doubt factor.
Very few will be able to put all the required efforts and fully realise. Hence, they will be able to manifest successfully that they are great mountaineer by summiting enough difficult mountains. Very few because majority of people have very little or no realisation, so they will be moving against a large force. So only very very few people will be able to sustain it and put required efforts. We are considering this because we have to consider exceptions who can sustain against large force. And very very few people will move against very little force because they have more amount of realisation, so they will be able to sustain it easily and hence put required efforts. Majority of people won’t be able to sustain and put all the required efforts. Depending on at what point they give up, they will be considered as dreamer, or as average mountaineer, or as an above-average mountaineer but not great mountaineer.
Training was supposed to fully realise those who didn’t fully realised yet. But since training process is not natural, only the ones who already fully realised are able to complete successfully and also those rare people who are able to sustain effortfulness. Shouldn’t curiosity be the only prerequisite to become great mountaineer? And able to climb any difficult mountain shouldn’t be just for those people who already fully realised. It should be for everyone who are curious.
For curiosity be the only prerequisite, one have to make the process of realisation natural. A person should feel like he/she should be able to put all required efforts and there shouldn’t be any doubt factor. Then only putting efforts will be effortless.
Why naturalness? So that anyone who is curious, moves forward without any force against them, so that they will be able to move forward successfully without anyone giving up in between. And those giving up are the only ones who didn’t want to move forward at the first place and were not curious at all.
If there were “help” which include stairs to the summit of a difficult mountain with overall bearable environment till summit, then anyone who wants to summit (the “curious” ones) will be able to summit. Because they are curious to climb a difficult mountain and by looking at stairs and bearable environment till summit, they will think they can definitely put the required efforts because it will satisfy their curiosity successfully. Since there is no doubt factor, they will be able to put required efforts effortlessly. Hence now it is natural for them to climb the difficult mountain.
Once they successfully summit, they would immediately know that they could have done it naturally with lesser help i.e. with more steep stairs and less bearable environment. And once they try with lesser help, they would again think that they could have done with even less help and so on. Slowly, they would realise their potential that they can climb naturally with lesser and lesser help. And eventually after enough summits with lesser and lesser help, they will fully realise that they have a great mountaineer inside them and they don’t require help at all.
There is very similar situation in AI. In this blog, we will talk about similar “help” but for mastering AI. Help in the context of AI has the same philosophy. It will be for all curious people and process of mastering will be natural. We will be calling the help “G-Ramps”. It is the potential way through which any curious student can naturally become a non-PhD “great” AI researcher where only prerequisite is curiosity or enthusiasm or excitement. One do not need to be intelligent or should be good at required maths. Intelligence is a state which currently-not-so intelligent student can reach to with experience given that they already have enough curiosity.
- We want to serve all
- We want to make them best, we want insurance for this.
Outline of the blog

We will first talk about who is our audience. Then we will talk about our goal. Then we will be defining some terms like “AI”, “AI Mastery” and “Learning Resources” for this blog. Then we will talk about how a learner gonna learn and explain in detail the situation we are dealing with. Then we will start to deal with the situation. And at last, we will propose the detailed way and will explain in detail how it will deal with the situation successfully.
[talk about the abstractness of the blog]
- Introduction
- mainly talk about mountaineering analogy,
- motivate the solution,
- why we need support
- Learners
- about intelligence
- talk about both X and Y learners
- introduction of why Y learners need support
- why they want to do non-phd research
- “fact” about school-level knowledge
- Y learners need crazy support and there is no such support for them.
- not ideal situation for Y learners
- Goal
- about goal
- prerequisite = curiosity
- process = natural
- target = AI mastery
- AI
- What do we mean by AI
- means all AI algos
- about research
- about AI algo
- about great AI researcher
- introduced HD maths tools and why it is difficult
- Types of ML
- What do we mean by AI
- How research is done?
- how to propose logical solution
- “solving from first principle”
- “sequence of logical steps”.
- introduce “great AI researcher”
- AI Mastery
- ai mastery = great AI researcher.
- introduce “stage 1”
- “AI fundamentals”
- “stage 2”
- now define “ai mastery” = stage 1 and stage 2
- first stage
- with stage 1 capabilities, second stage
- How to master a topic?
- introduces “learning from first principle”
- or “active prob solving”
- and “passive learning”
- Prove mastery by either giving some examples of questions which one can answer.
- about X learner can do active problem solving
- about Y cant do active prob solving
- given partial few key steps, one figures out all the steps.
- introduces “learning from first principle”
- Learning Resource
- now we know we need to master AI fundamentals to reach to stage 1. And then master enough AI algos to reach to final stage, stage 2 of AI mastery.
- And in previous section, we talked about how to master a topic in general and to master it we need a learning resource.
- So, we will be needing learning resources for AI fundamentals and research papers.
- In this section, we will be talking in detail about these learning resources required to go to each stage.
- types of learning resources:
- Standard
- contains basic ML to some popular AI algos for each problem type.
- Non-Standard
- Standard
- introduce “ramp” word
- ramp classification
- steep and gentle
- from where sequence in ramp should start?
- define gentle, steep ramp. And complete, sparse sequence.
- break complete seq in two halves.
- all available ramps are steep.
- because of prerequi
- given steps are only partially given.
- importance of prerequi
- ramps for HD maths
- Process of going to stage 1
- how Y learner can’t learn HD maths even after AI intro?
- some likes bottom up approach
- shuttle
- some likes top down
- Whether they do bottom up or top down, they will feel not like intelligent and do not deserve mastering it.
- So either they are doing bottom-up or top-down, they will end up into passive learning or just leave mastering AI to master something else.
- passive learning is same as shallow learning
- some likes bottom up approach
- how Y learner can’t learn HD maths even after AI intro?
-
How Y learner learn naturally?
-
G-Ramps
-
Process of going to stage 2
- Conclusion
Learners
Let’s first talk about who is our audience. Our audience is anyone who is curious to master AI and become great AI researcher. We will be calling such curious people “learners” in this blog. For this blog, “mastering AI” or becoming a “great AI researcher” is same thing and will be used interchangeably.
- There are students who wants to deeply become an AI researcher (not because of AI hype these days) but simply out of curiosity and kind of world a “Safe AI” could offer.
What I strongly believe is that intelligence is a state and all human beings can become intelligent if they are already not in that state. Some people are born intelligent and some become intelligent over time. There are definitely no binary states of “intelligent” and “not-intelligent” but a continuum over many states. For simplicity, let’s divide the continuum in two sets of states. First set contain single state which is “intelligent” and second set contain rest of the states of continuum. Meaning second set contain states from “not-intelligent” to having little intelligence to close to being an intelligent but not intelligent. Let’s name the learners falling in the first set “X” and those falling in second set “Y”. X learners are those who join the top PhD programmes and becomes great researchers. Trajectory of Y learners varies, some give up after trying and some become average AI researcher after a lot of struggle and very few become great AI researcher after transitoning to a “intelligent” state.
- There are only few percentage of learners who are intelligent and close to becoming next hinton, lecun, bengio, welling, sutton, sutskever etc. Among rest of the learners, majority are not-so intelligent and others are close-to intelligent.
What I mean by intelligence here (which I may be wrong about, if so, I will change the description but for now, let’s go with this). Instead of defining it, which is difficult, let me mention how it manifests, which should be fine for this blog.
The way it manifests in X learners is that they can learn anything naturally, even if there is very little curiosity for it. Learning naturally means able to put the required efforts effortlessly. Because they know for sure their efforts won’t go to waste, i.e. they will master successfully because either they have good knowledge base by learning similar things. Or they just enjoy being in deep focus state therefore they just pour themselves in effortlessly without caring about the result. They just enjoy putting the efforts which in turn let them put the required efforts effortlessly. After putting the required efforts, they will come out of it either mastering the topic or confidently coming with a conclusion that this topic is arbitrary and can not be derived logically and hence can not be learned. Overall, they are able to invest themselves effortlessly because either they know they will get return or they just simply enjoy in being invested or they have both - good knowledge base and deep focus ability.
Y learners can not learn anything naturally. They can only learn naturally when there is enough curiosity for a topic, i.e., it should be more than required efforts. They have only curiosity to drive them to learn naturally. If curiosity is more than required efforts then there is no doubt factor that whether they will be able to put all required efforts or not. And hence they will be able to put efforts effortlessly. Because they know their curiosity will drive them to put all required efforts.
Otherwise if there is no enough curiosity i.e. required efforts for a topic is more than their curiosity for that topic, then putting efforts will feel effortful. Why not enough curiosity? This could happen in two ways. In one way, curiosity is high but estimated required efforts seem so much more than curiosity. In other way, a learner’s curiosity just drops low because it is not getting satisfied right way and also they may not be sure whether it is ever going to satisfy or not.
Why feels effortful when there is no enough curiosity? Because they are in doubt whether there efforts will be worthwhile in terms of whether they will be able to satisfy their original curiosity of mastering or not. They are in doubt because for reasons explained in previous paragraph, there is no enough curiosity to drive them to put all required efforts. They are not sure whether they will be able to put required efforts before giving up to short-term fun. If they couldn’t put required efforts and give up early then whatever efforts they have already put will go to waste, because they could not satisfy their curiosity of mastering. They are continuously doubting whether their investment of efforts, is worthwhile or not. They are not all in, in terms of their mental capacity. There is a force against what they have to do. It feels effortful because they need to go against this doubt factor or feeling of giving up and do something else which is doable. Because of this effortfulness, they may feel stuck, or may take long time or some may are able to easily give up that topic for some other interesting stuff which they know will give instant gratification.
- mention about exponential distribution across states of Y learners.
- two parts of brain forcing each other to work together. One part wants to stop putting efforts because it thinks putting efforts wont’t be worthwhile, it feels that there is no enough curiosity. While other part is working against it and feels like, there is enough curiosity and wants to put efforts. Since other part is working against the will of first part, putting efforts feel effortful.
What I mean by putting efforts? Anything there is to learn can be framed as problem-solution pair. We all humans have innate problem solving skill. Curiosity to learn something drives us to use this skill. Using this skill is what I meant by putting efforts.
Learners are those people who want to become great AI researchers, so either they want to join top PhD programs or just want to become a “non-PhD” AI researcher.
There are very few learners who will be able to join PhD program, not to mention top PhD programs. Simple reason is that there are just too many learners and there are not as many seats across the world because of limited funding, limited professors etc. So, therefore a good PhD programs requires a good profile as a prerequisite for screening which includes good academic record with good mathematics, research projects etc. As explained above, X learners are ideal candidates for PhD because they will most probably have good academic record and they will also be able to create good research profile and will be able to meet other requirements. X learners will think that these PhD programs are for them only. If they do not get in one cycle, they will keep trying and they will be able to join it. Many of the Y learners who thought to join a PhD program won’t be able to join when they come to know about difficult requirement. Either they won’t have good academic record. Or even if they have, they would just won’t be sure that they will be able to build good research projects so to be able to compete at the global level, so they won’t be able to build good research projects naturally. Except for few exceptional Y learners who will be able to put required efforts effortfully, many Y learners just will give up and won’t be able to join a PhD program.
- Y learners will think either they can’t get it or do not deserve it. You are not willing to put efforts required to build research profile for application to universities. Even if a Y learner join somehow, they know they can’t study under high pressure which a PhD program typically have.
Rest of the learners which are majority of the learners will try to become “non-PhD” AI researchers because they are still curious and want to satisfy their curiosity by still trying to master AI and become an AI researcher. They either want to be independent AI researchers (not affiliated with university or any company) or want to do research in a company. Even if they do not able to publish papers because of its cost, but they will be able to write high quality papers publishable at top conferences. That is, they will atleast be able to improve or create novel algorithms so that they can use those algorithms to solve new problems or solve existing problems in better way at their company.
All these learners are Y learners except for few X learners. Some are those Y learners who wanted to join a PhD program but couldn’t join it as explained in previous para. Others are those X and Y learners who do not want to join a PhD program at the first place. These are learners who are either in a developing or under developed country and do not want to get into financial burden by moving to expensive city in a new country or just can’t leave the country. Or they are in a school or undergrad college. Or can’t go away from home. Or because of age. Or a X learner who can’t find appropriate PhD programme. Because of these reasons, or for some other reasons, they wanna work and research in a company or on their own without doing higher studies.
These learners who want to do “non-PhD” research do not have access to a good research guide except for very few, highly unlikely cases who can join few good companies which have good researchers as a guide and also where their interest aligns with the company goal.
Let’s talk about importance of a research guide. Without a research guide, you can get stuck in gaps, and it becomes harder to fill the gaps in your understanding. This will become clear in following sections. A good research guide can help you in that. Without research guide, a part of your brain can become guide, and it just guide you to do simple stuff, and won’t let you put efforts where required. And since you are not good at research, therefore this brain guide is not a good research guide for you. If there is a good real research guide and you trust him/her then putting efforts for you become natural, you can put efforts effortlessly.
- Intelligent knows or have right confident that they can do it. While rest, in varying quantity, they do not have right confidence that they can do it. So, if there is a good guide who can tell them that they can do it then then they will start doing it and slowly they will become intelligent or they will realise that they are already intelligent. But in most of the cases, these students do not have a good guide.
-
So, either they will find something else to master or they will just spend too much time in shallow learning and waste their precious time.
- Talk about LLMs as research guide.
How can one become an AI master or a great AI researcher without a good research guide who can guide them. Learners have access to plethora of learning resources available online in exchange of inexpensive internet. X learners who plan to become “non-PhD” AI researchers can naturally learn from these available learning resource and become an AI master successfully. But unfortunately learning and mastering from these available learning resources for Y learners is not natural which is a similar situation we mentioned in our analogy in the “Introduction” section.
- But AI is hard to master because it is based on unintuitive maths foundation.
- Only rare students can naturally or effortlessly learn this maths foundation and there are high chances that they are already in a top university or can easily join such university.
-
But for majority of the students, learning maths will be effortful. Depending on their state of intelligence, this effortfulness will be more or less but it will be effortful. Effortfulness is like a friction. Depending on the amount of friction and will power, students will give up at different points in their journey towards AI mastery. Some will give up very early. Some will give up towards the end. But there will be only rare students who won’t give up and will become a master. And they will realise that they are also intelligent.
- Such students would never be able to feel confident in maths. Shouldn’t the curiosity be the only prerequisite to become a researcher.
So, therefore we need similar “help” for Y learners in the context of AI. So that curiosity is the only prerequisite which means nothing but every learner should be able to master AI naturally. This is our goal and we will talk about it in detail in next and in upcoming sections.
To create such a help, we need to know what our learners are already good at. We are assuming that learners are already good at school-level maths. This is a very weak assumption and there are so high chances of it being true that we can safely call it a “fact”. And for the rare case, where a learner is not even familiar with school-level maths can learn school maths “easily” because it is intuitive. There are also high chances that many of the learners are only good at school level maths because maths beyond school-level maths is unintuitive and difficult. Curiosity of all learners can be equally high but state of intelligence can not be same.
- – And there are also high chances that a learner is familiar with high-dimensional maths but not confident about or good at it. So it is very safe to assume that learners are only good at school-level maths for inclusivity, so that we can cover
To make sure that curiosity is the only prerequisite to take “help”, we will be using this fact. By curiosity as only prerequisite, we ensure inclusivity because all learners are curious (this is the definition of “learners”) but not necessarily intelligent.
Goal
Goal is to create a help such that by using it any learner can master AI naturally where only prerequisite is curiosity.
Let’s break it. AI mastery is the output after mastering by taking this help.
There is some redundancy in our above statement describing goal. Help is for any learner means curiosity has to be the only prerequisite. Since all learners have curiosity to master AI, therefore all learners already satisfy the prerequisite of this learning resource.
And process of mastering by taking help should be natural or effortless. Since it is for any learner and a learner can have only curiosity and nothing else like intelligent. So, to ensure that every learner successfully master AI, a learner should be able to naturally master, i.e. without feeling effortfulness. So, there is no thought trying to stop them to master. Naturalness ensures that a learner who starts mastering, will persevere, won’t give up in between and successfully achieve it. More about “Why naturalness?” in one of later section.
Inspiration for such a goal came from Y learners. Such a way will greatly benefit Y learners while X learners will have small but non-zero benefit with respect to already existing learning resources. So, X learner do not care much for such a resource but Y learners care a lot. And since X learners are rare, this learning resource will cater majority of the learners.
- Such a goal is possible because all Y learners are curious and curiosity should be only requirement for becoming researcher.
How such help will help all kind of learners? Such help not only help in creating “non-PhD” AI researchers but also help all X and Y learners. E.g. Very few of the Y learner will like to join a top PhD programme, but they are either not confident about this or struggling in making a research profile. This will help them in that. Or for those who are not “learners” yet but wanna try it and see what it is like.
In following sections, we will first try to understand the goal in detail before moving to how to achieve it.
AI
By “AI”, we mean by all AI algorithms. And in the context of this blog, we not only mean by “AI algorithm” as a way to simulate intelligence (talked about it in one of previous blog titled “AI - Mathematical Formulation of Intelligence”), but actually we are using the phrase to represent any of the discoveries (research papers) in AI.
Let’s first talk about AI research. Technology research or AI research is about finding “logical” solutions to problems. Logical solutions are proposed by researchers in research papers and expands research community’s knowledge logically. “Logical” here is nothing but whatever they are proposing has to be newer i.e, different than proposals in existing papers and is either better or has the potential to be better than other existing solutions. Whenever some researcher’s paper is reviewed for publication in a conference or journal, it is checked that whatever the paper is proposing is logical or not.
There is always an algorithm along with solution in an AI research paper. Either solution is the algorithm itself or there is algorithm to support the solution. In any case, we are calling the algorithm an “AI algorithm” just because it is there in AI research paper.
If solution is AI algorithm itself then that algorithm is either intelligence-simulating algorithm (ex: new model, new loss, new optimiser etc.) or that algorithm is an improvement over existing AI algorithms (recursive definition). Improvement could either be in explanation, evaluation, efficiency or performance. Some examples of algorithms for explanation are: explaining a layer (like batch norm) or activation functions (like why relu over sigmoid) or optimisers (like why SGD generalise better than Adam in some cases) or why neural network can learn and generalise etc.
If solution is not algorithm itself but some theoretical findings, then in this case, algorithm is there to support the theoretical findings, i.e., algorithm is used to do experimentation to check whether theoretical findings are true or not by finding evidence. If true then to what extent it is true. We are focusing on algorithm because “mastering” it meaning including whole research paper and solution. This will get more clear in next section.
Depending on how new is the solution, it can be classified as iterative or novel research.
- “novel” - significant novel change in an existing algo to improve it or completely new algorithm.
-
Novel is with respect to whole research community’s knowledge base.
- [Talk about applied and fundamental researcher.]
- [Make a point that [great?] researcher can do all kind of AI research - applied, fundamental, iterative, novel, proposing intelligence-simulating algorithm or improving existing algorithms.]
We will talk about how to do research in next section. Understanding how to do research will let us understand easily what we mean by “mastering” anything.
Now, let’s talk in detail about “AI algorithm”. Any AI algorithm is nothing but an advanced machine learning (ML) algorithm. ML is a way via computers can learn. “ML algorithm” is nothing but implementation of ML concepts. And any ML algorithm is nothing but an application or combination of high-dimensional (or “HD” for this blog) maths tools. High-dimensional is anything which is more than 1 dimensional.
Note:- Mathematics required to master ML and AI is Linear Algebra, Probability, Statistics, Multivariate Calculus, Optimisation etc. I have used the term “maths fundamentals” in my previous blogs to refer to this required maths. Here I will be using another term for “maths fundamentals”, which is “high-dimensional maths”. Both terms represent the same maths and can be used interchangeably. Not all but most of maths fundamentals is high-dimensional. “High-dimensional” term is just used to emphasis the main difficulty in mastering maths fundamentals.
Basic ML algorithms are simple algorithms like linear regression, logistic regression, shallow neural networks etc. They do not simulate human-like intelligence but they have very useful many real-world applications like house price predictions, forecasting etc. If one is not interested in their application then still one should understand these algorithms because they can help to easily understand ML concepts.
Advanced ML Algorithms or “AI algorithms” are ML algorithms, so same ML concepts are used. Advanced ML algorithms are similar to basic ML algorithms. The only difference is that there is a lot more use of HD maths in AI algorithms. Because AI algorithm implement human-like learning and simulate human intelligence which is more complex than simple problems solved by basic ML. Complexity which AI algorithm deals with is high dimensional highly nonlinear functions whose either input or output or both lies in HD spaces. These HD spaces can be of images, text, speech, music, videos, etc. To deal with such complex functions, one need to use a lot more HD maths. More use of HD maths tools will be like more application of matrix multiplication (E.g., in deep neural networks), use of function approximation which is taylor approximation (E.g. for approximating loss surface for analysis or for second-order optimisation which involves hessian) or use of multivariate normal distribution for approximating high-dimensional probability distribution etc.
There are only few parts of high dimensional maths for which we can draw analogies with 1 dimensional or 2 or 3 dimensional maths and hence they are intuitve. Rest (majority) of parts of high dimensional maths are highly unintuitive, which makes it difficult to grasp, which requires significant efforts to understand.
- In our context, Not all AI algos are ML algos!!!!!! Since, we are using much broader definition for AI algorithm.
- When solution is the AI algorithm itself then it does not mean whole AI algorithm is new. It just mean that solution lies in AI algorithm. Only its part could be new. Most of the time, only one or few parts are new.
How research is done?
Note:- This section may get heavily edited in later versions.
In this section, we will talk about how research is done to better understand what we mean by mastering a topic.
To solve a problem and find a logical solution, a researcher has to first choose some tools. Depending on how much new or different the solution a researcher want, we will chose the tools for solving. Choosing the tools means assuming that these are the only [best?] tools available to solve the problem. And we have to find the better solution given these tools.
More complex the tools we choose, more closer we are to logically solve the problem. More complex the tools, more heavylifting is outsourced to the tools and we have to only do small tweaks. More complex the tools we choose, more potential other complex tools we are discarding. More complex the tools we choose, more cons it will have along with more pros because finding the ideal tool out of exponentially large number of all possible complex tools is very difficult. Hence, more complex the tools, more our assumption is strong and less of it is true. Hence, more complex the tools, lesser the solution is logical - less new as well as less better. Less new because with only small tweak we are able to get to the solution. Less better because of only small tweak, we are only iteratively improving the solution, hence only able to suppress the cons little bit and not entirely.
More simpler choosen tools are, less weaker the assumption is. Meaning, less number of total tools exist, so more chances that we are picking the ideal tool or whatever tools which are picked they are closer to being ideal. Which means, we are more away from solving the problem, bigger solution space, more control we have in our hand to find more logical solution, more control we have in our hand to fix cons of existing complex tools. Because we know the existing complex tools available and their cons, so keeping this in mind, we can first find different and better complex tools for our problem using simpler tools. And then using better complex tools to solve the main-problem in more logical way.
At the simplest level, very few tools exist and all are ideal, perfect tools with no scope of improvement, meaning these are the tools which is going to solve any problem perfectly and lead to most logical solution ever exist. If tools are simplest then we are solving the problem from first principle because then we are not assuming, we are sure that this is the only way to solve the problem logically and therefore solving in most logical way possible.
More simpler the tools, we are more closer to solving the problem from first principle. But at the same time, it becomes difficult to find the logical solution. But even closer to solving from first principle is enough, i.e., using simpler tools and not simplest, is enough. Below even if a problem is solved from simpler tools and not simplest, we will call it “solving from first principle”. In other way, by solving from first principle, we mean that we are not using the most complex tools.
[continuum between applied to fundamental] In research, every paper do choose some tools and then by using it, researchers propose something new on top of it. Depending on the level of complexity of the choosen tools, the “assumption” as explained above will be weaker or stronger and hence the corresponding paper could be part of “fundamental” research or be part of “applied” research.
Applied research is application specific where researchers want to solve a given real-world problem in best way possible from whatever complex tools available. So, they will choose most complex tools which are available like transformer, a complex neural network and assume that they are good enough in themselves and uses it to solve some real-world problem like disease diagonosis.
On the other hand, researchers in fundamental research do not focus on solving any problem from given most complex tools. Rather, they try to come up with better complex tools so that any real-world problem can be solved in a more logical way (more newer and more better). That is, they try to solve a problem from first principle by choosing simpler tools. E.g. they will not assume that current transformers are good enough, they will rather just assume that neural network, a less complex tool or a simpler tool, are good and try propose something which improves upon current transformers. A even more fundamental research wont asssume that neural network are good and they will propose some alternative of neural networks. Some other fundamental research will assume neural network and but propose a totally different way of training them. More simpler the tools, more fundamental the research is.
- Different level of complexity of tools keeps the research going by allowing iterative development. Without different level of tools complexity, only very few number of good fundamental papers will come out. Here, goodness is in terms of whether it can be used in applied research or not.
How solving from first principle looks like? We first find the problem to solve. Let’s call it as “main-problem”. Then we decide what tools we are using and assuming that this is what we know as best available. Given these tools, we find the solution logically. And given these tools, solution is logical. But could have been more logical if tools were less complex.
- If main-problem is complex with respect to tools we have selected then one needs to solve it in iterations or steps. In every step, solution gets improved and gets complex. Next step comes by first finding a problem (let’s call it “sub-problem”) in current solution. Then next step is nothing but the solution of the sub-problem in current solution. Given the current solution, next solution is more complex but simple enough that any learner can come up with it by solving the sub-problem. [but each sub-problem is simple enough that any learner can solve it and hence come up with next solution]
- This iterative problem solving forms a “sequence of logical steps”. Final step is the solution to the main-problem. This sequence is long enough such that solution of main-problem is logical enough.
-
Every step moves closer to main solution by either improving one of its parts from previous step or adding one of its part to previous step solution. Sub-problem from the current solution is nothing but looks like this: what is missing in the current solution in order to solve main problem or current solution solves the main problem but how can we improve it?
- standard part, no options
- arbitrary part, chosen randomly from many options
- a part coming from inspiration, which can be logically compared to other options via mathematics but final comparison is done via experimentations.
- a part choosen by mathematically comparing to its options.
- [Example of sequence of logical step]
-
Let’s call each step a “logical step”. It is “logical” because solution is sensible in terms of other options available or it is solving the problem to some extent as opposed to not solving it at all. So, one has to first find a problem and then “logically” solve it. And a problem can have multiple logically valid solutions. The algorithm depends on which solution one picks at each logical steps.
- what is sequence?
- What is topic, main-problem, sub-problem, solution?
- In engineering or tech research, anything which a student learns, let’s call it “topic”. Any topic is a part of problem-solution pair where topic is a solution to some given problem. Let’s call the problem which a topic solves as “main-problem”. While deriving the final solution or the topic, problems one find in current solution are called “sub-problems”. “Solution” is solution to any sub-problem or main-problem. The final solution is the “topic”.
- By sequential problem solving where every next problem is easy and solvable for any kind of learner and next problem comes from the existing solution.
- If this is the next problem (solvable) then this is the solution. This solution is better than earlier solutions but still have some problem, so now solve this problem. It is an iterative approach where every next solution is better than before.
- And a point comes where your solution matches the algorithm or concept you are learning.
- There could be multiple sequences of logical steps.
- [solving from first principle]
- As you solve the problem, you are “creating” the knowledge in the form of solution to the problem. You are creating something new because you are moving in a direction not taken before.
AI Mastery
In this blog, AI mastery is nothing but able to do “great” AI research. So, a master in AI or “great” AI researcher is same thing.
To become a researcher, one has to first master community’s existing knowledge, so that one can extend community’s knowledge logically, i.e., find newer and better algorithms or algorithms which has potential to be better. And also to get inspiration about what problem to solve or how to solve or both. Mastering community’s existing knowledge is nothing but mastering existing AI algorithms. There is a dedicated section below about how one becomes a researcher after mastering enough AI algorithms.
There are lots of algos and one should master enough AI algorithms, so therefore one should first master a skill which lets you master any AI algorithm naturally”. Let’s call it “stage 1”. Reaching to stage 1 means one have mastered this skill.
- This skill can be mastered by mastering “AI fundamentals”. In next sections, we will understand what we mean by previous statement.
- In upcoming sections, we will talk about how to master a topic and then how to go to stage 1.
After reaching stage 1, one can easily master existing community’s knowledge or AI algorithms. And after mastering enough AI algorithms, one will reach final stage where one is able to do great AI research. Let’s call the final stage “stage 2”.
Now let’s define “AI mastery” which is able to master any AI algorithm naturally and able to do great AI research.
- AI mastery is achieved in two stages. A learner needs to reach stage 1 first where he/she is a master in skill which lets him/her master any AI algo easily. One reaches to stage 1 by mastering AI fundamentals. If a learner is at stage 1 then we can say, he/she has half-mastered AI.
- After reaching to stage 1, one can easily move to stage 2. At stage 2, one is also able to do great AI research on top of skill learned at stage 1. If at stage 2 then we will say he/she has achieved AI mastery or is a master in AI.
How to master a topic?
- define “mastering a topic”
- Assuming there is already a master of that topic exist.
- Mastering means to have all knowledge what that master has of that topic.
- Mastering a topic means becoming a master of that topic. A master of a topic knows everything what whole research community knows about that topic.
- Able to answer all possible questions
- A master of a topic is able to answer all kind of questions related to that topic. Whatever your answers are, they are best with respect to community’s current knowledge and the questions which you can’t answer are not valid questions or their answers are not known by anyone.
- Questions are like: What problem it is solving, why it is solving, how it is solving and why the solution has the form it has. “How” and last “why” is difficult.
- [valid questions]
- Understanding why a part exists, why it exists in such form and not in other etc. Why it exists because it is solving some problem. Why such form? For this, one need to understand that problem.
- Also, do not just assume that problem just exist, question it, whether it really exist, if yes then first understand the problem deeply, why it exists etc. When you understand the problem (sub-problem) and try to solve it on your own. You will find that there exist multiple solution for it.
- So, Mastering is same as knowing sequence of logical steps which the creator of that topic has created. In this sense, creator of that topic is the first master.
- Creator of the topic have started the sequence from the tools which he has assumed and also mastered.
- Mastering the tools means creator already know the steps from basic point to tools he/she has assumed.
- Basic point is a knowledge which every learner who wants to master this topic have already mastered it. In the context of AI, basic point is school-level maths knowledge which we talked about in learners section.
- For others to master, they need to know whole sequence which starts from the point which they have mastered and end at the topic.
- We will also call “knowing the sequence of logical steps” as “logically deriving” that topic.
- So, the start of the sequence and hence its length and whole seq vary from one learner to another because different learner have different level of knowledge which they have mastered.
- A starting point of a learner may be behind the starting point of the creator. That is, a learner may have not mastered the tools used by the creator.
- knowing sequence of logical steps which connect your knowledge to knowledge of that topic. Connecting means this sequence fills the remaining knowledge which you do not know in order to master that topic.
- “Not assuming” [means starting from a point of mastery and not from anything else] is the core rule for mastering a topic. How “not assuming” helps you learn more deeply?
- Now we will go deep into what we mean by it and how do we do it.
- Basic point is a point which one can master easily. From this point, logically deriving the topic means, you are slowly combining the knowledge at basic point step by step. Since you know basic point and you yourselve combining it to form new knowledge at next step, you will master the new knowledge at the next step. You will be able to answer every questions one can answer after mastery as mentioned above. Because as you are combining knowledge at basic point by yourselve, you will know how knowledge is changing. Since you know basic point and you know how it is changing, you will know what has changed, what new knowledge has been created. Whatever knowledge there is at that step, you have created it all by yourselve so therefore you also have that knowledge, meaning you have mastered the step. You will know pros, cons, unique properties etc. of a solution at next step. Repeat the process for every next step until you reach the final step, which is the topic.
- Key thing to note is that you are yourself combining the knowledge. That is how one is supposed to master, to uncover hidden knowledge.
- You not only know the created knowledge, the step but also how it is created. That let’s you explain every behaviour at that step. Not only that, it also lets you create novel knowledge for some problem by yourselve.
- you would try different combination of basic knowledge. Since you are combining, you know what all nuances or behaviours you are inducing in the combined knowledge. Nuances means pros, cons, efficiency, to what extent it is solving problem etc. And you also know how exactly each of these nuances came into existence because as you were combining you could see how each of these nuances are coming to existence.
- Mastering a topic means able to logically derive every part of that topic from a point which you are already confident or master at.
- Meaning you do not assume any part of the topic or take anything for granted or question everything.
- Assuming a part meaning you have not connected that part logically to your existing knowledge.
-
Understanding of that assumed part is just combination of jargons for you, nothing more than that, meaning you do not truly understand that part at all.
- So, how a learner can master a topic?
- One obvious answer is that a master of that topic explicitly mention the complete sequence which starts from a very basic point such that all learners have mastered this point. But since length of the sequence from basic point to topic would be very long such that it would be laborous to mention it completely.
- even if one can do the labour and mention completely, it is not needed. Because one can assume that there are some people who have mastered beyond the basic point and also have skill to re-solve the main problem correspoding to the topic by themselves given the “learning resource” which contain minimal information about the topic - few key steps and that too partially.
- If a learner do not have skill to re-solve then they can be at this stage by following other learning resources.
- Prerequisite let a learner master a skill required to learn from first principle.
- Learning resource assumes prerequi because of two reasons - first,large number of steps and second, it is meant for specific set of learners. More about prerequi in next section.
- re-solving the main problem means re-solving it from first principle.
- Given incomplete but enough information about the algorithm, one can master an algorithm by re-solving it from first principle where you are only assuming the tools which you have already mastered.
- And using the given information to guide the problem solving. That is, we will be only assuming those parts of the algorithms which creators assumed. And we will be using only those options which creators use to compare and so on.
- Using the given information, we will figure out the hidden information/steps and therefore we will have complete information and hence we will master the algorithm. Let’s call this process of mastering “learning from first principles”.
- One expand their own knowledge logically until they master that topic.
- It is similar to solving from first principles.
- Only difference is that we “re-solve” the problem, i.e., we are arriving to same solution.
- Given a topic, learning it from first principles means you are yourself [active problem solving?] coming up to the topic logically by using your own existing knowledge and just taking help whenever needed. As opposed to shallow learning, where we only take the available help and do not figure out hidden steps.
- Logically in the case of mastering a topic means whatever solution/solutions you are coming up while learning from first principles, you should first think and try solving it for yourself. [active problem solving??].
- While learning a topic from first principles, you start from some starting knowledge which you have already mastered and from that knowledge, you try to solve the problem. As you solve the problem, you are “re-creating” the knowledge in the form of solution to the problem. You are re-creating something because you are moving on a path through which others have already moved.
- two ways to figure out missing steps: active problem solving vs passive learning
- In active problem solving, if one knows prerequisite of a steep ramp, then one acts like a creator. Like a creator, a learner will focus on the main problem and how it can be solved. The only difference is that you know the solution, the topic, you are curious about up front. And also know some of the steps leading to the topic in a steep ramp.
- As mentioned above, every step in sequence of logical steps is a solution to a subproblem. And a learner can use his/her problem solving skills to solve these subproblems and figure out missing steps in a sequence.
- By keeping the main problem in mind and using available hints, if a learner try to solve the main problem actively by putting required efforts then he/she will be able to figure out the missing steps and hence will be able to solve the main problem. Hints as mentioned above are the solution itself (the topic) and some of the steps (given steps) leading to it.
- If one does not know prerequisite, then one can apply the same active problem solving process to prerequisites. One can apply this process recursively. E.g. in cases where one does not know the prerequisite of prerequisite etc.
-
Why active problem solving? Because it is the most (or only?) effective and efficient way of learning. Because this is how a steep ramp expect a learner to learn. You will understand this when you will understand the other way of learning which is passive learing explained below.
-
In passive learning, ……
-
Given any topic, X learners will be able to successfully learn as an active problem solver. When X learners look at the topic and given steps in a steep ramp of that topic, they will immediately start to imagine the complete sequence. Or at least vaguely, they will know, how many steps are missing, and what are those missing steps. (They will even figure out some of the easy steps in their sub-conscious mind, so that they do not even have to write them somewhere or think consciously.) They will know, even before figuring out the complete sequence, that they could have come up with this topic on their own or they could have been the creator. And when they start learning, they will be able to naturlly, easily figure out all missing steps and master the topic.
- While learning by active problem solving for Y learners is difficult because as mentioned above, they can not put required efforts effortlessly. Since, all learners wants to master AI and nothing less than that. So, “required efforts” are the efforts required to master AI. And to put these efforts effortlessly, they need to be first sure that they can put these efforts. To make sure this, they first should know what efforts they would need to make. If they think they can put this, then only they will put all required efforts for it because then putting efforts will be effortlessly.
- To find whether they can achieve, they will learn passively, i.e., will keep looking for evidence. But they will always find steep ramps and therefore, they are always in doubt whether they can do it or not and hence it would be effortful to put required efforts.
===============================
===============================
- You can become a master of something by taking help from someone who is already a master. If a master is in direct contact with you either in person or digitally, then that is the easiest way to becoming a master. But since ratio of master to learners is very small, direct contact is not possible.
- [about learning resource]
- Master let other master this topic by sharing minimum knowledge in “learning resource” of that topic. And assumes that one will be able to master by learning from first principle by taking help from the learning resource.
- A good learning resource which can make one master is created by a master of that topic only. Masters usually pour their knowledge in “learning resources” like books, blogs, courses, lecture notes, videos etc.
- All knowledge about any topic could be very large and can’t be poured in a learning resource. Even if someone wants to be laborous and can pour the whole knowledge, still it is not the good idea because you want it modular. And learners who wants to master will have different level of their own knowledge.
- So, a learning resource has to assume some standard prerequisite for that topic. And given this prerequisite (level of learner’s knowledge or skill), they share only key knowledge about that topic such that using this learning resource, a learner can master that topic.
- learning resource helps us find what problem to solve, what options to compare with etc.
- What is available in learning resource in general.
- Creator share what problem it solve (what tools they are using), why it solve, how they are solving (what advanced patterns they are using, unique selling point of paper), why the solution is logical (how they are comparing to other options).
- sharing this information means sharing only few key steps and that too partially. These key steps are of two types. First type of step involve steps which talk about what problem they are solving and why they are solving. And second type of steps talk about how they are solving and why soln is logical. Rest of the steps are completely missing.
-
[modularity, move it to learning resource section] There two reasons to only share key knowledge. First of all, whole knowledge itself is just too big to share. And second, one would like to create learning resource by keeping in mind all the learners. If someone is following a curriculam seriously, then they must already be good at prerequisite. And if someone is not following, then there are learning resources available for prerequisite.
- Creator of the algorithm shares whatever they know about the algorithm, their thinking process by sharing minimum possible information (steps): what part of the algorithm is arbitrary, what is the main contribution, with what options each part is compared with, how each part is compared with other options etc. From this minimal information, one has to extract hidden information.
- Main contribution of the algorithm or paper is one of the most logical part because creators evaluate it well and make sure that it is better or atleast on par with other options available. Rest of the parts can be as logical or could be less logical as well because they are not the main parts and just supporting parts so they are not as well tested. It means less logical parts can even be replaced and still one will be able to prove the logicality of the main contribution.
- Creator also tell where they didn’t invest much time and where is the potential for improvement in the proposed algorithm, what are its limitations.
- Most of these thinking process for proving why solution is logical is hidden. So, therefore one have to think by themselve and prove it. Otherwise the solution will seem arbitrary and you won’t be able to answer all the valid questions. You wouldn’t have all the knowledge which research commnity have about it.
-
[introducing active solving active vs passive learning] - one needs to actively solve the problem as opposed to passively learn from given learning resource.
- here, active problem solving is nothing but re-solving the problem from first principle.
- And therefore, we can also say, active problem solving is “learning from first principle”.
- active prob solving and learning from first principle is same thing.
-
“Active” word is used to contrast against “passive” learning.
- Active prob solving let’s use our brain and since we are natural problem solver, we will figure out all the steps by utilising the help from given learning resource.
-
So, by active prob solving, we can actually extract all knowledge about that topic from our brain. And apparently, this is how one is meant to master anything.
- This is in contrast to passive learning where one is not using enough brain and extracting hidden knowledge but only learning about whatever knowledge easily available, i.e., only getting the knowledge which is available in learning resources and hence only shallow learning.
- Prove that learning from first principle lets you master a topic by either giving some examples of questions which one can answer.
- – These questions can be answered in deep by learning from first principles. What, why at the starting. Last why is the process. How is at the end.
- [more about standard and non-standard learning resource]
- To understand more about mastering, we should know how research community came to know everything about that topic.
- They came to know about it via the first master of that topic, creator of that topic, through his research paper.
-
Research paper is the first learning resource of any topic and it is meant for research community. And it is also the only learning resource for most of the topics. Research community consists of people who are atleast at stage 1 of AI mastery, which means that whole research community is master in AI fundamentals, which in turn means, they are also masters in HD maths [because it is a prerequisite of AI fundas]. So, “research paper” learning resource assumes AI fundamentals as a prerequisite. Hence, research paper only contain minimum steps and that too partially, such that the one who is good at AI fundamentals can figure out all the steps.
-
–So, subsequent masters (2nd, 3rd and so on) will come by going through a learning resource created by first master.
- –For the ones who are at stage 1 or stage 2 of AI mastery.
- –via research paper
-
- what about those who are not at stage 1.
- they have to come to stage 1.
- –about more gentler but still steep ramps, so that non-researchers can become master
- to go to stage 1, one needs to master AI fundamentals.
- To master AI fundamentals, [series of?] “Standard” learning resources are available.
- Standard learning resources assumes HD maths as a prerequi.
- given prerequi, share minimum steps and that too partially, such that rest of the steps can be figured out.
If a topic is part of AI fundamentals then they can master that topic by going
- reason about why research paper and other learning resources are steep
- To understand more about mastering, we should know how research community came to know everything about that topic.
===========================================================
[introducing “learning from first principle”]
- When mastering a topic, you learn to answer above questions not only for the topic but also about its parts. So, you master the parts (combo of HD maths tools) also.
- You master solution in each steps and not just the main solution.
- what is the sub-problem, why to solve it, how the solution is solving it and why its solution has a form the way it is.
- “How” tells you how efficient the solution is.
- “why such solution form” lets you understand the unique properties of the solution but also unique properties of other possible solutions. That is, you learn what differentiates one solution to another. You learn pros and cons of each solution.
- Given a sub-problem, you learn to choose best solution possible (on the basis of efficiency and how well it solves the problem).
- [active problem solving]
- You would try to think of combining your existing knowledge and try to come with different solutions by yourself. Then you will compare the solution and will find the best solution or would just filter out bad solution and be left with more than 1 viable solution.
- This thinking process is what expanding your own knowledge means. And coming up with best solution/solutions makes expanded knowledge logical because you just didn’t randomly come up with the solution, you compared and tested the solution.
- Since you are solving the sub-problem, doing this thinking process, coming up potential solutions and chosing the best ones is easy.
- If you can not come up with solution then check the algorithm and see how creator came up with solution. In most of these cases, creator must have some kind of inspiration which you will find logical but you wouldn’t be able to come with that solution. At other rare cases, solution is not coming from such inspiration and after looking at the solution, you would know immediately that you could have come up with the solution. You would be able to repeat the thinking process but this time you are able to think of solution as if you are coming up with it on your own.
- [About shallow learing]
- top-down approach, take for granted, not thinking much, assuming means you won’t be able to prove why it is logical. You convince yourself that you understood it by giving some thought about it but not really staying there and testing, countering your thought. This little thought at first makes you believe that solution is logical and you think that you have done your part, but in reality you didn’t understood much because you didn’t worked it out enough. Later on, when you revisit, at that time you would think differently and you would know that whatever you understood is either wrong or incomplete or you may find the correct solution completely illogical with respect to earlier thought process. Or you just consciously decide to assume and not go deeper.
-
Mastering uncovers missing steps in contrast to shallow learning or casual learning.
-
[about learning from first principle]
- Mastering something is similar to research in the sense that mastering something is nothing but learning from first principles.
- That is, there is some knowledge that you know you have mastered it. Apart from this mastered knowledge, do not assume that you know something unless you are very sure that you really know. While mastering you know that what you have not mastered yet. Difference between mastering and research is that in research, if a researcher knows that something is unknown then it is unknown by everyone. While mastering a topic, you do not know that topic but it is already mastered by some people therefore it is out there in learning resources for others to master.
- In mastering, answers are new with respect to your own existing knowledge and should be logical.
- What is logical in mastering? When you are able to ensure that the solution is best by comparing to other options. When problem is simple then we know that there is only one best solution.
- Why logical in mastering? Because it ensures that you master the topic.
- Mastering a algorithm is not just about learning that algorithm but coherently placing that algorithm among its options so that one knows pros and cons of each algorithm and when to use what etc.
- [about feeling as creator and no burden of remembering it]
- When you are able to logically derive something totally new, it means you are “creating” something. So, if one knows the sequence of logical steps of something existing then he/she will feel like a creator or will feel like they could have been the creator of it, if they have spent time and thought harder or if they have put required efforts. And also you will feel confident to create a new algorithm or propose an improvement.
- mention about the easiness of each step.
- When you feel like creator, you are not afraid that you may forget this difficult concept or you do not have a burden to remember it. This concept becomes intuitive, it just fits in your mind correctly, it is not out of place in yout mind. It connects to your existing knowledge via a sequence of logical steps. It is not just random number-manupulation. You can create it on the fly if you forget it or it slips through your mind.
Learning Resource or Ramp
[may be move it after the section “How to master a topic?”]
- lets first give new name - ramp
- let’s classify learning resource into gentle and steep.
- what we mean by steep and gentle?
- use the “fact” and learners should be able to master naturally.
- define steep and gentle for a general topic. It could be HD maths, AI fundas or AI algo
- and corresponding complete and sparse sequence.
- In next section, we will explain the importance of gentle ramp.
- more about the importance of prerequi.
- more about standard and non-standard learning resource
- standard
- need a proper name because by learning resource we mean “ramp” but standard learning resource is not a ramp but series of ramp.
- which contain ramps for basic ML and ramps for popular AI algos of each problem type.
- non-standard
- standard
- more about HD maths
=============================
- Let’s talk first briefly about “learning resources”:
- In this blog context, for AI, there are two types of learning resources: basic learning resource and advanced learning resource.
- Basic (Standard?) learning resource is for those who wants to learn or master AI. They are standard because they are specifically designed for learning purposes and hence are easy to learn from. This includes courses like Andrew Ng’s Coursera course or university courses on YouTube, lecture notes, blogs, YouTube videos like of 3blue1brown etc.
- Basic learning resource contain learning resources for both basic ML and advanced ML (AI) algorithms.
- Advanced (Non-standard) learning resource is for those who have reached stage 1 of AI mastery and wants to go to stage 2. Research papers are one of advanced learning resources.
- First of all, let me propose a new name “Ramp” for the term “learning resource”. A learning resource is a ramp because you can use a learning resource for a topic to go from a level of not-knowing that topic to a higher level of knowing that topic.
- Using a word “ramp” for any learning resource let us classify a learning resource into categories depending on how “steep” or “gentle” it is. From now on, I will be using the word “ramp” to refer a learning resource.
-
As mentioned in previous section, ramp “helps” us in coming up with a sequence of logical steps. Ramp does it by containing the sequence of logical steps.
- As mentioned in previous section, for AI mastery one needs to first master existing AI algorithms. To do that, a learner will be looking for a ramp for each AI algorithm and there are plethora of ramps for each AI algo.
-
Given a ramp for an AI algo, for inclusivity, so that every learners, especially every Y learners can master naturally, from where the sequence in a ramp should start? Given the “fact” in “Learners” section above that every learner is already good at school-level maths. And many of the learners are only good at school-level maths. And also because majority of learners are “Y” learners. Sequence should start from school-level maths. Such ramp provides most amount of help, a learner can get because a learner can get help at every step, right from the beginning (school-level maths) till end (mastery). So, even if a learner is at the other end of continuum, most far away from intelligent state, he/she will be able to master naturally. Why so? We will be explaining this in detail in one of later section.
- If a ramp for an AI algorithm contains sequence that starts from school-level maths knowledge and ends at that algorithm with no missing steps, then we call that ramp a “gentle ramp” and we call the sequence it contains a “complete sequence of logical steps”.
-
And if the ramp contain a sequence which is not complete in above sense, that is, sequence has missing steps, then we call that ramp a “steep ramp”. So, even if a single step is missing, then that ramp is steep but close to being a gentle. And let’s call this incomplete sequence a “sparse sequence of logical steps”.
-
Complete sequence of logical steps for an AI algo can be broken into two halves. First half connects school-level maths knowledge to high-dimensional maths. And second half connects high-dimensional maths to AI algorithm which is nothing but application or combination of HD maths tools.
- About importance of prerequisites but there are cons.
- Do not want to recreate the wheel.
- if there already exists great resources explaining some steps in a complete ramp, so there is no need to repeat those steps.
- Every resource make sure that it is contributing something new, i.e., some new steps.
- Many students are following same trajectory. That is, they first pass schools, then bachelor’s degree, then masters, then PhD or direct PhD after bachelors. In each degree, some specific set of subjects are taught. As one goes towards, higher degree, subject become complex and subject in one degree uses the knowledge from previous degrees. And universities ensures that a student enrolled in a master’s or PhD degree has already passed bachelor’s degree. So, they won’t be teaching anything from bachelor’s degree.
- Because of following this trajectory, a student going through a resource of a subject online has high chances that he/she already know subjects on which that subject depends on. Therefore, that online resource will have those subjects as prerequisite.
- It makes the ramp focused on one topic and hence makes overall content on internet modular. Modularity caters to both kind of learners, one who is already familiar with prerequisite and the one who is not. If not modular, then the one who is already familiar with prerequisite have to again go through it and it will waste their time.
- Do not want to recreate the wheel.
- Almost all the ramps for AI algorithms are steep ramp. Because first of all, they assume high-dimensional maths as a prerequisite, which means first half is missing. Secondly, second half will have missing steps.
- After satisfying prerequisites of a steep ramp, a steep ramp contains enough steps, so that any learner can figure out rest of the missing steps, after looking at the given steps.
- Although some (like courses) are less steeper than others (like papers).
- More about missing steps and given steps? Here missing or given is not always black and white, it could be less black and less white too. Meaning, when only very little information about the step is given, then also we are considering them missing. And when only little information is missing from a step, then also we are considering that step as given otherwise we are considering as missing.
- So one have to not only figure out “missing” steps but also all the little information of “given” steps whose only little info is missing.
- Steepness of ramps is a common knowledge and all learners know this. Learners know that HD maths is a prerequisite for AI algo ramps. So, given such resources how a learner gonna learn.
- The way AI is taught in general is by assuming that a student is well versed with high-dimensional maths and then a “steep” ramp is provided for AI algorithms. And it is assumed that a student will be able to go through this ramp.
- Why steepness?
- reason is because a gentle ramp could be longer and also difficult or time consuming to build.
- because high-dimensional maths is taught in undergrad. And maths subject is a generic not just for AI. It is a standalone toolbox subject which is used in many subjects like physics, AI etc to formalise ideas. So, there are dedicated ramps for maths which just teaches maths, so that one can do multiple stuff after going over maths ramps. A learner usually expects to go over such ramps in undergrad. So, ramps for AI algorithms expects that a learner should already be gone over HD maths ramps.
- steepness or gentlenss is the property of a learning resource, not the algorithm.
- And learning resource should be for an AI algo.
- Above we defined steep and gentle ramp for AI algorithm. But same classification is applied to any ramp. If the starting point of ramp is same as our level or below it and if there are no missing steps, then it is a gentle ramp. Otherwise it is a steep ramp. And corresponding sequences are called complete and sparse sequences respectively.
- Ramps for HD maths:
- HD maths contains lots of topics and each topic is complex and large.
- Each topic are like tools which are used to create AI algorithms.
- Books and free online courses are the primary learning sources and which are just collection of ramps on each topic. Each book will try to teach all the topics but not in detail. Meaning, it will contain steep ramps for each topic.
- There can be two reason why they won’t teach in detail - First, book will get lengthy and out of focus. Second, they expect learners to find missing steps for themselves.
- Professors, teachers expects one to find the missing steps by themselves and go to them personally if they can’t figure out.
- But a self-learner can’t ask doubts to these professors easily.Self-learners (“learners” in this blog) can find the missing steps by searching on internet. Either, they will find on stackexchange or in notes by a professor.
- There are gentle ramps too but not that popular. But can be found easily in todays time because of better search.
- Some teach maths specifically for AI or ML.
- HD maths contains lots of topics and each topic is complex and large.
- extra-stuff:
- Let’s call a learning resource a “ramp” because sequence of logical steps is analogous to sequence of steps you take on a ramp where every logical step is as simple as taking a step on a real ramp. These resources take you from one level of not-knowing the algorithm to another higher (in terms of amount of knowledge) level of knowing the algorithm via sequence of logical steps just like a ramp.
- If a ramp does not exactly take us to our goal of deeply understanding the derivation, then we will say it is still a ramp but a steep one. The one where you have to do some hard work or face difficulty while reaching your goal.
-
When one goes over a ramp then he/she learns the algorithm from first principles.
- ramp like (sequence of logical steps with no missing steps) instead of staircase with larger step heights.
- Let’s call a learning resource a “ramp” because sequence of logical steps is analogous to sequence of steps you take on a ramp where every logical step is as simple as taking a step on a real ramp. These resources take you from one level of not-knowing the algorithm to another higher (in terms of amount of knowledge) level of knowing the algorithm via sequence of logical steps just like a ramp.
Going to Stage 1
- In this section let’s understand in detail what we mean by stage 1 and process of going to it.
- Reaching to stage 1 meaning one have mastered a skill which lets you master any AI algorithm naturally.
- conclude that, to reach stage 1, one needs to be intelligent.
- There are lots of AI algorithms out there and total number is only increasing every year. The skill lets you master “any” AI algo meaning a learner could choose a very old algo, or freshly minted algo or an applied AI algo or fundamental algo or theoretical algo. And almost all the algo which one will choose are very difficult. And skill lets you master any algo “naturally”. Which means if one have this skill then he/she is nothing but a X learner and hence in intelligent state in the context of AI because only a X learner can learn naturally anything. So process of reaching to stage 1 must make a Y learner, a X learner in the context of AI. How is it possible? We want a learner to be X learner only in the context of AI because a learner could be a Y learner with respect to subjects apart from AI and it should not affect our goal achievement.
- easily: for X learners, it is already natural, so for them, easy means to take less time. For Y learners, easy means natural and also less time.
- There are lots of AI algorithms out there and total number is only increasing every year. The skill lets you master “any” AI algo meaning a learner could choose a very old algo, or freshly minted algo or an applied AI algo or fundamental algo or theoretical algo. And almost all the algo which one will choose are very difficult. And skill lets you master any algo “naturally”. Which means if one have this skill then he/she is nothing but a X learner and hence in intelligent state in the context of AI because only a X learner can learn naturally anything. So process of reaching to stage 1 must make a Y learner, a X learner in the context of AI. How is it possible? We want a learner to be X learner only in the context of AI because a learner could be a Y learner with respect to subjects apart from AI and it should not affect our goal achievement.
- As mentioned in “learners” section, what we mean by intelligent. There are three ways:
- deep focus
- good knowledge base (good with tools and patterns)
- or deep focus and good knowledge base
- [may be talk specifically about Y learner here that how Y learner will become X learner. And then about X learner later on] There are three ways to become intelligent. In the context of AI mastery, one (Y learner) “can” reaches to stage 1 and becomes a X learner, if not already is, by building a good knowledge base. And not generally by learning to deeply focus easily. A Y learner “can” build good knowledge base by mastering AI fundamentals, which we mentioned in one of previous section. And one can master AI fundamentals by going through standard learning resources.
- –extras:
- knowledge base involves HD maths and how to apply it.
- if one have good knowledge of this, then that learner is intelligent in the context of our goal.
- Because this knowledge base is the only difficult requirement for AI mastery. The only other requirement, which is inspiration, will be met very easily once we have a knowledge base. This other requirement, for most learners, is already partially fulfilled.
- So, for AI mastery, one needs to be very comfortable with maths fundamentals (or high dimensional maths).
- –extras:
- To understand knowledge base, we have to understand AI algorithm a little.
- All the steps in complete sequence of an AI algo can be classified into three types:
- steps deriving HD maths tools used for the AI algo
- steps which derive basic ML part of the AI algo
-
steps which derive advanced ML part of the AI algo
- Steps of first two types are completely missing from non-standard steep ramps. All the steps in a steep ramp is made only from some steps of third type and that too partially. Meaning, some steps of third type are also completely missing and whatever is present in a steep ramp is only partially available.
-
Reaching the stage 1 means able to naturally figure out all steps of any AI algo, where each step is one of three types above.
- Extras:
- already mentioned, AI algorithm as combination of HD maths tools and is also an advanced ML algorithm. Now mention patterns:
- basic patterns - coming from basic ML
- advanced patterns - used on top of basic so that the whole combo (basic+advanced) is suitable to solve AI problems.
- basic patterns are less in number.
- advanced patterns are large in number.
- How this knowledge base makes one intelligent? How mastering AI fundamentals exactly lead us to stage 1?
- Let’s now talk about transition from not-so intelligent states to intelligent state.
- HD maths is a prerequi of Standard learning resources.
- Standard learning resources makes you good at all basic patterns.
- And it realises you that you can figure out steps following advanced patterns given the “non-standard” learning resource of that algorithm if you know HD maths.
- can be figured by you by taking help from available steps.
- To master any easily, one have to first master AI fundamentals and HD maths which is assumed by creators and therefore they skip steps.
-
“Standard” learning resources makes you able to master any algo easily by bringing down the efforts required for any algo.
- X learners are already at stage 1.
- They are. But standard learning resource teach them basic patterns more easily, rather than learning it via non-standard learning resources. And once they learn about basic patterns, they can more naturally master any AI algo as opposed to not going through standard learning resource.
- Are X learners already at stage 1?
- Yes, but it just the matter of time. If they first master basic ML algorithms then they will be able to figure out some steps very easily as opposed to figuring out those same steps in AI algos which may take more time. And if they are followin standard curriculum, then they will first go through basic ML algorithms before AI algorithms.
- About learning HD maths? They may take bottom up or top-down approach or hybrid approach.
- they are type 1 X learner, after mastering AI fundas, they become type 3 X learner.
- Now talk about mastering AI fundamentals through standard learning resource.
- HD maths is a prerequi
- different possibilities about when each learner encounter HD maths.
- About X learner will learn HD maths
- About why Y learner can’t learn HD maths from given ramps.
- bottom up
- top down
- In next section, we will talk about how a Y learner can master their “first” AI algo and in next to next section, we will talk about how a Y learner can reach to stage 1, i.e., able to master any algo easily.
======================================================
- [about HD maths, basic and advanced patterns]
- To figure out first type of steps, one need to master HD maths.
- HD maths tools are just different combinations of school-level maths tools. And the way how school-level maths tools can be combined is restricted by rules or patterns from linear algebra, probability and multi-variate calculus. These rules are limited, so there are limited combinations and hence limited HD maths tools. But since HD maths tools are combination they will be more in number than school maths tools.
- And then learn to apply these tools to solve problems in the context of AI. Given an AI problem, the way algorithms solves a problem is by combining multiple different tools or also same tool multiple times, in a nested manner.
- In each AI algo, these tools are combined by following two types of patterns - basic and advanced patterns. A pattern here is some combination of HD maths tools which repeats across the algorithms. Basic patterns are limited in number and come from basic ML algorithms. They repeat across all ML algorithms - basic and advanced ML (AI). Advanced patterns are large in numbers and are used only in AI algos, on the top of basic patterns.
- What are advanced patterns?
- The “main problems” in AI are very complex. Basic patterns helps one build basic solution. This basic solution can be improved in many ways, i.e., there are multiple directons one can take from basic solution to improve upon. There are multiple directions because the main problem is very complex. To improve the basic solution, one can find different sets of sub-problems in the basic solution or different sets of solutions to same sub-problems.
- Advanced patterns are used to create solution to sub-problems in basic solution.
- Examples of advanced patterns are mixture of gaussians, EM algorithm, taylor approximation etc.
- To figure out first type of steps, one need to master HD maths.
- [stage 1 means intelligent]
- In the first stage, “any” algorithm means it can be a algorithm which a learner is seeing for the first time. It could be an old algorithm or a new algorithm. Since there are many algorithms out there. So one may not have mastered every algorithm while mastering AI fundamentals
- [–and it is not possible to have a gentle ramp for every algorithm out there because it is expensive to build.]
- So, one should be able to master easily a previously-not-seen algorithm whose only steep ramp is available (because atleast its research paper is available if not a blog or video).
- That is something which only a X learner can do. It means, after becoming a AI master, a learner should also become a X learner if he/she is not already.
- And as mentioned in “AI mastery” section above that one should be able to achieve AI mastery after mastering some existing AI algorithms.
- It means one should become a X learner after mastering some existing AI algos.
-
So, in this way, are X learners already in the first stage of AI mastery
- – To achieve AI mastery, one should be good at (or master) HD maths and how it can be applied. It means you know all the tools required for AI and how they can be used. In one way, after becoming good at this, a learner becomes a X learner.
- the derivation of an AI algo
- which means not assuming anything and trying to solve problem from whatever knowledge you have.
- we have assumed that we are only good at school-level maths.
- So, from this knowledge, we have to understand HD maths tools used by the algo and how these tools are applied.
- In this way, understanding few AI algos from first principles let us achieve our goal of becoming AI researcher. Because we do not only understand algorithms but also understand HD maths and how HD maths is applied for all possible algoriths or atleast we become good at learning new HD maths itself easily. We get a sense of HD maths in general.
- Mastering a algo is like squeezing as much information one can squeeze about that algorithm but also about how to better master next algo (going to stage 1) and once you master how to master each algo (stage 1),
- In one of previous section, we have seen how a learner gonna master a topic via a learning resource. Now we will see how a learner gonna master AI fundamentals via standard learning resources.
- Since all learners know that HD maths is a prerequisite of standard learning resources. There are two possibilities for each kind of learner about at what point they encounter HD maths and how they learn it. Below I will explain them in detail.
- X learner
- HD maths before getting introduced to AI
- may be not enough curious but still able to learn it because it was just easy for them to learn.
- HD maths after getting introduced to AI
- now there is curiosity and it is also easy to learn for them.
- mention two approach, bottom up and top-down.
- HD maths before getting introduced to AI
- Y learner
- HD maths before getting introduced to AI
- HD maths after getting introduced to AI
- How a X learner is able to go over HD maths without enough curiosity?
- Because as mentioned about X learners can learn anything naturally, easily, even if they are less curious about something. When they encounter HD maths in their undergrad before AI, HD maths ramps do give them some motivation about how these maths will be useful. For some, this motivation will be enough and for others they may not be as curious but still since this is one of the subject, there is already some motivation and they can learn naturally, so why not learn it.
- – A X learner will be able to go over a steep ramp with minimal efforts. Because while filling the gaps, they trust their intuition that they have choosen the right direction because of this trust they are able to switch to deep focus mode and work through the missing steps.
- Because as mentioned about X learners can learn anything naturally, easily, even if they are less curious about something. When they encounter HD maths in their undergrad before AI, HD maths ramps do give them some motivation about how these maths will be useful. For some, this motivation will be enough and for others they may not be as curious but still since this is one of the subject, there is already some motivation and they can learn naturally, so why not learn it.
- Why a Y learner is not curious about HD maths before AI intro?
- simply because they do not like its applications apart from AI.
- It seemed useless.
- Why a Y learner is not able to go over HD maths even after AI intro?
- Because they can’t find surity, hence can’t learn naturally.
- About hugeness of HD maths
- Many of the HD maths resources are for general purpose. That is, they are not specific to anyone subject like for Physics or for AI. They teach maths in general, so that it can be used in any more-specific subjects like AI. So, such resources contain lots of topics and each topic is taught in detail.
- There are also few AI specific maths resources like “Mathematics for Machine Learning” (MML) book by Faisal et al. Which is good, because such books have filtered out topics from general purpose maths, specific to AI.
- An AI algo uses only minimal tools and not every detail of tools is needed in the algo. Even if HD maths resources are specifically for AI or ML, they still will teach all the tools used by any AI algo in detail.
- But AI specific resources will contain all the topics required by any AI algo and that too in detail. And it will be in steep form.
- A learner will try to find required efforts for mastering HD maths
- Since these maths topics are complex (e.g. take linear algebra, it seems very random in first glance, not intuitive enough) and they have steep ramps, a Y learner will very easily estimate that these topics have long sequences.
- and then figure out whether he/she can put those efforts effortlessly. They find out that required efforts is way more than their curiosity.
- Main reason of not able to put efforts effortlessly because curiosity is varying. Curiosity decreases because it does not get satisfied when learning HD maths even if that HD maths is specifically for AI. It does not satisfied even while mastering AI specific HD maths because you still need to figure out steps of AI fundamentals. Or if there is surity that one will be able to figure out steps after mastering HD maths then ones curiosity will remain high while mastering HD maths even if it is not getting satisfied immediately because they know it will get satisfy. But this is not the case too, there is no surity that you will be able to master this and this specific algorithms after mastering HD maths.
- Even shallow learning take a lot of efforts such that a Y learner can shallow learn barely naturally. That is, required efforts are just little less than curiosity to learn it.
- Since, available steps in themselves are too many, that even going through them in shallow manner will take significant time.
- Since there are already too many topics and it seems that, each topics have long sequences of logical stpes. Mastering will take too much more efforts than shallow learning, so they definitely can’t master naturally.
- While considering for mastering HD maths they are not calculating required efforts just for mastering HD maths but total required efforts for mastering HD maths and for figuring out all steps in AI fundamentals complete sequences. But they haven’t seen AI fundamentals and their ramps.
- no HD maths resource is mentioning all the steps. One have to figure out all steps.
- Now, they will doubt whether they really need to master HD maths for mastering AI fundamentals. Standard learning resource do not mention that in how much detail HD maths should be learned, that is, do one have to really master HD maths or shallow learning will work.
- So, required efforts looks big enough that their curiosity for mastering AI is not enough to drive them to master the required maths. So, instead of mastering the required maths (figuing out complete sequence) what they can atleast do is that to go only through available steps in HD maths ramps without figuring out missing steps, i.e. learning in shallow manner. In short, learn whatever comes easily.
- And while learning in shallow manner, they will feel that such learning won’t be enough. so they may want to complete their shallow learning as soon as possible so that they can try learning AI and find out whether shallow learning is enough or not. And if not enough, then they will have to go through maths again.
- Even after giving minimal efforts, number of topics are so large that Y learner would feel satisfied after shallow learning maths in the sense that they have given enough efforts.
-
Then try to master AI fundamentals.
- Their curiosity for learning maths won’t boost.
- If there is proper reason how each tool is useful or is used in a algo, or why each tool is useful, then only a Y learner will have enough motivation to go over HD maths resource seriously before going over any AI algo ramp. But in most of the resources, there is no such reasons in enough detail.
- HD maths is vast, how one can keep continuing the maths without apparent progress on AI algorithms.
- So, a Y learner will try to find the enough reason or motivation so that he/she can go over HD maths seriously. One reason all learners could very easily find is the saying that “AI algorithm are based on maths, so you should be good at maths before you can learn a algorithm”. Overall, they won’t be able to find enough reason or be able to gather enough motivation to go over HD maths seriously.
- Some will try the top-down approach first instead of bottom-up or try top-down after bottom-up.
- look at standard learning resource first.
- overestimate the length of complete sequence.
- try to get more accurate estimate but your belief keeps getting stronger that your first overestimate is correct.
-
- What we have seen above is bottom-up approach of learning where it involves learning HD maths prerequisite before going over AI.
- Some learners, instead prefer top-down approach of learning which involves first going through the algorithm in whatever way possible, i.e, what algorithm does, why it does, how it can be applied etc. before going through how it does whatever it does. Learn first how to use blackbox before opening the blackbox and understand how it does. So that, one have better motivation to learn things in deep. So that one wont be lost in maths for long before they can be able to apply the algo. So that they can be able to apply the algo as soon as possible and feel useful.
- Top down approach to learning is easy (requiring less efforts) until you open a black box.
- Encountering the algo for the first time will make the learner realise that it is so complex. The learner will automatically estimate the length of complete sequence of logical steps of the algo. Since algo is complex, they will know for sure that the sequence will be too long for their curiosity to stick to it. This is the sequence whose first half is missing and second half is sparse with only steps partially available.
- One can find the HD maths tools used in an algo easily, but a learner won’t be sure whether learning them help him/her to figure out missing steps. Because learning the HD maths tools will itself involve figuring out missing steps in steep maths ramp. And then after that learner will have to figure out missing steps in algo ramp. Now since they have seen the complexity of the algo, they need to be sure whether after learning maths, they can figure out missing steps of algo. They need to sure that the amount of efforts they put should at least leads to as much curiosity satisfaction or more but not less.
- – You wont be able to convince yourself to put efforts to learn those tools because still you do not see “path” or sequence of logical steps which tells you this is how exactly you can use this tool to solve problem or this is how you will end up using this tool. You really need a strong case in order to convince yourself to learn HD tools, but there is no strong case available.
- and try to make sure whether mastering these tools will lead to master algorithm
- So, to make sure this, they may try to figure out missing steps in algo by doing passive learning. But most of the time, whatever ramps they will encounter for the algo will be steep. And some may even reveal some new complex details of that algo.
- In this passive learning, algorithm looks like a bad looking russian doll where each doll in a set looks bad (steep) and instead of inner doll being smaller always, here inner doll can be bigger. Because as mentioned above, the next ramp you encounter for that algo may reveal some new and even more complexity of the algo.
- So you may figure out some number of steps but you will also increase your estimation of the length of sequence. And increase in estimation will be more than the number of steps you figure out because of steep ramps you encounter and newly revealed complexities of the algorithm.
- They know while doing passive learning they will progress by figuring out missing steps [they won’t “figure out” missing steps because they simply haven’t mastered previous steps or HD tools]. But as mentioned above, it involves low confidence, frustration, time consuming and looks endless with no surity when you will be able to master.
- There are two problems, you have overestimated the sequence because of algo complexity and process of finding missing steps in it is also time consuming and frustrating.
- or first find easier algorithm to master to gain confidence first.
- Given this problem, the next obvious step is to learn that algorithm in a shallow manner and find the next algo which can be mastered easily.
- You just want to master a algo so that you get a confidence that you can learn it, your curiosity of learning this algos is valid. This realisation then let you continue the learning process and mastering other algos with renewed enthusiasm otherwise it will only become easier and easier to quit learning AI if can’t even master a single algo.
- The thing is, given only shallow knowledge of HD maths, it is very hard to find an AI algo which can be mastered “easily”, that is, which has a small sequence of logical steps.
- So, now time is spend in just finding the right algo where there is no surity when you will find this algo, and no surity whether in finding the algo and learning it in shallow manner, is actually taking you closer to being mastery of AI.
- [about same feeling mentioned in shuttling approach] In reality, this process may actually be taking you closer to “AI mastery” but very slowly and with disappointments. Because you find a algo and you decide you can not master it easily and jump to next algo. There comes a point where you may lose hope, stop the search, just get disappointed and take some break before starting the search. Because of this disappointment you won’t search with enthusiasm but very lazily, very passively.
- Shuttling approach
- You shuttle back and forth for one or two times then you starts to feel bad about yourselve that you are struggling, trying hard to master and makes you feel that you are not intelligent and do not deserve mastering it. This overall feeling increases with every shuttle, it means it slows you down more because you are having more and more second thoughts that you can’t master. Eventually it slows you down a point that you feel stuck. So, a learner may even stop learning and shallow learn or may even give up.
- – low confidence, time consuming, learning but very slow that you may not even figure out and think that you are stuck.
- Some may not give up and continue shutlling back and forth between maths and AI but after shuttling for some time, they will feel that they are not getting as good a reward as the amount of efforts or time they are putting in.
- Passive learning
- after failing from bottom-up, top-down, shuttling, they will try passive learning.
- look for surity
- about HD maths
- It’s huge, many topics, and that too in detail.
- can’t find in how much detail it should be learned.
- something
- There is enough curiosity but there is lot of unsurity (about length of sequence) whether one can put required efforts.
-
unsurity about how maths should be learned, that is, in how much detail.
- after shallow learning HD maths, now try learning AI algos
- [going through prerequi only makes you skilled at figuring out steps. One have to figure out all steps on their own after going through prerequi.]
- Now after completing HD maths (in shallow manner), when they try to learn AI algorithms from steep ramps, they would have hard time learning, as they won’t be able to figure out missing steps of second half easily. Required efforts will be more than curiosity. Now, they will think that even shallow learning maths required too much efforts and it didn’t led them to mastering AI algos.
- – Amount of HD maths you have to learn is huge before you can feel expert in AI immediately.
- And they know that giving more efforts to maths will take them close to mastery but how much more efforts? What if it requires too much efforts that they will never be able to put then all the already given efforts will go to waste in terms that they will fail to master AI and which will lead them to disappointment. So, to avoid that disappointment and to avoid that waste of time, they will never give more efforts. And they will end up having only shallow understanding of the algorithms.
- The problem is the unsurity about, in how much detail maths should be learned. Because of this unsurity, Y learners mind will always think the worse that the amount of detailed maths required such that they will never be able to learn it in such detail.
- –With this little motivation, they will go over maths in a shallow manner, because of which, they won’t be figure out all the missing steps of both the halves of any AI algo ramp. Which means, they won’t have full understanding of an AI algo, they won’t be able to understand it from first principle. They won’t feel as a creator. They won’t be able to easily improve any algo or propose new algorithm. They will simply accept this and say, “this is the best I can do, I am not good at maths”. Concluding this is easy, rather than keep trying and sticking to the goal of becoming an AI researcher. Because of this, majority of Y learner won’t become X learner and hence an AI researcher.
- shuttle or passive learning
- Many learners wont give up. But now instead of trying to learn as an active problem solver, they will do passive learning. Which involves figuring out missing steps in HD maths and AI ramps by going through different ramps and just look at them, without applying much of brain, thinking hard or applying much efforts, and try to find any new steps which they have not figured out yet.
- top-down approach
- about HD maths
- Can’t find surity, hence, can’t master HD maths.
- In both the ways, unsurity (about how much detail HD maths should be learned and about length of sequence for an algo) and apparent difficulty (huge amount of HD maths and complex algo and its long sequence) easily lead them to believe that required efforts can be way more than their curiosity. And they won’t be able to put those required efforts effortlessly. Or whatever efforts they can put effortlessly given their curiosity, would not be enough and will go to waste. So, either they need to force them to put efforts, which is very difficult or a simple solution is to just leave it and find something else to master. Or they may still stick to master via passive learning because they don’t know what else to do or they can’t do anything else. But because of lack of confidence and enthusiasm, they will move slowly and it will take a lot of time to master.
- lack of confi and enthu because they find many things which they currently can’t do than things which they can.
- hence, couldn’t master AI algo
- Now depending on their level of intuition and number of consequent missing steps, situation could be more unsure, unnatural or putting efforts feel more unnatural:
- Can’t even think of any possible next logical step. Feeling of stuck. Can’t even move one step forward.
- too much long missing initial sequence (compositionality, object-centric learning)
- You see a direction but not sure and uncertain
- not sure if putting efforts will be worthwhile because that direction may contain long sequence of steps and may be you are not sure the direction is right or may be you are doubting that this direction is easy for you.
- because there are too many missing steps, so you can not see that the sequence of steps you have taken is actually going to connect the given next subsequence.
- multiple directions
- examples of difficulty in finding right direction: why gaussian for denoising in diffusion model, or what problem reparameterization trick is solving or update step in boltzmann machine.
- cant deeply focus so time consuming
- you do not know how much efforts to put.
- most likely you will get tired finding the resources to fill the gap.
- best example is of jumping from one paper to another.
-
you won’t use LLMs because you cant trust it, they may hallucinate.
- learning from steep ramp does not feel natural, it feels effortful and tiring. So, much effortful that it may overtake curiosity. Situation is curiosity plus confusion. Curiosity is naturally driving you but there is friction coming from confusion and uncertainty.
- Number of missing steps a Y learner is able to figure out on their own depends on what kind of state he/she is in. If away from intelligence state then he/she may not be able to figure out even smaller sequence. Similarly, if closer to being an intelligent then he/she may be able to figure out longer sequence of missing steps. But still there will be sequence of enough length that no Y learner can figure out otherwise he/she won’t be a Y learner.
- If Y learner can’t figure out missing steps then they will have to look for specific resources for those missing steps.
- Extra-stuff:
- Catering to a spectrum from not-so intelligent to close-to intelligent. Intelligent people are rare.
- Given a ramp, amount of efforts is inversly proportional to level of intelligence. E.g., a not-so intelligent have to put lots of efforts while an intelligent learner have to put minimal or no efforts. No efforts means, he/she already know the algorithm inside out.
- from the point of view of ramp, think of X learner as an experienced climber and Y learner as not-so experienced climber.
- If all the next problems are solvable then this way of learning is called natural learning because now you are using your innate problem solving skills to solve problems which do not require much efforts (because they are easy and easily solvable) but at the same time, they are enough difficult so that you feel a rewards after solving them. This reward motivates you to further improve the solution and you naturally move towards the goal.
How a Y learner can learn naturally? Answer: Gentle Ramp
- As explained in previous section, Y learner can’t learn “naturally” from steep ramps available in Standard learning resources and its prerequisite.
- Why naturally? To ensure mastering something difficult. So that more people should be able to master and not just few. Because mastering is difficult and requires a lot of efforts.
-
How a Y learner can learn naturally? From previous section, we can tell that a Y learner requires a gentle ramp to master naturally. We will understand this in detail in this section.
- So, for Y learnes to master AI fundamentals naturally, they need gentle ramps for AI fundamentals. Let’s call the learning resource for Y learners for mastering AI fundamentals which contain gentle ramps as Special standard learning resource (SSLR).
- SSLR contain gentle ramps for varied AI algos such that they cover all HD maths, all basic patterns and also realises them that they can figure out missing steps following advanced patterns by looking at given steps.
- Since they are only curious about AI and gentle ramp for AI algorithms will cover their respective basic ML and HD maths, so we will have gentle ramps for AI algorithms only.
- Why gentle ramp for AI algo and not for basic ML or HD maths?
- since a learner is only curious about AI, so why to have gentle ramp for basic ML and HD maths.
- And gentle ramp for AI algo will contain whatever a learner needs to learn before mastering AI algo, i.e., gentle ramp for AI algo will contain HD maths and basic ML.
- –
- Because of selective curiosity. Only curious for AI and gentle ramp involve picking the problem and solving it. If you are not curious about something then you do not know what problem to pick or simply wouldn’t like to do active problem solving if problem is given and hence learning won’t be “effective”.
- If there was a gentle ramp for a topic of HD maths, it will effortful to go over it because you are not curious about mastering that topic, even if you know that it is required by AI algos. Because until a learner start learning AI algo and feel the need of that topic, that topic will look useless to a learner because his/her brain will take the benefit of not trusting a human who is saying this topic is required.
- In the next section, we will talk about SSLR in detail. Now we will talk about why Y learners require gentle ramp to master naturally.
- In this section we will only talk about what it is like to go over a gentle ramp of an AI algo.
- Why a Y learner require gentle ramp for an AI algo to master it naturally?
- Gentle ramp solves the overestimation problem because a Y learner can accurately find the required efforts to master that algorithm by looking at its gentle ramp.
- Gentle ramp for an AI algo contain all knowledge there is to master that algo. So, it solves unsurity problem which let a Y learner to overestimate the efforts required for an algo or the efforts required to learn whole HD maths in detail.
- It does not have any prerequisite apart from having curiosity to learn because it uses the “fact” mentioned in “Learners” section above that all learners are already good at school-level maths. Gentle ramp starts from a knowledge level of a learner or below it but not from above it. Makes one master naturally from whatever level they are. Secondly, it contains all the logical steps till the algorithm. That is, it does not contain any missing steps.
- After finding the required efforts, a Y learner will know that his/her curiosity is way more than required efforts and hence he/she will be able to mastery naturally.
- By looking at a gentle ramp for an AI algo, any Y learner will suddenly realise that they have definitely enough curiosity to learn all this. Now your other part of the brain can not influence you into believing that you can not put required efforts because now curiosity takes the driver’s seat.
-
In contrast to steep ramps, where they don’t know how much efforts are required and they overestimates the required efforts which feels more than their curiosity and hence it feels effortful.
- well analogy
-
[rewrite this para] first talk about steep ramp as looking at a dark well. It feels like never ending [overestimation]. But when the well is lit, and when you can see the bottom, then you are like, yeah I can go down. Even if it is very very deep but you know its finitely deep and not infinitely deep. Equivalent of lit well is gentle ramp. Gentle ramp contains all the steps, i.e., complete sequence of logical steps. By looking at complete sequence, a learner will know, it is a finite sequence and a learner’s curiosity is lot more than this.
- Mastering means learning from first principle or active problem solving.
- They will do active prob solving even after all steps are available because gentle ramp are not available for all the algos. They need to become a X learner and they can become after active problem solving only.
- Gentle ramp solves the overestimation problem because a Y learner can accurately find the required efforts to master that algorithm by looking at its gentle ramp.
- gentle ramp lets one know whether they are curious or not.
- Even after looking at a gentle ramp, if someone feels like he/she can’t put all efforts then that person is not at all curious about AI and hence he/she is not a “learner” (our audience) we are talking about in this blog. And our goal is not for such people who are not curious about AI.
- Gentle ramp is the most support [“help”] a person can get to learn, after this, a person has to put efforts on his/her own. A learning resource can not give more support than a gentle ramp.
- So, a gentle ramp also let someone knows if he/she is curious about mastering AI. If going through a gentle ramp feels effortful then that person should know that he/she is not curious about “mastering” AI at all but may be curious about just shallow learning. Because gentle ramp only requires little curiosity to go over it naturally [mention that the one who wants to become a great AI researcher or achieve AI mastery, they have a lot of curiosity.] and even after the most support one can get, a person do not want to learn then he/she is not curious at all.
- why natural when curio>req effs?
- As mentioned in “Learners” section, natural meaning putting efforts effortlessly. And a learner is able to do that when he/she is sure that curiosity is enough to drive him/her to put all the required efforts. And curiosity is only enough when required efforts “feels” not bigger than curiosity, i.e., required efforts should feel like less than or equal to curiosity. Required efforts only feel less than curiosity when as you put efforts, you are satisfying your curiosity and this is why, in this case curiosity is enough because you will keep putting efforts and keep satisfying your curiosity until you put all required efforts and completely satisfy your curiosity.
- Since your curiosity is getting satisfied as you are putting efforts, so you know that your curiosity will remain high and will drive you to put all required efforts. This surity that you will put all required efforts is what lets you put efforts effortlessly which is nothing but you are naturally mastering.
- Curiosity always remain high because as you actively solve, curiosity is constantly getting satisfied.
-
When putting efforts does not satisfy your curiosity then it feels like you are putting unnecessary efforts and eventually your curiosity will drop and required efforts will feel more than curiosity.
- How one can be sure that curiosity is enough? A learner will be able to make sure that curiosity is enough after knowing that with every effort, curiosity should be constantly getting satisfied.
- In the case of a Y learner mastering an AI algo through gentle ramp, a Y learner is able to constantly satisfy their curiosity with every effort they put. A Y learner’s curiosity is to master AI algorithms. Mastering via gentle ramp means a learner is not going through prerequisite first. But right from the beginning, the learner is focusing on figuring out steps of complete sequence of that algorithm i.e. doing active problem solving for that algorithm.
- It is constantly getting satisfied because you are only focused in solving sub-problems which leads to an AI algo. You are only focused in figuring out steps which leads to AI algo and not HD maths.
- And any Y learner with the “help” of gentle ramp will be able to figure out all the steps one by one starting from the begining, which means as they put efforts, they will be able to figure out steps meaning they are constantly satisfying their curiosity of mastering a AI algorithm.
- It is constantly getting satisfied as you actively solve because whatever help a Y learner may require to successfully figure out steps, is available in a gentle ramp because it contains complete sequence. Help from the gentle ramp makes Y learner’s efforts fruitful.
- [path analogy]It feels like moving on a path which you know will lead you to your destination. This is opposed to a case where you need to first move on several different paths which does not lead you to your destination before you can move on a path which leads to your destination.
- As mentioned in “Learners” section, natural meaning putting efforts effortlessly. And a learner is able to do that when he/she is sure that curiosity is enough to drive him/her to put all the required efforts. And curiosity is only enough when required efforts “feels” not bigger than curiosity, i.e., required efforts should feel like less than or equal to curiosity. Required efforts only feel less than curiosity when as you put efforts, you are satisfying your curiosity and this is why, in this case curiosity is enough because you will keep putting efforts and keep satisfying your curiosity until you put all required efforts and completely satisfy your curiosity.
-
How a Y learner will do active problem solving or learning from first principle? And how they will learn HD maths and basic ML while mastering via a gentle ramp?
- advantage of active prob solving apart from effective and efficient
====================
- For Y learners, natural learning only happens when you are curious about some problem and wanna solve it. [That curiosity allows you to put efforts effortlessly?]. But curiosity should be enough to overcome the amount of efforts required.
- And in mastering AI, your curiosity could be very high but AI is so complex and hence its learning resources that amount of efforts required easily overcome curiosity. So, to let curiosity be the driver and not get overtaken, a gentle ramp is required.
- Given the complete sequence in gentle ramp, gentle ramp allows the Y learner to learn as active problem solving.
- Gentle ramp converts the passive learner into active problem solver/creator.
- It does not let you do something (going over HD maths resources) which you are not sure about. Everything, you do while going over a gentle ramp satisfies your curiosity. You do not do anything for which you are not curious about.
- After every step you take, you know you are satisfying your curiosity, which keeps driving you to keep taking the next step.
- And do not let you worry whether you have required knowledge for even trying to attempt it.
- Gentle ramp tells you, you try to come up with steps in a sequence and do not worry at all whether you can do it or not. It means, Y learner do not even have to worry about learning HD maths or worry about finding missing steps on the internet. It just tells you to focus on steps you need to take to come up with the solution (algorithm) from your current knowledge for the problem at hand.
- For every next step, you would feel, you could have think of that step easily. And this is how you take a step after a step and reach the top of the ramp naturally, as if you have created the algorithm.
- Because the learner is now out of worry whether he/she be able to master the algorithm or not. Or whether they are able to put required efforts or not.
- So, what magic a gentle ramp does? It lets you focus on your curiosity of how the problem can be solved or how an AI algo solves the problem, while it does all the heavylifting. We as a human being are a natural problem solver and curiosity drives us to solve a problem.
- [about active prob solving]
- As mentioned above, every next step is a solution to a immediate next sub-problem. Because a Y learner knows that this ramp contains all the logical steps, he/she won’t worry of their efforts going to waste.
- They will try to find the next sub-problem and solving it as a curious natural problem solver. If they think, they have tried enough, they will look up the next sub-problem in a ramp and would think that “I could have easily thought of that sub-problem and its solution” (when they first think this, they are now curious not only for knowing the solution of a problem but also that they could have solved this sub-problem, meaning they are now curious whether they can solve all the subsequent sub-problems and feel as a creator).
- Now, curiosity will naturally drive them to a next step and they would repeat the same process. Sometimes they will come up with the next step and other times, ramp will provide them the next step. Everytime, ramp provides them the next step, they will always think that they could have come up with that step easily. Let’s call “ramp providing the next step” as ramp “helping” the learner.
- So, a good Y learner will now try to use his/her innate problem solving skills to come up with the steps by himself.
- And Y learner will do that because he knows his efforts won’t go to waste because gentle ramp has done all the heavylifting of providing the most help.
- That means, Y learner will be able to put efforts effortlessly, that is, he will learn naturally.
- Not only learning via active problem solving is effective and efficient as mentioned in previous section. For Y learners, active problem solving makes the learner feel like intelligent because they themselves are solving problem and learning and not dependent on many resources. While passive learning makes the learner feels like dependent on many resources, not learning by themselves and hence, feels like less intelligent.
- Feeling like intelligent is important because any learners knows that AI is hard to master. If one feels intelligent than the journey of mastering looks easy otherwise it will look so difficult and time consuming that one will keep feeling disappointed, less confident and will either give up to master something else or will only learn in a shallow manner.
- Importance of active problem solving (for Y learner?):
- It brings a learner closer to AI mastery much quickly because of three reasons.
- First, feeling of being intelligent.
- Secondly, it actually improves your problem solving skills which you will need when improving the existing algorithms or creating novel algorithms. It does this because a learner first “actively” try to solve the main problem and all sub-problems by themselves. And when they have given enough time, they will look at the given steps and the algorithm and then will get some feedback on how they tried to solve problems and how they should have done it.
- Thirdly, it makes one feel like a creator after going through the complete sequence of that algorithm. When they look at the given steps, they will realise that they could have come up with those steps. Feeling of being a creator is important because such a feeling is a sign that you have mastered the algorithm.
- –Gentle ramp makes Y learners feel the creator of the algorithm and that too effortlessly, naturally.
- Overall, it boost confidence and with this renewed encouragement, one will be able to put more efforts effortlessly in later AI algo ramps, which means, gentle ramp makes a Y learner move to a better state of intelligence.
- This is the power of gentle ramp.
- While solving the main problem, a Y learner has to first think of high-dimensional objects and working with them. They have to think of high-dimensional counterparts of the single-dimensional maths tools they know. So, initial steps the learner will work on by taking help (partly or full help depending on the learner) from ramp are steps to derive the HD maths from school-level maths.
- When they derive the HD maths, every aspect of required HD maths tools suddenly makes sense as opposed to earlier. A Y learner understand that these tools could not be any different, or couldn’t be less complex. They understand that why the complexity of these tools is appropriate and needed to solve the problem and why not other tools with lesser complexity. They understand that why these tools are, the way they are. They understand this and they are sure of this because they themselves with the help of ramp come up with these tools logically [i.e., they would have tried to think of all possible options that could solve the problem which these HD tools are solving but they couldn’t find other options, and hence HD tools makes sense to them].
- When they come up with the HD maths, they know that this is not just complex random number-manupulation because they know that these tools are logical combinations of school-level maths tools and this logical combination is stored in their mind in the form of sequence of logical steps connected to their school-level maths knowledge. They needed this logical combination of school-level maths tools because they needed to work with HD objects. Now, they do not find the HD maths complex or unnecessary because they know why it is the way it is.
-
Similarly, in the second half of a ramp, the complexity of combination of HD maths tools makes sense because a learner would come up with this combination by himself/herself logically. A learner will know why the algorithm is the way it is. And how creator of the algorithm would have come up with this algorithm.
- Y learners need confidence that they can do it. If for once, they feel like creator they will continue and won’t give up. Once they go over their first gentle ramp, they will know what experts know about that algorithm, they will know what is meant to be an expert and how to be one. So, now they are not only curious to master next algorithms, but are also curious to become an expert. This will drive them happily in continuing going over next gentle ramps.
“G-Ramps” to achieve the Goal
- talk about how a Y learner will become intelligent.
- In this section, we will talk about special standard learning resource. How many gentle ramps it needs and its difficulty to build it.
- We will see that we do not need every ramp to be gentle.
- about next ramp can be little less gentle.
- this forms “series of ramps”.
- after going through this series, a Y learner becomes intelligent.
Note:- Mention that one do not even need to revise school-level maths. G-ramps should cover enough such maths so that one can revise it while going over a g-ramp.
=============================
- In this section, we will finally talk about how we will achieve our goal.
- Our goal, as mentioned in “Goal” section above is to “create a learning resource such that any learner can master AI naturally where only requirement is curiosity”.
- As explained in previous sections, for X learners, this goal is already achieved as there are standard learning resources which fulfils this goal for them. But this goal is still not fulfilled for Y learners as they need special learning resource which does not exist yet.
-
As mentioned in one of above section that a learner can reach to stage 1 of AI mastery by first mastering AI fundamentals.
- As mentioned in previous section, for learning and mastering AI, one of the (or only?) perfect [standard??] learning resource for Y learners is gentle ramps for AI algos. It is perfect because of two reasons. First, curiosity is the only prerequisite and second, it allows Y learners to master naturally.
- As explained in previous section, after going over a gentle ramp, a Y learner is in better state of intelligence.
- Here is more explanation of this which is more specific to AI. Since patterns for HD maths are limited and hence HD maths tools are also limited, they will repeat from one AI algo to next algo. Given a next algo and its gentle ramp, you may not take much help from it because some patterns and tools will repeat from previous gentle ramps or ramp.
- Same is true for basic patterns in second half.
- After going through some gentle ramps, you will have mastered all HD maths tools and basic patterns. It means you have mastered the prerequisite of AI and basic ML. It means you have figured out first half and some of the steps of second half of complete sequence of all AI algos.
- If already figured out means, we do not need to take help from a ramp for these steps. So, therefore these steps are missing in non-standard learning resources / ramps.
- It means we do not need series of gentle ramps but “series of ramps” where first ramp is gentle and each subsequent ramp is little steeper than previous ramp. Each subsequent ramp can have missing steps without affecting the natural or effortless learning of Y learner.
- What about rest of the missing steps in non-standard ramps which follow advanced patterns. Advanced patterns are large in number, hence less likely to repeat than basic patterns. E.g., they may repeat just within a domain and not outside of it. In gen AI domain, an advanced pattern [mention pattern?] in VAE repeats in Diffusion model.
- Because of this, it is even hard or next to impossible for even an AI master to figure these missing steps in non-standard steep ramps.
- And this is why any steep ramp have just enough given steps so that any X learner can figure out such kind of missing steps.
- While going over a gentle ramp of any algorithm in “series of ramps”, and doing active problem solving, a Y learner should not only look at the algorithm first but also look at given steps in any non-standard steep ramp of that algorithm.
- [May be do not even look at the algorithm, just look at given steps]
- This given steps are intermediate solutions which solves sub-problems.
- When a Y learner actively doing problem solving at any step, i.e., trying to figure out next step. The Y learner should look at the next given step, which could be few missing steps ahead. The next given step will tell the learner what sub-problem to solve and also about its solution (combo of some HD maths tools), which is present indirectly inside the bigger solution which is that given step itself.
- Based on how many ramps you have gone through in “series of ramps” and your knowledge of HD maths tools, you will perform accordingly in figuring out the missing steps in the current ramp. You may or may not find what next sub-problem to solve. If you know the next sub-problem then you may or may not know why to solve. If you know what and why, then you may or may not know how to solve the problem. You may or may not be able to propose some solution by yourself while doing active problem solving. You may or may not be able to find the solution from the next given step.
- Learning about finding what next sub-problem to solve and why to solve is very easy, and one will learn this very early in their journey through “series of ramps”.
- As they master more and more HD maths tools, they will be able to propose some solution to a sub-problem. They will also start to find solution from the next given step if the solution contain HD maths tools which they have already mastered. They are able to do this because if they have mastered some tools then they also deeply know their combination, which is nothing but nested use of tools where one tools is an input to other. You know how your input gets transformed as it moves through the tools in the combination.
- As they master more tools, they will be able to propose better solution and also be able to find actual solution from the next given step. And then they will also be able to compare their solution with actual solution, by doing this, they will learn why the actual solution has such a form and its pros and cons with respect to other solution/solutions.
- When for the first time, they are able to find what problem to solve, why to solve, and able to propose a solution and also able to find the actual solution, compare their solution without taking help from a ramp in a series of ramp, then they will realise for the first time that they can figure out steps following advanced patterns.
- As they continue going through “series of ramps”, they will keep realising this. Their realisation will only grow stronger and stronger. And after going through series, they will know for sure that they can figure out missing steps following advanced patterns by taking only help from non-standard steep ramps which are readily available.
-
Ramps in the “series of ramps” will get steeper also by dropping steps following advanced pattern. Given an algorithm in a series, if previous ramps makes Y learner master in some set of HD maths tools. And if these tools are used in some steps of the sequence of this algorithm, then these steps can be safely dropped from the ramp without hurting the natural or effortless learning of Y learner.
- You may or may not figure out next missing step by using next given step.
- So, “series of ramps” can’t tell you those missing steps for all AI algos explicitly but it will make you good at figuring out such steps by yourself.
- To understand this, let’s understand the second importance in previous section for Y learners of gentle ramp in the context of AI.
- As you go through “series of ramps” and master HD maths tools, you will be able to solve sub-problems whose solution involve advanced patterns on your own in active problem solving mode in more better way. Because solution to a sub-problem involves the combination of HD maths tools. So, as you master HD maths tools, you will b.
- So, instead of making you good at all advanced patterns, “series of ramps” makes a learner realise that once they master HD maths, they will be able to naturally, effortlessly figure out missing steps which follow advanced patterns on their own in active problem solving mode.
-
As they go from one ramp to another in a “series of ramps”, they will keep realising this. After going through all the ramps in series, they will fully realise this.
- One will realise that they can do this. This realisation will “help” in figuring out missing steps which follow advanced patterns.
- [another way to reason about “ramps”]
- How we will create such a learning resource? We already know the prerequisite which is curiosity meaning it is for all “learners”, which means all Y and X learners. We also know from “AI mastery” section above that the process of mastering AI involves mastering some existing AI algorithms.
- And our goal says that the process of mastering should be natural. Meaning a learner should be able to master some existing AI algorithms naturally.
- As we know that Y learners have inspired such a goal and majority of the learners are Y learners. And if we take care of Y learners then X learners are already taken care of.
- Previous section explains that a gentle ramp allows a Y learner to master naturally and it has only curiosity as a prerequisite. So therefore, our learning resource is nothing but gentle ramps for some existing AI algorithms.
- – “Master” AI “naturally” means we need to build a gentle “ramp” for every AI algorithm, until we do not require a ramp (meta-skilled).
- About creating gentle ramp:
- About assumptions: Given an algorithm, there are multiple ways we can break the complex assumptions and go down and touch school-level maths. So, instead of learning HD maths from multiple ways in a single algo, we will be learning through one or only couple of ways per algo. E.g. we will be assuming that neural networks are universal function approximator.
- About implementation and playing around the algo, to get more insights about where it works and where it fails, how it fails, so that one can find the problems in it, which will inspire them to learn better algorithms.
-
As explained in the previous section, a Y learner can only learn naturally when there is a gentle ramp, i.e., full sequence of logical steps.
- AI mastery via multiple ramps: Meta-skill: As you go from one AI algorithm ramp to another AI algo ramp, every next ramp can be more shorter and steeper without hurting the natural learning. That means, every next ramp becomes easier to build and at the same time, your intuition becomes better with every ramp.
- It is because there are some limited number of patterns via which high dimensional maths tools (first sub-ramp) and AI algorithms (second sub-ramp) are created. So, as you go through each ramp, you come to know these limited number of patterns. More the number of patterns you know, more steeper the next ramp can be walked on easily. The more you become good at HD maths, so your starting point for every next algo also gets further away from school-level maths and gets nearer towards algorithm.
- AI mastery via sequence of ramps where first ramp is gentle. And subsequent ramp can be less gentle or more steeper.
- This should get more clear with some examples.
- Initial ramps could be much longer and this could be also the reason that some people may not continue mastering the AI and change the field.
- forget about the pattern or intuition getting better, combination of school-level maths and combination of HD maths tool are going to repeat. This will happen right from the second ramp.
- So, initially-not-so-intelligent learner can become intelligent effortlessly if there is proper aid.
- “going over a ramp” is same as “taking help from a ramp.”
- Before starting a ramp, a learner have to go over assumptions taken by the ramp and description of an algorithm the ramp is for.
- assumptions would be like: go over previous ramps in the sequence of ramps or assuming that NN are universal function approximator.
- description of an algorithm would be like: it is a generative model and this is how it looks like.
- Before going over a gentle ramp, one can also look at the given steps in any easily available steep ramp.
- Since these are the steps typically available means this steps are needed to master the algorithm even for the one who is already master in AI.
- Meaning, without these steps, it will be either impossible, or very difficult or just time consuming to master the algorithm even for AI master.
- starting point of the sequence of logical steps depends on the given assumptions. If algorithm depends on the assumptions only then sequence starts from assumptions otherwise it will start from school-level maths.
- Now go over a ramp. what you mean by going over a ramp?
- going over a ramp simply means taking “help” (explained below) of a ramp.
- figuring out all the steps in complete sequence.
- from the current step, there can be many logically valid next steps. Assumptions of a ramp and description of an algorithm narrows down the possible number of logically valid next steps.
- also, this rule is followed that solution for the given problem goes from simple to complex.
In first principles understanding, we will be going only one level down, which will be enough to connect it to school-level maths. And this should make the ramp “smoother” than almost all the tutorials or blogs available on any topics.
- That is, we will be assuming something but number of assumptions will be very less than standard resources.
- E.g., if we are learning about generative models from first priciples and we have assumed that neural networks are universal function approximator, then we won’t learn neural networks from first principles.
- Note:- We are using the word “assumption” because we are assuming that we “know” some topics, so that we can avoid delving into them and to avoid making the sequence of logical steps further longer.
-
If we do not assume anything than a sequence becomes too large and not focused. For example, if we are interested in learning generative model, we simply want to assume universal approximation theorem so that we stay focus on generative model.
- Extra-Stuff:
- First sub-ramp only teaches you the high dimensional maths tools required for that algorithm. This required maths do not define the algorithm, it is just the prerequisites. That is, this maths tools could be used in any other AI algorithm.
-
Second sub-ramp teaches you how this maths tool could be combined to form an algorithm.
-
Via a ramp for a algorithm, you not only learn that algorithm but how that algorithm may have created.
-
What are the topics the ramp made of and how they are structured or ordered?
- Spoon Feeding
Going to Stage 2
Note:- First draft
- We have seen how our learners both X and Y can reach to stage 1 naturally.
- And in the previous section, we understood how Y learner goes to stage 1 in detail. Same detail is true for X learner except they uses standard learning resource.
- In this section, we will see how they will go to stage 2 and achieve AI mastery.
- Once each learner reaches stage 1, they become a X learner if they are not already.
- So, the process of going to stage 2 is already natural.
- In this section, we just need to understand how the process of going to stage 2 makes one an AI master.
- Learners at stage 1 have already mastered HD maths, basic ML, some advanced patterns like standard neural networks (like CNN, transformers etc.), standard optimizers (like SGD, Adam etc.), losses, taylor approximation etc.
- clear the confusion between tool and advanced pattern. E.g. NN is an advanced pattern and is used as a tool too.
- pattern is a tool and tool is a pattern
- NN can be considered as “assumed complex tool”.
- As mentioned in previous sections, process of going to stage 2 from stage 1 is just about mastering enough AI algorithms. Let’s understand more about what we mean by it.
- In the process, intially learner is reading different papers and exploring AI research just to find more specific research area which is interesting to him/her within AI. AI is broad research area, where main-problem is very broad and general but eventually this is what we really want to achieve. Since it is broad and general, it is hard to solve.
- So, learners look for specific research area within AI, which mean more specific main-problems.
- After exploring the broad AI research, they will find a specific area which interests them more. E.g., it could just be focusing on optimisation part of AI algos.
- Now, read research papers within that specific research area. You will know what main-problems researchers have solved and upto what extent each main-problem is solved. And they will be able to find “open main-problems”, main-problems yet to be attempted or main-problems which are attempted, i.e. solved but there is scope for better solution. They can do this because they know about what an ideal AI algo should be like and therefore they can find the gap between current AI algos and ideal AI algo.
- read research papers within that specific research area
- you will find many “open main-problems” and many new advanced patterns.
- mention that learners are able to master these advanced patterns, meaning they do not need to remember this patterns. They will be able to recall appropriate advanced pattern as they focus on solving the given sub-problem.
=====================
- now talk about how a Y and X learner will go to stage 2 from stage 1.
- multiple factors:
- practise. realisation you can solve or you can solve better.
- inspiration for problem, find problems in existing algos, or find totally new problems.
- able to ask right questions.
- copy advanced pattern as it is like NN, or “diffusion process” in diffusion model.
- not directly copy but as they try to solve the given problem, they can think of multiple solutions because they have seen many solutions in different AI algos. As they try to solve, they will be able to retrieve viable advanced pattern which may not be applied in current situation, hence making the solution new. If they really think, it will work, then they will try to test it via experimentation and compare to other options, if it comes out better, then this becomes the logical solution and hence publishable.
- or if it has the potential to be better after fixing some fixable limitations of it.
- library of advanced patterns
- as you master more and more AI algos, you build the library of advanced patterns which is not same as library of AI algos.
- Given a main-problem, as you solve one of its sub-problem, you will automatically find a suitable advanced pattern from the huge library effortlessly, as if it is just clicking to you and you are not feeling strained in searching for suitable advanced patterns. It is effortless because you understand HD maths and as you solve the problem, suitable HD maths tools is coming to mind. Once the set of tools comes to your mind, you will then easily think of suitable advanced pattern [also mention about NN like advanced pattern, which you are considering as tools]. As you keep solving sub-problems of the main-problem, you will find that you have build a novel algorithm whose parts in itself are not novel but algorithm is still novel because these parts were not used to solve this main-problem before.
- Many or apparently most of research happens like this. Some examples are:
- Diffusion model - diffusion
- normalising flows - change of variable
- vae - change of variable
- object centric learning (slot model) - recurrent model
- SGD flatness analysis - taylor approximation
- bengio’s compositionality theory paper - VAE
- As you keep mastering AI algos, there comes a point where as you solve sub-problems, you may think of some advanced patterns which does not exist yet and you test it, if it gives better result then it will be the most novel algorithm. Where algorithm is novel and advanced patterns used in it are also novel.
- How mastering existing algorithms exactly lead us to be able to improve them or create novel algorithm?
- ??
- After reaching to stage 1. A learner will be able to improve existing algorithms or create novel algorithms only if a learner is confident enough in his/her proposed solution to sub-problems that they may test it via experimentation. And if experiments gives good result then only learner will try to propose it in a research paper.
- Whether a research paper improves existing algorithm or create new algorithm depends on how many steps which follow advanced patterns are newly proposed with respect to previous algorithms dealing with same main-problem. If there are only few new steps while rest of the steps are copied from another algorithm, then we will say a research paper improves upon existing algorithm. If there are many new steps then we will say, a research paper proposed new algorithm or researcher/researchers created new algorithm.
- After reaching stage 1, as they master more algorithms via non-standard ramps, there will be a point where they will be able to propose solutions logically to sub-problems which matches original solution without looking at available ramps. This will boost their confidence that they can propose correct solution. As they continue proposing solution logically without taking help, there will be a point where their proposed solution is different than the given solution and they will have enough confidence to not reject their solution as wrong but rather test it via experimentations.
- Going to stage 2 from stage 1
- This involves mastering existing AI algorithms mainly via non-standard learning resources and building the confident to improve existing algos or create novel algos.
-
This involves mastering enough existing AI algorithms so that one feels at being stage 2. If so, try to improve or create novel algorithms. If not, repeat the process of mastering existing algorithms and trying to improve or create.
- Before improving existing algorithms or creating novel algorithms, one has to master “some” existing AI algorithms or already proposed algorithms. So to avoid proposing what is already being proposed (e.g. creating the same algorithm like MLP or proposing same improvement like SGD with momentum over vanilla SGD). Or to get inspiration (e.g. you find some problems in existing algorithms which inspires you to improve it or you find something in one subarea which can help in other subarea). Or to just learn how current algorithms are made.
- Inspiration will also come from understanding human intelligence or brain.
- – You not only know AI algorithms deeply but also know how this algorithm are possibly created and has ability to create new algorithms.
- [Example of repeatation of advanced patterns]
- Give example of learning change of variable in MVN in VAE ramp is also applied in normalising flows algorithm.
- mastering algos teach you how to do research (going to stage 2).
- That is, you learn to answer above questions not only for the algorithm but also about its parts. So, you master the parts (combo of HD maths tools) also.
- You master solution in each steps and not just the main solution.
- what is the sub-problem, why to solve it, how the solution is solving it and why its solution has a form the way it is.
- “How” tells you how efficient the solution is.
- “why such solution form” lets you understand the unique properties of the solution but also unique properties of other possible solutions. That is, you learn what differentiates one solution to another. You learn pros and cons of each solution.
- Given a sub-problem, you learn to choose best solution possible (on the basis of efficiency and how well it solves the problem).
- Mastering a part means not just remembering some set of jargons but deeply knowing how exactly it differs from other parts. Given a part, you know what you need to change in it to make other parts. You do not learn parts in isolation but also learn deeply the change in their form as one moves from one part to other. As one changes a part bit by bit to a other part, they can see the changes in one set of pros and cons to others. They learn how each bit of the part contributes to the unique properties of the part, to its pros and cons. Learning this lets you create new parts. Given a situation, you can create custom made parts.
- –about how to combine HD maths tools. You will learn that given a situation, you learn how each combo, when to apply them by learning about their pros and cons in detail.
- That is, you learn to answer above questions not only for the algorithm but also about its parts. So, you master the parts (combo of HD maths tools) also.
- researchers doubt everything - problem and its solution, if there is not enough explanation in existing papers. Take the case of batch normalisation layer. The original paper mention that there exist internal covariate shift problem in deep neural networks and proposes a solution, a batch normalisation layer for it. Later on, some researchers were not convinced enough that the batch norm is solving internal covariate shift, so they tried to investigate it and found that the success of batch norm was not due to solving internal covariate shift but solving some other problem. They were able to publish this finding in NeurIPS, one of the best AI conference. So, this is how researcher publishes papers by not assuming something and doubting it first, then coming up with new and better finding.
- If someone wants to learn deeply or master something or learning from first priciples, then do not assume anything, be it a problem or its solution, even if something is very much evident and true for AI masters. For e.g., AI masters know that so and so problem exists in current AI algorithms. That problem may be complex enough that from school-level maths knowledge, a learner wouldn’t be able to understand how they come up to a conclusion that problem exist. If a learner just assume that problem exist then he/she wouldn’t be able to master it solution (why such form) and wouldn’t be able to find new problems because they do not know the process of finding problems.
- More you assume, the less you will learn or less your creative space will be or less you will be able to explore or less novel you will be or less fundamental your proposal will be.
- E.g., in VAE, people say derivative w.r.t. phi cant be pushed inside the expectation or “it is difficult to differentiate through this stochastic component” and to fix this, they introduce reparameterization trick. But why can not be it pushed inside or why it is difficult? Understand this in detail. Stay a bit there. You will understand the problem (sub-problem) instead of just simply understanding the solution (reparameterization trick). After understanding the problem or figure out missing steps, you will find that derivative can be pushed inside but the problem is of high variance of gradient estimate. And this high variance problem is there because of the kind of main problem creators chose to solve. If there was other but similar main-problem, i.e. same problem in other situation, then may be pushing gradient inside is more logical solution then reparameterisation trick.
- Understanding the “problem”, leads you to other solutions of it apart from what is proposed. More than one solution of a same problem, opens up your mind more and gives you more stuff to think about the problem from different perspectives, which may lead you to propose a new solution. Or better understand the proposed solutions (reparametrisation trick) by understanding their pros and cons with respect to each other. This will help you decide when to use which solution.
- Other example, do not assume ELBO is the only possible solution. Or try to solve the problem apart from ELBO.
- which means not only understanding a specific solution but understanding the whole class of solution.
- two factors - first factor, knowing the community’s knowledge so that one can propose “different” algo, or what problem exist and to what extent it is solved and also compare to existing algos.
- Second factor, to easily propose “better”, one can use community’s knowledge as compared to using only your brain to get ideas. By mastering existing algos, you get lots of ideas about how other researchers are solving problems in AI. You learn how each HD maths tools combo solve problems. Learning about problem solving and not just looking at the solution.
- should be able to compare existing algorithms know pros and cons of existing algorithms.
- two things: inspiration (NN, Boltzmann Machine) and realisation (nested HD maths tools)
- one has to master enough AI algos until one feel confident enough to improve or create.
- By looking at many examples, one just learns how to use HD tools and combine them to solve problems in AI.
- they will learn how HD maths tools are applied in AI context.
- They have mastered HD maths tools already. Now, they will learn to find problems in AI. Once they learn to find problems, they will then find new problem. Given any new problem, they will be able to solve it because its solution involves combining HD maths tools and they know this.
- Second factor, to easily propose “better”, one can use community’s knowledge as compared to using only your brain to get ideas. By mastering existing algos, you get lots of ideas about how other researchers are solving problems in AI. You learn how each HD maths tools combo solve problems. Learning about problem solving and not just looking at the solution.
- BM uses the advanced patterns from physics.
Conclusion
- We talked about our goal.
- We broke AI mastery into two stages.
- By doing this, we found that to achieve our goal we just need to create learning resource such that any Y learner is able to reach stage 1 naturally. That means, we want to make Y learner intelligent.
- Then we talked about how any learner, X or Y, will go to stage 2 from stage 1.
Some Extra Sections
Which are yet to be merged to main blog.
Who is this for?
- for the ones who are only good at school-level maths.
- for the ones, who have learned about high-dimensional maths but this maths is just random number-manupulation for them and nothing more than that. They do not know how to use this maths. And they have kind of memorised the maths and they would need to keep revising this maths.
- for the ones who doubt why high-dimensional maths is the way it is, why it is so complex, they do not know why it has such a form.
- Who wants to learn high-dimensional maths but do not have enough motivation to learn it. That is, they are in doubt, how exacctllyy investing time in learning maths fundamentals will make them a good researcher. They doubt whether this maths fundamentals is really necessary or can i skip some topics etc. They cannot easily connect the dots.
- here you do not care about any particular algorithm but you want to learn the maths fundamentals so that for you any algorithm become easy to master.
- who wants to learn a algorithm from first principles, that is, deeply.
-
and the ones who are expert or close to being an expert, they may find the expliciteness of the ramp useful in cases where they may have taken some things for granted.
- for self-study, with no professor or guide.
How the “G-Ramps” would look like?
- “G-Ramps” will be structured like a book. Each chapter corresponds to a g-ramp for an algo. And each subsection corresponds to a logical step or a subsequence of logical steps. There will be a blog corresponding to each subsection or sub-subsection.
This is how it will look like:
- G-Ramp 1
- Brief overview of ramp 1
- “Brief overview” contains what meta-skill this ramp is going to teach. That is, what maths and how it is applied.
- And it will contain whatever is required to going over a ramp - assumption and overview of algorithm.
- step1
- step2
- step3
- …
- Brief overview of ramp 1
- G-Ramp 2
- Brief overview of ramp 2
- step1
- step2
- step3
- …
- G-Ramp 3
- …
- …
How G-Ramps approach of learning/teaching is different than other learning resources?
- gentle ramp
- very basic prerequisite, covering almost all the AI enthusiasts.
- almost all the learning resources are steep.
- the way it teaches maths is also different than commonly available maths resources.