“Learning” in the age of LLMs
Do we need to change the way we learn new things now what we have tools like GPT-4 and LLama3?
Large language models are excellent information retrieval agents, while their reasoning ability still has isn’t on par with human capability, there’s no question that language models are orders of magnitude more efficient at extracting highly relevant information from large corpus’s of data.
What does this mean for human learning, especially in specialized, domain-specific tasks? Let’s look at an example using computer programming, in a context-specific domain.
This code outputs the array “[1]”, but it isn’t immediately obvious why. An experienced python programming will immediately be able to tell that the program is behaving as expected, but a novice may need to build some understanding. Here is how the context-building process may go for a beginner, in google searches
→ “Python variables of object affected by other members of same class”
→ “Python default arguments changing between executions”
→ “Reference vs value types python”
→ “Are default arguments stored as variables”
→ “How are default arguments stored in python classes”
→ “How to read all attributes of a class python”
→ “Are default arguments function variables python”
→ “Read default arguments from function object”
Time taken: ~30 minutes
This may be a bit verbose, but for a new programmer trying to gain context on a programming context, it’s the best way to gain broad domain knowledge. The downside however, is that it takes forever. The more context-heavy the task, the longer the the context-building process takes to truly understand what you’re doing. This was a simple example, but in domains like biology, computer engineering, physics, literature, etc. A lot of context contains cross-dependancies (often forcing you to learn two or more topics at once to gain understanding, regardless of however many you’re interested in).
Now, let’s take a look at solving the above problem, using an LLM (specifically GPT-4-Turbo).
Prompt:
Why does this python code output "[1]", when the "data" attribute of e2 has not been modified?
class Example:
def init(self, data=[]):
self.data = data
e1 = Example()
e2 = Example()
e1.data.append(1)
print(e2.data)
Result:
Obviously, GPT-4 was able to answer and explain the behaviour correctly, but let’s pay attention to the remarkable information retrieval that it had to perform to arrive at the answer, and compare it with our imaginary student’s debugging process.
Both the student and the LLM identified that
Lists in python are reference types, and modifications persist between different aliases
Default arguments in python are mutable
Default arguments are associated function objects, function closures use the same list reference
Moreover, the LLM also identified that
Methods are class attributes and persist between instances
Variables within function closures do not persist between instances
Time taken: <30 seconds
There is no comparison to be made in terms of speed. Using the LLM to perform the context gathering task when debugging is much faster. When trying to iterate quickly and ship, these gains compound rapidly, I’ve personally seen my development workflow completely transformed by LLMs like GPT.
But what about learning from first principles? Is the former method of context gathering more effective? More memorable in the long term? What topics are worth diving deep into, and which ones should just be delegated to an LLM?
tbh, I have no idea.
It’s important to know however, that LLM’s can be wrong, catastrophically wrong at times. And specialized knowledge, learned from first-principles is the most effective way to see through the times when LLM’s are wrong. So traditional methods of learning are still important, but as we just showed below, can be highly augmented my LLMs.
This is also widely different across domains. Simply due to the distribution of data that they’re trained on, LLMs still suck at reasoning in certain domains like advanced scientific search, ethical issues, geospatial reasoning, highly technical engineering problems, etc.