- what if the human is anxious or deferential to authority
- how should the human think of the AI, perhaps as a lawyer speaks to a paralegal?
- question about the context window and how much lookback it has
Thank you for this useful model about how to engage with the LLM to gain understanding and knowledge. Perhaps you could speak to some of the human challenges of engaging in critical thinking.
- some people are afraid to make a decision
- some people want to take the first answer so they can be done with it
- some people have only budgeted the time and capacity for a 30-second conversation, rather than a 30-minute reflection
- some people come to the prompt with anxiety - anxiety that they don't know what to say - but also anxiety that their question is incomplete - or that the topic is more complex and less definitive
- for some people, critical thinking is a hard thing. Perhaps part of that is that their circumstances expect deferral to authority. And that they are the authority, not the LLM.
- it might be hard for some people to prompt the AI with several very explicit and specific sentences. When talking to another equal person, you wouldn't speak like that.
What are the mental hurdles a person would have in talking to an AI in a critical mode?
One thing I have a question about is the context window. How much does it remember about what you have previously talked about?
If I were to do what you said, to have several sessions, generate several points of view, and paste them into a single session - is the AI capable of processing an unlimited amount of text that you dump in there?
All your comments are super insightful, definitely critical thinking is hard skill to master, even before AI. I'll have to think more deeply how to touch some of these themes in the near future, thanks for the suggestion!
Regarding the context question, everything you type during a conversation is, in principle, in the "short-term memory" of the model, which just means it's immediately accessible because the whole conversation is fed to the model for every message you sent. So in a sense it doesn't have to really "remember" anything, the whole conversation is always there.
Now, once you restart the conversation, all of that context is lost. Some apps do try to store some concrete insights from a conversation for the future, what's being called "memory" in ChatGPT for example. But this is a very crude form of long term memory, because it's like tiny summaries of conversations. There is no real remembering, so to speak. This is why LLMs are amnesiac, everytime you start over, you start over, just like Memento.
And to your last question, no, the context is not infinite, but it's veeery long, on the order of 100 thousand words or more for top models, so, you're safe pasting stuff in. The problem is that even with such a long context, the model can get lost and confused and not really use the whole context. There is no guarantee that, even if some key idea is there, it will find it.
I assume most people have become as addicted to this stuff as I have, and so have eventually fallen into one of the traps you mention here. Software development has become the easy target of 'get rid of everyone and let the LLM do it' and eventually I ran into the reality that we aren't there yet, we are just building shittier software using cheaper labor. Learning to use the cheap LLM labor well is an important skill, and your framework provides a good starting point.
...we arrive at the most pressing question: How do we actually use the powerful new tools it has produced?
an equally important question is how do these tools use us?
I have some thoughts today on:
- what are the human challenges of engagement
- what if the human is anxious or deferential to authority
- how should the human think of the AI, perhaps as a lawyer speaks to a paralegal?
- question about the context window and how much lookback it has
Thank you for this useful model about how to engage with the LLM to gain understanding and knowledge. Perhaps you could speak to some of the human challenges of engaging in critical thinking.
- some people are afraid to make a decision
- some people want to take the first answer so they can be done with it
- some people have only budgeted the time and capacity for a 30-second conversation, rather than a 30-minute reflection
- some people come to the prompt with anxiety - anxiety that they don't know what to say - but also anxiety that their question is incomplete - or that the topic is more complex and less definitive
- for some people, critical thinking is a hard thing. Perhaps part of that is that their circumstances expect deferral to authority. And that they are the authority, not the LLM.
- it might be hard for some people to prompt the AI with several very explicit and specific sentences. When talking to another equal person, you wouldn't speak like that.
What are the mental hurdles a person would have in talking to an AI in a critical mode?
One thing I have a question about is the context window. How much does it remember about what you have previously talked about?
If I were to do what you said, to have several sessions, generate several points of view, and paste them into a single session - is the AI capable of processing an unlimited amount of text that you dump in there?
All your comments are super insightful, definitely critical thinking is hard skill to master, even before AI. I'll have to think more deeply how to touch some of these themes in the near future, thanks for the suggestion!
Regarding the context question, everything you type during a conversation is, in principle, in the "short-term memory" of the model, which just means it's immediately accessible because the whole conversation is fed to the model for every message you sent. So in a sense it doesn't have to really "remember" anything, the whole conversation is always there.
Now, once you restart the conversation, all of that context is lost. Some apps do try to store some concrete insights from a conversation for the future, what's being called "memory" in ChatGPT for example. But this is a very crude form of long term memory, because it's like tiny summaries of conversations. There is no real remembering, so to speak. This is why LLMs are amnesiac, everytime you start over, you start over, just like Memento.
And to your last question, no, the context is not infinite, but it's veeery long, on the order of 100 thousand words or more for top models, so, you're safe pasting stuff in. The problem is that even with such a long context, the model can get lost and confused and not really use the whole context. There is no guarantee that, even if some key idea is there, it will find it.
Hope it helps :)
I like your method.
I assume most people have become as addicted to this stuff as I have, and so have eventually fallen into one of the traps you mention here. Software development has become the easy target of 'get rid of everyone and let the LLM do it' and eventually I ran into the reality that we aren't there yet, we are just building shittier software using cheaper labor. Learning to use the cheap LLM labor well is an important skill, and your framework provides a good starting point.
...we arrive at the most pressing question: How do we actually use the powerful new tools it has produced?
an equally important question is how do these tools use us?