What Does “Expertise” Mean for Agent Skills?
When I first started building Agent Skills, I had a simple thought.
One of the historical inspirations behind neural networks was the abstraction of biological nervous systems.
Of course, an Agent is not an LLM, and an LLM is not a brain.
But it gave me a question:
If I want an Agent to reuse experience, how do humans reuse experience?
How do people use experience?
Suppose I have dealt with a problem before.
The next time I encounter the same situation, I usually do not analyze everything again from zero.
I notice a few important features and quickly think:
I have seen this before.
Then previous experience starts to work almost automatically.
But the difficult question is:
What if I recognized the situation incorrectly?
Maybe this case only looks similar.
Maybe one condition that actually matters has changed.
Then experience does not merely fail to help.
It can make me reach the wrong answer faster—and with more confidence.
That made me think that expertise is not the same thing as “having done something many times.”
At least in the way I currently understand it, expertise also includes:
knowing when previous experience applies.
That is also what I eventually want Agent Skills to approach.
The point is not to claim that my own Skills already use experience like an expert.
The question is:
If I want to reproduce the way an expert uses experience, what exactly am I trying to reproduce?
It turns out this question has been studied
Later I came across a concept that was very close to this idea:
conditionalized knowledge.
Put simply, it is not just remembering:
“When X happens, do Y.”
It means understanding knowledge together with the conditions under which it applies:
Under what conditions is X actually X?
When is Y useful?
When a condition changes, when should I stop applying Y?
In its discussion of research on experts and novices, the National Academies describes expert knowledge as being connected to the contexts in which it is useful. That connection helps experts retrieve knowledge that is relevant to the current task. Knowledge that has not been conditionalized in this way may be something a person already knows, yet still fail to be retrieved appropriately when it is needed.
Original discussion of conditionalized knowledge: How People Learn — How Experts Differ from Novices
This was very close to what I had been thinking about Skills.
If a Skill records only “how to do something,” it captures only part of the experience.
The other part is:
Under what conditions is this experience actually valid?
Experts may not even see the same problem
Chess experts are an intuitive example.
When they look at a real game position, they can quickly recognize meaningful patterns.
What helps them move faster is not simply that they have memorized more positions.
They are quicker to recognize:
what kind of situation this is.
That changed how I thought about Skills.
A Skill is not just “how to do it”
The simplest form of experience reuse looks like:
Situation A → Skill A
But real problems are rarely that clean.
What I actually want is closer to:
What situation am I in now?
Which past experiences are relevant?
What is different this time?
Which experiences can I reuse, which need to be modified, and which should be discarded?
So even a very complete Skill is only one part of the system.
If an Agent simply does this:
Looks like A
→ find A
→ execute A
then having more Skills does not necessarily make the system more expert.
It may simply become better at finding an old answer that looks correct.
The harder question is:
Is this really A?
So what do I mean by “expertise”?
Not always being correct.
And not having the largest number of Skills.
At least for me, the kind of expertise I want to reproduce is closer to:
recognizing the situation, drawing on experience, while keeping the ability to question that experience.
Past experience can make me faster.
But when an important condition changes, I want the system to be able to notice:
This time, I may not be able to do exactly what I did before.
So the question in the title—what “expertise” means for Agent Skills—applies to my own Skills as well.
I do not know how close my own system is to that kind of judgment yet.
It is more like a direction I want to move toward:
not making an Agent remember more and more procedures, but making it increasingly capable of judging when a past procedure deserves to be trusted.