Do I Know What I Built?

A while back I built a tool with Claude that pulls student data out of PowerSchool and rosters it into ManageBac. I kept picking at it — spotting patterns in the behavior that weren’t quite right — and each pass hardened it a little. In the end, the vibe-coded project was able to sync PowerSchool to ManageBac.

Then I forgot about it entirely. Other problems were calling. Opened up more Claude sessions to solve them …

A month later, another school called for the same solution. I located the codebase, set up the credentials, and told Claude to let her rip. It worked almost flawlessly. I didn’t even need to remember the sequence of commands.

Great, I thought. Time to productionize this so any school can use it, not just the ones I set up from my laptop. In other words, it was time to hand off to the professional development team something that worked for a few schools, so that it worked for all of them.

My colleague’s first question was, “How does it work?” And I couldn’t tell him — not to the depth or with the confidence I usually exhibit. What I had was a vague memory, the sort you have about what you ate last week. Was it the pasta on Tuesday or the hamburger? Didn’t I have a Claude session that did it this way, or am I thinking of the other one? It all just got blurred together.

The nagging bit

I could have found out in ninety seconds, by you know, reading the docs that were automatically derived. (Did I even read them in the first place?) Or I could have just asked Claude to walk me through it, which it would have done better than my memory ever could. (But that would be yet another session to open.)

There was something nagging me. I should just know this. Reaching for the machine to explain my own work felt like a crutch. I shouldn’t need a GPS to drive myself to my favorite restaurants and back again either, right?

This has become my defining experience of agentic coding. I’m learning a lot. I’m completing a lot (not just doing, completing), but if I don’t remember it well enough did I even do it? It’s as if this ongoing parade of projects is giving me a form of amnesia.

Or maybe it’s challenging the concept of ‘knowledge’?

Maybe the discomfort I feel is because I’m worried it doesn’t allow me to pursue knowledge or achieve fulfillment in the way that I’m used to.

Socrates worried about something similar. In the Phaedrus he explains how he regards knowledge by telling the story of the god who invents writing and presents it to the king as a recipe for memory and wisdom. The king is unimpressed and so is Socrates. This will produce forgetfulness in the souls of those who learn it, they agree, because we will trust external marks rather than remembering from within.

Elsewhere in the same dialogue, Socrates admits he hardly ever leaves the city walls, because the fields and the trees have nothing to teach him — only the people in town do. His problem with writing is if you ask it a question it will just repeat itself. His lamentation is insightful, but now, those external marks are bits and bytes — and they can talk back.

So far, I side on thinking that AI isn’t yet a crutch, but I’m not ready to say it’s fine, either.

Too many options

Instead of spending the time excavating how that tool works, I’ve written this article, asked AI about how philosophers define knowledge, re-read the Phaedrus, and worked on two other completely new coding projects.

Today, I don’t know how I feel about being able to multi-task to the point of amnesia. I oscillate between relishing in the productivity to deeply considering the conceptual implications.

But I do know this: Being articulate is what Socrates sought nearly 2,500 years ago, and I can’t see that ever getting deprecated. Thus, it seems to me that debate and deliberation have to be the cornerstone of learning. It’s not just “talk back” – there should be “push back” too. That’s what makes discussion authentic.

Meanwhile, my colleague is still waiting for that answer.