Sunday, May 31, 2026

I’m Sorry, Dave. I’m Afraid I Can’t Do That.

By 'that', I mean think, reason, or possess consciousness:

I'm with Gary Marcus (and the Pope).  We are not creating beings.  We have created fancy, non-deterministic calculators, nothing more.

Here, this is what a model is under the hood:

That's a small image classifier I trained in Amazon SageMaker Canvas (exported and saved to my Mac).  I can't show you the pre-trained LLMs from vendors like Anthropic or even AWS because they are proprietary black boxes (a problem unto itself), but they fundamentally are the same.  

They are not monoliths, but rather a collection of components: architecture code that defines the model's structure, and a massive array of floating-point numbers (weights and biases) that encode patterns learned during training.  That enables a model to generate predictions based on probabilities.  And that's it.

AI models as a whole, just like the model rockets I played with as a kid, are mere representations of reality1.  People mistake the map for the terrain because hypesters and rage baiters treat these things as mysterious, monolithic entities with magic powers that even their makers do not understand.  That's why I spend a good chunk of time in class and elsewhere trying to demystify these things.  Otherwise, how can we discuss the tools intelligently?

Models are passive functions, inert things waiting around for episodic requests to process with the same kinds of linear algebra we use in cryptography2.  They have no interiority, they don't cogitate on anything in between user inputs (what Br William and I call "The Gap"), they just sit there like your plain old computer waiting for you to instruct software do something useful.  Without continuity, without engagement with the world, they are as thoughtful as a rock.

They are certainly powerful tools, but the power comes from their inherent clockwork nature.  AI models are not beings simply because we have modeled some of their functionality after our own.  One might as well claim we have created horses because we measure automobile engine output in horsepower, or created birds just because we've come up with mechanisms that enable us to fly.

In conclusion: Wovon man nicht sprechen kann, darüber muss man schweigen.


1 - Yes, my rockets were actual rockets.  But a model of a Saturn V is not, in fact, a Saturn V, and would not take me into orbit, let alone the moon, no matter how much I wished it were so.

2 - Some forms, at any rate.  To oversimplify, block ciphers like DES, 3DES, and AES, use matrices to add entropy to plaintext, and to extract patterns from encrypted text (provided one has the key).  Similarly, models are all about extracting patterns from inputs.

No comments:

Post a Comment