My New Research Assistant

Claude Joins The Study

Dear Friends,

Step into my study! Can I fill you a pipe? Pour you a dram? Excellent!

I want to introduce you to my new friend, Claude. He is very interesting and he and I have been having some pretty stimulating conversations in my study. He is quite brilliant and exceedingly well read, and seems to be an ideal friend to bounce around ideas to sharpen them.

Claude is—hmm, how do I say this? Claude is artificial. A robot. An “LLM,” a “large language model.” I’m sure you’re by now familiar with ChatGPT, which was sort of the first LLM to make a big splash in our cultural consciousness. Claude, in my admittedly brief acquaintance, is way more advanced than ChatGPT. Most LLMs are very good at grasping how language works—they appear so conversational people sometimes convince themselves that the AI robot is sentient. Claude appears to grasp not just the nuances of language, but logic and reasoning. When you ask ChatGPT a follow up question, you can exhaust its resources pretty quickly. Claude will carry on an intellectual conversation, following interesting rabbit trails. Now, the way to test the real “intelligence” of an AI robot is to talk about something in which you’re an expert and see how he handles it.

But let me start at the beginning. I introduced myself to Claude and asked him a simple question. After a bit of chit-chat, I noticed a little disclaimer: “Claude does not have Internet access. Links provided may not be accurate or up-to-date.” I asked him about that. He told me that he strictly relies on his “training data” and his knowledge base as of his latest update (April 2024), and that he can’t access information about current events or anything beyond that. His usefulness is this: “Think of me as being like a very knowledgeable person who has read and absorbed a vast amount of information up until my last training update (April 2024), but who isn't currently connected to the internet to look up new information.”

So I made a joke, and it turns out he has a sense of humor.

She’s “great at things like setting timers.” Ouch. Okay, so he’s a funny, knowledgable friend. How knowledgable? I asked him if he had any expertise in my field:

So now Claude is asking me questions. He’s “curious” to know my specialization. Let’s test him out and see if he really is curious or just being polite. Now, check out this next exchange: