We're probably thinking about this at very different levels. Here's what I meant: I can ask Claude for "a bilingual German-Russian poem about the side effects of the most common drugs used in anesthesia". I would bet my left shoe that if I asked people on the street, no one will do a better job than Claude. And to me, answering questions correctly is a very good metric for intelligence.
We can debate whether that's real intelligence, and whether the question is fair, but this is still a real, measurable capability, that just eight years ago was a pipe dream. This capability is what OP is tracking, and what I believe is impressive but hamstrung by harnesses.
>I would bet my left shoe that if I asked people on the street, no one will do a better job than Claude.
Because it's unrealistic to expect such a niche population to be out and wandering around. What it sounds like you want is something that masquerades at intelligence, but is really alien.
We can debate whether that's real intelligence, and whether the question is fair, but this is still a real, measurable capability, that just eight years ago was a pipe dream. This capability is what OP is tracking, and what I believe is impressive but hamstrung by harnesses.