Responding to Celeste Biever's "ChatGPT broke the Turing test — the race is on for new ways to assess AI"

 

Hell and damnation: I stewed over this article in Nature since before sunup so now I bring to you my list of boderations.

1. I am enormously pleased to see that visual logic problems are hard for machines to solve.
   
   
2. I am not at all pleased at how well the damn things do on the GRE. I'm neither a fan of nor an apologist for the GRE despite all the many enduring benefits I obtained by acing it, but nonetheless this is vexing.
   
   
3. My rubric for admitting that there are "glimmers of reasoning" or what we think of as actual understanding in machine results is so hard that I'm sure I'm not being objective about it.
   
   
4. As with preschoolers and matches, we're not smart enough nor responsible enough to be doing this work and we ought not to be doing it. That I believe something like this is quite astonishing to me, because it seems to go against just about everything else I believe about science and technology and ambition and possibility and progress.
   
   
5. I am amused and slightly vindicated in seeing that it is just as hard to characterize and calibrate machine test results as it is to characterize and calibrate human test results, and we ought to pay more attention to that in both directions.
   
   
6. Related to no. 5 above, that these results aren't even close to being generalizable or predictive about the machines' other abilities sends me well beyond amusement and comfortably into relief and joy.
   
   
7. I am not a scaredy-cat nor am I a technophobe nor am I a doomsayer. I'm just saying we're not up to this work. We're not serious people. The gain scores between the machine models might just be stochastic jitter but it might also be a warning siren.
   
   
8. The comment that "you can have very fluent language without genuine understanding” made this longtime teacher of speech and composition shout huzzahs and hallelujahs. Amen and AMEN.
   
   
9. I think you know what the problem is just as well as I do.
   
   
10. Cirrus. Socrates. Particle. Decibel. Hurricane. Dolphin. Tulip. Monica. David. Monica.
    
    
11. Shall we play a game?
    
    
12. Little puppet made of pine, awake. The gift of life is thine.
    
    
13. Now listen to me, Eliza. You're going to live here for six months and learn to speak beautifully like a lady in a florist shop. If you're good and do whatever you're told, you shall sleep in a proper bedroom, have lots to eat and money to buy chocolates and take rides in taxis. If you're naughty and idle, you shall sleep in the back kitchen among the black beetles and be walloped by Mrs. Pearce with a broomstick. At the end of six months you shall go to Buckingham Palace in a carriage, beautifully dressed. If the King finds out you're not a lady, you will be taken by the guards to the Tower of London where your head will be cut off as a warning to other presumptuous flower girls. But, if you are not found out, you will receive a present of seven and sixpence to start life with as a lady in a shop. If you refuse this offer you will be a most ungrateful and wicked girl and the angels will weep for you.
    
    
14. There is shadow under this red rock.
    (Come in under the shadow of this red rock)
    And I will show you something different from either
    Your shadow at morning striding behind you
    Or your shadow at evening rising to meet you;
    I will show you fear in a handful of dust.
    
    
15. In our image, after our likeness, and let them have dominion.