Stupid AI on key signatures (slightly OT)

ShiroKuro

So I was googling key signatures (long story) and googled "key of g minor."

Among other things, Google's AI Overview said of G minor:

Popularity: The 6th most popular minor key and the 14th most popular key overall

What does that even mean, the 6th most or 14th most? There are only 24, for starters. And most popular among whom? And how did they arrive at this popularity ranking? And who is they, anyway??

For all we know, this could be just another AI hallucination. Or maybe "most popular" here is based on something totally legitimate, like a review of all musical pieces on record (??) and G minor is the 6th most commonly used. If that were the data behind this blurb, it would be interesting.

But I seriously doubt it. (for one thing, there's the methodological problem with trying to survey "all musical pieces on record," among other things....)

I hate AI.

😛

Sgisela

The Google AI thing is, IMO, not really ready for primetime. It’s supposed to be a better version of a search engine, but an answer like this is just meaningless to me.

This is going further OT, but people keep talking about how powerful AI is and how quickly it’s progressing. I’m sure that’s true in some respects. and yet it seems like there are some very fundamental issues with it that seem to me very persistent. Like clearly not understanding the assignment when you were googling a key signature.

Yesterday I was trying to search some basic information for my job. I wanted to know the genome length of two different organisms. For one of the organisms, it gave me the right answer, but for the other it said something with great certainty that was clearly completely wrong. I tried it again and it gave me the same very very wrong answer. I have no idea why it got one right and the other wrong, let alone why it came up with the answer it did for the very wrong one (did it just make it up? Did it misunderstand something that was reported somewhere? Was it pulling false information from somewhere?

Anyways, I don’t understand why it can’t just say, ‘I don’t know,’ if it doesn’t know.

ShiroKuro

Sgisela I have no idea why it got one right and the other wrong, let alone why it came up with the answer it did for the very wrong one (did it just make it up? Did it misunderstand something that was reported somewhere? Was it pulling false information from somewhere?

As @iternabe said, AI is based on "large language models" that basically operate on a very sophisticated statistical process of predicting what word should come next, given all the words that have come up thus far. This means the AI out is programmed to align with the topic, as well as grammar, of the prompt. A lot of times, that results in some pretty amazing, and amazingly accurate, responses.

But AI doesn't have the ability to assess the truth value of its own output (or of the prompt, for that matter), which is why is output is just hit or miss.

Oh, and when the output is factually wrong, it's not necessarily because the AI is pulling from some source data that's wrong, although that can happen as well. It could just be because it assembled a plausible sentence, without regard for its accuracy.

Sorry, this is yet more OT I guess 😃

iternabe

I looked it up. Apparently the statement was made by the very authoritative website HookTheory.com drawn from over 30,000 songs in their one TheoryTab database.

That particular flavor of AI is in fact just a LLM (Large Language Model) whose sole ability is to predict what's the most likely next word to come up given a partially constructed sentence. "Intelligence" is a stretch, although the model faked it seemingly quite well.

BartK

I know a bit about the subject as I work in IT and IMO the whole AI thing is overblown. Current "AI" isn't much more than glorified pattern recognition. Sure, it can learn to recognize patterns very well and fake intelligence by imitating those patterns but it's not intelligent. Currently, AI has a big problem reconciling the pattern/language abilities with the logic/math abilities that computers are traditionally considered good at. Ask an AI any non-trivial math problem and you will see how hilarious the answers are. AI will confidently give you the wrong answer because it's just faking language patterns not really understanding the logic.

Don't even get me started on self-awareness. People are scared of conscious AI but that is still complete Sci-Fi. We have exactly zero understanding of what consciousness even is let alone reproducing it in software.

ShiroKuro

BartK AI will confidently give you the wrong answer because it's just faking language patterns not really understanding the logic.

Exactly. I do a unit on AI with one of my classes, and one of the things I point out to my students is that AI output “looks legit” — this is where the confidence comes from, it mimics well-formed language because it’s programmed to produce grammatically correct sentences. But unless you have the background knowledge to truly evaluates that output, you have no way of knowing whether the actual content is accurate, relevant, quality information… But again, because it’s grammatically accurate, it’s very easy to assume it’s factually accurate.

People are scared of conscious AI

I’m not scared of AI becoming conscious, I’m more worried about people believing AI without doing any kind of double-checking, and then that leading them to make bad decisions and undertake misguided actions. I’m sure this is already happening…

keystring

ShiroKuro Exactly. I do a unit on AI with one of my classes, and one of the things I point out to my students is that AI output “looks legit” — this is where the confidence comes from, it mimics well-formed language because it’s programmed to produce grammatically correct sentences. But unless you have the background knowledge to truly evaluates that output, you have no way of knowing whether the actual content is accurate, relevant, quality information… But again, because it’s grammatically accurate, it’s very easy to assume it’s factually accurate.

You have put into words succinctly what I've been trying to say for a while.