Actions
Forum Thread Details
  • Replies 31 replies
  • Subscribers 579 subscribers
  • Views 728 views
  • Users 0 members are here
Related

Is General Artificial Intelligence a Breakthrough or just a Financial Hype?

HKPhysicist

According to my understanding, AI hype big corps spend a whole country's energy resource to produce learning software and hardware which can do a very little things for us.  Laughing

On the other hands, a parrot can learn how to talk with men with just a few grams of nutrition.  Also, my little puppy learnt how to do simple house hood things for me in just a few minutes and a few grams fo nutrition. Blush

Which one of these examples are more intelligent and energy efficient?

Had we better use those astronomical sum of money and resources on investigating deeper natural laws such as room temperature superconductivity, novel material and crystal, higher energy experiments beyond the Large Hadron Collider, nuclear fusion experiments, etc.?

Parents

  • Hello all Friends who have joined the discussion.

    Here is another viewpoint published on Nature about recent AI and LLM development:

    AI models collapse when trained on recursively generated data

    Nerd

  • in reply to HKPhysicist

    That's another problem with generative AI; The output is derivative. And once the input becomes the output of another model, you start seeing a steady decline in quality. It's a lossy process. Like compressing a JPEG, and then compressing it again, and again, and again. Eventually the image would be a mess of blurry blobs and nothing like the original picture.

    Part of this has been shown to happen already in the world of programming with code-oriented LLMs; Several studies were published last year like this one,  this one or this one. All seem to agree that while the speed at which programmers are writing code has increased with the help of AI, the quality is decreasing fast.

    What can be seen as a problem of entropy, is another major problem with that technology. Most people assume that AI "can only get better", when in reality, as human input gets diluted in a sea of AI-generated training data, it may actually get worse and worse.

Reply
  • in reply to HKPhysicist

    That's another problem with generative AI; The output is derivative. And once the input becomes the output of another model, you start seeing a steady decline in quality. It's a lossy process. Like compressing a JPEG, and then compressing it again, and again, and again. Eventually the image would be a mess of blurry blobs and nothing like the original picture.

    Part of this has been shown to happen already in the world of programming with code-oriented LLMs; Several studies were published last year like this one,  this one or this one. All seem to agree that while the speed at which programmers are writing code has increased with the help of AI, the quality is decreasing fast.

    What can be seen as a problem of entropy, is another major problem with that technology. Most people assume that AI "can only get better", when in reality, as human input gets diluted in a sea of AI-generated training data, it may actually get worse and worse.

Children
  • in reply to battlecoder
    battlecoder said:
    All seem to agree that while the speed at which programmers are writing code has increased with the help of AI, the quality is decreasing fast.

    I think we've seen a decrease in quality with human programming as well. Nothing beats the good old days, when it was important for your code to be written such that where the heads on the drum will be when the next instruction needs to be fetched, will contain that instruction, after all.

    So we may just be seeing a sped up time-lapse movie of where programming is headed regardless. 256K words (a moby) of memory ought to be enough for anyone.

  • in reply to bradfordmiller

    Fair, but the decline in human code quality is more related to the abuse of bloated frameworks, libraries and languages with higher level abstractions, and the constant prioritization of "delivering fast" over creating performant and optimized apps (a priority imposed by most tech companies). The code itself is not that much worse, but there's a clear over-reliance in just shoving down "magic" dependencies into a project without caring about how efficient, lean, or optimized they are. With this they can get an app done fast, at the cost of being slow as hell, being a million MBs in size, or using a truckload of RAM (and sometimes all of the above).

    The metrics used to measure the decline in quality due to use of AI however are slightly different; They have measured the abundance of duplicated blocks of code, the complexity of the code created even for the simplest tasks, the prevalence of "copy and paste" sections that have a bunch of unnecessary stuff, etc. All of which are a lot worse now with the help of AI than they were before.