element14 Community
element14 Community
    Register Log In
  • Site
  • Search
  • Log In Register
  • About Us
  • Community Hub
    Community Hub
    • What's New on element14
    • Feedback and Support
    • Benefits of Membership
    • Personal Blogs
    • Members Area
    • Achievement Levels
  • Learn
    Learn
    • Ask an Expert
    • eBooks
    • element14 presents
    • Learning Center
    • Tech Spotlight
    • STEM Academy
    • Webinars, Training and Events
    • Learning Groups
  • Technologies
    Technologies
    • 3D Printing
    • FPGA
    • Industrial Automation
    • Internet of Things
    • Power & Energy
    • Sensors
    • Technology Groups
  • Challenges & Projects
    Challenges & Projects
    • Design Challenges
    • element14 presents Projects
    • Project14
    • Arduino Projects
    • Raspberry Pi Projects
    • Project Groups
  • Products
    Products
    • Arduino
    • Avnet Boards Community
    • Dev Tools
    • Manufacturers
    • Multicomp Pro
    • Product Groups
    • Raspberry Pi
    • RoadTests & Reviews
  • Store
    Store
    • Visit Your Store
    • Choose another store...
      • Europe
      •  Austria (German)
      •  Belgium (Dutch, French)
      •  Bulgaria (Bulgarian)
      •  Czech Republic (Czech)
      •  Denmark (Danish)
      •  Estonia (Estonian)
      •  Finland (Finnish)
      •  France (French)
      •  Germany (German)
      •  Hungary (Hungarian)
      •  Ireland
      •  Israel
      •  Italy (Italian)
      •  Latvia (Latvian)
      •  
      •  Lithuania (Lithuanian)
      •  Netherlands (Dutch)
      •  Norway (Norwegian)
      •  Poland (Polish)
      •  Portugal (Portuguese)
      •  Romania (Romanian)
      •  Russia (Russian)
      •  Slovakia (Slovak)
      •  Slovenia (Slovenian)
      •  Spain (Spanish)
      •  Sweden (Swedish)
      •  Switzerland(German, French)
      •  Turkey (Turkish)
      •  United Kingdom
      • Asia Pacific
      •  Australia
      •  China
      •  Hong Kong
      •  India
      •  Korea (Korean)
      •  Malaysia
      •  New Zealand
      •  Philippines
      •  Singapore
      •  Taiwan
      •  Thailand (Thai)
      • Americas
      •  Brazil (Portuguese)
      •  Canada
      •  Mexico (Spanish)
      •  United States
      Can't find the country/region you're looking for? Visit our export site or find a local distributor.
  • Translate
  • Profile
  • Settings
Code Exchange
  • Technologies
  • More
Code Exchange
Forum AI induced errors in your code
  • Blog
  • Forum
  • Documents
  • Events
  • Polls
  • Files
  • Members
  • Mentions
  • Sub-Groups
  • Tags
  • More
  • Cancel
  • New
Join Code Exchange to participate - click to join for free!
Actions
  • Share
  • More
  • Cancel
Forum Thread Details
  • Replies 19 replies
  • Subscribers 44 subscribers
  • Views 1183 views
  • Users 0 members are here
Related

AI induced errors in your code

michaelkellett
michaelkellett 4 months ago

There's an interesting article here:

https://www.eenewseurope.com/en/llms-used-to-code-can-introduce-serious-errors/?hash=93e5355d8bf7ef1fe62975a87f9d012aac4b4f7c7d1099e932443336d0aa983e

The key point of the article is:

"discovered that 440,445 out of 2.23 million code samples generated in Python and JavaScript using LLM models referenced hallucinated packages"

Bad actors can identify popular hallucinations and make real malware packages which will then be loaded into your code.

So unless you have a bomb proof way of checking that any packages referenced in AI generated code you may have serious problems.

Worth thinking about !

MK

  • Sign in to reply
  • Cancel

Top Replies

  • Jan Cumps
    Jan Cumps 4 months ago +6
    what a time to be alive
  • Jan Cumps
    Jan Cumps 4 months ago in reply to shabaz +6
    shabaz said: Some orgs have mechanisms that would avoid that from being introduced into code. Mine does. Our engine is vetted on a fenced set of information. And we have a process to introduce new knowledge…
  • shabaz
    shabaz 4 months ago +5
    I've noticed that, sometimes the AI will choose unexpected libraries, and there is the temptation to just accept it and see where it goes with the code! Some orgs have mechanisms that would avoid that…
  • Jan Cumps
    Jan Cumps 4 months ago

    what a time to be alive

    • Cancel
    • Vote Up +6 Vote Down
    • Sign in to reply
    • Cancel
  • BigG
    BigG 4 months ago

    Indeed. Hence I'm viewing LLM's as currently only useful for driver development on embedded systems, as you always have the datasheet for reference and you can also sense check functions used against OEM SDK (assuming that's used). Probably also safer by sticking with C or if you have a good understanding of C++ go with that to develop your driver class.

    Using LLM's for large scale Python code development is a receipt for disaster, IMHO, unless you have a super rigorous testing programme.

    • Cancel
    • Vote Up +4 Vote Down
    • Sign in to reply
    • Cancel
  • obones
    obones 4 months ago in reply to BigG

    Have a look at "vibe coding" especially applied to website development and enjoy the show.

    • Cancel
    • Vote Up +2 Vote Down
    • Sign in to reply
    • Cancel
  • shabaz
    shabaz 4 months ago

    I've noticed that, sometimes the AI will choose unexpected libraries, and there is the temptation to just accept it and see where it goes with the code! Some orgs have mechanisms that would avoid that from being introduced into code. Example is financial orgs, they won't allow any library or API or middleware to be used without an 'on-boarding' process, which can easily drag out for months on end (and six months to a year is not unknown), because it needs approval from multiple departments and they will have a documented process to do that so that things are not ignored. As a result, there's massive incentive to try to use libraries/APIs etc that are already in use elsewhere in the org, simply to get the code released quicker.

    Another now popular thing is to modularize the code substantially, and then each portion has it's own enforced rules (for instance, which destinations the portion is allowed to communicate to, and over which ports etc), in other words there's a virtual firewall in-between almost every tiny portion of code, and anomalies are then easily noticed. This doesn't apply all that much to embedded apps unfortunately, although, the Microsoft SDK for a microcontroller (Azure Sphere, it was for some MediaTek chip) which was released around 2019 did contain separate areas for code to execute in, and a sort of rules that could be configured and enforced, for what communication was allowed between those two areas, i.e. a super-tiny firewall (but I don't remember much about it).

    • Cancel
    • Vote Up +5 Vote Down
    • Sign in to reply
    • Cancel
  • Jan Cumps
    Jan Cumps 4 months ago in reply to shabaz
    shabaz said:
    Some orgs have mechanisms that would avoid that from being introduced into code.

    Mine does. Our engine is vetted on a fenced set of information. And we have a process to introduce new knowledge.

    It 'll come at a cost - we miss a whole lot of training that's available in public. 
    On the other hand, we avoid to get unfriendly code in. And it avoids that our trade secrets leak out to the public.

    • Cancel
    • Vote Up +6 Vote Down
    • Sign in to reply
    • Cancel
  • mp2100
    mp2100 4 months ago in reply to Jan Cumps

    This makes perfect sense, but is ironic.  (Some) People are using AI to write their code.  But if you want safe, secure, reliable code, you have to check everything before it goes into production.  Is there a time savings?  Maybe.  Maybe not.  But if you don’t want safe reliable code, of course, there is a time savings.

    • Cancel
    • Vote Up +1 Vote Down
    • Sign in to reply
    • Cancel
  • Fred27
    Fred27 4 months ago

    Ai is making a lot of incompetent people think they're competent. I'm not a fan of it at all.

    I remember someone asking on a StreamDeck forum where to download the Python SDK. When I pointed out that there wasn't one I was told "ChatGPT says there is. Where is it?" When I pointed out that ChatGT was wrong I was told that I clearly didn't know what I was talking about and AI would take the jobs of dinosaurs like me.

    The best fit I've seen for AI is generating background scenery in Unreal. I feel that if you're making something that doesn't need to be right and that "good enough" will do, then maybe AI can help. If it needs to be exactly right, or you need to be able to understand why it came up with what it did and make fine adjustments then it's not much use.

    • Cancel
    • Vote Up +1 Vote Down
    • Sign in to reply
    • Cancel
  • Jan Cumps
    Jan Cumps 4 months ago in reply to Fred27

    I've changed my mind somewhat. It can make people that are skilled in using AI, more efficient.

    I've seen it in use (and have helped feed the engine) in  a professional setting now - with the current state of the technology. 

    • Cancel
    • Vote Up 0 Vote Down
    • Sign in to reply
    • Cancel
  • Fred27
    Fred27 4 months ago in reply to Jan Cumps

    I'll try to keep an open mind and see how things develop.

    Completely coincidentally, I just came across this. I have a lot of respect for Dylan Beattie (although he does seem to like getting his guitar out at conferences a lot).

    dylanbeattie.net/.../the-problem-with-vibe-coding.html

    • Cancel
    • Vote Up 0 Vote Down
    • Sign in to reply
    • Cancel
  • BigG
    BigG 4 months ago in reply to Jan Cumps

    We all make mistakes. The strange thing is that it's harder to spot our own mistakes.

    As such, I view AI as a helper, as you can either get AI to find your own mistakes or if AI has crafted some initial code, you'll find it easier to spot the AI mistakes. Win-win, in my opinion.

    • Cancel
    • Vote Up 0 Vote Down
    • Sign in to reply
    • Cancel
>
element14 Community

element14 is the first online community specifically for engineers. Connect with your peers and get expert answers to your questions.

  • Members
  • Learn
  • Technologies
  • Challenges & Projects
  • Products
  • Store
  • About Us
  • Feedback & Support
  • FAQs
  • Terms of Use
  • Privacy Policy
  • Legal and Copyright Notices
  • Sitemap
  • Cookies

An Avnet Company © 2025 Premier Farnell Limited. All Rights Reserved.

Premier Farnell Ltd, registered in England and Wales (no 00876412), registered office: Farnell House, Forge Lane, Leeds LS12 2NE.

ICP 备案号 10220084.

Follow element14

  • X
  • Facebook
  • linkedin
  • YouTube