AI wrote my code and all I got was this broken prototype

AI wrote my code and all I got was this broken prototype

/in

AI wrote my code and all I got was this broken prototype

Welcome to this week’s edition of the Threat Source newsletter. 

Vulnerabilities within software are a persistent challenge. Software engineers inadvertently tend to make the same mistakes repeatedly, with the same entries appearing in the annual top 25 list of Common Weakness Enumerations each year. 

The truth is, writing software is difficult. Software engineering is a craft demands concentration, knowledge and time, all coupled with extensive testing. Even the most skilled software engineer can get distracted or have a bad day, leading to a hidden vulnerability inadvertently making its way into a production codebase. 

Identifying vulnerabilities early in the software development process is one of the promises of AI. The idea being that an AI agent would write perfect code under the direction of a software engineer or verify and correct code written by a human. 

Last weekend, I decided to put this premise to the test. As a somewhat rusty software engineer, I resolved to see if AI could assist me with a personal software project. Initially, I was impressed, the AI agent offered an engaging discussion about high-level architecture and the trade-offs of various approaches. I was amazed at the lines of code that the AI generated on request. All the software for my project written at the press of a button! 

Then came the testing. Although the code looked convincing, it failed to interface with the required libraries. Parameters were incorrect, it tried to call fictional functions. It seemed that the way the AI imagined the library to work didn’t reflect reality or the available documentation. Similarly, there were less sanity checks or verification of variable values than I was comfortable with; especially since many of these were derived from external inputs. 

To be fair, the AI code resolved a tricky threading issue that had defeated me, and the ‘boilerplate’ code necessary to form the skeleton structure of the software was flawless. I felt that I achieved a productivity boost from the AI’s exposure to ‘frequently encountered’ coding issues. However, when it came to more esoteric APIs with which I was moderately familiar, the AI was unable to generate functional code or correctly diagnose reported errors. 

After some debugging and manual rewriting, I managed to create a working prototype. The code is clearly not bulletproof, but then again, I hadn’t explicitly asked for code that was secured against all potential hacks. Like many software engineers, myself and my AI assistant focused on quickly delivering the desired functionality, rather than considering the long-term operation of the code in a potentially hostile environment. 

I remain optimistic that AI assisted coding is the pathway to a software vulnerability free future. However, my recent limited personal experience leads me to think that we still have a considerable journey ahead before we can definitively resolve software vulnerabilities for good. 

I hope you all have a tremendous time at Summer Camp, see a lot of old friends and make new ones and most importantly that you shower and use deodorant. Conference season is a marathon, it’s long, it’s arduous, it’s sweaty – be the hygienic change you want to see in the world.  

The one big thing 

Continuing the AI theme, Guilherme describes how AI LLM models can be used to assist in the reverse engineering of malware. Used correctly, LLMs can provide valuable insights and facilitate the analysis of malware. 

Why do I care? 

Reverse engineering malware is the often time-consuming task of identifying the execution path of malicious software. Frequently malware writers obfuscate their code to make it difficult to understand and follow what their code is doing. Advances in technology that can speed up this process make fighting malware easier.  

So now what? 

Investigate if the tools and approaches described in the blog can be used to improve your reverse engineering process, or as a means to begin learning about reverse engineering. 

Top security headlines of the week 

As ransomware gangs threaten physical harm, ‘I am afraid of what’s next,’ ex-negotiator says

In an effort to increase the pressure on victims, ransomware gangs are now using threats of physical violence. (The Register)

‘Shadow AI’ increases cost of data breaches, report finds

Unmanaged and unsecured use of AI is leading to data breaches. (Cybersecurity Dive)

Enough to drive a cybersecurity officer mad: one rule here, a different rule there

Chief information security officers call for less fragmentation in global cybersecurity regulations. (ASPI)

UK Online Safety Act promotes insecurity

The implementation of the UK Online Safety Act requiring age verification for content deemed harmful to children introduces some security quandaries. (Tech HQ)

Can’t get enough Talos? 

Cyber Analyst Series: Cybersecurity Overview and the Role of the Cybersecurity Analyst

A series of videos on the profession of cybersecurity analysts made in conjunction with the Ministry of Digital Transformation of Ukraine for Diia.Education (available in English and Ukrainian languages). Watch here.

Tales from the Frontlines

Join the Cisco Talos Incident Response team to hear real-world stories from the frontlines of cyber defense. Reserve your spot.

Vulnerability roundup

Cisco Talos’ Vulnerability Discovery & Research team recently disclosed seven vulnerabilities in WWBN AVideo, four in MedDream, and one in an Eclipse ThreadX module. Read more.

Talos Takes

Hazel is joined by threat intelligence researcher James Nutland to discuss Cisco Talos’ latest findings on the newly emerged Chaos ransomware group. Listen here.

Upcoming events where you can find Talos 

It’s the summer. We’ll be on the beach. 

Most prevalent malware files from Talos telemetry over the past week  

SHA 256: 9f1f11a708d393e0a4109ae189bc64f1f3e312653dcf317a2bd406f18ffcc507 
MD5: 2915b3f8b703eb744fc54c81f4a9c67f 
VirusTotal: https://www.virustotal.com/gui/file/9f1f11a708d393e0a4109ae189bc64f1f3e312653dcf317a2bd406f18ffcc507 
Typical Filename: VID001.exe 
Claimed Product: N/A 
Detection Name: Win.Worm.Coinminer::1201  

SHA 256: a31f222fc283227f5e7988d1ad9c0aecd66d58bb7b4d8518ae23e110308dbf91    
MD5: 7bdbd180c081fa63ca94f9c22c457376  
VirusTotal: https://www.virustotal.com/gui/file/a31f222fc283227f5e7988d1ad9c0aecd66d58bb7b4d8518ae23e110308dbf91/details  
Typical Filename: IMG001.exe   
Detection Name: Simple_Custom_Detection 

SHA 256: 41f14d86bcaf8e949160ee2731802523e0c76fea87adf00ee7fe9567c3cec610 
MD5: 85bbddc502f7b10871621fd460243fbc  
VirusTotal: https://www.virustotal.com/gui/file/41f14d86bcaf8e949160ee2731802523e0c76fea87adf00ee7fe9567c3cec610/details 
Typical Filename: N/A 
Claimed Product: Self-extracting archive 
Detection Name: Win.Worm.Bitmin-9847045-0 

SHA256: 47ecaab5cd6b26fe18d9759a9392bce81ba379817c53a3a468fe9060a076f8ca  
MD5: 71fea034b422e4a17ebb06022532fdde   
VirusTotal: https://www.virustotal.com/gui/file/47ecaab5cd6b26fe18d9759a9392bce81ba379817c53a3a468fe9060a076f8ca/details  
Typical Filename: VID001.exe   
Claimed Product: N/A   
Detection Name: Coinminer:MBT.26mw.in14.Talos  

SHA 256: 59f1e69b68de4839c65b6e6d39ac7a272e2611ec1ed1bf73a4f455e2ca20eeaa  
MD5: df11b3105df8d7c70e7b501e210e3cc3  
VirusTotal: https://www.virustotal.com/gui/file/59f1e69b68de4839c65b6e6d39ac7a272e2611ec1ed1bf73a4f455e2ca20eeaa/details  
Typical Filename: DOC001.exe  
Claimed Product: N/A  
Detection Name: Win.Worm.Coinminer::1201 

Cisco Talos Blog – ​Read More