Cybersecurity Snapshot: ChatGPT Gets So-So Grade in Code Analysis Test, while JCDC Pledges To Focus on Protecting Critical Infrastructure

Check out why ChatGPT’s code analysis skills left Carnegie Mellon researchers unimpressed. Plus, JCDC will put special focus on critical infrastructure security in 2024. Meanwhile, CISA and OpenSSF shine a spotlight on the security of software package repositories. And scammers leveraged tech tools to steal $10 billion from U.S. consumers last year. And much more!

Dive into six things that are top of mind for the week ending February 16.

1 - ChatGPT’s code analysis skills? Not great

Thinking of using ChatGPT to detect flaws in your code? You’ll have to double-check that its findings are accurate.

That’s the conclusion of a research team from Carnegie Mellon University. The researchers, from the CERT Division of the university’s Software Engineering Institute (SEI), tested ChatGPT 3.5’s ability to examine noncompliant software code examples using the SEI CERT C Coding Standard.

The results show that “while ChatGPT 3.5 has promise, there are clear limitations,” Mark Sherman, one of the researchers, wrote in the blog post “Using ChatGPT to Analyze Your Code? Not So Fast.”

So how did ChatGPT 3.5 fare at error detection? As the graph below shows, it succeeded less than half of the time.

^{(Source: CERT Division of Carnegie Mellon University’s Software Engineering Institute, February 2024)}

The research also shows that ChatGPT 3.5's code-analysis accuracy varies depending on the type of coding error it encounters.

ChatGPT 3.5’s Rate of Discovery and Correction of Specific Coding Mistakes

^{(Source: CERT Division of Carnegie Mellon University’s Software Engineering Institute, February 2024)}

So what’s the takeaway? Review ChatGPT 3.5’s output. Don’t trust it blindly.

Meanwhile, the researchers expect ChatGPT and other generative AI tools to get better at code analysis. For example, in preliminary testing, ChatGPT 4.0 performs better than ChatGPT 3.5, Sherman wrote.

To get all the details, read the blog post “Using ChatGPT to Analyze Your Code? Not So Fast.”

For more information about using AI tools and technology for cybersecurity tasks:

• “Evaluate the risks and benefits of AI in cybersecurity” (TechTarget)
• “The role for AI in cybersecurity” (Cybersecurity Dive)
• “AI Is About To Take Cybersecurity By Storm” (Tenable)
• “From Malware Detection to Predictive Analytics, 5 Ways AI Enhances Cybersecurity” (Acceleration Economy)
• “New Generative AI Tools Aim to Improve Security” (Dark Reading)

2 - Protecting critical infrastructure: A key JCDC priority for 2024 

Defuse advanced persistent threat (APT) attacks against critical infrastructure. Improve critical infrastructure’s cybersecurity foundation. Accelerate cybersecurity innovation to curb emerging technology threats against critical infrastructure.

Notice a theme?

Those are the three main areas of focus this year for the Joint Cyber Defense Collaborative (JCDC), the group of government and private-sector organizations launched in 2021 by the U.S. Cybersecurity and Infrastructure Security Agency (CISA) to collaborate on cybersecurity.

“These priorities will further expand the breadth and depth of our partnership to tackle more challenging, forward-leaning cyber risks that could evolve in the future, not just the immediate risks,” Clayton Romans, the JCDC’s Associate Director, wrote in a blog post this week.

More specific priorities include:

• Improve the ability of critical infrastructure providers to fend off APT attacks, particularly from groups affiliated with the Chinese government.
• Reduce ransomware’s impact on critical infrastructure
• Lower the risk of AI-boosted cyberattacks against critical infrastructure

To get more details, check out:

• The blog “Extending the Breadth and Depth of our Partnerships: JCDC 2024 Priorities”
• A page about the JCDC’s 2024 priorities
• The JCDC’s home page

For more information about protecting critical infrastructure from cyberattacks, check out these Tenable resources:

• “Building Resilient OT Environments: Safeguards for Electric Utilities” (blog)
• “Critical Infrastructure Cybersecurity: Disrupt OT Attack Vectors” (white paper)
• “How to Tackle OT Challenges: Asset Inventory and Vulnerability Assessment” (blog)
• “2024 OT Security Trends: From the Boardroom to the Industrial Floor” (webinar)
• “Critical Infrastructure Orgs Must Beware of China-backed Volt Typhoon, Cyber Agencies Warn” (blog)
• “Shoring Up Water Security: Industry Leaders Testify Before Congress” (blog)

VIDEO

The Business Risk From a Ransomware Attack on OT Systems 

3 - Security of software package repositories on the spotlight

Involved with boosting the security of your software supply chain? You may be interested in new guidance issued this week for better securing software package repositories.

Published by the Open Source Security Foundation (OpenSSF) in collaboration with CISA, the “Principles for Package Repository Security” framework aims to help those in charge of repositories assess and improve their security. The guidance could also help organizations better evaluate the security of the repositories they use.

“Compromises of widely used open source dependencies can have widespread consequences. Package repositories are at a critical point in the open source ecosystem to help prevent or mitigate such attacks,” reads a blog co-authored by OpenSSF and CISA representatives.

The framework, now in version 0.1, outlines four core areas of repository security – authentication, authorization, general capabilities, and command-line interface tooling. It also details four levels of security maturity – from level zero to level three – for each area.

For example, a package repository would be considered to have level-three security authentication if it:

• Supports passwordless authentication
• Requires multi-factor authentication (MFA) for all of its maintainers
• Requires phishing-resistant MFA for critical packages

To get all the details, check out:

• The blog “OpenSSF Securing Software Repositories Working Group Releases Principles for Package Repository Security”
• The “Principles for Package Repository Security” framework

For more information about software supply chain security:

• “Security of Software Update Systems in 2023” (The New Stack)
• “6 most common types of software supply chain attacks explained” (CSO)
• “9 Supply Chain Security Best Practices” (The New Stack)
• “CISA, NSA push SBOM adoption to beef up supply chain security” (Tenable)
• “Securing the Software Supply Chain: Recommended Practices Guide for Customers” (CISA)
• “Why open-source software supply chain attacks have tripled in a year” (CSO)

4 - A temperature check on VM practices

During our webinar “When it Comes to Vulnerabilities, ‘Critical’ Doesn’t Always Mean ‘Critical,’” we took the opportunity to poll attendees about their vulnerability management programs. Check out what they said about their main VM challenges and their vulnerability-identification methods.

^{(286 respondents polled by Tenable in January 2024)}

^{(321 respondents polled by Tenable in January 2024)}

Watch the “When it Comes to Vulnerabilities, ‘Critical’ Doesn’t Always Mean ‘Critical’” webinar on-demand and learn how to establish an efficient and smooth patching process.

5 - Tech tools help drive U.S. consumer fraud to record highs

$10 billion. That’s how much consumers in the U.S. lost in 2023 to fraud, according to the U.S. Federal Trade Commission (FTC).

The losses, up 14% from 2022, are a new record, as fraudsters increasingly use technology to improve the speed, precision and sophistication of their scams.

“Digital tools are making it easier than ever to target hard-working Americans, and we see the effects of that in the data we're releasing today,” Samuel Levine, Director of the FTC’s Bureau of Consumer Protection, said in a statement.

In total, the FTC fielded 2.6 million fraud reports. Investment scams generated the most losses with $4.6 billion, an increase of 21%.

^{(Credit: Federal Trade Commission, February 2024)}

And the top method used to target consumers? That’d be email, followed by good, old-fashioned phone calls, with text messages in third place.

To get more details, check out:

• The FTC’s announcement “As Nationwide Fraud Losses Top $10 Billion in 2023, FTC Steps Up Efforts to Protect the Public”
• The FTC infographic “A Scammy Snapshot of 2023”

6 - Got info on the Hive ransomware gang? It could net you millions

The U.S. government is offering up to $10 million for information that can lead to identifying or locating leaders of the Hive ransomware gang.

There’s another $5 million reward for information that leads to the arrest of anyone who participated, or tried to participate, in Hive ransomware activity.

The rewards are being offered by the U.S. State Department’s Transnational Organized Crime Rewards Program.

“We will continue to work with allies and partners to disrupt and deter ransomware actors that threaten the backbone of our economies and critical infrastructure,” the State Department said in a statement.

After striking more than 1,500 victims globally, Hive had its operations disrupted and dismantled in 2022 by the FBI in collaboration with international law enforcement agencies.

For more information about the ransomware threat:

• “How to develop a cloud backup ransomware protection strategy” (TechTarget)
• “What’s new for ransomware in 2024?” (SC Magazine)
• “Understanding the Ransomware Ecosystem: From Screen Lockers to Multimillion-Dollar Criminal Enterprise” (Tenable)
• “How to prevent ransomware in 6 steps” (TechTarget)
• “7 Steps to Help Prevent & Limit the Impact of Ransomware” (Center for Internet Security)