July 16, 2024
ChatGPT can write smart contracts, just don’t use it as a security auditor
AI

ChatGPT: Write Smart Contracts, Not Security Auditor

Researchers from Salus Security, a blockchain security company with a global presence, have recently conducted research highlighting GPT-4’s capabilities in parsing and auditing smart contracts. Despite its proficiency in generating and parsing code, GPT-4 is not deemed suitable for the role of a security auditor.

“GPT-4 can be a useful tool in assisting with smart contract auditing, especially in code parsing and providing vulnerability hints. However, given its limitations in vulnerability detection, it cannot fully replace professional auditing tools and experienced auditors at this time.”

The researchers utilized a dataset comprising 35 smart contracts, known as the SolidiFI-benchmark vulnerability library, containing a total of 732 vulnerabilities. This dataset was used to assess GPT-4’s ability to identify security weaknesses across seven common vulnerability types. The findings indicate that ChatGPT performs well in detecting true positives, reaching over 80% precision in testing.

True positives refer to actual vulnerabilities that would merit investigation outside a testing environment. However, GPT-4 exhibits a notable challenge in generating false negatives, as reflected in its “recall rate,” which, in the Salus team’s experiments, was as low as 11% (where a higher rate is desirable). The researchers concluded that GPT-4’s vulnerability detection capabilities are lacking, with the highest accuracy reaching only 33%.

Consequently, they recommend the use of dedicated auditing tools and traditional human expertise for auditing smart contracts until AI systems like GPT-4 can be enhanced to meet the necessary standards. The study underscores the importance of a cautious approach to relying solely on AI for security audits in complex systems like smart contracts, emphasizing the continued need for human oversight and specialized tools to ensure thorough and accurate evaluations.

Photo by Andrew Neel

Disclosure Statement: Miami Crypto does not take any external funding, or support to bring crypto news to the readers. We do not have any conflicts of interest while writing news stories on Miami Crypto.

Related posts

Trump Claims One of His Speeches Was Altered by AI

Cheryl  Lee

Explore with Alexa: Amazon’s Kid-Friendly Answer to Consumer AI

Kevin Wilson

UK Regulators Probe Big Tech’s AI Investments

Robert Paul

This website uses cookies to improve your experience. We'll assume you're ok with this, but you can opt-out if you wish. Accept Read More

Please enter CoinGecko Free Api Key to get this plugin works.