arXiv:2504.05515v2 Announce Type: replace Abstract: Smart contract vulnerabilities pose significant security risks to the Ethereum ecosystem, driving the development of automated tools for detection and mitigation. Smart contracts are written in Solidity, a rapidly evolving programming language that continues to add features and improvements to enhance smart contract security. New versions of Solidity change the compilation process, potentially affecting how tools interpret and analyze smart contract code. We use Solidity as a case study to investigate a broader problem in software analysis: whether tool effectiveness remains stable as the underlying programming language and execution environment evolve. In this continuously evolving landscape, we investigated the compatibility of detection tools with different Solidity versions. More specifically, we analyzed detection tools by empirically assessing (i) their crash resilience, (ii) their detection effectiveness, and (iii) their execution time across different versions of Solidity. We collected 2,500 real-world smart contracts to construct a balanced wild dataset and created a separate injected-vulnerability dataset comprising 270 contracts. We extended SmartBugs, a framework that enables the execution of vulnerability detection tools for Solidity, adding 11 new tools. Results indicate that most tools are crash-resilient to version 0.4. However, 20% of the evaluated tools have been updated to remain compatible. Tool performance varies across Solidity versions and is affected by Solidity changes; execution time is higher in version 0.4.x, which was used for evaluation. We present a list of Solidity changes that affect tool performance and offer actionable takeaways for both researchers and practitioners.
Continue to arXiv Software Engineering →You are being forwarded in 6s — stay on this page