Greetings dear readers! Today’s essay is the next in a series exploring creative solutions to challenges that an Auditor can encounter. In our previous article, we described Slither and how to use it; but today, we’d like to discuss fuzzing and the tool for it, Echidna.
Read an original article via the link below:
First and foremost, we would like to express our sincere gratitude to the creators of this tool, everyone who supports it, the authors of all the resource materials, and of course our staff auditors who have helped us by revealing much-needed information and lifting the curtain of secrecy. And today, dear readers, it will be made available to you.
In this series we will focus only on those aspects that can be really useful for auditing and bug bounty hacking and that are not described anywhere! We can confidently say that such tips can be read publicly in a few places, and our blog is one of those places!
Please follow us right now, check out our recent article about Slither, and let’s get started! By the way, there are some vacancies now so if your project needs an audit — feel free to write to us, visit our public reports page here.
I — Fuzzing: Starting Up
Let’s familiarize ourselves with the fundamentals before moving on to the challenging material. What is this fuzzing, and how might it be used by security researchers, ethical hackers, and bug bounty hunters?
Fuzz testing, often known as fuzzing, is an automated software testing technique used to identify application flaws, security risks, and vulnerabilities. Using a fuzzing tool, the goal is to introduce erroneous, improper, or unexpected inputs into the application and track how the system responds to them (e.g., exceptions, leakage of information, or application crashes).
Fuzzers for traditional software (such as AFL or LibFuzzer) are known to be efficient tools to find bugs. Beyond the purely random generation of inputs, there are many techniques and strategies to generate good inputs, including:
-
Obtain feedback from each execution and guide generation using it. For example, if a newly generated input leads to the discovery of a new path, it makes sense to generate new inputs closest to it.
-
Generate input with respect to a structural constraint. For example, if your input contains a header with a checksum, it makes sense to let the fuzzer generate input validating the checksum.
-
Use known inputs to generate new inputs. If you have access to a large dataset of valid input, your fuzzer can generate new inputs from them, rather than starting from scratch for each generation. These are usually called seeds.
Echidna is part of a fuzzing family called property-based fuzzing. It is a Haskell program designed for fuzzing/property-based testing of Ethereum smart contracts which uses sophisticated grammar-based fuzzing campaigns based on a contract ABI to falsify user-defined predicates or Solidity assertions.
Rather than looking for crashes like a traditional fuzzer, Echidna will try to break user-defined invariants.
Features of Echidna:
-
Generates inputs tailored to your actual code.
-
Optional corpus collection, mutation, and coverage guidance to find deeper bugs.
-
Powered by Slither to extract useful information before the fuzzing campaign.
-
Source code integration to identify which lines are covered after the fuzzing campaign.
-
Curses-based retro UI, text-only, or JSON output.
-
Automatic test case minimization for quick triage.
-
Seamless integration into the development workflow, but, according to our experience in this case, the tool can behave unpredictably and success is not guaranteed.
-
Maximum gas usage reporting of the fuzzing campaign.
II — Fuzzing with Echidna
In smart contracts, invariants are Solidity functions, that can represent any incorrect or invalid state that the contract can reach, including:
-
Incorrect access control: the attacker became the owner of the contract.
-
Incorrect state machine: the tokens can be transferred while the contract is paused.
-
Incorrect arithmetic: the user can underflow its balance and get unlimited free tokens.
Echidna can test contracts compiled with different smart contract build systems, including Hardhat, Foundry, Truffle, and even those written in Vyper, — everything supported by crytic-compile. Echidna also supports two modes of testing complex contracts:
-
Firstly, one can describe an initialization procedure with Truffle and Etheno and use that as the base state for Echidna.
-
Secondly, the Echidna can call into any contract with a known ABI by passing in the corresponding solidity source in the CLI.
-
Echidna supports three different output drivers. There is the default text driver, a json driver, and a none driver, which should suppress all stdout output.
III — Installation & Combat Use of the Echidna
The information presented below is mostly based on the data we got from our audits, a few of them are detailed enough for you to study them and to better understand the essence of our work and the application of fuzzing in it. Keep in mind a few important statements. They will come in handy when you start the program:
-
Slither is required to run Echidna correctly.
-
Installation is done depending on the OS used (you can choose a specific operating system for your work).
-
Pay attention to the version to be installed, always install the latest version (and don’t forget to update).
We recommend this procedure to fuzz contracts with Echidna. It detects configuration errors early and helps to run Echidna smoothly:
-
Compile the target project.
-
Run Slither. If it does not start, make changes to the project/solve the problem in any other way.
-
Launch Echidna. Run echidna without any property as follows: «
— echinda-test . — contract contract — test-mode assertion. » -
If it does not start, modify/update echidna according to terminal messages.
-
Create a test contract. Inherit the target contract, write a simple basic property, try it again.
-
Write a full-fledged property and then start fuzzing.
Also check out how to optimize Echidna in this tutorial article from Trail of Bits! Keep in mind that optimizing Haskell programs is very different from optimizing imperative programs since the order of execution is often very different from the order in which the code is written!
IV — Limitations & Known Issues
Echidna has quite a few limitations in the latest release. Some of these are inherited from hevm while some are results from design/performance decisions or simply bugs in the code.
The issues are listed as follows:
-
Debug information can be insufficient.
-
Vyper support is limited.
-
Limited library support for testing.
-
If the contract is not linked properly, Echidna will crash.
-
Assertions are not detected in internal transactions.
-
Value generation can fail in multi-abi mode since the function hash is not precise enough.
Also keep in mind that Brownie builds interfaces in its own way, so it is not possible to run Slither directly on this framework (slither-side issue) due to crytic-compile errors. The solution is to migrate to another framework (Truffle, Hardhat, Foundry).
The following problems may also occur:
-
Test mode assertion does not work on ^0.8 contracts if the Echidna version which is older than 2.0.0 is installed.
-
If the project uses libraries, all functions in them must be internal.
-
Slither struggles with ternary expressions, you will often have to rewrite them as regular if .
-
If project doesn’t find error in simple property, then it’s require/revert inside project. Try to reconfigure the project so that require passes.
-
The Echidna IllegalOverflow error appears when you want to limit the parameters of gas costs for function calls in the config.
With that being said, it is also to be noted that no testing/fuzzing tool out there is perfect as EVM emulation and testing is a hard nut to crack. However, Echidna is a great tool when dealing with complex smart contracts.
V — Additional Tips
The advice above is enough to get you started. If you have any problems, don’t hesitate to ask us about it in the comments!
If you want to get more info on Echidna tricks, then browse this repository, watch this video, and choose a specific operating system for your work.
Go through all listed resources — most of them are real gems and provide great info on how to run Echidna with examples.
Use other instruments as well! E.g. Mantictore (more info) — or just look at this selection and separately at this ultra-specific tool (2); there are so many of them.
Most importantly, even admirably well-done manual unit tests don’t find the kind of weird edge-case bugs attackers are looking for.
Below you will find a list of resources used — they contain many good manuals that will help you with your auditor’s work. I take this opportunity to pay my respects to the Authors!
Sources:
-
blog.pessimistic.io/slither-an-auditors-cornucopia-a8793ea96e67
-
github.com/crytic/building-secure-contracts/tree/master/program-analysis/echidna
-
blog.finxter.com/ethereum-smart-contract-fuzz-testing-with-echidna
-
immunebytes.com/blog/effective-usable-and-fast-echidna-a-smart-contract-fuzzing-tool
-
github.com/crytic/building-secure-contracts/tree/master/program-analysis/echidna#echidna-tutorial
-
blog.trailofbits.com/2020/08/17/using-echidna-to-test-a-smart-contract-library
-
researchgate.net/publication/343048222_Echidna_effective_usable_and_fast_fuzzing_for_smart_contracts
-
blog.trailofbits.com/2022/03/02/optimizing-a-smart-contract-fuzzer
VI — Conclusion
In conclusion, we would like to say that we hope that this article was informative and useful for you! Thank you for reading! 🙂
The most important thing we wanted to tell you and show you is that a new instrument does not mean good and an old instrument does not mean bad. It’s all about knowing how to use it and being willing to look for something new. We hope we’ve got you interested in that!
What instruments should we review? What would you be interested in reading about? Please leave your comments, we will be happy to answer them, and the best answers and questions may be included in the next article!
By the way, there are some vacancies now so if your project needs an audit — feel free to write to us, visit our public reports page here.