Introduction
Humans are prone to error. In every project built, there are some errors & bugs and there will always be. As applications and lines of code increase, it becomes challenging for developers and security professionals to investigate code. Complexity of application makes it hard to catch bugs in the test environment. Because of these reasons, fuzzing was invented.
Fuzzing is a technique used in software testing primarily aimed at uncovering coding mistakes and defects. Fuzzers are automated software testing tools designed to find bugs by randomly generating and inputting a wide range of data into a program to see if it causes crashes or other failures.
An important thing about fuzzing is that one needs to suit fuzz testing according to the application logic as in all automated testing. Because computers are just machines that the user requested to do, users need to tailor fuzzing for their needs, application logic and so on. Otherwise, the fuzzer will just give an error or in the best case would not find anything useful.
Benefits & Limitations
Let’s start with the benefits of the fuzzers. The modern fuzzing tools implement advanced algorithms to generate intelligent and diverse input data to improve the quality of testing. This situation can be very helpful for users. Imagine you want to check the transaction signing feature. You need to construct transaction data that matches the expected structure in order to trigger the internal logic of the transaction signing process. Using fuzzers, it is possible to test not only the signing process but also all the related logic that is called internally. For example, errors in the encoding/decoding routines, misuse of third-party APIs, and so on, can be identified. This approach enables the implementation of comprehensive testing for the entire feature using solely the fuzzer.
Fuzzers can run for a long time, and therefore they can trigger rare code paths and identify vulnerabilities in the edge cases. This approach is called continuous fuzzing. Its idea is to continuously run fuzzers for the production code to uncover possible errors introduced in the development process. It can also be helpful for organisations to measure the quality of software they have. In a continuous fuzzing environment, it is more likely to catch bugs before the malicious actors.
It is also important to mention limitations of the fuzzers. Typically, it needs much more time to obtain interesting findings using the fuzzing approach. It might take days or even weeks. Moreover, the fuzzing algorithms typically require modern hardware for execution.
It also takes time to design a good fuzzing environment, which requires an excellent knowledge of the tested system and understanding of the tools in use. But that’s an important step, because it optimises fuzzers for specific needs and improves the quality of testing.
Fuzzers also have a variety of false positive bugs. One needs to check if these crashes are caused really from the input or other factors like configuration etc. After investigating the cause of the crash, one can confirm that a bug actually exists. Each of these crashes will require time and assessment. Poorly configured systems can create multiple crashes which cause organisations to lose time with false positive findings.
Fuzzing is used by open source projects in multiple years. According to Google, Since 2016 Google’s free OSS-Fuzz code testing service has helped get over 8800 vulnerabilities and 28,000 bugs fixed across 850 projects.
Use of Fuzzer for the Polkadot Ecosystem
Fuzzers are useful tools that can be used to test complex applications. With tailored solutions according to needs, they can find critical bugs in a low cost and continuous way. Implementing a fuzzer for a project ensures its security, integrity and quality.
Parity takes fuzzers as a serious business. In the publicly accessible GitHub repositories, have implemented a suite of fuzzers, several of which are integrated with various fuzzer engines. The fuzzers are used to test the specific features in the projects, they extend test suites and are used in property-based tests. Thanks to the partnership with SRLabs new fuzzers have been developed and made available to the ecosystem. Such runtime fuzzers have been used to support the Substrate Builder Program. In addition, learnings and Q&A sessions have been conducted to clarify all the quirks and features of the Substrate Runtime Fuzzer as it is one of the biggest and heaviest (runs can take several days). Due to the nature of some fuzzers, it is important to have and define clarity about what and when to execute them for maximum positive results.
Please find below a list of the different fuzzers available
So to conclude, don’t hesitate to use them and to share your successes and challenges in order for the polkadot community to be more secure.