https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/Recent content in Introduction on Testing HandbookHugoen-ushttps://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/static-analysis/codeql/installation/Mon, 01 Jan 0001 00:00:00 +0000https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/static-analysis/codeql/installation/This section explains the process of installing CodeQL, as well as how to build and analyze a CodeQL database.https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/static-analysis/semgrep/installation/Mon, 01 Jan 0001 00:00:00 +0000https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/static-analysis/semgrep/installation/This section explains the process of installing Semgrep and how to start using it.https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/fuzzing/c-cpp/libfuzzer/Mon, 01 Jan 0001 00:00:00 +0000https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/fuzzing/c-cpp/libfuzzer/libFuzzer # libFuzzer is the clear and easy choice if you need to fuzz your C/C++ program, because it is part of the LLVM project and is available on most platforms. We recommend fuzzing on Linux if possible because it is the platform with the best support for libFuzzer (e.g., it is not preinstalled in XCode with macOS). Microsoft’s MSVC compiler has recently gained support for libFuzzer. Note that libFuzzer has been in maintenance-only mode since late 2022, so no new features will be added.https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/fuzzing/techniques/writing-harnesses/Mon, 01 Jan 0001 00:00:00 +0000https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/fuzzing/techniques/writing-harnesses/Writing harnesses # The following section showcases some techniques to successfully write a fuzzing harness—the most important part of any fuzzing setup. If written poorly, critical parts of your application may not be covered. Beyond byte arrays # Often the code you want to fuzz not only takes a plain byte array as input, but has more complex input. A very basic example is the following fuzz test that contains a division by 0.https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/fuzzing/c-cpp/aflpp/Mon, 01 Jan 0001 00:00:00 +0000https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/fuzzing/c-cpp/aflpp/AFL++ # The AFL++ fuzzer is a fork from the AFL fuzzer. It offers better fuzzing performance and more advanced features while still being a very stable alternative to libFuzzer. A major benefit over libFuzzer is that AFL++ has stable support for running fuzzing campaigns on multiple cores (see Multi-core fuzzing). This section of the Testing Handbook is based on fuzzing binaries written in C/C++ on Ubuntu on x64_64. AFL++ supports different environments like macOS, but there are caveats.https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/fuzzing/rust/cargo-fuzz/Mon, 01 Jan 0001 00:00:00 +0000https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/fuzzing/rust/cargo-fuzz/cargo-fuzz # The cargo-fuzz tool is the de facto choice for fuzzing your Rust project when using Cargo. It uses libFuzzer as the back end. Note that if you are not using Cargo, you cannot use the cargo-fuzz tool. By installing the cargo-fuzz crate, a Cargo subcommand is installed. Therefore, cargo-fuzz depends on using Cargo. The subcommand also automatically enables relevant compilation flags for your Rust project and even supports enabling sanitizers like AddressSanitizer.https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/fuzzing/c-cpp/techniques/coverage-analysis/Mon, 01 Jan 0001 00:00:00 +0000https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/fuzzing/c-cpp/techniques/coverage-analysis/Coverage analysis # Gaining confidence in your code coverage archived during fuzzing is essential for two reasons. Firstly, you want to assess which parts of your applications your fuzzing harnesses execute. For example, a magic value check, like the one shown in the following figure, may be hard for a fuzzer to overcome. Discovering such a check is important so that the values can be provided to the fuzzer through a dictionary or test cases in the seed corpus.https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/fuzzing/rust/techniques/coverage-analysis/Mon, 01 Jan 0001 00:00:00 +0000https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/fuzzing/rust/techniques/coverage-analysis/Coverage analysis # Gaining confidence in your code coverage archived during fuzzing is essential for two reasons. Firstly, you want to assess which parts of your applications your fuzzing harnesses execute. For example, a magic value check, like the one shown in the following figure, may be hard for a fuzzer to overcome. Discovering such a check is important so that the values can be provided to the fuzzer through a dictionary or test cases in the seed corpus.https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/fuzzing/techniques/dictionary/Mon, 01 Jan 0001 00:00:00 +0000https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/fuzzing/techniques/dictionary/Fuzzing dictionary # A dictionary can be used to guide the fuzzer. A dictionary is usually passed as a file to the fuzzer. The simplest input accepted by libFuzzer is a ASCII text file where each line consists of a quoted string. Strings can contain escaped byte sequences like “\xF7\xF8". Optionally, a key-value pair like hex_value="\xF7\xF8" can be used for documentation purposes. Comments are supported by starting a line with #.https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/fuzzing/c-cpp/techniques/obstacles/Mon, 01 Jan 0001 00:00:00 +0000https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/fuzzing/c-cpp/techniques/obstacles/SUT patching: Overcoming obstacles # Codebases are often not fuzzing-friendly. This can happen if, for example, the code uses checksums or depends on a global state like a system-time seeded PRNG (i.e., by using rand) that causes the code to behave differently for the same input. Refer to Practical harness rules to learn more about potential problems in your SUT. If you encounter checksums or a global state in your SUT, you may want to apply fuzzing-specific patches to change the behavior of the program during fuzzing, as shown in the following paragraphs.https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/fuzzing/rust/techniques/obstacles/Mon, 01 Jan 0001 00:00:00 +0000https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/fuzzing/rust/techniques/obstacles/SUT patching: Overcoming obstacles # Codebases are often not fuzzing-friendly. This can happen if, for example, the code uses checksums or depends on a global state like a system-time seeded PRNG (i.e., from the rand) that causes the code to behave differently for the same input. Refer to Practical harness rules to learn more about potential problems in your SUT. If you encounter checksums or a global state in your SUT, you may want to apply fuzzing-specific patches to change the behavior of the program during fuzzing, as shown in the following paragraphs.https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/fuzzing/rust/techniques/writing-harnesses/Mon, 01 Jan 0001 00:00:00 +0000https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/fuzzing/rust/techniques/writing-harnesses/Writing harnesses # In the following, we will go over Rust specific tips to optimize the results from your harnesses. For general advice, refer to Writing harnesses. Structure-Aware fuzzing with the arbitrary crate # The arbitrary crate simplifies writing fuzzing harnesses. By deriving a macro, Rust structs can be targeted for fuzzing. For example, the following code requires constructing a Name struct that owns a String. We derived the Arbitrary macro to facilitate the construction of such a Name.https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/fuzzing/techniques/asan/Mon, 01 Jan 0001 00:00:00 +0000https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/fuzzing/techniques/asan/AddressSanitizer # AddressSanitizer (ASan) is a widely adopted tool in the realm of software testing, particularly during fuzzing. Fuzzing greatly benefits from the use of ASan because it helps detect memory errors that might otherwise go unnoticed, such as buffer overflows and use-after-free errors. While ASan is a standard practice in fuzzing due to its effectiveness in identifying such vulnerabilities, it does come with certain drawbacks. One significant downside is that it can make the fuzzing process approximately 2–4 times slower.https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/fuzzing/c-cpp/libafl/Mon, 01 Jan 0001 00:00:00 +0000https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/fuzzing/c-cpp/libafl/LibAFL # The LibAFL fuzzer implements features from AFL-based fuzzers like AFL++. Similarly to AFL++, LibAFL provides better fuzzing performance and more advanced features over libFuzzer. However, with LibAFL, all functionality is provided in a modular and customizable way—in fact, LibAFL is a library that can be used to implement custom fuzzers. Because LibAFL is a library, there is no single-line command to install LibAFL like there is with libFuzzer (apt install clang) and AFL++ (apt install afl++).https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/fuzzing/python/Mon, 01 Jan 0001 00:00:00 +0000https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/fuzzing/python/Python # We recommend using Atheris to fuzz Python code. Installation # Atheris supports 32-bit and 64-bit Linux, and macOS. We recommend fuzzing on Linux because it’s simpler to manage and often faster. If you’d like to run Atheris in a Linux environment on a Mac or Windows system, we recommend using Docker Desktop. If you’d like a fully operational Linux environment, see the Dockerfile section below. If you’d like to install Atheris locally, first install a recent version of clang, preferably the latest release, then run the following command:https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/fuzzing/techniques/environments/Mon, 01 Jan 0001 00:00:00 +0000https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/fuzzing/techniques/environments/Fuzzing environments # Like any software, the choice of fuzzer will depend on factors such as the operating system, architecture, software versions, and hardware. This section will review factors that influence the choice of the environment used for fuzzing. Choice of hardware. If your fuzzer supports running on multiple cores, choose hardware that has many cores available. Renting or purchasing a bare-metal server might be worthwhile if you plan to run campaigns regularly.https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/fuzzing/ruby/Mon, 01 Jan 0001 00:00:00 +0000https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/fuzzing/ruby/Ruby # We recommend using Ruzzy to fuzz Ruby code. Installation # Ruzzy supports Linux x86-64 and AArch64/ARM64. If you’d like to run Ruzzy on a macOS or Windows, you can build the Dockerfile and/or use the development environment. Ruzzy requires a recent version of clang (tested back to 14.0.0), preferably the latest release. Install Ruzzy with the following command: MAKE="make --environment-overrides V=1" \ CC="/path/to/clang" \ CXX="/path/to/clang++" \ LDSHARED="/path/to/clang -shared" \ LDSHAREDXX="/path/to/clang++ -shared" \ gem install ruzzy There’s a lot going on here, so let’s break it down:https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/fuzzing/techniques/faq/Mon, 01 Jan 0001 00:00:00 +0000https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/fuzzing/techniques/faq/FAQ (Fuzzily Asked Questions) # The fuzzer is showing crashes, but when I run the SUT on the test cases outside of the fuzzer, then no crash is shown. What is happening? # There are several potential reasons: The SUT is behaving nondeterministically. The SUT depends on global state (e.g., it may read or write to disk or use in-memory singletons). You are experiencing crashes because your system is running out of memory and killing processes.https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/fuzzing/oss-fuzz/Mon, 01 Jan 0001 00:00:00 +0000https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/fuzzing/oss-fuzz/OSS-Fuzz # OSS-Fuzz is an open-source project developed by Google that aims to improve the security and stability of open-source software by providing free distributed infrastructure for continuous fuzz testing. Using a pre-existing framework like OSS-Fuzz has many advantages over manually running harnesses: it streamlines the process and facilitates simpler modifications. Although only select projects are accepted into OSS-Fuzz, because the project’s core is open-source, anyone can host their own instance of OSS-Fuzz and use it for private projects!https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/fuzzing/snapshot-fuzzing/Mon, 01 Jan 0001 00:00:00 +0000https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/fuzzing/snapshot-fuzzing/Snapshot fuzzing enables security engineers to effectively test software that is traditionally difficult to analyze, such as kernel-level software (though the technique is not limited to such software). Whether you’re auditing drivers or other kernel-mode components, including antivirus software, snapshot fuzzing provides a robust way to discover critical vulnerabilities. Consult this section for a walkthrough on how to conduct snapshot fuzzing on your system.https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/fuzzing/resources/Mon, 01 Jan 0001 00:00:00 +0000https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/fuzzing/resources/Additional resources # Learning # Awesome fuzzing list 1: GitHub-hosted list focused on scientific publications about fuzzing. Awesome fuzzing list 2: GitHub-hosted list about fuzzers and fuzzing related books. Fuzzing handbook: A fuzzing handbook written from an academic perspective. CNCF-Fuzzing handbook: Handbook created by the CNCF. Fuzzing101: Tutorial and training for various fuzzing methods by GitHub Security Lab. “How to Spot Good Fuzzing Research”: ToB blogpost 39c3 “Demystifying Fuzzer Behaviour”: interesting perspective on fuzz testing How to Build a Fuzzing Corpus Fuzzing corpus data # https://github.https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/static-analysis/codeql/advanced/Mon, 01 Jan 0001 00:00:00 +0000https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/static-analysis/codeql/advanced/This section describes how to create and test custom CodeQL queries and query packs.https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/static-analysis/semgrep/advanced/Mon, 01 Jan 0001 00:00:00 +0000https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/static-analysis/semgrep/advanced/This section explores the advanced usage of Semgrep, including how to create new rules.https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/languages/c-cpp/lang-c-cpp-bug-classes/Mon, 01 Jan 0001 00:00:00 +0000https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/languages/c-cpp/lang-c-cpp-bug-classes/C/C++ bug classes # Below is a list of common vulnerability types for C/C++ programs. This list does not include comprehensive details; rather, it is intended to be broad and high level. Use it as a starting point during security reviews of C/C++ programs. Buffer overflow and underflow, spatial safety Off-by-one mistakes Invalid computation of object sizes Misunderstanding of data-moving functions’ semantics Data comparison using out-of-bounds lengths Copying of raw memory instead of the object Out-of-bounds iterators Use after free, temporal safety Use after free Example: Two shared_ptrs point to the same object, and one of them decrements its reference count (refcount) to 0 and frees the object.https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/languages/rust/lang-rust-security-overview/Mon, 01 Jan 0001 00:00:00 +0000https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/languages/rust/lang-rust-security-overview/Rust security overview # Safety and security # The Rust compiler guarantees the memory safety of Rust programs: no undefined behavior or data race will happen during runtime, no matter the inputs. Therefore, when security-testing Rust programs, it’s important to understand what is and what is not considered undefined behavior (UB). There is no sense in looking for double-free bugs in (safe) Rust, right? For the guarantees made by the Rust compiler, see the “Behavior considered undefined” Rust Reference page.https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/languages/c-cpp/lang-c-cpp-linux-usermode/Mon, 01 Jan 0001 00:00:00 +0000https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/languages/c-cpp/lang-c-cpp-linux-usermode/Linux usermode # This list covers common checks for and footguns of C/C++ standard libraries when used in Unix environments. Run checksec to learn about the executable’s exploit mitigations. Check for uses of NX, PIE, stack cookies, RELRO, FORTIFY_SOURCE, stack clash protector, SafeStack, ShadowCallStack, and other mitigations. Check that production releases do not contain debug information. Check for uses of non-thread-safe functions in multi-threaded programs. Some of these functions—such as gethostbyname, inet_ntoa, strtok, and localtime—may return pointers to static data.https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/languages/c-cpp/lang-c-cpp-linux-kernelmode/Mon, 01 Jan 0001 00:00:00 +0000https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/languages/c-cpp/lang-c-cpp-linux-kernelmode/Linux kernel # This list includes basic checks for Linux kernel drivers and modules. Check that all uses of data coming from the userspace are validated. Look for the __user tag in the code. Not necessarily all userspace data is marked with the tag, but it is a good start. Uses of copy_from_user(to, from, size) should be validated: The length should be validated correctly. There should be no uninitialized to addresses.https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/languages/c-cpp/lang-c-cpp-windows-usermode/Mon, 01 Jan 0001 00:00:00 +0000https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/languages/c-cpp/lang-c-cpp-windows-usermode/Windows usermode # This list covers common checks for and footguns of C/C++ standard libraries when used in Windows environments. Run binskim to check mitigation opt-in and other issues in the application binaries. DEP (NX), ASLR (and HiASLR on x86_64), Control Flow Guard (CFG), and SafeSEH (on x86_32 only) should always be enabled, and executables should be signed. Shadow stack (CET) and Spectre mitigations may be enabled for additional security.https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/languages/c-cpp/lang-c-cpp-windows-kernelmode/Mon, 01 Jan 0001 00:00:00 +0000https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/languages/c-cpp/lang-c-cpp-windows-kernelmode/Windows kernel # This list includes basic checks for Windows kernel drivers and modules. Run CodeQL on the application’s driver. Microsoft has published CodeQL support and security query packs for Windows drivers. If you can build the driver from source, CodeQL is a high-value SAST approach. If you cannot build the driver from source, you may still be able to run CodeQL with --build-mode=none on the CodeQL CLI during database creation, but coverage and accuracy will be significantly diminished.https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/static-analysis/codeql/continuous-integration/Mon, 01 Jan 0001 00:00:00 +0000https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/static-analysis/codeql/continuous-integration/This section describes the process of integrating CodeQL into your continuous integration and continuous delivery (CI/CD) pipeline.https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/static-analysis/semgrep/continuous-integration/Mon, 01 Jan 0001 00:00:00 +0000https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/static-analysis/semgrep/continuous-integration/This section explains the process of integrating Semgrep into your continuous integration and continuous delivery (CI/CD) pipeline.https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/languages/rust/lang-rust-dynamic-analysis/Mon, 01 Jan 0001 00:00:00 +0000https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/languages/rust/lang-rust-dynamic-analysis/Rust dynamic analysis # There are two categories of dynamic analysis: fuzz testing and “standard” testing, like unit and functional testing. In this chapter, we will focus on standard testing. This is the basic level of testing that every project should have implemented. While basic, standard testing can be built up with a lot of security-wise improvements. To read about Rust fuzzing, see the Rust fuzzing chapter of this handbook.https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/languages/c-cpp/lang-c-cpp-seccomp/Mon, 01 Jan 0001 00:00:00 +0000https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/languages/c-cpp/lang-c-cpp-seccomp/Seccomp # Sandboxing is hard. But if it is needed and seccomp is used, ensure the following is done: The BPF filter checks the architecture (e.g., x86-64 versus i386). The BPF filter checks the ABI/calling convention (e.g., x86-64 versus x32 ABIs for x86-64 architecture). This means checking for syscalls with numbers greater than 0x40000000 (the __X32_SYSCALL_BIT flag). Syscalls implemented in vsyscall and VDSO are handled correctly. This is important only for optimized syscalls like gettimeofday, time, getcpu, and clock_gettime.https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/static-analysis/semgrep/in-your-organization/Mon, 01 Jan 0001 00:00:00 +0000https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/static-analysis/semgrep/in-your-organization/This section discusses the process of introducing Semgrep to your organization.https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/languages/rust/lang-rust-static-analysis/Mon, 01 Jan 0001 00:00:00 +0000https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/languages/rust/lang-rust-static-analysis/Rust static analysis # Static analysis involves analyzing source code without executing it. Clippy # Clippy is the fundamental linter. Just use it. Clippy lints are categorized into groups and levels. Groups categorize lints by the types of issues they detect. Levels indicate what to do when a lint finds an issue: Allow: Ignore the issue Warn: Print a message to stdout Deny: Return an error code (useful in CI pipelines) Clippy is a wrapper over the rustc compiler, so when Clippy is run, it executes its own set of lints in addition to rustc’s.https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/crypto/wycheproof/wycheproo_example/Mon, 01 Jan 0001 00:00:00 +0000https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/crypto/wycheproof/wycheproo_example/Wycheproof testing harness example in Python # The following section will showcase how to write a simple testing harness to test a Python library implementing AES in GCM mode. We will use the cryptography package as an example of an AES-GCM implementation. For testing, we will use the pytest package, one of the most popular testing frameworks in Python. Prerequisites # Add the Wycheproof repository as a submodule to your existing repository.https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/languages/rust/lang-rust-gotchas-and-footguns/Mon, 01 Jan 0001 00:00:00 +0000https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/languages/rust/lang-rust-gotchas-and-footguns/Rust gotchas and footguns # This section provides a checklist that can be used during manual Rust code reviews. The list represents common issues we have encountered during our audits. It is not comprehensive, but it is a good starting point to quickly bootstrap an audit. For safe code # Check string comparisons. Often partial-match (starts_with, ends_with, contains) is used instead of equality. Case (in)sensitivity of comparisons often results in issues.https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/crypto/constant_time_tool/dudect/Mon, 01 Jan 0001 00:00:00 +0000https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/crypto/constant_time_tool/dudect/Dudect # Overview # Dudect is a statistical constant-time analysis tool that measures the execution time of a specific code section for two input classes and aims to find the statical difference between the measurements of the two classes. If the timing measurements for the two input classes deviate from one another, it would suggest that the code is dependent on the input and, therefore, not constant time. The two most commonly used input classes are:https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/languages/rust/lang-rust-memory-zeroization/Mon, 01 Jan 0001 00:00:00 +0000https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/languages/rust/lang-rust-memory-zeroization/Rust memory zeroization # Zeroization in the presence of optimizing compilers is difficult. In Rust, it is particularly tricky because of the constraints the compiler imposes on memory management. The compiler can infer significant information about aliasing information that allows unexpected copies of secret data to appear on the stack. For example, consider the pitfall described in the blog post “A pitfall of Rust’s move/copy/drop semantics and zeroing data”.https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/crypto/constant_time_tool/timecop/Mon, 01 Jan 0001 00:00:00 +0000https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/crypto/constant_time_tool/timecop/Timecop (Valgrind) # Timecop is a wrapper around Valgrind designed to detect potential timing leaks dynamically. It allows developers to mark memory regions as secret, and if during runtime, a branching instruction or memory access is performed that is dependent on the secret memory region, Valgrind will report the behavior, helping to identify timing vulnerabilities. Overview # Timecop is a C macro wrapper around functions provided by Valgrind. Setup # To use Timecop, you must first install Valgrind.https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/languages/rust/lang-rust-model-checking/Mon, 01 Jan 0001 00:00:00 +0000https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/languages/rust/lang-rust-model-checking/Rust model checking # Model checking means verifying that a program works correctly for all possible inputs. Instead of testing with a single value (like with unit testing) or with a set of values (like with property testing), we check all possible values—and hope that smart algorithms will make it possible to finish testing in a reasonable time. Prusti # Prusti is based on Viper, a framework for building verification tools.https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/languages/rust/lang-rust-specialized-testing/Mon, 01 Jan 0001 00:00:00 +0000https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/languages/rust/lang-rust-specialized-testing/ Rust specialized testing # Concurrency testing # Shuttle Unsound, but scalable Works analogously to property testing Loom Sound, but slow Works analogously to model checkers Fault injections # MadSim Replaces the tokio and tonic crates with simulated versions Injects faults and increases randomness fail-rs Needs you to manually add fail_point! macros into the codehttps://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/languages/rust/lang-rust-supply-chain-analysis/Mon, 01 Jan 0001 00:00:00 +0000https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/languages/rust/lang-rust-supply-chain-analysis/Rust supply chain analysis # Vetting # The tools in this section are more for “understanding” than “checking.” That is, running them does not produce bug reports, but can help you assess the maturity and security of dependencies. The tools below are quantitative rather than qualitative; you will need to perform manual, in-depth review of the outputs to extract any solid evidence about the maturity. cargo-supply-chain # The tool reveals who you are implicitly trusting via dependencies.https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/static-analysis/codeql/resources/Mon, 01 Jan 0001 00:00:00 +0000https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/static-analysis/codeql/resources/Learn more about CodeQL using the external resources listed on this page.https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/static-analysis/semgrep/resources/Mon, 01 Jan 0001 00:00:00 +0000https://trailofbits.github.io/testing-handbook-preview/pr-preview/pr-128/docs/static-analysis/semgrep/resources/Learn more about Semgrep using the external resources listed on this page.