Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: upgrade rustc, libafl, and clap to the latest version #201

Merged
merged 7 commits into from
Sep 29, 2023
Merged

refactor: upgrade rustc, libafl, and clap to the latest version #201

merged 7 commits into from
Sep 29, 2023

Conversation

jacob-chia
Copy link
Contributor

Fix #200

The CHANGELOG of migration from LibAFL 0.8.2 to 0.11.1 is as follows:

1 Implemented New Traits

  1. Implemented UsesInput for custom corpora/states: IndexedInMemoryCorpus, FuzzState, InfantStateState
  2. Implemented HasTestcase for custom corpora/states: IndexedInMemoryCorpus, FuzzState, InfantStateState
  3. Implemented UsesState for custom components:
    • schedulers: SortedDroppingScheduler, MoveVMStateScheduler, MoveTestcaseScheduler
    • executors: FuzzExecutor
    • stages: CoverageStage, ConcolicStage
    • fuzzers: ItyFuzzer
    • observers: None
    • event_managers: None
  4. Implemented UsesObservers for FuzzExecutor
  5. Implemented RemovableScheduler for SortedDroppingScheduler and MoveVMStateScheduler
  6. Implemented HasLastReportTime for FuzzState

2 The Scheduler

The type of scheduler and infant_scheduler are changed due to the constraints of new Scheduler trait. The types are changed as follows:

  • dyn Scheduler -> generic Scheduler;
  • Immutable reference -> Owned or Mutable reference.

For example:

// Schedulers are owned by structs
pub struct EVMCorpusInitializer<'a, SC, ISC> {
    // scheduler: &'a dyn Scheduler<EVMInput, EVMFuzzState>, // Before
    // infant_scheduler: &'a ISC,                            // Before
    scheduler: SC,                                           // After
    infant_scheduler: ISC,                                   // After
}

// Schedulers are borrowed as mutable in functions
fn add_infant_state<SC>(&mut self, scheduler: &mut SC, /* other args */) -> usize { ... }

2.1 Why cannot Schedulers be made into trait objects?

Because the Scheduler of new version is NOT object-safe. Let's look into the definition of Scheduler:

// libafl-0.11.1
pub trait Scheduler: UsesState
where
    Self::State: HasCorpus,
{
    // Except for the receiver (i.e. `&mut self`), if there is an argument type related to `Self`, it is NOT dispatchable.
    // The second argument type `&mut Self::State` is NOT dispatchable, so `Scheduler` is NOT object safe.
    fn next(&mut self, state: &mut Self::State) -> Result<CorpusId, Error>;
    fn on_add(&mut self, _state: &mut Self::State, _idx: CorpusId) -> Result<(), Error>;
}

2.2 Why mutable?

As you can see in the above, all the methods of Scheduler take &mut self as the receiver, so the schedulers have to be mutable.

2.3 Why owned?

Because a mutable reference cannot be borrowed more than once at a time, and clone a scheduler is cheap and safe since it is an empty struct.

#[derive(Debug, Clone)]
pub struct QueueScheduler<S> {
    phantom: PhantomData<S>,
}

#[derive(Debug, Clone)]
pub struct SortedDroppingScheduler<S> {
    phantom: std::marker::PhantomData<S>,
}

So, it's safe that structs own schedulers and pass them to functions as mutable references.

3 The Type of CorpusId

The type of corpus_id in traits has changed from usize to CorpusId, but I didn't change the layout of existing structs, so there are some conversions between usize and CorpusId.

4 HasMetadata and HasNamedMetadata

The method names have been changed: metadata/metadata_mut -> metadata_map/metadata_map_mut, and named_metadata/named_metadata_mut -> named_metadata_map/named_metadata_map_mut.

5 libalf::bolts -> libafl_bolts

The libafl::bolts module is separated from libafl to libafl_bolts.

// libafl-0.8.2
use libafl::{impl_serdeany, prelude::{Named, Rand, ShMemProvider}, ...};

// libafl-0.11.1
use libafl_bolts::{impl_serdeany, Named, bolts_prelude::{Rand, ShMemProvider}, ...};

6 Fine-Tuned Existing Implementations

Fine-Tuning Existing Implementations by removing reduntant generics and adding associated types and so on.

@shouc shouc requested a review from 0xAWM September 25, 2023 00:54
@shouc
Copy link
Contributor

shouc commented Sep 25, 2023

lgtm, thanks!

@fuzzland-bot
Copy link

Found: 7

Project Name Vulnerability Found Time Taken Log
DYNA_exp.txt ‼️ Crashed -1 Log File
SEAMAN_exp.txt ‼️ Crashed -1 Log File
BIGFI_exp.txt ✅ Price Manipulation 0h-0m-8s Log File
BEGO_exp.txt ‼️ Crashed -1 Log File
Yyds_exp.txt ✅ Fund Loss 0h-0m-1s Log File
AUR_exp.txt ‼️ Crashed -1 Log File
Annex_exp.txt ‼️ Crashed -1 Log File
PLTD_exp.txt ‼️ Crashed -1 Log File
ApeDAO_exp.txt ✅ Price Manipulation 0h-0m-3s Log File
Axioma_exp.txt ‼️ Crashed -1 Log File
SELLC03_exp.txt ‼️ Crashed -1 Log File
Novo_exp.txt ‼️ Crashed -1 Log File
VerilogCTF.txt ‼️ Crashed -1 Log File
THB_exp.txt ‼️ Crashed -1 Log File
CS_exp.txt ‼️ Crashed -1 Log File
EAC_exp.txt ‼️ Crashed -1 Log File
GSS_exp.txt ‼️ Crashed -1 Log File
SellToken_exp.txt ‼️ Crashed -1 Log File
cftoken_exp.txt ‼️ Crashed -1 Log File
GPT_exp.txt ‼️ Crashed -1 Log File
OLIFE_exp.txt ‼️ Crashed -1 Log File
MintoFinance_exp.txt ✅ Arbitrary Call 0h-0m-1s Log File
MBC_ZZSH_exp.txt ‼️ Crashed -1 Log File
ROI_exp.txt ✅ Fund Loss 0h-0m-10s Log File
HEALTH_exp.txt ✅ Price Manipulation 0h-0m-2s Log File
Shadowfi_exp.txt ‼️ Crashed -1 Log File
Carrot_exp.txt ✅ Arbitrary Call 0h-0m-9s Log File
RFB_exp.txt ‼️ Crashed -1 Log File

@jacob-chia
Copy link
Contributor Author

WOW! It's an amazing tool. I'll fix these errors.

@0xAWM
Copy link
Contributor

0xAWM commented Sep 26, 2023

I'll be working on these issues together as well, feel free to contact me

@fuzzland-bot

This comment has been minimized.

@fuzzland fuzzland deleted a comment from fuzzland-bot Sep 28, 2023
@0xAWM
Copy link
Contributor

0xAWM commented Sep 28, 2023

CI is down. I will manual trigger it when ready.

@jacob-chia
Copy link
Contributor Author

Some of the errors reported by the bot are unrelated to this PR, so I have created a new issue: #211.

The new changelog is as follows:

1 MaxMapFeedback

  • libafl-0.8.2: When is_interesting is called, it will merge the current JMP_MAP into the MapFeedbackMetadata::history_map.
  • libafl-0.11.1: The history_map is updated in the append_metadata function.

But the MaxMapFeedback is wrapped by Sha3WrappedFeedback and ConcolicFeedbackWrapper, so the two wrappers should implement the append_metadata function and forward the call to MaxMapFeedback. Take Sha3WrappedFeedback for example:

impl<I, S, VS, F, SC> Feedback<S> for Sha3WrappedFeedback<I, S, VS, F, SC> {
    fn append_metadata<OT>(/* args */) -> Result<(), Error> {
        self.inner_feedback.as_mut().append_metadata(state, observers, testcase)
    }
}

2 InMemoryCorpus

  • libafl-0.8.2: The storage of testcases is Vec, so the corpus_id is continuous.
  • libafl-0.11.1: The storage of testcases is HashMap, so the corpus_id is NOT continuous. But the trait Corpus defines new methods (next and last) to meet the needs of getting the next/last testcase.

Thus, the cov_stage should be updated as below:

// src/evm/cov_stage.rs

// Before
fn perform(/* args */) -> Result<(), Error> {
    // Start from `self.last_corpus_idx` because it has not been processed yet
    for i in self.last_corpus_idx..total { /* ... */ }

    // total is the next future idx that is not in the corpus
    self.last_corpus_idx = total;
}

// After
fn perform(/* args */) -> Result<(), Error> {
    let mut current = Some(CorpusId::from(self.last_corpus_idx));
    // Start from `state.corpus().next(current_idx)`
    // because the current_idx has been processed in the last round.
    while let Some(i) = state.corpus().next(current_idx) {
        /* logic */

        idx = state.corpus().next(i);
    }

    // last_idx is the last idx in the corpus
    self.last_corpus_idx = last_idx;
}

Note that the meaning of self.last_corpus_idx is different in the two versions:

  • Before: The next future idx that is NOT processed yet
  • After: The last processed idx in the corpus, because we don't know the next future idx.

@fuzzland-bot
Copy link

Found: 23

Project Name Vulnerability Found Time Taken Log
DYNA_exp.txt ✅ Price Manipulation 0h-0m-50s Log File
SEAMAN_exp.txt ✅ Fund Loss 0h-1m-25s Log File
BIGFI_exp.txt ✅ Price Manipulation 0h-1m-28s Log File
BEGO_exp.txt ✅ Fund Loss 0h-0m-31s Log File
Yyds_exp.txt ✅ Fund Loss 0h-0m-57s Log File
AUR_exp.txt -1 Log File
Annex_exp.txt -1 Log File
PLTD_exp.txt ✅ Price Manipulation 0h-0m-56s Log File
ApeDAO_exp.txt ✅ Price Manipulation 0h-0m-36s Log File
Axioma_exp.txt ✅ Fund Loss 0h-0m-56s Log File
SELLC03_exp.txt ✅ Fund Loss 0h-1m-17s Log File
Novo_exp.txt ✅ Price Manipulation 0h-0m-54s Log File
VerilogCTF.txt ✅ Fund Loss 0h-0m-27s Log File
THB_exp.txt ✅ Fund Loss 0h-0m-32s Log File
CS_exp.txt ✅ Price Manipulation 0h-0m-59s Log File
EAC_exp.txt -1 Log File
GSS_exp.txt ✅ Price Manipulation 0h-1m-50s Log File
SellToken_exp.txt ✅ Fund Loss 0h-0m-41s Log File
cftoken_exp.txt ✅ Price Manipulation 0h-0m-21s Log File
GPT_exp.txt -1 Log File
OLIFE_exp.txt -1 Log File
MintoFinance_exp.txt ✅ Arbitrary Call 0h-0m-35s Log File
MBC_ZZSH_exp.txt ✅ Price Manipulation 0h-0m-28s Log File
ROI_exp.txt ✅ Fund Loss 0h-0m-24s Log File
HEALTH_exp.txt ✅ Price Manipulation 0h-0m-17s Log File
Shadowfi_exp.txt ✅ Price Manipulation 0h-2m-32s Log File
Carrot_exp.txt ✅ Arbitrary Call 0h-0m-20s Log File
RFB_exp.txt ✅ Fund Loss 0h-0m-43s Log File

@0xAWM 0xAWM merged commit 47e25c3 into fuzzland:master Sep 29, 2023
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

rustc, libafl, and clap should be upgraded to meet the requirements of "sui_support" feature.
4 participants