Solana localnet with ledger copied from mainnet-beta - solana

Question

I'm trying to test a program on localnet which makes numerous cross-program invocations (CPIs).
Is there an easy way to initialize a localnet cluster with all the accounts copied over from mainnet-beta?
I know there is a clone flag on the solana-test-validator command however it would be impractical to use clone for all the accounts I need copied over.

Answer 1

It is impractical to invoke solana-test-validator from the command line to do this.
The approach I've taken is to use solana account to get the accounts to local files, then use "in code" initialization of the solana test validator to load those accounts and then test.
For the first part, you could rig up a script to invoke:
solana account -o LOCALFILE.json --output json-compact PUBLIC_KEY where it will fetch account associated with PUBLIC_KEY and put in LOCALFILE.json
Then, in rust (just an example using 2 accounts but it could be many more. More than likely you'd want to walk a well known directory to load from and just loop that to build the input Vec:
fn load_stored(tvg: &mut TestValidatorGenesis) -> &mut TestValidatorGenesis {
let mut avec = Vec::<AccountInfo>::new();
for i in 0..2 {
let akp = get_keypair(USER_ACCOUNT_LIST[i]).unwrap();
avec.push(AccountInfo {
address: akp.pubkey(),
filename: USER_STORED_LIST[i],
});
}
tvg.add_accounts_from_json_files(&avec)
}
/// Setup the test validator with predefined properties
pub fn setup_validator() -> Result<(TestValidator, Keypair), Box<dyn error::Error>> {
let vwallet = get_keypair(WALLET_ACCOUNT).unwrap();
std::env::set_var("BPF_OUT_DIR", PROG_PATH);
let mut test_validator = TestValidatorGenesis::default();
test_validator.ledger_path(LEDGER_PATH);
test_validator.add_program(PROG_NAME, PROG_KEY);
load_stored(&mut test_validator);
// solana_logger::setup_with_default("solana=error");
let test_validator =
test_validator.start_with_mint_address(vwallet.pubkey(), SocketAddrSpace::new(true))?;
Ok((test_validator, vwallet))
}

Answer 2

You can launch the validator with -um -c ADDRESS to preload accounts with the content of mainnet-beta. In practice that's often not feasible, as you simply would need to many accounts, but for small programs it does work.

Answer 3

As another alternative, you can try using this fork of the Solana monorepo, which aims to clone the entire state of the ledger from mainnet, and spins up a validator from it: https://github.com/DappioWonderland/solana
Note that I haven't used it and haven't audited it to be sure it doesn't do anything shady, but if it lives up to the promise, it should be exactly what you need!