svm: optimize filter_executable_program_accounts
#4574
Closed
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem
presently,
filter_executable_program_accounts
makes an expensive accounts-db lookup for every account in every transaction batch. this used to be an unfortunate necessity because transaction account loading could not happen until the local program cache was built. a recent feature gate activation has severed that dependency, giving us freedom to improve thisSummary of Changes
initialize the
AccountLoader
before constructing the program cache, and changefilter_executable_program_accounts
to use it instead of accounts-db. this eliminates all expensive calls toaccount_matches_owners
. this change does not increase the number of calls toget_account_shared_data
because the sameAccountLoader
object is then used for transaction loadingwe also remove the program owner from
program_accounts_map
. this was removed in an early version of 2.0 but had to be added back; we can now definitively remove it, since it is no longer required for owner validation in transaction loadingwe also fix an incidental bug where usage counts of builtin programs were reset to zero
Notes
the initial attempt at this pr (#4553) tried to move program cache creation after account loading, creating one local program cache per transaction. however we cannot easily do this: under normal operation,
config.check_program_modification_slot
isfalse
. this makes it impossible to determine whether a loaderv3 program should be tombstoned because it was modified in-entry. presently this behaves correctly as a side effect of reusing the same local program cachein the long term we do want to move in this direction, cutting back the functionality of the local program cache into a much simpler passthrough to retrieve compiled/verified program bytecode, with extremely minimal internal state-tracking and no merge/update logic. this might be straightforward if we can simply remove
check_program_modification_slot
and make it behave as if it were alwaystrue
, but this may require a feature gatein the interim we may also be able to remove accounts-db lookups from
replenish_program_cache
and its children by usingAccountLoader
instead of the callback. this should not require major redesign but it is complicated by the fact thatload_program_with_pubkey
andload_program_accounts
are required byBank
. in theory we could justimpl TransactionProcessingCallback
forAccountLoader
but this has soundness concerns because it itself contains aTransactionProcessingCallback
type and merits further discussion