Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[factory] simplify interface for archive mode #4474

Merged
merged 3 commits into from
Nov 25, 2024
Merged

[factory] simplify interface for archive mode #4474

merged 3 commits into from
Nov 25, 2024

Conversation

dustinxie
Copy link
Member

@dustinxie dustinxie commented Nov 5, 2024

Description

make the Factory interface more natural to support archive mode

  1. functions needed to support archive-mode are now implemented by the new WorkingSetAtHeight() interface
  2. removed 2 duplicate methods and obsolete historyfactory.go

Fixes #(issue)

Type of change

Please delete options that are not relevant.

  • [] Bug fix (non-breaking change which fixes an issue)
  • [] New feature (non-breaking change which adds functionality)
  • [] Code refactor or improvement
  • [] Breaking change (fix or feature that would cause a new or changed behavior of existing functionality)
  • [] This change requires a documentation update

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

  • [] make test
  • [] fullsync
  • [] Other test (please specify)

Test Configuration:

  • Firmware version:
  • Hardware:
  • Toolchain:
  • SDK:

Checklist:

  • [] My code follows the style guidelines of this project
  • [] I have performed a self-review of my code
  • [] I have commented my code, particularly in hard-to-understand areas
  • [] I have made corresponding changes to the documentation
  • [] My changes generate no new warnings
  • [] I have added tests that prove my fix is effective or that my feature works
  • [] New and existing unit tests pass locally with my changes
  • [] Any dependent changes have been merged and published in downstream modules

@dustinxie dustinxie requested review from CoderZhi, Liuhaai, envestcc and a team as code owners November 5, 2024 01:05
@dustinxie dustinxie force-pushed the sfarchive branch 3 times, most recently from fbadbad to b365980 Compare November 5, 2024 05:39
ArchiveStateSimulator interface {
StateReader
SimulateExecution(context.Context, address.Address, action.Envelope, ...SimulateOption) ([]byte, *action.Receipt, error)
ReadContractStorage(context.Context, address.Address, []byte) ([]byte, error)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these 2 are needed for archive-node API

}
return state.NewIterator(keys, values)
}

Copy link
Member Author

@dustinxie dustinxie Nov 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

included in WorkingSetAtHeight funcs

Comment on lines 24 to 25
sf.mutex.RLock()
defer sf.mutex.RUnlock()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why does it share the same mutex with factory?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, I think reading on archive state can be lock-free

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated in latest commit

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SimulateExecution and ReadContractStorage also need re-implement, they are not currently based on the specified height, but rather on the latest height

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok just added, was thinking to add it later in API PR

timer := sf.timerFactory.NewTimer("Commit")
sf.mutex.Unlock()
Copy link
Member Author

@dustinxie dustinxie Nov 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when reviewing the usage of sf.mutex, found it's not really needed here

cfg, err := processOptions(opts...)
if err != nil {
return 0, err
}
if cfg.Keys != nil {
return 0, errors.Wrap(ErrNotSupported, "Read state with keys option has not been implemented yet")
}
if sf.height > sf.currentChainHeight {
return 0, errors.Errorf("query height %d is higher than tip height %d", sf.height, sf.currentChainHeight)
}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move the check to AtHeight()

return sf.height, sf.stateAtHeight(sf.height, cfg.Namespace, cfg.Key, s)
}

func (sf *factoryWithHeight) stateAtHeight(height uint64, ns string, key []byte, s interface{}) error {
if !sf.saveHistory {
return ErrNoArchiveData
}
Copy link
Member Author

@dustinxie dustinxie Nov 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the check is already there in AtHeight()

defer sf.mutex.RUnlock()
if sf.height > sf.currentChainHeight {
return 0, nil, errors.Errorf("query height %d is higher than tip height %d", sf.height, sf.currentChainHeight)
}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move the check to AtHeight()

@@ -87,8 +87,6 @@ type (
NewBlockBuilder(context.Context, actpool.ActPool, func(action.Envelope) (*action.SealedEnvelope, error)) (*block.Builder, error)
PutBlock(context.Context, *block.Block) error
DeleteTipBlock(context.Context, *block.Block) error
StateAtHeight(uint64, interface{}, ...protocol.StateOption) error
StatesAtHeight(uint64, ...protocol.StateOption) (state.Iterator, error)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these 2 are included in WorkingSetAtHeight funcs

@dustinxie dustinxie changed the title [facotry] add factoryWithHeight for archive mode [facotry] simplify interface for archive mode Nov 19, 2024
@dustinxie dustinxie changed the title [facotry] simplify interface for archive mode [factory] simplify interface for archive mode Nov 19, 2024
if err != nil {
return nil, 0, err
}
d, h, err := p.ReadState(ctx, historySR, methodName, arguments...)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can combine codes after getting the specified StateReader

rootKey = fmt.Sprintf("%s-%d", ArchiveTrieRootKey, height)
createTrie = false
}
store, err := newFactoryWorkingSetStore(sf.protocolView, flusher, rootKey, createTrie)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Splitting into the following two functions seems more clear:

  • newFactoryWorkingSetStore(view, flusher), which inner use ArchiveTrieRootKey and true
  • newFactoryWorkingSetStoreAt(view, flusher, height), which inner use ArchiveTrieRootKey-height and false

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does it should use ArchiveTrieRootKey or ArchiveTrieRootKey-height if height equals currentChainHeight?

Copy link
Member Author

@dustinxie dustinxie Nov 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 cases for height equals currentChainHeight

  1. createGenesisStates calls newWorkingSet(ctx, 0), which is write at current tip
  2. archive-mode trying to read at current tip height > 0, at the same time PutBlock may still be in progress so ArchiveTrieRootKey-height may not exist at the moment

both cases should use ArchiveTrieRootKey

@@ -333,7 +333,7 @@ func (ws *workingSet) Commit(ctx context.Context) error {
return err
}
ws.Reset()
return nil
return ws.store.Stop(ctx)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why add this?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

newWorkingSet() creates a factoryWorkingSetStore instance and calls Start() inside, but its Stop() is never called.

Copy link
Member Author

@dustinxie dustinxie Nov 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After checking, workingSetStore.Finalize() calls store.tlt.RootHash(), which calls tlt.flush(), which is the effective Stop(), we may need another PR to clean-up this

@@ -343,6 +343,9 @@ func (ws *workingSet) State(s interface{}, opts ...protocol.StateOption) (uint64
if err != nil {
return ws.height, err
}
if cfg.Keys != nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why add this?

Copy link
Member Author

@dustinxie dustinxie Nov 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's in the removed stateAtHeight() func, it's a safety check, there's a similar check in States() at L361

Copy link

sonarcloud bot commented Nov 22, 2024

@dustinxie dustinxie merged commit be618b8 into master Nov 25, 2024
3 checks passed
@dustinxie dustinxie deleted the sfarchive branch November 25, 2024 01:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants