Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: Specification update to clean up and improve documentation for proto message definitions #388

Merged
merged 4 commits into from
Dec 11, 2024

Conversation

jsync-swirlds
Copy link
Member

@jsync-swirlds jsync-swirlds commented Jul 3, 2024

Massive update to all services proto files to update and correct in-place
specification and documentation.

Note, the rendered markdown is removed from this PR to ease review.
The full rendered markdown documents may be found here.

  • Added svg diagrams in a branch for most state objects to assist in review.
  • Cleaned up copyright headers to remove non-printing characters.
  • Began reworking documents to produce clear documentation via the most
    common document processor for protobuf
    • There are limitations that encourage a slightly different style.
    • Markdown produces the best output (with slight modifications to the template),
      so that is selected.
  • Added a modified markdown template (to work around the "sanitization" bug in
    the docs processor for markdown).
    • Now we can add most HTML tags in the markdown, allowing things like line breaks
      in attribute descriptions.
    • This allows using a line with only <br/> or <p> in place of blank
      lines (that break the markdown tables).
  • Thorough update of many proto files in state.
  • Committed a fully generated set of docs in a separate branch for review
    purposes.
  • Moved document outputs to documents/api/services to allow for other document sets
  • Removed unused imports to quiet warnings.
  • Replicate "service" documentation to the message body entries for clarity
    • Message bodies were just "see ..." references, which do not link well,
      and push content specification to the service API, which muddies both.
  • Rewrite message body documentation to content specification
  • Rewrite "service" API documentation to API specification.
  • Adjust common fields to have consistent descriptions.
  • Updated copyright header to match services standard.
    • Corrected incorrect copy-paste of older years in some files.
    • Removed hidden zero-width characters.
    • Removed unnecessary additions.
  • Adjusted query header and query response header text to be more uniform.
  • Thorough update across services proto definitions
    • State messages complete
    • Consensus service complete
    • Smart Contract service complete.
    • Crypto Service complete.
    • Miscellaneous protos complete.
    • File Service complete.
    • Freeze Service complete.
    • Transaction and records complete.
    • Synthetic node_stake_update transaction complete.
    • Query and Response complete.
    • General Queries complete.
      • Harmonized crypto_get_info and get_account_details
      • Discovered no service interface defines getByKey, we may be able to
        completely remove that query, with it's handler in network admin service,
        and just reserve the field number in query and hedera functionality henceforth.
      • Found a few more unsupported queries and marked them deprecated in the
        query.proto one-of block.
    • Network Service complete
    • Schedule Service complete
    • Token Service complete
      • Fixed the new token_reject file, which was not created according to
        the content in the HIP.
    • Utility Service complete.
    • Address Book Service complete.
      • This was a separate PR.
    • Response Code
      • Separate due to size (over 360 fields) and frequent changes.
        Will be addressed in a separate PR and HIP to minimize conflict.

@jsync-swirlds jsync-swirlds self-assigned this Jul 3, 2024
@jsync-swirlds jsync-swirlds changed the title docs: Documentation Update to clean up and improve documentation for proto message definitions docs: Specification update to clean up and improve documentation for proto message definitions Jul 11, 2024
@jsync-swirlds jsync-swirlds force-pushed the pbj-storage-spec-review branch 6 times, most recently from fb4bc88 to df235a5 Compare July 19, 2024 00:51
@jsync-swirlds jsync-swirlds force-pushed the pbj-storage-spec-review branch 4 times, most recently from 658ad21 to e4dfc42 Compare July 24, 2024 23:38
@jsync-swirlds jsync-swirlds force-pushed the pbj-storage-spec-review branch 3 times, most recently from 85a8dc1 to 99947b7 Compare August 1, 2024 18:02
@jsync-swirlds jsync-swirlds force-pushed the pbj-storage-spec-review branch 2 times, most recently from 030315e to 3c26875 Compare August 9, 2024 19:33
Copy link
Member Author

@jsync-swirlds jsync-swirlds left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.

@jsync-swirlds jsync-swirlds force-pushed the pbj-storage-spec-review branch 7 times, most recently from 22a1201 to 5ffdb1c Compare August 14, 2024 23:58
@jsync-swirlds jsync-swirlds force-pushed the pbj-storage-spec-review branch 2 times, most recently from 8abac61 to 243d22f Compare August 22, 2024 21:21
…message definitions

* Added svg diagrams for most state objects.
* Clean up copyright headers to remove non-printing characters.
* Began reworking documents to produce clear documentation via the most common document processor for protobuf
   * There are limitations that encourage a slightly different style.
   * Markdown produces the best output (with slight modifications to the template), so that is selected.
* Added a modified markdown template (to work around the "sanitization" bug in the docs processor for markdown).
   * Now we can add most HTML tags in the markdown, allowing things like line breaks in attribute descriptions.
   * The results are much better, might be something worth submitting as a PR upstream (along with some non-trivial examples)...
   * This allows using a line with only `<br/>` or `<p>` in place of blank lines (that break the markdown tables).
   * Ideally the go code would do this, but that's a bigger lift, this works for now.
* Thorough update of many proto files in state.
* Committed a fully generated set of docs in this branch for review purposes, will remove before completing the task.
* Moved document outputs to `documents/api/services` to allow for other document sets
* Removed unused imports to quiet warnings.
* Replicate "service" documentation to the message body entries for clarity
   * Message bodies were just "see ..." references, which do not link well, and push content specification to the service API, which muddies both.
* Rewrite message body documentation to *content* specification
* Rewrite "service" API documentation to *API* specification.
* Adjust common fields to have consistent descriptions.
Mass update of the copyright header.
* Updated copyright header to match services standard.
   * Corrected incorrect copy-paste of older years in state.
   * Removed hidden zero-width characters.
   * Removed unnecessary additions.
* Adjusted query header and query response header text to be more uniform.
* Thorough update across services proto definitions
   * State messages complete
   * Consensus service complete
   * Smart Contract service complete.
   * Crypto Service complete.
   * Miscellaneous protos complete.
   * File Service complete.
   * Freeze Service complete.
   * Transaction and records complete.
   * Synthetic node_stake_update transaction complete.
   * Query and Response complete.
   * General Queries complete.
      * Harmonized crypto_get_info and get_account_details
      * Discovered no service interface defines `getByKey`, we may
        be able to completely remove that query, with it's handler
        in network admin service, and just reserve the field number
        in query and hedera functionality henceforth.
      * Found a few more unsupported queries and marked them deprecated in
        the query.proto one-of block.
   * Network Service complete
   * Schedule Service complete
   * Token Service complete
      * Fixed the new `token_reject` file, which was not created according to
        the content in the HIP.
   * Utility Service complete.
   * Address Book Service complete.
      * Minor updates and clarification only.
   * HIP 904 additions
      * Updated all airdrop and pending airdrop entries.
   * Response Code
      * Separate due to size (iover 360 fields) and frequent
        changes.  Will be addressed in a separate PR to minimize
        conflict with ongoing services work.

Signed-off-by: Joseph Sinclair <[email protected]>
* Documentation is moved to the hip-1037-documentation branch

Signed-off-by: Joseph Sinclair <[email protected]>
 * Fixed incorrect reference in block header.
 * Improvements to style guidelines file.
 * Small review fixes.

Signed-off-by: Joseph Sinclair <[email protected]>
@jsync-swirlds jsync-swirlds force-pushed the pbj-storage-spec-review branch from 763c39a to b0f1ea6 Compare December 10, 2024 17:37
 * Many suggestions slightly rewritten to better match style
   guidelines.
 * Harmonized use of the `reserved` keyword.
 * Added several clarifications to style guidelines.
 * Changed wording for `memo` fields to refer to the system property that
   governs the length limit
 * Fixed field ordering in transaction body to work around PBJ limitation.
 * Adjusted wording and corrected misunderstandings in both
   File and network admin services.
 * Fixed inconsistend description for ledger ID.
 * Clarified what is a "system" file and the requirement that
   "system" files cannot be deleted.
 * Deprecated several obsolete and unsupported queries and
   transactions.
 * Minor changes to accomodate JavaDoc's broken HTML processing.
    * Apparently, services insists on generating javadoc for generated code.
 * Added purpose description for `SignaturePair.pubKeyPrefix` based
   on a question raised recently, and answers given.

Signed-off-by: Joseph Sinclair <[email protected]>
@jsync-swirlds jsync-swirlds force-pushed the pbj-storage-spec-review branch from b0f1ea6 to db889af Compare December 10, 2024 17:43
@Nana-EC Nana-EC requested a review from lukelee-sl December 10, 2024 18:36
Copy link
Member

@Neeharika-Sompalli Neeharika-Sompalli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks @jsync-swirlds

@jsync-swirlds jsync-swirlds merged commit 759eaea into main Dec 11, 2024
1 check passed
@jsync-swirlds jsync-swirlds deleted the pbj-storage-spec-review branch December 11, 2024 16:12
jsync-swirlds added a commit to hashgraph/hedera-services that referenced this pull request Dec 11, 2024
 * Merged all changes from [PR 388](hashgraph/hedera-protobufs#388) in `hedera-protobufs`

Signed-off-by: Joseph Sinclair <[email protected]>
jsync-swirlds added a commit to hashgraph/hedera-services that referenced this pull request Dec 11, 2024
 * Merged all changes from [PR 388](hashgraph/hedera-protobufs#388) in `hedera-protobufs`

Signed-off-by: Joseph Sinclair <[email protected]>
jsync-swirlds added a commit to hashgraph/hedera-services that referenced this pull request Dec 11, 2024
 * Merged all changes from [PR 388](hashgraph/hedera-protobufs#388) in `hedera-protobufs`

Signed-off-by: Joseph Sinclair <[email protected]>
jsync-swirlds added a commit to hashgraph/hedera-services that referenced this pull request Dec 11, 2024
 * Merged all changes from [PR 388](hashgraph/hedera-protobufs#388) in `hedera-protobufs`

Signed-off-by: Joseph Sinclair <[email protected]>
jsync-swirlds added a commit to hashgraph/hedera-services that referenced this pull request Dec 11, 2024
 * Merged all changes from [PR 388](hashgraph/hedera-protobufs#388) in `hedera-protobufs`

Signed-off-by: Joseph Sinclair <[email protected]>
jsync-swirlds added a commit to hashgraph/hedera-services that referenced this pull request Dec 11, 2024
 * Merged all changes from [PR 388](hashgraph/hedera-protobufs#388) in `hedera-protobufs`

Signed-off-by: Joseph Sinclair <[email protected]>
jsync-swirlds added a commit to hashgraph/hedera-services that referenced this pull request Dec 11, 2024
 * Merged all changes from [PR 388](hashgraph/hedera-protobufs#388) in `hedera-protobufs`

Signed-off-by: Joseph Sinclair <[email protected]>
jsync-swirlds added a commit to hashgraph/hedera-services that referenced this pull request Dec 11, 2024
 * Merged all changes from [PR 388](hashgraph/hedera-protobufs#388) in `hedera-protobufs`

Signed-off-by: Joseph Sinclair <[email protected]>
jsync-swirlds added a commit to hashgraph/hedera-services that referenced this pull request Dec 11, 2024
 * Merged all changes from [PR 388](hashgraph/hedera-protobufs#388) in `hedera-protobufs`
 * Removed references in test clients to obsolete removed functionality
 * Removed three unused imports.

Signed-off-by: Joseph Sinclair <[email protected]>
jsync-swirlds added a commit to hashgraph/hedera-services that referenced this pull request Dec 12, 2024
 * Merged all changes from [PR 388](hashgraph/hedera-protobufs#388) in `hedera-protobufs`
 * Removed references in test clients to obsolete removed functionality
 * Removed three unused imports.
 * Removed HAPI tests for unsupported queries that are removed.

Signed-off-by: Joseph Sinclair <[email protected]>
jsync-swirlds added a commit to hashgraph/hedera-services that referenced this pull request Dec 12, 2024
 * Merged all changes from [PR 388](hashgraph/hedera-protobufs#388) in `hedera-protobufs`
 * Removed references in test clients to obsolete removed functionality
 * Removed three unused imports.
 * Removed HAPI tests for unsupported queries that are removed.

Signed-off-by: Joseph Sinclair <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.