Skip to content

Commit

Permalink
Documentation Update to clean up and improve documentation for proto …
Browse files Browse the repository at this point in the history
…message definitions

* Added svg diagrams for most state objects.
* Clean up copyright headers to remove non-printing characters.
* Began reworking documents to produce clear documentation via the most common document processor for protobuf
   * There are limitations that encourage a slightly different style.
   * Markdown produces the best output (with slight modifications to the template), so that is selected.
* Added a modified markdown template (to work around the "sanitization" bug in the docs processor for markdown).
   * Now we can add most HTML tags in the markdown, allowing things like line breaks in attribute descriptions.
   * The results are much better, might be something worth submitting as a PR upstream (along with some non-trivial examples)...
   * This allows using a line with only `<br/>` or `<p>` in place of blank lines (that break the markdown tables).
   * Ideally the go code would do this, but that's a bigger lift, this works for now.
* Thorough update of many proto files in state.
* Committed a fully generated set of docs in this branch for review purposes, will remove before completing the task.
* Moved document outputs to `documents/api/services` to allow for other document sets
* Removed unused imports to quiet warnings.
* Replicate "service" documentation to the message body entries for clarity
   * Message bodies were just "see ..." references, which do not link well, and push content specification to the service API, which muddies both.
* Rewrite message body documentation to *content* specification
* Rewrite "service" API documentation to *API* specification.
* Adjust common fields to have consistent descriptions.
Mass update of the copyright header.
* Updated copyright header to match services standard.
   * Corrected incorrect copy-paste of older years in state.
   * Removed hidden zero-width characters.
   * Removed unnecessary additions.
* Adjusted query header and query response header text to be more uniform.
* Thorough update across services proto definitions
   * State messages complete
   * Consensus service complete
   * Smart Contract service complete.
   * Crypto Service complete.
   * Miscellaneous protos complete.
   * File Service complete.
   * Freeze Service complete.
   * Transaction and records complete.
   * Synthetic node_stake_update transaction complete.
   * Query and Response complete.
   * General Queries complete.
      * Harmonized crypto_get_info and get_account_details
      * Discovered no service interface defines `getByKey`, we may
        be able to completely remove that query, with it's handler
        in network admin service, and just reserve the field number
        in query and hedera functionality henceforth.
      * Found a few more unsupported queries and marked them deprecated in
        the query.proto one-of block.
   * Network Service complete
   * Schedule Service complete
   * Token Service complete
      * Fixed the new `token_reject` file, which was not created according to
        the content in the HIP.
   * Utility Service complete.
   * Address Book Service complete.
      * Minor updates and clarification only.
   * HIP 904 additions
      * Updated all airdrop and pending airdrop entries.
   * Response Code
      * Separate due to size (iover 360 fields) and frequent
        changes.  Will be addressed in a separate PR to minimize
        conflict with ongoing services work.

Signed-off-by: Joseph Sinclair <[email protected]>
  • Loading branch information
jsync-swirlds committed Aug 22, 2024
1 parent 0047255 commit 243d22f
Show file tree
Hide file tree
Showing 286 changed files with 40,300 additions and 5,064 deletions.
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,7 @@ secring.*

### Custom items
.idea
**/*.iml
*.sw*
*.iml

14 changes: 6 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,10 @@ from the _*.proto_ files in the following repositories:
## Overview of services

There are five primary service families, which inter-operate on entities
controlled by one (or more) Ed25519 keypairs:
controlled by one (or more) Ed25519 or ECDSA(secp256k1) keypairs:
1. The [cryptocurrency service](services/crypto_service.proto),
for cryptocurrency accounts with transfers denominated
in [hBar (ℏ)](https://help.hedera.com/hc/en-us/articles/360000674317-What-are-the-official-HBAR-cryptocurrency-denominations-).
in [hbar (ℏ)](https://help.hedera.com/hc/en-us/articles/360000674317-What-are-the-official-HBAR-cryptocurrency-denominations-).
2. The [consensus service](services/consensus_service.proto), for
fast and unbiased ordering of opaque binary messages exchanged on
arbitrary topics.
Expand All @@ -38,7 +38,7 @@ privileged accounts to suspend network operations during a maintenance window.
It is important to note that most network services are gated by fees which
must be paid in ℏ from a cryptocurrency account. The payer authorizes a
fee by signing an appropriate transaction with a sufficient subset of the
Ed25519 key(s) associated to their account.
cryptographic key(s) associated to their account.

## Overview of state
State directory and its subdirectories contain the protobuf files that define the state of the network.
Expand All @@ -51,11 +51,9 @@ The state directory and its subdirectories are in preview and are subject to ch
# For Developers

## Branching
This repository uses a simple branching model with only two distinguished branches:
1. The head of `main` points to the latest protobufs blessed for deployment to mainnet.
This repository uses a simple branching model with only one distinguished branch;
`main` points to the latest protobufs intended for the next deployment to mainnet.
Tags such as `v0.12.0` mark commits used for testnet and mainnet deploys.
2. The head of `develop` points to the latest candidate for the next tag in `main`.
Tags suffixed with `-alpha.x` mark commits used for previewnet deploys.

## PBJ Package Support
Proto files in this repo are generated into two kinds of Java files, one using **protoc** and other using
Expand All @@ -65,4 +63,4 @@ need to have an option of the form:
// <<<pbj.java_package = "com.hedera.hapi.node.token">>> This comment is special code for setting PBJ Compiler java package
```
To specify the Java package for PBJ generated model objects. This option is ignored by **protoc**. PBJ ignores the default
option `option java_package = "com.hederahashgraph.api.proto.java";` which is used by **protoc** to generate Java files.
option `option java_package = "com.hederahashgraph.api.proto.java";` which is used by **protoc** to generate Java files.
35 changes: 27 additions & 8 deletions block/block_service.proto
Original file line number Diff line number Diff line change
Expand Up @@ -19,17 +19,19 @@
* >> due to issue 262
* >
* > Issue 240
* >> These files currently cause PBJ integration tests to fail if included
* >> due to issue 240.
* >> These files may cause PBJ integration tests to fail if included
* >> due to issue 240. This was worked around by specifying different
* >> values for java_package and pbj.java_package.
* >
* > Issue 218
* >> These files have the same value for package and java_package. Ideally
* >> we would not specify `java_package` or the pbj comment in that situation,
* >> but Issue 218 prevents eliding the unnecessary directives.
* >> These files _should_ have the same value for package and java_package.
* >> Ideally we would not specify `java_package` or the pbj comment in that
* >> situation, but Issue 218 prevents eliding the unnecessary directives.
* >
* > Issue 217
* >> These files may cause PBJ to fail compilation due to comments preceeding
* >> the `syntax` keyword.
* >> the `syntax` keyword. Currently this is a warning, but in projects that
* >> make warnings errors, this will cause compilation to fail.
* >
* > Issue 216
* >> These files would do well with validation support, but cannot make
Expand Down Expand Up @@ -790,24 +792,35 @@ message BlockNodeVersions {
}

/**
* Remote procedure calls (RPCs) for the Block Node.
* Remote procedure calls (RPCs) for the Block Node ancillary services.
*/
service BlockStreamService {
service BlockNodeService {
/**
* Read the status of this block node server.
* <p>
* A client SHOULD request server status prior to requesting block stream
* data or a state snapshot.
*/
rpc serverStatus(ServerStatusRequest) returns (ServerStatusResponse);
}

/**
* Remote procedure calls (RPCs) for the Block Node block services.
*/
service BlockAccessService {
/**
* Read a single block from the block node.
* <p>
* The request SHALL describe the block number of the block to retrieve.
*/
rpc singleBlock(SingleBlockRequest) returns (SingleBlockResponse);
}

/**
* Remote procedure calls (RPCs) for the Block Node state snapshot
* and query services.
*/
service StateService {
/**
* Read a state snapshot from the block node.
* <p>
Expand All @@ -818,6 +831,12 @@ service BlockStreamService {
*/
rpc stateSnapshot(StateSnapshotRequest) returns (StateSnapshotResponse);

}

/**
* Remote procedure calls (RPCs) for the Block Node stream services.
*/
service BlockStreamService {
/**
* Publish a stream of blocks.
* <p>
Expand Down
6 changes: 5 additions & 1 deletion block/stream/output/consensus_service.proto
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,11 @@
* ### Topic Running Hash Calculation
* Submitted messages include a topic running hash. This value has changed
* over time, with the known versions detailed in the `RunningHashVersion`
* enumeration.
* enumeration. The running hash version SHALL NOT be transmitted in the
* Block Stream, however, as it is a fixed value that can only be changed
* with a new release of consensus node software following a HIP to update
* the specification, so repeating that fixed value in the block stream is
* wasteful and unnecessary.
*
* ### Keywords
* The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
Expand Down
2 changes: 1 addition & 1 deletion block/stream/output/smart_contract_service.proto
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ option java_package = "com.hedera.hapi.block.stream.output.protoc";
// <<<pbj.java_package = "com.hedera.hapi.block.stream.output">>> This comment is special code for setting PBJ Compiler java package
option java_multiple_files = true;

import "contract_call_local.proto";
import "contract_types.proto";
import "sidecar_file.proto";

/**
Expand Down
1 change: 1 addition & 0 deletions block/stream/output/state_changes.proto
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@ import "state/token/account.proto";
import "state/token/account_pending_airdrop.proto";
import "state/token/network_staking_rewards.proto";
import "state/token/nft.proto";
import "state/platform_state.proto";
import "state/token/staking_node_info.proto";
import "state/token/token.proto";
import "state/token/token_relation.proto";
Expand Down
92 changes: 92 additions & 0 deletions buf/validate/expression.proto
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
// Copyright 2023 Buf Technologies, Inc.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

syntax = "proto3";

package buf.validate;

option go_package = "buf.build/gen/go/bufbuild/protovalidate/protocolbuffers/go/buf/validate";
option java_multiple_files = true;
option java_outer_classname = "ExpressionProto";
option java_package = "build.buf.validate";

// `Constraint` represents a validation rule written in the Common Expression
// Language (CEL) syntax. Each Constraint includes a unique identifier, an
// optional error message, and the CEL expression to evaluate. For more
// information on CEL, [see our documentation](https://github.com/bufbuild/protovalidate/blob/main/docs/cel.md).
//
// ```proto
// message Foo {
// option (buf.validate.message).cel = {
// id: "foo.bar"
// message: "bar must be greater than 0"
// expression: "this.bar > 0"
// };
// int32 bar = 1;
// }
// ```
message Constraint {
// `id` is a string that serves as a machine-readable name for this Constraint.
// It should be unique within its scope, which could be either a message or a field.
string id = 1;

// `message` is an optional field that provides a human-readable error message
// for this Constraint when the CEL expression evaluates to false. If a
// non-empty message is provided, any strings resulting from the CEL
// expression evaluation are ignored.
string message = 2;

// `expression` is the actual CEL expression that will be evaluated for
// validation. This string must resolve to either a boolean or a string
// value. If the expression evaluates to false or a non-empty string, the
// validation is considered failed, and the message is rejected.
string expression = 3;
}

// `Violations` is a collection of `Violation` messages. This message type is returned by
// protovalidate when a proto message fails to meet the requirements set by the `Constraint` validation rules.
// Each individual violation is represented by a `Violation` message.
message Violations {
// `violations` is a repeated field that contains all the `Violation` messages corresponding to the violations detected.
repeated Violation violations = 1;
}

// `Violation` represents a single instance where a validation rule, expressed
// as a `Constraint`, was not met. It provides information about the field that
// caused the violation, the specific constraint that wasn't fulfilled, and a
// human-readable error message.
//
// ```json
// {
// "fieldPath": "bar",
// "constraintId": "foo.bar",
// "message": "bar must be greater than 0"
// }
// ```
message Violation {
// `field_path` is a machine-readable identifier that points to the specific field that failed the validation.
// This could be a nested field, in which case the path will include all the parent fields leading to the actual field that caused the violation.
string field_path = 1;

// `constraint_id` is the unique identifier of the `Constraint` that was not fulfilled.
// This is the same `id` that was specified in the `Constraint` message, allowing easy tracing of which rule was violated.
string constraint_id = 2;

// `message` is a human-readable error message that describes the nature of the violation.
// This can be the default error message from the violated `Constraint`, or it can be a custom message that gives more context about the violation.
string message = 3;

// `for_key` indicates whether the violation was caused by a map key, rather than a value.
bool for_key = 4;
}
41 changes: 41 additions & 0 deletions buf/validate/priv/private.proto
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
// Copyright 2023 Buf Technologies, Inc.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

syntax = "proto3";

package buf.validate.priv;

import "google/protobuf/descriptor.proto";

option go_package = "buf.build/gen/go/bufbuild/protovalidate/protocolbuffers/go/buf/validate/priv";
option java_multiple_files = true;
option java_outer_classname = "PrivateProto";
option java_package = "build.buf.validate.priv";

extend google.protobuf.FieldOptions {
// Do not use. Internal to protovalidate library
optional FieldConstraints field = 1160;
}

// Do not use. Internal to protovalidate library
message FieldConstraints {
repeated Constraint cel = 1;
}

// Do not use. Internal to protovalidate library
message Constraint {
string id = 1;
string message = 2;
string expression = 3;
}
Loading

0 comments on commit 243d22f

Please sign in to comment.