Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace gogo protobuf with google's protobuf v2 compiler #119

Merged
merged 61 commits into from
Nov 21, 2023
Merged
Show file tree
Hide file tree
Changes from 22 commits
Commits
Show all changes
61 commits
Select commit Hold shift + click to select a range
e344d3e
Point API at my fork of temporalio/api
tdeebswihart Sep 27, 2023
a372766
Remove proxygenerator's reliance on gogo/protobuf
tdeebswihart Sep 27, 2023
2bef229
Update serviceerror package and regenerate Go code
tdeebswihart Sep 27, 2023
e660f0b
Delete the proxy for now
tdeebswihart Sep 27, 2023
205b263
Testing makefile changes
tdeebswihart Sep 28, 2023
ce9a5b2
Use the new buf-generated API code
tdeebswihart Sep 28, 2023
69c53d5
Keep just enough of the proxy to please the sdk
tdeebswihart Sep 29, 2023
13f2c44
Fix the awful enum naming
tdeebswihart Sep 29, 2023
716a83c
Update interceptor to not copy locks
tdeebswihart Sep 29, 2023
f9fb8c6
Fix mock generation
tdeebswihart Oct 2, 2023
6fdbb4c
Port history loading over from the SDK
tdeebswihart Oct 3, 2023
389ae08
Document LoadFromJSON
tdeebswihart Oct 3, 2023
65454a6
Port remaining tests from sdk-go
tdeebswihart Oct 3, 2023
7efc384
Remove options pattern from history.LoadFromJSON
tdeebswihart Oct 3, 2023
6cda8db
Add proto-conversion helpers for Time and Durations
tdeebswihart Oct 3, 2023
de61875
Improve enum rewriting
tdeebswihart Oct 3, 2023
7c4c150
I missed a few references to history.LoadOption, whoops
tdeebswihart Oct 3, 2023
72ba345
Point at the appropriate api branch
tdeebswihart Oct 3, 2023
b7493bd
Merge remote-tracking branch 'upstream/master' into nomo-gogo
tdeebswihart Oct 3, 2023
cbe2bb4
Fix history load tests and actually run them
tdeebswihart Oct 3, 2023
d410a2a
Bump versions and remove extra makefile print
tdeebswihart Oct 3, 2023
428211c
Add Ptr() methods to duration and timestamp packages
tdeebswihart Oct 5, 2023
dd417ed
Move codegen helpers to this repo and update
tdeebswihart Oct 9, 2023
b5b4a15
Correctly install proto plugins
tdeebswihart Oct 9, 2023
1ca3f12
Correctly translate JSON enums for types which contain themselves
tdeebswihart Oct 9, 2023
62b7da4
Remove type helpers
tdeebswihart Oct 9, 2023
ec12a7f
Remove dependencies on cmp and protocmp
tdeebswihart Oct 9, 2023
7515ef0
Add new jsonpb package
tdeebswihart Oct 10, 2023
17191bd
Cleanup
tdeebswihart Oct 10, 2023
16ae34d
Rename jsonpb to temporalproto
tdeebswihart Oct 10, 2023
910c300
Add JSON prefix to json-specific types
tdeebswihart Oct 10, 2023
837039d
Bump
tdeebswihart Oct 10, 2023
aae8509
May go 1.16 rest in peace
tdeebswihart Oct 10, 2023
1d57552
Generate functions to convert enum values from string
tdeebswihart Oct 10, 2023
8435832
Lower-case my error string
tdeebswihart Oct 10, 2023
47d4e43
Improve errors slightly
tdeebswihart Oct 11, 2023
4c04e0d
Add functionality to go from enum to shorthand string
tdeebswihart Oct 11, 2023
6e958a1
I don't know why I made a function for this. Delete
tdeebswihart Oct 11, 2023
2c522e4
Remove shorthandName generation. I'm hard-coding the one server needs
tdeebswihart Oct 11, 2023
229d40f
Rewrite proto enum String methods
tdeebswihart Oct 12, 2023
0909eae
Rename the enum rewriter
tdeebswihart Oct 12, 2023
6469411
Move commands into cmd/
tdeebswihart Oct 16, 2023
3a4d6e3
Port reflection helpers over from the server
tdeebswihart Oct 18, 2023
ef5a505
Remove debugging
tdeebswihart Oct 18, 2023
966fdd4
Fix (and test) a bug involving nils
tdeebswihart Oct 18, 2023
a2111ee
Merge reflect and proto extensions
tdeebswihart Oct 18, 2023
a9bd390
Bump api submodule
tdeebswihart Oct 20, 2023
f1f1432
Merge remote-tracking branch 'upstream/master' into nomo-gogo
tdeebswihart Oct 20, 2023
a4867f8
Merge remote-tracking branch 'upstream/master' into nomo-gogo
tdeebswihart Oct 24, 2023
447c31e
Rebuild
tdeebswihart Oct 27, 2023
bfe668b
Merge remote-tracking branch 'upstream/master' into nomo-gogo
tdeebswihart Oct 27, 2023
098edea
Go mod tidy
tdeebswihart Oct 27, 2023
872ab2d
Address PR comments
tdeebswihart Oct 30, 2023
ba122e5
Remove temporalproto.PublicMethods
tdeebswihart Oct 30, 2023
c5f08f0
Address even more review comments
tdeebswihart Oct 30, 2023
e3f2b0b
Add public method rgx back in
tdeebswihart Oct 30, 2023
9088f4d
Update proto structs
tdeebswihart Nov 19, 2023
59e4eda
Inline strcase dependency
MichaelSnowden Nov 21, 2023
2507a24
Merge remote-tracking branch 'upstream/master' into nomo-gogo
tdeebswihart Nov 21, 2023
f8d3c66
Point everything at the official API repo
tdeebswihart Nov 21, 2023
080d999
Merge remote-tracking branch 'upstream/master' into nomo-gogo
tdeebswihart Nov 21, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .gitmodules
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
[submodule "proto/api"]
path = proto/api
url = https://github.com/temporalio/api
url = https://github.com/tdeebswihart/temporal-api
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Goes without saying I'm sure, but reminder to replace this when temporalio/api#317 merged

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💯

branch = nomo-gogo
63 changes: 27 additions & 36 deletions Makefile
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not required here, but I guess we never had a CI step that did a rebuild + diff/status to confirm generated source is what remains in the repo (e.g. someone doesn't make a script alteration without regen). Also gives CI a chance to confirm that nobody messes up the build process. We can open an issue if we don't want to tackle now.

Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
$(VERBOSE).SILENT:
all: install test
############################# Main targets #############################
# Install everything, update submodule, and compile proto files.
install: grpc-install mockgen-install goimports-install update-proto
install: mockgen-install goimports-install update-proto

# Compile proto files.
proto: grpc goimports proxy grpc-mock copyright
Expand All @@ -22,15 +23,9 @@ COLOR := "\e[1;36m%s\e[0m\n"

PINNED_DEPENDENCIES := \


PROTO_ROOT := proto/api
PROTO_FILES = $(shell find $(PROTO_ROOT) -name "*.proto")
PROTO_DIRS = $(sort $(dir $(PROTO_FILES)))
PROTO_ENUMS := $(shell grep -R '^enum ' $(PROTO_ROOT) | cut -d ' ' -f2)
PROTO_OUT := .
PROTO_IMPORTS = \
-I=$(PROTO_ROOT) \
-I=$(shell go list -modfile build/go.mod -m -f '{{.Dir}}' github.com/temporalio/gogo-protobuf)/protobuf \
-I=$(shell go list -m -f '{{.Dir}}' github.com/grpc-ecosystem/grpc-gateway)/third_party/googleapis

$(PROTO_OUT):
mkdir $(PROTO_OUT)
Expand All @@ -40,24 +35,32 @@ update-proto-submodule:
printf $(COLOR) "Update proto-submodule..."
git submodule update --init --force --remote $(PROTO_ROOT)


##### Compile proto files for go #####
grpc: gogo-grpc fix-path
grpc: go-grpc fix-path fix-enums copy-helpers

gogo-grpc: clean $(PROTO_OUT)
printf $(COLOR) "Compiling for gogo-gRPC..."
$(foreach PROTO_DIR,$(PROTO_DIRS),\
protoc --fatal_warnings $(PROTO_IMPORTS) \
--gogoslick_out=Mgoogle/protobuf/any.proto=github.com/gogo/protobuf/types,Mgoogle/protobuf/wrappers.proto=github.com/gogo/protobuf/types,Mgoogle/protobuf/duration.proto=github.com/gogo/protobuf/types,Mgoogle/protobuf/descriptor.proto=github.com/gogo/protobuf/protoc-gen-gogo/descriptor,Mgoogle/protobuf/timestamp.proto=github.com/gogo/protobuf/types,plugins=grpc,paths=source_relative:$(PROTO_OUT) \
--grpc-gateway_out=allow_patch_feature=false,paths=source_relative:$(PROTO_OUT) \
$(PROTO_DIR)*.proto;)
go-grpc: clean $(PROTO_OUT)
# Relative to PROTO_ROOT
PROTO_OUT=../.. make -C $(PROTO_ROOT) go-grpc

fix-path:
fix-path: go-grpc
mv -f $(PROTO_OUT)/temporal/api/* $(PROTO_OUT) && rm -rf $(PROTO_OUT)/temporal
# Also copy the payload JSON helper

copy-helpers:
# Also copy the payload and history JSON helpers
cp $(PROTO_OUT)/internal/temporalcommonv1/payload_json.go $(PROTO_OUT)/common/v1/
cp $(PROTO_OUT)/internal/temporalhistoryv1/load*.go $(PROTO_OUT)/history/v1/

# The generated enums are go are just plain terrible, so we fix them
# by removing the typename prefixes. We already made good choices with our enum
# names, so this shouldn't be an issue
fix-enums: fix-path
printf $(COLOR) "Fixing enum naming..."
$(foreach PROTO_ENUM,$(PROTO_ENUMS),\
$(shell grep -Rl "$(PROTO_ENUM)" $(PROTO_OUT) | grep -E "\.go" | xargs -P 8 sed -i "" -e "s/$(PROTO_ENUM)_\(.*\) $(PROTO_ENUM)/\1 $(PROTO_ENUM)/g;s/\.$(PROTO_ENUM)_\(.*\)/.\1/g"))

# All generated service files pathes relative to PROTO_OUT.
PROTO_GRPC_SERVICES = $(patsubst $(PROTO_OUT)/%,%,$(shell find $(PROTO_OUT) -name "service.pb.go"))
PROTO_GRPC_SERVICES = $(patsubst $(PROTO_OUT)/%,%,$(shell find $(PROTO_OUT) -name "service_grpc.pb.go"))
service_name = $(firstword $(subst /, ,$(1)))
mock_file_name = $(call service_name,$(1))mock/$(subst $(call service_name,$(1))/,,$(1:go=mock.go))

Expand All @@ -75,21 +78,9 @@ goimports:
goimports -w $(PROTO_OUT)

##### Plugins & tools #####
grpc-install: gogo-protobuf-install
printf $(COLOR) "Install/update gRPC plugins..."
go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest

gogo-protobuf-install: go-protobuf-install
go install github.com/temporalio/gogo-protobuf/protoc-gen-gogoslick@latest
go install -modfile build/go.mod github.com/temporalio/gogo-protobuf/protoc-gen-gogoslick
go install github.com/grpc-ecosystem/grpc-gateway/protoc-gen-grpc-gateway@latest

go-protobuf-install:
go install github.com/golang/protobuf/protoc-gen-go@latest

mockgen-install:
printf $(COLOR) "Install/update mockgen..."
go install github.com/golang/mock/mockgen@v1.6.0
go install -modfile=build/go.mod github.com/golang/mock/mockgen

goimports-install:
printf $(COLOR) "Install/update goimports..."
Expand All @@ -111,19 +102,19 @@ gomodtidy:

##### Test #####

test:
go test ./proxy ./serviceerror
test: copy-helpers
go test ./serviceerror/... ./proxy/... ./history/...

##### Check #####

generatorcheck:
printf $(COLOR) "Check generated code is not stale..."
(cd ./cmd/proxygenerator && go mod tidy && go run ./ -verifyOnly)
#(cd ./cmd/proxygenerator && go mod tidy && go run ./ -verifyOnly)

check: generatorcheck

##### Clean #####
clean:
printf $(COLOR) "Deleting generated go files..."
# Delete all directories with *.pb.go and *.mock.go files from $(PROTO_OUT)
find $(PROTO_OUT) \( -name "*.pb.go" -o -name "*.mock.go" \) | xargs -I{} dirname {} | sort -u | xargs rm -rf
find . \( -name "*.pb.go" -o -name "*.mock.go" -o -name "*.go-helpers.go" \) | xargs -I{} dirname {} | sort -u | xargs rm -rf
224 changes: 224 additions & 0 deletions batch/v1/message.go-helpers.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,224 @@
// The MIT License
//
// Copyright (c) 2022 Temporal Technologies Inc. All rights reserved.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
// THE SOFTWARE.

package batch

import "google.golang.org/protobuf/proto"

func (val *BatchOperationInfo) Marshal() ([]byte, error) {
tdeebswihart marked this conversation as resolved.
Show resolved Hide resolved
return proto.Marshal(val)
}

func (val *BatchOperationInfo) Unmarshal(buf []byte) error {
return proto.Unmarshal(buf, val)
}

func (val *BatchOperationInfo) Size() int {
return proto.Size(val)
}

// Equal returns whether two BatchOperationInfo values are equivalent by recursively
// comparing the message's fields.
// For more information see the documentation for
// https://pkg.go.dev/google.golang.org/protobuf/proto#Equal
func (this *BatchOperationInfo) Equal(that interface{}) bool {
if that == nil {
return this == nil
}

var that1 *BatchOperationInfo
switch t := that.(type) {
case *BatchOperationInfo:
that1 = t
case BatchOperationInfo:
that1 = &t
default:
return false
}

return proto.Equal(this, that1)
}
func (val *BatchOperationTermination) Marshal() ([]byte, error) {
return proto.Marshal(val)
}

func (val *BatchOperationTermination) Unmarshal(buf []byte) error {
return proto.Unmarshal(buf, val)
}

func (val *BatchOperationTermination) Size() int {
return proto.Size(val)
}

// Equal returns whether two BatchOperationTermination values are equivalent by recursively
// comparing the message's fields.
// For more information see the documentation for
// https://pkg.go.dev/google.golang.org/protobuf/proto#Equal
func (this *BatchOperationTermination) Equal(that interface{}) bool {
if that == nil {
return this == nil
}

var that1 *BatchOperationTermination
switch t := that.(type) {
case *BatchOperationTermination:
that1 = t
case BatchOperationTermination:
that1 = &t
default:
return false
}

return proto.Equal(this, that1)
}
func (val *BatchOperationSignal) Marshal() ([]byte, error) {
return proto.Marshal(val)
}

func (val *BatchOperationSignal) Unmarshal(buf []byte) error {
return proto.Unmarshal(buf, val)
}

func (val *BatchOperationSignal) Size() int {
return proto.Size(val)
}

// Equal returns whether two BatchOperationSignal values are equivalent by recursively
// comparing the message's fields.
// For more information see the documentation for
// https://pkg.go.dev/google.golang.org/protobuf/proto#Equal
func (this *BatchOperationSignal) Equal(that interface{}) bool {
if that == nil {
return this == nil
}

var that1 *BatchOperationSignal
switch t := that.(type) {
case *BatchOperationSignal:
that1 = t
case BatchOperationSignal:
that1 = &t
default:
return false
}

return proto.Equal(this, that1)
}
func (val *BatchOperationCancellation) Marshal() ([]byte, error) {
return proto.Marshal(val)
}

func (val *BatchOperationCancellation) Unmarshal(buf []byte) error {
return proto.Unmarshal(buf, val)
}

func (val *BatchOperationCancellation) Size() int {
return proto.Size(val)
}

// Equal returns whether two BatchOperationCancellation values are equivalent by recursively
// comparing the message's fields.
// For more information see the documentation for
// https://pkg.go.dev/google.golang.org/protobuf/proto#Equal
func (this *BatchOperationCancellation) Equal(that interface{}) bool {
if that == nil {
return this == nil
}

var that1 *BatchOperationCancellation
switch t := that.(type) {
case *BatchOperationCancellation:
that1 = t
case BatchOperationCancellation:
that1 = &t
default:
return false
}

return proto.Equal(this, that1)
}
func (val *BatchOperationDeletion) Marshal() ([]byte, error) {
return proto.Marshal(val)
}

func (val *BatchOperationDeletion) Unmarshal(buf []byte) error {
return proto.Unmarshal(buf, val)
}

func (val *BatchOperationDeletion) Size() int {
return proto.Size(val)
}

// Equal returns whether two BatchOperationDeletion values are equivalent by recursively
// comparing the message's fields.
// For more information see the documentation for
// https://pkg.go.dev/google.golang.org/protobuf/proto#Equal
func (this *BatchOperationDeletion) Equal(that interface{}) bool {
if that == nil {
return this == nil
}

var that1 *BatchOperationDeletion
switch t := that.(type) {
case *BatchOperationDeletion:
that1 = t
case BatchOperationDeletion:
that1 = &t
default:
return false
}

return proto.Equal(this, that1)
}
func (val *BatchOperationReset) Marshal() ([]byte, error) {
return proto.Marshal(val)
}

func (val *BatchOperationReset) Unmarshal(buf []byte) error {
return proto.Unmarshal(buf, val)
}

func (val *BatchOperationReset) Size() int {
return proto.Size(val)
}

// Equal returns whether two BatchOperationReset values are equivalent by recursively
// comparing the message's fields.
// For more information see the documentation for
// https://pkg.go.dev/google.golang.org/protobuf/proto#Equal
func (this *BatchOperationReset) Equal(that interface{}) bool {
if that == nil {
return this == nil
}

var that1 *BatchOperationReset
switch t := that.(type) {
case *BatchOperationReset:
that1 = t
case BatchOperationReset:
that1 = &t
default:
return false
}

return proto.Equal(this, that1)
}
Loading
Loading