All Projects → peter-edge → Protoeasy Go

peter-edge / Protoeasy Go

Licence: mit
Simpler usage of protoc. Deprecated.

Programming Languages

go
31211 projects - #10 most used programming language

Projects that are alternatives of or similar to Protoeasy Go

Rules protobuf
Bazel rules for building protocol buffers and gRPC services (java, c++, go, ...)
Stars: ✭ 206 (+59.69%)
Mutual labels:  grpc, grpc-gateway, protobuf, protocol-buffers
agentgo
Hi! Agentgo is a tool for making remote command executions from server to client with golang, protocol buffers (protobuf) and grpc.
Stars: ✭ 15 (-88.37%)
Mutual labels:  protobuf, protocol-buffers, grpc
protoc-plugin
A protoc compiler plugin for Clojure applications
Stars: ✭ 28 (-78.29%)
Mutual labels:  protobuf, protocol-buffers, grpc
docker-protobuf
An all-inclusive protoc Docker image
Stars: ✭ 105 (-18.6%)
Mutual labels:  protobuf, grpc, grpc-gateway
Go Grpc Examples
This repo contains examples and implementations of different types of GRPC services and APIs using Golang.
Stars: ✭ 180 (+39.53%)
Mutual labels:  grpc, protobuf, protocol-buffers
Clay
Proto-first minimal server platform for gRPС+REST+Swagger APIs
Stars: ✭ 212 (+64.34%)
Mutual labels:  grpc, grpc-gateway, protobuf
Protoc Gen Struct Transformer
Transformation functions generator for Protocol Buffers.
Stars: ✭ 105 (-18.6%)
Mutual labels:  grpc, protobuf, protocol-buffers
Prototool
Your Swiss Army Knife for Protocol Buffers
Stars: ✭ 4,932 (+3723.26%)
Mutual labels:  grpc, protobuf, protocol-buffers
Protobuf
[Looking for new ownership] Protocol Buffers for Go with Gadgets
Stars: ✭ 4,998 (+3774.42%)
Mutual labels:  grpc, protobuf, protocol-buffers
Kroto Plus
gRPC Kotlin Coroutines, Protobuf DSL, Scripting for Protoc
Stars: ✭ 400 (+210.08%)
Mutual labels:  grpc, protobuf, protocol-buffers
Buf
A new way of working with Protocol Buffers.
Stars: ✭ 3,328 (+2479.84%)
Mutual labels:  grpc, protocol-buffers, protobuf
Grpc Jersey
gRPC<->Jersey bridge
Stars: ✭ 23 (-82.17%)
Mutual labels:  grpc, grpc-gateway, protobuf
Go Micro Boilerplate
The boilerplate of the GoLang application with a clear microservices architecture.
Stars: ✭ 147 (+13.95%)
Mutual labels:  grpc, protobuf, protocol-buffers
grpc-chat
Simple Chat Server/Client implemented with gRPC
Stars: ✭ 107 (-17.05%)
Mutual labels:  protobuf, protocol-buffers, grpc
Grpc Example
An example of using Go gRPC and tools from the greater gRPC ecosystem together with the GoGo Protobuf Project.
Stars: ✭ 352 (+172.87%)
Mutual labels:  grpc, grpc-gateway, protobuf
Protoreflect
Reflection (Rich Descriptors) for Go Protocol Buffers
Stars: ✭ 651 (+404.65%)
Mutual labels:  grpc, protobuf, protocol-buffers
Protodot
transforming your .proto files into .dot files (and .svg, .png if you happen to have graphviz installed)
Stars: ✭ 107 (-17.05%)
Mutual labels:  grpc, protobuf, protocol-buffers
Pufferdb
🐡 An Android & JVM key-value storage powered by Protobuf and Coroutines
Stars: ✭ 91 (-29.46%)
Mutual labels:  protobuf, protocol-buffers
Protobuf Nim
Protobuf implementation in pure Nim that leverages the power of the macro system to not depend on any external tools
Stars: ✭ 90 (-30.23%)
Mutual labels:  protobuf, protocol-buffers
Go Shopping
A sample suite of services built on the go-micro framework
Stars: ✭ 98 (-24.03%)
Mutual labels:  grpc, protobuf

CircleCI Go Report Card Docker Repository on Quay GoDoc MIT License

Protoeasy

Deprecated.

Protoeasy is intended to make compiling protocol buffers files easier, and to optionally offload the compilation to a server for consistency and so that protoc and any associated plugins do not have to be installed locally.

Instead of having to specify individual plugins and files, protoeasy operates on the concept of languages and directories. Compilation is recommended to happen in a provided docker container. The primary language developed for protoeasy was Go, and all gogo plugins are supported as well, but support also exists for C++, C#, Objective-C, Python, and Ruby. gRPC support is built in for all supported languages. Proto3 is used as the default compiler, however a proto2 docker container is also available, but support is limited.

Quick Start

This assumes you are familiar with Go.

Note that client/server mode is optional - you can use protoeasy locally if you have protoc and all relevant plugins installed.

# install binary
go get -v go.pedge.io/protoeasy/cmd/protoeasy
# OPTIONAL - if you have all the required binaries installed (protoc, plugins, etc), then you can use protoeasy locally
docker pull quay.io/pedge/protoeasy
docker run -d -p 6789:6789 quay.io/pedge/protoeasy
# Assuming you are running the docker daemon locally, DOCKER_ADDRESS is 0.0.0.0.
# Assuming you are running the docker daemon in a VM, DOCKER_ADDRESS is DOCKER_HOST without the port.
# Note that if you are running the docker daemon in a VM or remotely, port 6789 needs to be open for you to connect to
# DO NOT SET IF RUNNING LOCALLY
export PROTOEASY_ADDRESS="${DOCKER_ADDRESS}:6789"
# doing go as an example
# you do not need to cd in, but this is as an example
cd github.com/user/your-go-project
# this assumes your protocol buffers import paths are relative to the root of your go project
protoeasy --go --grpc --go-import-path github.com/user/your-go-project .
# compile for c++ and ruby too
protoeasy --go --grpc --go-import-path github.com/user/your-go-project --cpp --ruby .
# exclude protocol buffers files in foo/*
protoeasy --go --grpc --go-import-path github.com/user/your-go-project --exclude foo .
# this assumes your protocol buffers files are in ext/proto, and import paths are relative to ext/proto
protoeasy --go --grpc --go-import-path github.com/user/your-go-project/ext/proto ext/proto
# this assumes you have a protoeasy.yaml file in your current directory, see the file here as an example
protoeasy

Motivation

Have you ever had one of the following issues:

  • You're the resident protobuf expert on your team, and every time there is a change to a protocol buffers package or you get a new teammate, you already know what the next hour of your life will be.
  • You move to a new development machine, and can't remember how you got protoc installed properly, and now your old build scripts do not work anymore.
  • You upgrade proto3 on your computer, and suddenly protoc is broken.
  • One of your teammates upgrades proto3 on their computer, and suddently protoc is broken.
  • You did go get -u google.golang.org/grpc, or even worse, google.golang.org is a dependency of some package and is updated, and suddenly new, incompatible .pb.go files are created.
  • You have gRPC working on your mac, but linux is a mess.
  • You use Mosh and proto3, but since Mosh is still on proto2, you're in trouble.
  • You're sick of resolving all relative paths to absolute paths or vice versa for -I.
  • You really don't like having to edit a Makefile or script every time you add a protocol buffer file or package.
  • You never can figure out a good scheme for a large amount of protocol buffers files and how to do imports properly.
  • You think this protobuf thing is really cool, but are sick of maintaining large bash scripts or Makefile directives to use protoc, and are sick of having to get everything installed on your development machine, and you know there must be an easier way to do it but haven't found it yet.

Then protoeasy is for you!

Tutorial

Installation

Install protoeasy using make install, assuming ${GOPATH}/bin is on your ${PATH}. You can also do go get -v go.pedge.io/protoeasy/cmd/protoeasy.

Basics

Protoeasy compiles entire directories of protocol buffer files, as opposed to individual files. To use protoeasy:

  • All protocol buffers files in the same sub-directory must have the same protocol buffers package.
  • All imports must be relative to the base directory.

Both of these are already good practices. Assuming this structure, all you have to do is pass in a base directory, and the rest is done for you. All -I directives are automatically figured out, and all protocol buffers files in the same sub-directory are compiled together.

Instead of specifying individual output directives, protoeasy breaks compilation into separate languages, optionally doing gRPC compilation for each language.

Assume we have a single file in our current directory called foo.proto.

syntax = "proto3";

package foo;

message One {
  int64 i = 1;
}

To compile for ruby, we would do:

protoeasy --ruby .
#protoc \
#  -I/tmp/protoeasy-input353662656 \
#  -I/go/src/go.pedge.io/protoeasy/vendor/go.pedge.io/pb/proto \
#  --ruby_out=/tmp/protoeasy-output452425503 \
#  /tmp/protoeasy-input353662656/foo.proto

To compile for ruby and c++, we would do:

protoeasy --ruby --cpp .
#protoc \
#  -I/tmp/protoeasy-input138072562 \
#  -I/go/src/go.pedge.io/protoeasy/vendor/go.pedge.io/pb/proto \
#  --cpp_out=/tmp/protoeasy-output084691625 \
#  /tmp/protoeasy-input138072562/foo.proto
#protoc \
#  -I/tmp/protoeasy-input138072562 \
#  -I/go/src/go.pedge.io/protoeasy/vendor/go.pedge.io/pb/proto \
#  --ruby_out=/tmp/protoeasy-output084691625 \
#  /tmp/protoeasy-input138072562/foo.proto

If we wanted to generate gRPC code for both, we would do:

protoeasy --ruby --cpp --grpc .
#protoc \
#  -I/tmp/protoeasy-input924418036 \
#  -I/go/src/go.pedge.io/protoeasy/vendor/go.pedge.io/pb/proto \
#  --cpp_out=/tmp/protoeasy-output335846083 \
#  --grpc_out=/tmp/protoeasy-output335846083 \
#  --plugin=protoc-gen-grpc=/usr/local/bin/grpc_cpp_plugin \
#  /tmp/protoeasy-input924418036/foo.proto
#protoc \
#  -I/tmp/protoeasy-input924418036 \
#  -I/go/src/go.pedge.io/protoeasy/vendor/go.pedge.io/pb/proto \
#  --ruby_out=/tmp/protoeasy-output335846083 \
#  --grpc_out=/tmp/protoeasy-output335846083 \
#  --plugin=protoc-gen-grpc=/usr/local/bin/grpc_ruby_plugin \
#  /tmp/protoeasy-input924418036/foo.proto

What is going on here:

  • For each language, a protoc command is generated. Each language is run separately because some protoc flags overlap (--plugin specifically).
  • The /tmp/protoeasy-input... directories are the directory where foo.proto is located. These compilations were done on a remote server, explained later.
  • If --grpc is specified, --grpc_out and --plugin flags are specified for each associated language, except for Go which has different logic.
  • Other typical imports are added automatically, explained later.

Supported languages are C++, C#, Objective-C, Python, Ruby, and Go. Protoeasy was primarily developed for Go (which is what I use it for), so if there are idioms for other languages, let me know.

Go

Go has special handling. Let's assume we have our original foo.proto, and another file bar/bar.proto in the sub-directory bar, and let's assume these are in the Go package github.com/alice/helloworld.

syntax = "proto3";

import "google/api/annotations.proto";
import "google/protobuf/empty.proto";
import "foo.proto";

package bar;

message Two {
  foo.One one = 1;
  int64 j = 2;
}

service API {
  rpc Do(Two) returns (google.protobuf.Empty) {
    option (google.api.http) = {
      post: "/do"
      body: "*"
    };
  }
}

Let's compile these with go, grpc, and grpc-gateway, and do them all now without building up the example so that this README does not get too long.

protoeasy --go --go-import-path=github.com/alice/helloworld --grpc --grpc-gateway .
#protoc \
#  -I/tmp/protoeasy-input035188496 \
#  -I/go/src/go.pedge.io/protoeasy/vendor/go.pedge.io/pb/proto \
#  --go_out=Mbar/bar.proto=github.com/alice/helloworld/bar,Mfoo.proto=github.com/alice/helloworld,Mgoogle/api/annotations.proto=github.com/grpc-ecosystem/grpc-gateway/third_party/pb/go/google/api,Mgoogle/api/http.proto=github.com/grpc-ecosystem/grpc-gateway/third_party/pb/go/google/api,Mgoogle/datastore/v1beta3/datastore.proto=go.pedge.io/pb/go/google/datastore/v1beta3,Mgoogle/datastore/v1beta3/entity.proto=go.pedge.io/pb/go/google/datastore/v1beta3,Mgoogle/datastore/v1beta3/query.proto=go.pedge.io/pb/go/google/datastore/v1beta3,Mgoogle/devtools/cloudtrace/v1/trace.proto=go.pedge.io/pb/go/google/devtools/cloudtrace/v1,Mgoogle/example/library/v1/library.proto=go.pedge.io/pb/go/google/example/library/v1,Mgoogle/iam/v1/iam_policy.proto=go.pedge.io/pb/go/google/iam/v1,Mgoogle/iam/v1/policy.proto=go.pedge.io/pb/go/google/iam/v1,Mgoogle/longrunning/operations.proto=go.pedge.io/pb/go/google/longrunning,Mgoogle/protobuf/any.proto=go.pedge.io/pb/go/google/protobuf,Mgoogle/protobuf/api.proto=go.pedge.io/pb/go/google/protobuf,Mgoogle/protobuf/descriptor.proto=github.com/golang/protobuf/protoc-gen-go/descriptor,Mgoogle/protobuf/duration.proto=go.pedge.io/pb/go/google/protobuf,Mgoogle/protobuf/empty.proto=go.pedge.io/pb/go/google/protobuf,Mgoogle/protobuf/field_mask.proto=go.pedge.io/pb/go/google/protobuf,Mgoogle/protobuf/source_context.proto=go.pedge.io/pb/go/google/protobuf,Mgoogle/protobuf/struct.proto=go.pedge.io/pb/go/google/protobuf,Mgoogle/protobuf/timestamp.proto=go.pedge.io/pb/go/google/protobuf,Mgoogle/protobuf/type.proto=go.pedge.io/pb/go/google/protobuf,Mgoogle/protobuf/wrappers.proto=go.pedge.io/pb/go/google/protobuf,Mgoogle/pubsub/v1/pubsub.proto=go.pedge.io/pb/go/google/pubsub/v1,Mgoogle/pubsub/v1beta2/pubsub.proto=go.pedge.io/pb/go/google/pubsub/v1beta2,Mgoogle/rpc/code.proto=go.pedge.io/pb/go/google/rpc,Mgoogle/rpc/error_details.proto=go.pedge.io/pb/go/google/rpc,Mgoogle/rpc/status.proto=go.pedge.io/pb/go/google/rpc,Mgoogle/type/color.proto=go.pedge.io/pb/go/google/type,Mgoogle/type/date.proto=go.pedge.io/pb/go/google/type,Mgoogle/type/dayofweek.proto=go.pedge.io/pb/go/google/type,Mgoogle/type/latlng.proto=go.pedge.io/pb/go/google/type,Mgoogle/type/money.proto=go.pedge.io/pb/go/google/type,Mgoogle/type/timeofday.proto=go.pedge.io/pb/go/google/type,plugins=grpc:/tmp/protoeasy-output215726383 \
#  --grpc-gateway_out=Mbar/bar.proto=github.com/alice/helloworld/bar,Mfoo.proto=github.com/alice/helloworld,Mgoogle/api/annotations.proto=github.com/grpc-ecosystem/grpc-gateway/third_party/pb/go/google/api,Mgoogle/api/http.proto=github.com/grpc-ecosystem/grpc-gateway/third_party/pb/go/google/api,Mgoogle/datastore/v1beta3/datastore.proto=go.pedge.io/pb/go/google/datastore/v1beta3,Mgoogle/datastore/v1beta3/entity.proto=go.pedge.io/pb/go/google/datastore/v1beta3,Mgoogle/datastore/v1beta3/query.proto=go.pedge.io/pb/go/google/datastore/v1beta3,Mgoogle/devtools/cloudtrace/v1/trace.proto=go.pedge.io/pb/go/google/devtools/cloudtrace/v1,Mgoogle/example/library/v1/library.proto=go.pedge.io/pb/go/google/example/library/v1,Mgoogle/iam/v1/iam_policy.proto=go.pedge.io/pb/go/google/iam/v1,Mgoogle/iam/v1/policy.proto=go.pedge.io/pb/go/google/iam/v1,Mgoogle/longrunning/operations.proto=go.pedge.io/pb/go/google/longrunning,Mgoogle/protobuf/any.proto=go.pedge.io/pb/go/google/protobuf,Mgoogle/protobuf/api.proto=go.pedge.io/pb/go/google/protobuf,Mgoogle/protobuf/descriptor.proto=github.com/golang/protobuf/protoc-gen-go/descriptor,Mgoogle/protobuf/duration.proto=go.pedge.io/pb/go/google/protobuf,Mgoogle/protobuf/empty.proto=go.pedge.io/pb/go/google/protobuf,Mgoogle/protobuf/field_mask.proto=go.pedge.io/pb/go/google/protobuf,Mgoogle/protobuf/source_context.proto=go.pedge.io/pb/go/google/protobuf,Mgoogle/protobuf/struct.proto=go.pedge.io/pb/go/google/protobuf,Mgoogle/protobuf/timestamp.proto=go.pedge.io/pb/go/google/protobuf,Mgoogle/protobuf/type.proto=go.pedge.io/pb/go/google/protobuf,Mgoogle/protobuf/wrappers.proto=go.pedge.io/pb/go/google/protobuf,Mgoogle/pubsub/v1/pubsub.proto=go.pedge.io/pb/go/google/pubsub/v1,Mgoogle/pubsub/v1beta2/pubsub.proto=go.pedge.io/pb/go/google/pubsub/v1beta2,Mgoogle/rpc/code.proto=go.pedge.io/pb/go/google/rpc,Mgoogle/rpc/error_details.proto=go.pedge.io/pb/go/google/rpc,Mgoogle/rpc/status.proto=go.pedge.io/pb/go/google/rpc,Mgoogle/type/color.proto=go.pedge.io/pb/go/google/type,Mgoogle/type/date.proto=go.pedge.io/pb/go/google/type,Mgoogle/type/dayofweek.proto=go.pedge.io/pb/go/google/type,Mgoogle/type/latlng.proto=go.pedge.io/pb/go/google/type,Mgoogle/type/money.proto=go.pedge.io/pb/go/google/type,Mgoogle/type/timeofday.proto=go.pedge.io/pb/go/google/type:/tmp/protoeasy-output215726383 \
#  /tmp/protoeasy-input035188496/bar/bar.proto
#protoc \
#  -I/tmp/protoeasy-input035188496 \
#  -I/go/src/go.pedge.io/protoeasy/vendor/go.pedge.io/pb/go/google/protobuf \
#  --go_out=Mbar/bar.proto=github.com/alice/helloworld/bar,Mfoo.proto=github.com/alice/helloworld,Mgoogle/api/annotations.proto=github.com/grpc-ecosystem/grpc-gateway/third_party/pb/go/google/api,Mgoogle/api/http.proto=github.com/grpc-ecosystem/grpc-gateway/third_party/pb/go/google/api,Mgoogle/datastore/v1beta3/datastore.proto=go.pedge.io/pb/go/google/datastore/v1beta3,Mgoogle/datastore/v1beta3/entity.proto=go.pedge.io/pb/go/google/datastore/v1beta3,Mgoogle/datastore/v1beta3/query.proto=go.pedge.io/pb/go/google/datastore/v1beta3,Mgoogle/devtools/cloudtrace/v1/trace.proto=go.pedge.io/pb/go/google/devtools/cloudtrace/v1,Mgoogle/example/library/v1/library.proto=go.pedge.io/pb/go/google/example/library/v1,Mgoogle/iam/v1/iam_policy.proto=go.pedge.io/pb/go/google/iam/v1,Mgoogle/iam/v1/policy.proto=go.pedge.io/pb/go/google/iam/v1,Mgoogle/longrunning/operations.proto=go.pedge.io/pb/go/google/longrunning,Mgoogle/protobuf/any.proto=go.pedge.io/pb/go/google/protobuf,Mgoogle/protobuf/api.proto=go.pedge.io/pb/go/google/protobuf,Mgoogle/protobuf/descriptor.proto=github.com/golang/protobuf/protoc-gen-go/descriptor,Mgoogle/protobuf/duration.proto=go.pedge.io/pb/go/google/protobuf,Mgoogle/protobuf/empty.proto=go.pedge.io/pb/go/google/protobuf,Mgoogle/protobuf/field_mask.proto=go.pedge.io/pb/go/google/protobuf,Mgoogle/protobuf/source_context.proto=go.pedge.io/pb/go/google/protobuf,Mgoogle/protobuf/struct.proto=go.pedge.io/pb/go/google/protobuf,Mgoogle/protobuf/timestamp.proto=go.pedge.io/pb/go/google/protobuf,Mgoogle/protobuf/type.proto=go.pedge.io/pb/go/google/protobuf,Mgoogle/protobuf/wrappers.proto=go.pedge.io/pb/go/google/protobuf,Mgoogle/pubsub/v1/pubsub.proto=go.pedge.io/pb/go/google/pubsub/v1,Mgoogle/pubsub/v1beta2/pubsub.proto=go.pedge.io/pb/go/google/pubsub/v1beta2,Mgoogle/rpc/code.proto=go.pedge.io/pb/go/google/rpc,Mgoogle/rpc/error_details.proto=go.pedge.io/pb/go/google/rpc,Mgoogle/rpc/status.proto=go.pedge.io/pb/go/google/rpc,Mgoogle/type/color.proto=go.pedge.io/pb/go/google/type,Mgoogle/type/date.proto=go.pedge.io/pb/go/google/type,Mgoogle/type/dayofweek.proto=go.pedge.io/pb/go/google/type,Mgoogle/type/latlng.proto=go.pedge.io/pb/go/google/type,Mgoogle/type/money.proto=go.pedge.io/pb/go/google/type,Mgoogle/type/timeofday.proto=go.pedge.io/pb/go/google/type,plugins=grpc:/tmp/protoeasy-output215726383 \
#  --grpc-gateway_out=Mbar/bar.proto=github.com/alice/helloworld/bar,Mfoo.proto=github.com/alice/helloworld,Mgoogle/api/annotations.proto=github.com/grpc-ecosystem/grpc-gateway/third_party/pb/go/google/api,Mgoogle/api/http.proto=github.com/grpc-ecosystem/grpc-gateway/third_party/pb/go/google/api,Mgoogle/datastore/v1beta3/datastore.proto=go.pedge.io/pb/go/google/datastore/v1beta3,Mgoogle/datastore/v1beta3/entity.proto=go.pedge.io/pb/go/google/datastore/v1beta3,Mgoogle/datastore/v1beta3/query.proto=go.pedge.io/pb/go/google/datastore/v1beta3,Mgoogle/devtools/cloudtrace/v1/trace.proto=go.pedge.io/pb/go/google/devtools/cloudtrace/v1,Mgoogle/example/library/v1/library.proto=go.pedge.io/pb/go/google/example/library/v1,Mgoogle/iam/v1/iam_policy.proto=go.pedge.io/pb/go/google/iam/v1,Mgoogle/iam/v1/policy.proto=go.pedge.io/pb/go/google/iam/v1,Mgoogle/longrunning/operations.proto=go.pedge.io/pb/go/google/longrunning,Mgoogle/protobuf/any.proto=go.pedge.io/pb/go/google/protobuf,Mgoogle/protobuf/api.proto=go.pedge.io/pb/go/google/protobuf,Mgoogle/protobuf/descriptor.proto=github.com/golang/protobuf/protoc-gen-go/descriptor,Mgoogle/protobuf/duration.proto=go.pedge.io/pb/go/google/protobuf,Mgoogle/protobuf/empty.proto=go.pedge.io/pb/go/google/protobuf,Mgoogle/protobuf/field_mask.proto=go.pedge.io/pb/go/google/protobuf,Mgoogle/protobuf/source_context.proto=go.pedge.io/pb/go/google/protobuf,Mgoogle/protobuf/struct.proto=go.pedge.io/pb/go/google/protobuf,Mgoogle/protobuf/timestamp.proto=go.pedge.io/pb/go/google/protobuf,Mgoogle/protobuf/type.proto=go.pedge.io/pb/go/google/protobuf,Mgoogle/protobuf/wrappers.proto=go.pedge.io/pb/go/google/protobuf,Mgoogle/pubsub/v1/pubsub.proto=go.pedge.io/pb/go/google/pubsub/v1,Mgoogle/pubsub/v1beta2/pubsub.proto=go.pedge.io/pb/go/google/pubsub/v1beta2,Mgoogle/rpc/code.proto=go.pedge.io/pb/go/google/rpc,Mgoogle/rpc/error_details.proto=go.pedge.io/pb/go/google/rpc,Mgoogle/rpc/status.proto=go.pedge.io/pb/go/google/rpc,Mgoogle/type/color.proto=go.pedge.io/pb/go/google/type,Mgoogle/type/date.proto=go.pedge.io/pb/go/google/type,Mgoogle/type/dayofweek.proto=go.pedge.io/pb/go/google/type,Mgoogle/type/latlng.proto=go.pedge.io/pb/go/google/type,Mgoogle/type/money.proto=go.pedge.io/pb/go/google/type,Mgoogle/type/timeofday.proto=go.pedge.io/pb/go/google/type:/tmp/protoeasy-output215726383 \
#  /tmp/protoeasy-input035188496/foo.proto

Whoa! There's a lot going on there, let's walk through it:

  • The -I directives are the same as other languages.
  • For gRPC, instead of having --grpc_out and --plugin, Go does --go_out=plugins=grpc
  • We specified that we are in the github.com/alice/helloworld package using --go-import-path and the modifiers Mbar/bar.proto=github.com/alice/helloworld/bar,Mfoo.proto=github.com/alice/helloworld were generated for us. Therefore bar.pb.go has the import statement import foo "github.com/alice/helloworld". If, instead, all protocol buffers files were in a sub-directory proto in github.com/alice/helloworld, we would have done protoeasy --go --grpc --grpc_gateway --go-import-path github.com/alice/helloworld/proto proto
  • We have the modifiers for a bunch of other files, but specifically Mgoogle/api/annotations.proto=github.com/grpc-ecosystem/grpc-gateway/third_party/googleapis/google/api,Mgoogle/protobuf/empty.proto=go.pedge.io/pb/go. More on this later.
  • The directive --grpc-gateway_out created a bar/bar.pb.gw.go file for us, for grpc-gateway. You should really check grpc-gateway out :)

Other Go flags:

  • --go-protoc-plugin=: specifiy a plugin other than protoc-gen-go. This is useful for gogo.
  • --go-modifier: specify an additional modifier of the form Mfile=package.
  • --go-no-default-modifiers: do not use the default modifiers for google/protobuf and googleapis.

Automatically Imported Packages

Protoeasy automatically imports the protocol buffers files in:

Track https://github.com/golang/protobuf/issues/50 for more details.

More Examples

See make proto and make example-complete for two more example usages. I also use protoeasy for all my public Go projects on GitHub that use protocol buffers (the command using protoeasy is in the Makefile in each project):

Client/Server Setup

Protoeasy works on your local machine by default, however this is not how it is intended to be used. Protoeasy provides a Docker image quay.io/pedge/protoeasy, created from the Dockerfile in this repository, that has everything necessary for compilation already installed, and which runs the protoeasyd daemon. The protoeasy binary will automatically connect to this daemon and delegate compilation if the environment variable PROTOEASY_ADDRESS is set.

How this works:

  • The client (protoeasy) tarballs all .proto files in the specified directory and uploads this tarball to the server, along with the directives given from the flags.
  • The server creates two temporary directories, one for the uploaded tarball, and one to output generated files.
  • The server untars the uploaded tarball, runs the protoeasy logic, and outputs the generated files.
  • The server tarballs the generated files, and sends them back to the client.
  • The client untars the generated files in the specified directory.

To use this, first get the Docker image:

docker pull quay.io/pedge/protoeasy

Or, build it from this repository:

make docker-build

Then launch the daemon:

docker run -d -p 6789:6789 quay.io/pedge/protoeasy # or do make docker-launch

My local setup is a Macbook Pro with Docker running in a VirtualBox VM with a private network at 192.168.10.10.

# a simple version from my Vagrantfile at https://github.com/peter-edge/dotfiles/blob/master/vagrant/common/base_provision.rb
Vagrant.configure(2) do |config|
  ...
  config.vm.network "private_network", ip: "192.168.10.10"
end
# on my vagrant host, this was setup using https://github.com/peter-edge/dotfiles/blob/master/bin/setup_docker_http_upstart.sh
ps -ef | grep docker
#root     16450     1  0 Nov28 ?        00:06:01 /usr/bin/docker daemon -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock

Many Docker VirtualBox setups are similar to this, including I believe the one created from Docker Machine.

In my ~/.bash_aliases, I have:

export PROTOEASY_ADDRESS=192.168.10.10:6789

launch-protoeasy() {
  docker rm -f protoeasy || true
  docker run -d -p 6789:6789 --name=protoeasy quay.io/pedge/protoeasy
}

Whenever protoeasy is stopped, or when I want to restart it, I just run launch-protoeasy. Since I have PROTOEASY_ADDRESS exported, the protoeasy binary always connects to the running Docker container.

In this manner, I never actually compile protocol buffers files on my Macbook directly.

Q: Why not just do your builds directly in a Docker container, i.e. docker run --volume $(pwd):/compile --workdir /compile pedge/proto3grpc protoc --go_out=. foo.proto https://hub.docker.com/r/pedge/proto3grpc A: This is what I used to do, but if your Docker host is NOT local (i.e. you are not using Docker locally in Linux, or in a local VM), this means that you cannot have a host volume linking. Protoeasy should soon have a deployment option once I get TLS added. Also, protoeasy is a little faster, but it's not really important.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].