Python Protobuf interface

Package

Enumerations

An protobuf enumeration such as

enum EnumType {
  UNKNOWN = 0,
  STARTED = 1,
  RUNNING = 2
};

will produce functions

var_bool = EnumType_IsValid(var_int)
var_string = EnumType_Name(var_int)
[var_bool, value] = EnumType_Parse(var_string)

which respectively:

• check whether an integer var_int represents a valid enumerated value,
• convert integer values to the stringified name of the enumerated value, and
• convert stringified names back to integer values.

Where var_bool is a Python boolean value (True or False), var_int is an Python integer variable and var_string is a Python string variable. In addition two package or class constants are defined giving the minimum and maximum integer values in the enum.

var_int = EnumType_MIN
var_int = EnumType_MAX

For the example given above we would have,

>>> print(EnumType_MIN)
0
>>> print(EnumType_MAX)
2
>>> print(EnumType_IsValid(1))
True
>>> print(EnumType_IsValid(3))
False
>>> print(EnumType_Name(1))
STARTED
>>> print(EnumType_Name(3))

>>> print(EnumType_Parse('RUNNING'))
[True, 2]
>>> print(EnumType_Parse('BLAHBLAHBLAH'))
[False, 0]

If an enum is defined within a containing message, rather than in the global space, these functions and constants are part of the containing message (i.e. they are class functions and class variables).

Messages

Protobuf messages produce Python classes that wrap the underlying C++ code. Messages are all derived from the base class google.Message from which they inherit the following member functions:

m.CopyFrom(m_from)
m.MergeFrom(m_from)
var_int = m.SpaceUsed()

var_string = m.DebugString()
var_string = m.ShortDebugString()
var_string = m.GetTypeName()
m.Clear()
var_bool = m.IsInitialized()
var_int = m.ByteSize()
var_bool = m.ParseFromString(var_bytes)
var_bool = m.ParsePartialFromString(var_bytes)
var_bytes = m.SerializeAsString()
var_bytes = m.SerializePartialAsString()

The meanings of these functions can be deduced from the Google Protobuf documentation site.

Message fields

Access to each field in the Protobuf message is given by Python class member functions that are generated automatically from the definition of the message. The specific Python functions produced depend on the type of field, as described in the sections below. The names of the functions are all based on the name of the field in the .proto definition; for example a repeated field named telescopes will have a field telescopes_size() which gives the number of entries in the repeated field.

Singular Numeric, String and Bytes Fields

A message with a simple singular numeric or string field, such as

message SimpleMessage {
  int32 i = 1;
}

will produce the following Python member functions to get, set and clear the field i:

m = SimpleMessage()
var_int = m.i()
m.set_i(var_int)
m.clear_i()

where var_int is a Python variable. The correspondence between Protobuf and Python types is given in the table below:

Protobuf type	Python 3 type
bool	bool
uint32, sint32, fixed32, sfixed32	int
uint64, sint64, fixed64, sfixed64	int
float	float
double	float
string	str
bytes	bytes

Singular Enum Fields

An enum field of type EnumType, such as

EnumType e = 1;

produces Python member functions to get, set, and clear e,

var_int = m.e()
m.set_e(var_int)
m.clear_e()

In this case var_int is an integer type.

Singular Message Fields

As in the C++ implementation embedded message fields work differently to the data simple types above. They do not have traditional setter functions that take an Message as an input, but rather there is a mutable accessor that returns a proxy that can be use manipulate the sub-message.

Singular message fields such as:

message SubMessageType {
  int32 i = 1;
}    
message MessageType {
  SubMessageType sm = 1;
}

the Python code for the MessageType class will have the following member functions:

var_bool has_sm()
var_proxy = m.const_sm()
var_proxy = m.mutable_sm()
var_proxy = m.sm()
m.clear_sm()

where var_proxy is a Python proxy for the C++ instance of the SubMessageType. The function m.has_sm() can be used to test whether the field sm is set within m or not. The function m.clear_sm() clears any instance of sm in m. The other three functions provide access to the sub-field. Of these the recommended accessor is the simple function m.sm() which provides read and write access to the sub-message. The function m.const_sm() is intended to provide read-only access to the sub-field but unfortunately this is not enforced by SWIG/Python and hence the user is not forbidden from invoking non-const functions on var_proxy - the consequences of doing so on the underlying C++ implementation could be unfortunate, and hence use of the const functions are not recommended and they may be removed. The function m.mutable_sm() is equivalent to m.sm().

Repeated Numeric and String Fields

A Protobuf field such as,

repeated int32 vec_i = 1;

can be accessed from Python using the following functions:

var_int = m.vec_i_size()
var_int = m.vec_i(index)
m.add_vec_i(var_int)
m.set_vec_i(index, var_int)
array_int = m.vec_i()
m.set_vec_i(array_int)
m.clear_vec_i()

• m.vec_i_size returns the number of the items in the repeated field
• m.vec_i(index) returns the element referred to by index. An assertion is thrown if index is not in the range of -m.vec_i_size() <= index < m.vec_i_size().
• m.add_vec_i(var_int) appends the value of var_int to the field
• m.set_vec_i(index, var_int) sets the value of the element referred to by index to var_int. An assertion is thrown if index is not in the range of -m.vec_i_size() <= index < m.vec_i_size().
• m.vec_i() returns a numpy array of all elements.
• m.set_vec_i(array_int) clears any existing elements in the vector and adds all those in array_int which must be a numpy array or a list.
• m.clear_vec_i() clears all elements in the vector.

Repeated Bytes Fields

Repeated Enum Fields

Repeated Message Fields