[protobuf] Inheriting protobuf messages

[Saurabh Kumar]

Hi,

This question is regarding inheritance in C++ protobuf library. I will
explain by giving a concrete example.

message MSG1
{
required string account = 0;
required string symbol = 1;
}

message MSG2
{
required string account = 0;
required string symbol = 1;

}

[Saurabh Kumar]

Hi,

This question is regarding inheritance in protobuf C++ library. I will
explain what I am looking for with a concrete example.

I have this message definition:

message MSG1
{
required string account = 0;
required string symbol = 1;
}

Now I want to extend this message and define a new message like this:
message MSG2
{
required string account = 0;
required string symbol = 1;
required int32 id = 2;
}

You will notice that first two fields of MSG2 are exactly same as MSG1
(they are intended to be like that). But here I had to copy paste the
common fields again.
Can I do something like this?

message MSG2 extends MSG1
{
required int32 id = 2;
}

I have already thought about using it like:
message MSG2
{
required MSG1 msg1 = 0;
required int32 id = 2;
}
But this is not really what I want.

What's the best way to achieve this?

Thanks,
Saurabh

['Feng Xiao' via Protocol Buffers]

Post by Saurabh Kumar
Hi,
This question is regarding inheritance in protobuf C++ library. I will
explain what I am looking for with a concrete example.
message MSG1
{
required string account = 0;
required string symbol = 1;
}
message MSG2
{
required string account = 0;
required string symbol = 1;
required int32 id = 2;
}
You will notice that first two fields of MSG2 are exactly same as MSG1
(they are intended to be like that). But here I had to copy paste the
common fields again.
Can I do something like this?
message MSG2 extends MSG1
{
required int32 id = 2;
}
message MSG2
{
required MSG1 msg1 = 0;
required int32 id = 2;
}
But this is not really what I want.
What's the best way to achieve this?

Protobuf doesn't support inheritance. Having a common header and using
composition is the best solution.

Post by Saurabh Kumar
Thanks,
Saurabh
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

[Saurabh Kumar]

Thanks for the reply. What exactly do you mean by common header?

Post by Saurabh Kumar
Hi,
This question is regarding inheritance in protobuf C++ library. I will
explain what I am looking for with a concrete example.
message MSG1
{
required string account = 0;
required string symbol = 1;
}
message MSG2
{
required string account = 0;
required string symbol = 1;
required int32 id = 2;
}
You will notice that first two fields of MSG2 are exactly same as MSG1
(they are intended to be like that). But here I had to copy paste the
common fields again.
Can I do something like this?
message MSG2 extends MSG1
{
required int32 id = 2;
}
message MSG2
{
required MSG1 msg1 = 0;
required int32 id = 2;
}
But this is not really what I want.
What's the best way to achieve this?
Protobuf doesn't support inheritance. Having a common header and using
composition is the best solution.
Thanks,
Saurabh
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

['Feng Xiao' via Protocol Buffers]

I meant something like:

message Header {
string account = 1;
string symbol = 1;
}

message Msg1 {
Header header = 1;
...
}

message Msg2 {
Header header = 1;
...
}

Post by Saurabh Kumar
Thanks for the reply. What exactly do you mean by common header?

Post by Saurabh Kumar
Hi,
This question is regarding inheritance in protobuf C++ library. I will
explain what I am looking for with a concrete example.
message MSG1
{
required string account = 0;
required string symbol = 1;
}
message MSG2
{
required string account = 0;
required string symbol = 1;
required int32 id = 2;
}
You will notice that first two fields of MSG2 are exactly same as MSG1
(they are intended to be like that). But here I had to copy paste the
common fields again.
Can I do something like this?
message MSG2 extends MSG1
{
required int32 id = 2;
}
message MSG2
{
required MSG1 msg1 = 0;
required int32 id = 2;
}
But this is not really what I want.
What's the best way to achieve this?
Protobuf doesn't support inheritance. Having a common header and using
composition is the best solution.
Thanks,
Saurabh
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.
--

You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

[Saurabh Kumar]

Understood but this is not what I wanted in the first place.

Does someone has any idea about what makes it difficult to implement this?
Also, is there a clever way to have the same behaviour?
Basically, here I want to avoid copy pasting same fields over and over
again (makes code less maintainable).

Any ideas are welcome.

Post by 'Feng Xiao' via Protocol Buffers
message Header {
string account = 1;
string symbol = 1;
}
message Msg1 {
Header header = 1;
...
}
message Msg2 {
Header header = 1;
...
}

Post by Saurabh Kumar
Thanks for the reply. What exactly do you mean by common header?

Post by 'Feng Xiao' via Protocol Buffers

Post by Saurabh Kumar
Hi,
This question is regarding inheritance in protobuf C++ library. I will
explain what I am looking for with a concrete example.
message MSG1
{
required string account = 0;
required string symbol = 1;
}
message MSG2
{
required string account = 0;
required string symbol = 1;
required int32 id = 2;
}
You will notice that first two fields of MSG2 are exactly same as MSG1
(they are intended to be like that). But here I had to copy paste the
common fields again.
Can I do something like this?
message MSG2 extends MSG1
{
required int32 id = 2;
}
message MSG2
{
required MSG1 msg1 = 0;
required int32 id = 2;
}
But this is not really what I want.
What's the best way to achieve this?

Protobuf doesn't support inheritance. Having a common header and using
composition is the best solution.

Post by Saurabh Kumar
Thanks,
Saurabh

--

Post by Saurabh Kumar
You received this message because you are subscribed to the Google
Groups "Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send
<javascript:>.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an
<javascript:>.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

[Marcelo Cantos]

It probably wouldn’t be difficult to implement, but it’s not, afaik, a
design goal for protocol buffers because it is almost never (if ever)
necessary.

There are two composition approaches available, depending on what your
needs are. Contain the common stuff:

message Common {
required string account = 1;
required string symbol = 2;
}
message MSG1 {
required common = 1
}
message MSG2 {
required common = 1
required int32 id = 2;
}

Or contain the variable stuff:

message MSG {
required string account = 1;
required string symbol = 2;
optional Extra1 extra1 = 3;
optional Extra2 extra2 = 4;

message Extra1 {
required int32 id = 1;
}

message Extra2 {
required string foo = 1;
}
}

with proto3, you can do slightly better:

message MSG {
string account = 1;
string symbol = 2;
oneof extra { Extra1 extra1 = 3; Extra2 extra2 = 4;
}

message Extra1 {
int32 id = 1;
}

message Extra2 {
string foo = 1;
}
}

If composition is not what you want, then why not? What real-world problem
do you have that cannot be effectively solved with one of the above
strategies?

On Thursday, 20 October 2016 17:50:22 UTC+11, Saurabh Kumar wrote:

Understood but this is not what I wanted in the first place.

Post by Saurabh Kumar
Does someone has any idea about what makes it difficult to implement this?
Also, is there a clever way to have the same behaviour?
Basically, here I want to avoid copy pasting same fields over and over
again (makes code less maintainable).
Any ideas are welcome.

Post by 'Feng Xiao' via Protocol Buffers
message Header {
string account = 1;
string symbol = 1;
}
message Msg1 {
Header header = 1;
...
}
message Msg2 {
Header header = 1;
...
}

Post by Saurabh Kumar
Thanks for the reply. What exactly do you mean by common header?

Post by 'Feng Xiao' via Protocol Buffers

Post by Saurabh Kumar
Hi,
This question is regarding inheritance in protobuf C++ library. I will
explain what I am looking for with a concrete example.
message MSG1
{
required string account = 0;
required string symbol = 1;
}
message MSG2
{
required string account = 0;
required string symbol = 1;
required int32 id = 2;
}
You will notice that first two fields of MSG2 are exactly same as MSG1
(they are intended to be like that). But here I had to copy paste the
common fields again.
Can I do something like this?
message MSG2 extends MSG1
{
required int32 id = 2;
}
message MSG2
{
required MSG1 msg1 = 0;
required int32 id = 2;
}
But this is not really what I want.
What's the best way to achieve this?

Protobuf doesn't support inheritance. Having a common header and using
composition is the best solution.

Post by Saurabh Kumar
Thanks,
Saurabh

--

Post by Saurabh Kumar
You received this message because you are subscribed to the Google
Groups "Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google
Groups "Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

​

[Saurabh Kumar]

Hi,

Thanks for the suggestions below. I agree with you that these approaches
address most of the real life scenarios. What I am really looking for is
more of a syntactic sugar. In my case, I want to avoid any kind of nesting
as a personal preference. I just want to avoid copy pasting same members at
multiple places and have the overhead of keeping them updated everywhere.

I don't even mind if we had a certain kind of preprocessor to achieve this
i.e.
#define common_fields required int32 a; \
required int32 b; \

message msg1 {
common_fields;
required int32 c;
}


message msg2 {
common_fields;
required int32 d;
}

Does it make sense?

Post by Marcelo Cantos
It probably wouldn’t be difficult to implement, but it’s not, afaik, a
design goal for protocol buffers because it is almost never (if ever)
necessary.
There are two composition approaches available, depending on what your
message Common {
required string account = 1;
required string symbol = 2;
}
message MSG1 {
required common = 1
}
message MSG2 {
required common = 1
required int32 id = 2;
}
message MSG {
required string account = 1;
required string symbol = 2;
optional Extra1 extra1 = 3;
optional Extra2 extra2 = 4;
message Extra1 {
required int32 id = 1;
}
message Extra2 {
required string foo = 1;
}
}
message MSG {
string account = 1;
string symbol = 2;
oneof extra { Extra1 extra1 = 3; Extra2 extra2 = 4;
}
message Extra1 {
int32 id = 1;
}
message Extra2 {
string foo = 1;
}
}
If composition is not what you want, then why not? What real-world problem
do you have that cannot be effectively solved with one of the above
strategies?
Understood but this is not what I wanted in the first place.

Post by Saurabh Kumar
Does someone has any idea about what makes it difficult to implement
this? Also, is there a clever way to have the same behaviour?
Basically, here I want to avoid copy pasting same fields over and over
again (makes code less maintainable).
Any ideas are welcome.

Post by 'Feng Xiao' via Protocol Buffers
message Header {
string account = 1;
string symbol = 1;
}
message Msg1 {
Header header = 1;
...
}
message Msg2 {
Header header = 1;
...
}

Post by Saurabh Kumar
Thanks for the reply. What exactly do you mean by common header?

Post by 'Feng Xiao' via Protocol Buffers

Post by Saurabh Kumar
Hi,
This question is regarding inheritance in protobuf C++ library. I
will explain what I am looking for with a concrete example.
message MSG1
{
required string account = 0;
required string symbol = 1;
}
message MSG2
{
required string account = 0;
required string symbol = 1;
required int32 id = 2;
}
You will notice that first two fields of MSG2 are exactly same as
MSG1 (they are intended to be like that). But here I had to copy paste the
common fields again.
Can I do something like this?
message MSG2 extends MSG1
{
required int32 id = 2;
}
message MSG2
{
required MSG1 msg1 = 0;
required int32 id = 2;
}
But this is not really what I want.
What's the best way to achieve this?

Protobuf doesn't support inheritance. Having a common header and using
composition is the best solution.

Post by Saurabh Kumar
Thanks,
Saurabh

--

Post by Saurabh Kumar
You received this message because you are subscribed to the Google
Groups "Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it,
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google
Groups "Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

​

[Marcelo Cantos]

I know exactly what you're trying to achieve, but you haven't presented a
strong case for why you want this. Both the approaches I've suggested avoid
the copy pasting you're worried about, and personal preference isn't an
argument.

Post by Saurabh Kumar
Hi,
Thanks for the suggestions below. I agree with you that these approaches
address most of the real life scenarios. What I am really looking for is
more of a syntactic sugar. In my case, I want to avoid any kind of nesting
as a personal preference. I just want to avoid copy pasting same members at
multiple places and have the overhead of keeping them updated everywhere.
I don't even mind if we had a certain kind of preprocessor to achieve this
i.e.
#define common_fields required int32 a; \
required int32 b; \
message msg1 {
common_fields;
required int32 c;
}
message msg2 {
common_fields;
required int32 d;
}
Does it make sense?
It probably wouldn’t be difficult to implement, but it’s not, afaik, a
design goal for protocol buffers because it is almost never (if ever)
necessary.
There are two composition approaches available, depending on what your
message Common {
required string account = 1;
required string symbol = 2;
}
message MSG1 {
required common = 1
}
message MSG2 {
required common = 1
required int32 id = 2;
}
message MSG {
required string account = 1;
required string symbol = 2;
optional Extra1 extra1 = 3;
optional Extra2 extra2 = 4;
message Extra1 {
required int32 id = 1;
}
message Extra2 {
required string foo = 1;
}
}
message MSG {
string account = 1;
string symbol = 2;
oneof extra { Extra1 extra1 = 3; Extra2 extra2 = 4;
}
message Extra1 {
int32 id = 1;
}
message Extra2 {
string foo = 1;
}
}
If composition is not what you want, then why not? What real-world problem
do you have that cannot be effectively solved with one of the above
strategies?
Understood but this is not what I wanted in the first place.
Does someone has any idea about what makes it difficult to implement this?
Also, is there a clever way to have the same behaviour?
Basically, here I want to avoid copy pasting same fields over and over
again (makes code less maintainable).
Any ideas are welcome.
message Header {
string account = 1;
string symbol = 1;
}
message Msg1 {
Header header = 1;
...
}
message Msg2 {
Header header = 1;
...
}
Thanks for the reply. What exactly do you mean by common header?
Hi,
This question is regarding inheritance in protobuf C++ library. I will
explain what I am looking for with a concrete example.
message MSG1
{
required string account = 0;
required string symbol = 1;
}
message MSG2
{
required string account = 0;
required string symbol = 1;
required int32 id = 2;
}
You will notice that first two fields of MSG2 are exactly same as MSG1
(they are intended to be like that). But here I had to copy paste the
common fields again.
Can I do something like this?
message MSG2 extends MSG1
{
required int32 id = 2;
}
message MSG2
{
required MSG1 msg1 = 0;
required int32 id = 2;
}
But this is not really what I want.
What's the best way to achieve this?
Protobuf doesn't support inheritance. Having a common header and using
composition is the best solution.
Thanks,
Saurabh
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.
​
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

[Saurabh Kumar]

This post might be inappropriate. Click to display it.

[Marcelo Cantos]

Why do they "belong" on the same level? Does containment prevent you from
building the system you want (and how?), or are we'll still really talking
about personal preference?

Post by Saurabh Kumar
Sorry about that. I had written 'personal preference' because I thought it
is hard to convey why I have to do it.
I think simplest way to convey would be, there is one proto definition say
commonmsg.proto. I do not own this and can not make any changes to this.
Now I want to define many messages (that I own) and have some additional
optional/required parameters and common ones from commonmsg.proto. So the
message MSG { required string account = 1; required string symbol = 2;
optional Extra1 extra1 = 3; optional Extra2 extra2 = 4; message Extra1 {
required int32 id = 1; } message Extra2 { required string foo = 1; } }
Nesting (containing) is not what I want, because the nature of fields is
such that it does not make much sense. All of them belong to same level.
I guess it might still not make sense to you, but you know what I am
looking for and it could be a hack :)
I know exactly what you're trying to achieve, but you haven't presented a
strong case for why you want this. Both the approaches I've suggested avoid
the copy pasting you're worried about, and personal preference isn't an
argument.
Hi,
Thanks for the suggestions below. I agree with you that these approaches
address most of the real life scenarios. What I am really looking for is
more of a syntactic sugar. In my case, I want to avoid any kind of nesting
as a personal preference. I just want to avoid copy pasting same members at
multiple places and have the overhead of keeping them updated everywhere.
I don't even mind if we had a certain kind of preprocessor to achieve this
i.e.
#define common_fields required int32 a; \
required int32 b; \
message msg1 {
common_fields;
required int32 c;
}
message msg2 {
common_fields;
required int32 d;
}
Does it make sense?
It probably wouldn’t be difficult to implement, but it’s not, afaik, a
design goal for protocol buffers because it is almost never (if ever)
necessary.
There are two composition approaches available, depending on what your
message Common {
required string account = 1;
required string symbol = 2;
}
message MSG1 {
required common = 1
}
message MSG2 {
required common = 1
required int32 id = 2;
}
message MSG {
required string account = 1;
required string symbol = 2;
optional Extra1 extra1 = 3;
optional Extra2 extra2 = 4;
message Extra1 {
required int32 id = 1;
}
message Extra2 {
required string foo = 1;
}
}
message MSG {
string account = 1;
string symbol = 2;
oneof extra { Extra1 extra1 = 3; Extra2 extra2 = 4;
}
message Extra1 {
int32 id = 1;
}
message Extra2 {
string foo = 1;
}
}
If composition is not what you want, then why not? What real-world problem
do you have that cannot be effectively solved with one of the above
strategies?
Understood but this is not what I wanted in the first place.
Does someone has any idea about what makes it difficult to implement this?
Also, is there a clever way to have the same behaviour?
Basically, here I want to avoid copy pasting same fields over and over
again (makes code less maintainable).
Any ideas are welcome.
message Header {
string account = 1;
string symbol = 1;
}
message Msg1 {
Header header = 1;
...
}
message Msg2 {
Header header = 1;
...
}
Thanks for the reply. What exactly do you mean by common header?
Hi,
This question is regarding inheritance in protobuf C++ library. I will
explain what I am looking for with a concrete example.
message MSG1
{
required string account = 0;
required string symbol = 1;
}
message MSG2
{
required string account = 0;
required string symbol = 1;
required int32 id = 2;
}
You will notice that first two fields of MSG2 are exactly same as MSG1
(they are intended to be like that). But here I had to copy paste the
common fields again.
Can I do something like this?
message MSG2 extends MSG1
{
required int32 id = 2;
}
message MSG2
{
required MSG1 msg1 = 0;
required int32 id = 2;
}
But this is not really what I want.
What's the best way to achieve this?
Protobuf doesn't support inheritance. Having a common header and using
composition is the best solution.
Thanks,
Saurabh
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.
​
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

[SangemV]

I see two problems with this approach. It is neither cut & paste nor
personal preference issue as I see. The real issues I see are:

1) The base message (MSG in the example) and the extended message (MSG2 in
the example ) can belong to different package owned by different group/org.
The package defining base message (MSG) (call it Pkg1) does not event know
the existing of the package defining extended message (MSG2) (call it
Pkg2). How can the base message foresee all the extended message from it?
Even if Pkg1 and Pkg2 are owned by the same group/org, making Pkg1 aware of
Pkg2 is not a good idea as it can potentially create cyclic dependencies.

2) Polymorphic Lists: I have a list of MSG types which can potentially have
both MSG and MSG2 types. How is this modeled in proto3 using the proposed
solution?

These are real issues which I am facing right now. Any suggestions to
handle the above problems would be of great help.

​

[Marc Gravell]

I'm going to share some thoughts here simply for discussion purposes - I
don't expect them to be directly applicable.

FWIW, protobuf-net has spoofed inheritance for many many years. I'm able to
do this because protobuf-net only needs to target .NET, which has good
inheritance support.
I don't really expect the main "protobuf" project will add inheritance in
any timescale, because it might need to target platforms that don't have
inheritance support.
It would also require ".proto" language considerations.

That said... if anyone at Google does ever want to reignite the discussion
around inheritance, I'm all ears :)

---

The way it protobuf-net does this can be *essentially* represented by
"oneof" - so going back to the original example from (wow, a long time ago):

message MSG1
{
required string account = 1;
required string symbol = 2;
}
message MSG2 extends MSG1
{
required int32 id = 2;
}

the way protobuf-net does this is essentially:

message MSG1
{
required string account = 1;
required string symbol = 2;
oneof _subtype { // keep in mind that this is actually "zero-or-one-of"
MSG2 msg2 = 3; // this field-number needs to be unique in MSG1
}
}
message MSG2
{
required int32 id = 2; // no restrictions on this field-number
}

but protobuf-net does some tricks so that when deserializing a MSG1, it
might end up *actually* creating an instance of MSG2 instead (where MSG2 :
MSG1).
For LSP purposes, everything is *always* serialized from the outside in, so
if you had a list/array of MSG2, they would still write the MSG1 fields
first - essentially
it would be like a list/array of MSG1, but where everything *happens to be*
an MSG2. You're right to say that the base type needs to know about the
derived types,
or at least have lib support to help it there, with some kind of deferred
registration.

But: I wonder whether the same approach - just without the actual
inheritance in the generated types - might still be useful to you.

Concretely, from a code-first angle:
https://gist.github.com/mgravell/a33755800e823ee77ba01183be7084df

Or maybe it is completely unhelpful and you're worse off for reading
this... who knows!

Marc

Post by SangemV
I see two problems with this approach. It is neither cut & paste nor
1) The base message (MSG in the example) and the extended message (MSG2
in the example ) can belong to different package owned by different
group/org. The package defining base message (MSG) (call it Pkg1) does not
event know the existing of the package defining extended message (MSG2)
(call it Pkg2). How can the base message foresee all the extended message
from it? Even if Pkg1 and Pkg2 are owned by the same group/org, making Pkg1
aware of Pkg2 is not a good idea as it can potentially create cyclic
dependencies.
2) Polymorphic Lists: I have a list of MSG types which can potentially
have both MSG and MSG2 types. How is this modeled in proto3 using the
proposed solution?
These are real issues which I am facing right now. Any suggestions to
handle the above problems would be of great help.

​

--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

--
Regards,

Marc

[SangemV]

I see two problems with this approach. It is neither cut & paste nor
personal preference issue as I see. The real issues I see are:

1) The base message (MSG in the example) and the extended message (MSG2 in
the example ) can belong to different package owned by different group/org.
The package defining base message (MSG) (call it Pkg1) does not event know
the existing of the package defining extended message (MSG2) (call it
Pkg2). How can the base message foresee all the extended message from it?
Even if Pkg1 and Pkg2 are owned by the same group/org, making Pkg1 aware of
Pkg2 is not a good idea as it can potentially create cyclic dependencies.

2) Polymorphic Lists: I have a list of MSG types which can potentially have
both MSG and MSG2 types. How is this modeled in proto3 using the proposed
solution?

These are real issues which I am facing right now. Any suggestions to
handle the above problems would be of great help.

Post by Marcelo Cantos
It probably wouldn’t be difficult to implement, but it’s not, afaik, a
design goal for protocol buffers because it is almost never (if ever)
necessary.
There are two composition approaches available, depending on what your
message Common {
required string account = 1;
required string symbol = 2;
}
message MSG1 {
required common = 1
}
message MSG2 {
required common = 1
required int32 id = 2;
}
message MSG {
required string account = 1;
required string symbol = 2;
optional Extra1 extra1 = 3;
optional Extra2 extra2 = 4;
message Extra1 {
required int32 id = 1;
}
message Extra2 {
required string foo = 1;
}
}
message MSG {
string account = 1;
string symbol = 2;
oneof extra { Extra1 extra1 = 3; Extra2 extra2 = 4;
}
message Extra1 {
int32 id = 1;
}
message Extra2 {
string foo = 1;
}
}
If composition is not what you want, then why not? What real-world problem
do you have that cannot be effectively solved with one of the above
strategies?
Understood but this is not what I wanted in the first place.

Post by Saurabh Kumar
Does someone has any idea about what makes it difficult to implement
this? Also, is there a clever way to have the same behaviour?
Basically, here I want to avoid copy pasting same fields over and over
again (makes code less maintainable).
Any ideas are welcome.

Post by 'Feng Xiao' via Protocol Buffers
message Header {
string account = 1;
string symbol = 1;
}
message Msg1 {
Header header = 1;
...
}
message Msg2 {
Header header = 1;
...
}

Post by Saurabh Kumar
Thanks for the reply. What exactly do you mean by common header?

Post by 'Feng Xiao' via Protocol Buffers

Post by Saurabh Kumar
Hi,
This question is regarding inheritance in protobuf C++ library. I
will explain what I am looking for with a concrete example.
message MSG1
{
required string account = 0;
required string symbol = 1;
}
message MSG2
{
required string account = 0;
required string symbol = 1;
required int32 id = 2;
}
You will notice that first two fields of MSG2 are exactly same as
MSG1 (they are intended to be like that). But here I had to copy paste the
common fields again.
Can I do something like this?
message MSG2 extends MSG1
{
required int32 id = 2;
}
message MSG2
{
required MSG1 msg1 = 0;
required int32 id = 2;
}
But this is not really what I want.
What's the best way to achieve this?

Protobuf doesn't support inheritance. Having a common header and using
composition is the best solution.

Post by Saurabh Kumar
Thanks,
Saurabh

--

Post by Saurabh Kumar
You received this message because you are subscribed to the Google
Groups "Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it,
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google
Groups "Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

​

[SangemV]

I see two problems with this approach. It is neither cut & paste nor
personal preference issue as I see. The real issues I see are:

1) The base message (MSG in the example) and the extended message (MSG2 in
the example ) can belong to different package owned by different group/org.
The package defining base message (MSG) (call it Pkg1) does not event know
the existing of the package defining extended message (MSG2) (call it
Pkg2). How can the base message foresee all the extended message from it?
Even if Pkg1 and Pkg2 are owned by the same group/org, making Pkg1 aware of
Pkg2 is not a good idea as it can potentially create cyclic dependencies.

2) Polymorphic Lists: I have a list of MSG types which can potentially have
both MSG and MSG2 types. How is this modeled in proto3 using the proposed
solution?

These are real issues which I am facing right now. Any suggestions to
handle the above problems would be of great help.

Post by Marcelo Cantos
It probably wouldn’t be difficult to implement, but it’s not, afaik, a
design goal for protocol buffers because it is almost never (if ever)
necessary.
There are two composition approaches available, depending on what your
message Common {
required string account = 1;
required string symbol = 2;
}
message MSG1 {
required common = 1
}
message MSG2 {
required common = 1
required int32 id = 2;
}
message MSG {
required string account = 1;
required string symbol = 2;
optional Extra1 extra1 = 3;
optional Extra2 extra2 = 4;
message Extra1 {
required int32 id = 1;
}
message Extra2 {
required string foo = 1;
}
}
message MSG {
string account = 1;
string symbol = 2;
oneof extra { Extra1 extra1 = 3; Extra2 extra2 = 4;
}
message Extra1 {
int32 id = 1;
}
message Extra2 {
string foo = 1;
}
}
If composition is not what you want, then why not? What real-world problem
do you have that cannot be effectively solved with one of the above
strategies?
​

['Dov Wasserman' via Protocol Buffers]

Hello, Saurabh. I understand what you're trying to do, and that it seems to
make perfect sense. Every instance of a Msg2 also has all the fields of a
Msg1, so why can't it be a special subtype with just some extra data?
Ploymorphism is meant primarily for types where the difference among
parents and subtypes is behavior, but not for differering data fields. See
here <https://en.wikipedia.org/wiki/Circle-ellipse_problem> for a more
detailed explanation why.

Post by Saurabh Kumar
Hi,
This question is regarding inheritance in protobuf C++ library. I will
explain what I am looking for with a concrete example.
message MSG1
{
required string account = 0;
required string symbol = 1;
}
message MSG2
{
required string account = 0;
required string symbol = 1;
required int32 id = 2;
}
You will notice that first two fields of MSG2 are exactly same as MSG1
(they are intended to be like that). But here I had to copy paste the
common fields again.
Can I do something like this?
message MSG2 extends MSG1
{
required int32 id = 2;
}
message MSG2
{
required MSG1 msg1 = 0;
required int32 id = 2;
}
But this is not really what I want.
What's the best way to achieve this?
Thanks,
Saurabh

[Pankaj Garg]

I think it can result in in-efficiencies. One of the core feature of
porto-buffers is to pack things efficiently. The lower value of integer
assigned to a variable ensures optimised packing and secondly the integers
have to be unique but not necessarily continuous.
With multiple level of inheritance, it will be difficult to keep track of
the integers assigned to properties in the base structures.

Post by 'Dov Wasserman' via Protocol Buffers
Hello, Saurabh. I understand what you're trying to do, and that it seems
to make perfect sense. Every instance of a Msg2 also has all the fields of
a Msg1, so why can't it be a special subtype with just some extra data?
Ploymorphism is meant primarily for types where the difference among
parents and subtypes is behavior, but not for differering data fields. See
here <https://en.wikipedia.org/wiki/Circle-ellipse_problem> for a more
detailed explanation why.

Post by Saurabh Kumar
Hi,
This question is regarding inheritance in protobuf C++ library. I will
explain what I am looking for with a concrete example.
message MSG1
{
required string account = 0;
required string symbol = 1;
}
message MSG2
{
required string account = 0;
required string symbol = 1;
required int32 id = 2;
}
You will notice that first two fields of MSG2 are exactly same as MSG1
(they are intended to be like that). But here I had to copy paste the
common fields again.
Can I do something like this?
message MSG2 extends MSG1
{
required int32 id = 2;
}
message MSG2
{
required MSG1 msg1 = 0;
required int32 id = 2;
}
But this is not really what I want.
What's the best way to achieve this?
Thanks,
Saurabh

[Pankaj Garg]

I think it can result in in-efficiencies. One of the core feature of
porto-buffers is to pack things efficiently. The lower value of integer
assigned to a variable ensures optimised packing and secondly the integers
have to be unique but not necessarily continuous.
With multiple level of inheritance, it will be difficult to keep track of
the integers assigned to properties in the base structures.

Post by Saurabh Kumar
Hi,
This question is regarding inheritance in protobuf C++ library. I will
explain what I am looking for with a concrete example.
message MSG1
{
required string account = 0;
required string symbol = 1;
}
message MSG2
{
required string account = 0;
required string symbol = 1;
required int32 id = 2;
}
You will notice that first two fields of MSG2 are exactly same as MSG1
(they are intended to be like that). But here I had to copy paste the
common fields again.
Can I do something like this?
message MSG2 extends MSG1
{
required int32 id = 2;
}
message MSG2
{
required MSG1 msg1 = 0;
required int32 id = 2;
}
But this is not really what I want.
What's the best way to achieve this?
Thanks,
Saurabh

[John Lonergan]

The flexibility and expressiveness you are after can be found in Avro.
Having been involved in a pair of big APIs recently, one using proto and the other Avro, the groups found it significant easier and more sustainable when using avro.
A key advantage of avro was abstractions.

[Venkanna Sangem]

John,
Could you please throw some light on how the problem is solved in Avro?

Thanks,

Post by John Lonergan
The flexibility and expressiveness you are after can be found in Avro.
Having been involved in a pair of big APIs recently, one using proto and
the other Avro, the groups found it significant easier and more sustainable
when using avro.
A key advantage of avro was abstractions.
--
You received this message because you are subscribed to a topic in the
Google Groups "Protocol Buffers" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/protobuf/ojpYHqx2l04/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.