Issue 593 in protobuf: FatalException on calling SerializeAsString()

[p***@googlecode.com]

Status: New
Owner: ***@google.com
Labels: Type-Defect Priority-Medium

New issue 593 by ***@posteo.de: FatalException on calling
SerializeAsString()
http://code.google.com/p/protobuf/issues/detail?id=593

Constellation:
Protobuf 2.5.0
Ubuntu 12.04 LTS
C++ API

Problem:
Protobuf randomly throws a FatalException on calling SerializeAsString()
with this output:
[libprotobuf FATAL google/protobuf/message_lite.cc:224] CHECK
failed: !coded_out.HadError():
terminate called after throwing an instance
of 'google::protobuf::FatalException'
what(): CHECK failed: !coded_out.HadError()

This happens on every 10th or 100th call of SerializeAsString().

This is the source code in which the SerializeAsString is called:

QByteArray ProtobufSerializer::serialize(AbstractTransferable*
transferable) throw(NexusDataLinkException) {
ProtoEnvelop envelop;
ProtoPayloadWrapper* wrapper = envelop.add_payloadwrapper();

ChannelResponse* response = static_cast<ChannelResponse*>(transferable);
ProtoResponse*
allocatedProtoResponse(wrapper->mutable_channelresponse());
populateProtoResponse(response, allocatedProtoResponse,
transferable->getDataContainer());

envelop.set_destination_id(CommonDefinitions::SERVER_ID);
envelop.set_source_id(agentId);

// convert to byte array
std::string dataString = envelop.SerializeAsString();
QByteArray data = QByteArray(dataString.data(), dataString.size()); //
makes a deep copy of the data contained in dataString

return data;
}

What could be the cause of this error? I could not find anything similar on
google & Co.

Kind regards,
Manfred
--
You received this message because this project is configured to send all
issue notifications to this address.
You may adjust your notification preferences at:
https://code.google.com/hosting/settings

[p***@googlecode.com]

Comment #1 on issue 593 by ***@posteo.de: FatalException on calling
SerializeAsString()
http://code.google.com/p/protobuf/issues/detail?id=593

Additional notes:
The error seems to happen only for embedded messages that contain a bool
value. Other embedded messages seem to work as expected
--
You received this message because this project is configured to send all
issue notifications to this address.
You may adjust your notification preferences at:
https://code.google.com/hosting/settings

[p***@googlecode.com]

Comment #2 on issue 593 by ***@posteo.de: FatalException on calling
SerializeAsString()
http://code.google.com/p/protobuf/issues/detail?id=593

Did such an error occur before? Please let me know if you require
additional information or the .proto-files
--
You received this message because this project is configured to send all
issue notifications to this address.
You may adjust your notification preferences at:
https://code.google.com/hosting/settings

[p***@googlecode.com]

Comment #3 on issue 593 by ***@google.com: FatalException on calling
SerializeAsString()
http://code.google.com/p/protobuf/issues/detail?id=593

Program throwing exceptions indeterminately suggests multi-threading bugs.
--
You received this message because this project is configured to send all
issue notifications to this address.
You may adjust your notification preferences at:
https://code.google.com/hosting/settings

[p***@googlecode.com]

Comment #4 on issue 593 by ***@gmail.com: FatalException on calling
SerializeAsString()
http://code.google.com/p/protobuf/issues/detail?id=593


This exact same issue occurs for me too - and when I replace the usage of
all bools with uint32s, the problem goes away.

Looking through the source code, it seems that the only way in which the
_has_error flag can be set is if _Next returns false. The check that dies
is in message_lite.cc:224, and from a cursor examination the only code path
that would cause the error is if the return value of GetCachedSize() is
insufficient to serialise the message.

The reason this happens with bools is that a bool is always considered to
be one byte, but if your bool value is unitialised (as mine was) then when
protobuf actually tries to serialise the value it may end up using more
than one byte.

IMHO protobuf should assert if you try to set a value other than 0 or 1
into a bool. This would catch the error where it actually occurs, rather
than later on when it attempts to serialise the data.
--
You received this message because this project is configured to send all
issue notifications to this address.
You may adjust your notification preferences at:
https://code.google.com/hosting/settings

[ganumurali]

I recently observed the same.
Stack trace is below for reference. Following the suggestions in this
thread, changing bool -> int solved the problem.

Do we know if this issue needs to be addressed by changing bool->int or
protobuf library already has a fix for it?

"Program received signal SIGABRT, Aborted.
[Switching to Thread 0x7fffeca9c700 (LWP 2181)]
0x0000003f2f234ff9 in __GI_raise (sig=***@entry=6)
at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
56 return INLINE_SYSCALL (tgkill, 3, pid, selftid, sig);
(gdb) where
#0 0x0000003f2f234ff9 in __GI_raise (sig=***@entry=6)
at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1 0x0000003f2f238068 in __GI_abort () at abort.c:89
#2 0x00007ffff2f325b5 in _*gnu_cxx::*_verbose_terminate_handler ()
at
/data/users/test/workspace/daily/poky/build/tmp/work-shared/gcc-4.8.2-r0/gcc-4.8.2/libstdc+
-v3/libsupc+/vterminate.cc:95
#3 0x00007ffff2f30726 in _*cxxabiv1::*_terminate (handler=<optimized out>)
at
/data/users/test/workspace/daily/poky/build/tmp/work-shared/gcc-4.8.2-r0/gcc-4.8.2/libstdc+
-v3/libsupc+/eh_terminate.cc:38
#4 0x00007ffff2f30753 in std::terminate ()
at
/data/users/test/workspace/daily/poky/build/tmp/work-shared/gcc-4.8.2-r0/gcc-4.8.2/libstdc+
-v3/libsupc+/eh_terminate.cc:48
#5 0x00007ffff2f3097e in _*cxxabiv1::*_cxa_throw (obj=0x7fffdc028600,
tinfo=<optimized out>, dest=<optimized out>)
at
/data/users/test/workspace/daily/poky/build/tmp/work-shared/gcc-4.8.2-r0/gcc-4.8.2/libstdc+
-v3/libsupc+/eh_throw.cc:84
#6 0x00007ffff70ba618 in google::protobuf::internal::LogMessage::Finish (
this=0x7fffeca9b8f0) at google/protobuf/stubs/common.cc:195
#7 0x00007ffff70ba648 in google::protobuf::internal::LogFinisher::operator=
(
this=***@entry=0x7fffeca9b8af, other=...)
at google/protobuf/stubs/common.cc:203
#8 0x00007ffff70ce0da in
google::protobuf::MessageLite::SerializeWithCachedSizesToArray
(this=<optimized out>, target=0x7fffdc027c18 "\n\020\b")
--Type <return> to continue, or q <return> to quit--
at google/protobuf/message_lite.cc:224
#9 0x00007ffff70cdb91 in
google::protobuf::MessageLite::AppendPartialToString
(this=***@entry=0x7fffeca9ba30, output=***@entry=0x7fffeca9ba20)
at google/protobuf/message_lite.cc:283
#10 0x00007ffff70cdbf5 in google::protobuf::MessageLite::AppendToString (
this=***@entry=0x7fffeca9ba30, output=***@entry=0x7fffeca9ba20)
at google/protobuf/message_lite.cc:274
#11 0x00007ffff70cdd40 in google::protobuf::MessageLite::SerializeToString (
this=***@entry=0x7fffeca9ba30, output=***@entry=0x7fffeca9ba20)
at google/protobuf/message_lite.cc:292"
"

Post by p***@googlecode.com
SerializeAsString()
http://code.google.com/p/protobuf/issues/detail?id=593
This exact same issue occurs for me too - and when I replace the usage of
all bools with uint32s, the problem goes away.
Looking through the source code, it seems that the only way in which the
_has_error flag can be set is if _Next returns false. The check that dies
is in message_lite.cc:224, and from a cursor examination the only code path
that would cause the error is if the return value of GetCachedSize() is
insufficient to serialise the message.
The reason this happens with bools is that a bool is always considered to
be one byte, but if your bool value is unitialised (as mine was) then when
protobuf actually tries to serialise the value it may end up using more
than one byte.
IMHO protobuf should assert if you try to set a value other than 0 or 1
into a bool. This would catch the error where it actually occurs, rather
than later on when it attempts to serialise the data.
--
You received this message because this project is configured to send all
issue notifications to this address.
https://code.google.com/hosting/settings

[ganumurali]

I just observed the same and by following the workaround (changing
bool->int) solved it but...

Do we always have to do that or protobuf library has a fix for this?
If so which version?

"
at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
56 return INLINE_SYSCALL (tgkill, 3, pid, selftid, sig);
(gdb) where
#0 0x0000003f2f234ff9 in __GI_raise (sig=***@entry=6)
at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1 0x0000003f2f238068 in __GI_abort () at abort.c:89
#2 0x00007ffff2f325b5 in _gnu_cxx::_verbose_terminate_handler ()
at
/data/users/test/workspace/daily/poky/build/tmp/work-shared/gcc-4.8.2-r0/gcc-4.8.2/libstdc+-v3/libsupc+/vterminate.cc:95
#3 0x00007ffff2f30726 in _cxxabiv1::_terminate (handler=<optimized out>)
at
/data/users/test/workspace/daily/poky/build/tmp/work-shared/gcc-4.8.2-r0/gcc-4.8.2/libstdc+-v3/libsupc+/eh_terminate.cc:38
#4 0x00007ffff2f30753 in std::terminate ()
at
/data/users/test/workspace/daily/poky/build/tmp/work-shared/gcc-4.8.2-r0/gcc-4.8.2/libstdc+-v3/libsupc+/eh_terminate.cc:48
#5 0x00007ffff2f3097e in _cxxabiv1::_cxa_throw (obj=0x7fffdc028600,
tinfo=<optimized out>, dest=<optimized out>)
at
/data/users/test/workspace/daily/poky/build/tmp/work-shared/gcc-4.8.2-r0/gcc-4.8.2/libstdc+-v3/libsupc+/eh_throw.cc:84
#6 0x00007ffff70ba618 in google::protobuf::internal::LogMessage::Finish (
this=0x7fffeca9b8f0) at google/protobuf/stubs/common.cc:195
#7 0x00007ffff70ba648 in google::protobuf::internal::LogFinisher::operator=
(
this=***@entry=0x7fffeca9b8af, other=...)
at google/protobuf/stubs/common.cc:203
#8 0x00007ffff70ce0da in
google::protobuf::MessageLite::SerializeWithCachedSizesToArray
(this=<optimized out>, target=0x7fffdc027c18 "\n\020\b")
--Type <return> to continue, or q <return> to quit--
at google/protobuf/message_lite.cc:224
#9 0x00007ffff70cdb91 in
google::protobuf::MessageLite::AppendPartialToString
(this=***@entry=0x7fffeca9ba30, output=***@entry=0x7fffeca9ba20)
at google/protobuf/message_lite.cc:283
#10 0x00007ffff70cdbf5 in google::protobuf::MessageLite::AppendToString (
this=***@entry=0x7fffeca9ba30, output=***@entry=0x7fffeca9ba20)
at google/protobuf/message_lite.cc:274
#11 0x00007ffff70cdd40 in google::protobuf::MessageLite::SerializeToString (
this=***@entry=0x7fffeca9ba30, output=***@entry=0x7fffeca9ba20)
at google/protobuf/message_lite.cc:292"
"

Post by p***@googlecode.com
SerializeAsString()
http://code.google.com/p/protobuf/issues/detail?id=593
This exact same issue occurs for me too - and when I replace the usage of
all bools with uint32s, the problem goes away.
Looking through the source code, it seems that the only way in which the
_has_error flag can be set is if _Next returns false. The check that dies
is in message_lite.cc:224, and from a cursor examination the only code path
that would cause the error is if the return value of GetCachedSize() is
insufficient to serialise the message.
The reason this happens with bools is that a bool is always considered to
be one byte, but if your bool value is unitialised (as mine was) then when
protobuf actually tries to serialise the value it may end up using more
than one byte.
IMHO protobuf should assert if you try to set a value other than 0 or 1
into a bool. This would catch the error where it actually occurs, rather
than later on when it attempts to serialise the data.
--
You received this message because this project is configured to send all
issue notifications to this address.
https://code.google.com/hosting/settings

[lizhang zhan]

Thanks for your explanation. We just encountered this kind of issue.
However, I am not sure whether I have understand your solution

1. Should we replace the useage of all bools with uint32s in the proto
files involved?
2. Have this issue fixed in the latest version?

Thanks for your replay.

Post by p***@googlecode.com
SerializeAsString()
http://code.google.com/p/protobuf/issues/detail?id=593
This exact same issue occurs for me too - and when I replace the usage of
all bools with uint32s, the problem goes away.
Looking through the source code, it seems that the only way in which the
_has_error flag can be set is if _Next returns false. The check that dies
is in message_lite.cc:224, and from a cursor examination the only code path
that would cause the error is if the return value of GetCachedSize() is
insufficient to serialise the message.
The reason this happens with bools is that a bool is always considered to
be one byte, but if your bool value is unitialised (as mine was) then when
protobuf actually tries to serialise the value it may end up using more
than one byte.
IMHO protobuf should assert if you try to set a value other than 0 or 1
into a bool. This would catch the error where it actually occurs, rather
than later on when it attempts to serialise the data.
--
You received this message because this project is configured to send all
issue notifications to this address.
https://code.google.com/hosting/settings