--decode_raw errors with 'Failed to parse input'

[Vinay Wagh]

Hi,

I am debugging a core file in gdb and I have the raw encoded data of a
message. I want to decode this message to see the values. I tried using
--decode_raw with no success. Here is what I am doing

(gdb) p $12->send_ctx_.parts[0]
$23 = {_ = "0?)\224\002\177", '\000' <repeats 25 times>, "\200"}
(gdb) p $12->send_ctx_.parts[1]
$24 = {_ = "\000M0\224\002\177", '\000' <repeats 25 times>, "\200"}

parts is an array of serialized messages. Now I try

***@dev-01:~$ protoc --decode_raw
0?)\224\002\177
Failed to parse input.

I tried adding that data in a file and tried giving that as input

protoc --decode_raw < ~/data.txt
Failed to parse input.

I thought maybe I need to \000 repeated 25 times followed by a \200 so I
tried with that as well and still no luck. I then tried to use the decode
instead of decode_raw option by specifying the proto path and the proto
file and the message name but it still did not work.

I then tried to encode some data and decode it back to see if it works. I
entered the values in text form in a file
error:OUT_OF_MEMORY,object_id:123,size:4096

these are all valid values. I then ran the --encode option specifying the
proto path and file

protoc --proto_path=/path_to_proto_dir /path_to_proto_dir/proto_file.proto
--encode=MessageName < ~/proto.txt
?{?

I copied the text as is in to a file and rand --decode on it and it fails
with 'Failed to parse input". What am I missing ? How do I use the
decode_raw or decode option ? Is there a format the data has to be in for
it to be decoded ?

Thanks,
Vinay

[tarun sharma]

Hi Vinay

I am facing the same problem, I am reading protobuf data from a coredump
and I need to decode it to see what values it has, and I am getting the
same error ("Failed to parse input."). Did you find out a way out?

Thanks!
Tarun

Post by Vinay Wagh
Hi,
I am debugging a core file in gdb and I have the raw encoded data of a
message. I want to decode this message to see the values. I tried using
--decode_raw with no success. Here is what I am doing
(gdb) p $12->send_ctx_.parts[0]
$23 = {_ = "0?)\224\002\177", '\000' <repeats 25 times>, "\200"}
(gdb) p $12->send_ctx_.parts[1]
$24 = {_ = "\000M0\224\002\177", '\000' <repeats 25 times>, "\200"}
parts is an array of serialized messages. Now I try
0?)\224\002\177
Failed to parse input.
I tried adding that data in a file and tried giving that as input
protoc --decode_raw < ~/data.txt
Failed to parse input.
I thought maybe I need to \000 repeated 25 times followed by a \200 so I
tried with that as well and still no luck. I then tried to use the decode
instead of decode_raw option by specifying the proto path and the proto
file and the message name but it still did not work.
I then tried to encode some data and decode it back to see if it works. I
entered the values in text form in a file
error:OUT_OF_MEMORY,object_id:123,size:4096
these are all valid values. I then ran the --encode option specifying the
proto path and file
protoc --proto_path=/path_to_proto_dir /path_to_proto_dir/proto_file.proto
--encode=MessageName < ~/proto.txt
?{?
I copied the text as is in to a file and rand --decode on it and it fails
with 'Failed to parse input". What am I missing ? How do I use the
decode_raw or decode option ? Is there a format the data has to be in for
it to be decoded ?
Thanks,
Vinay

[Josh Haberman]

It looks like Vinay didn't get a response when posting in 2013. A couple
things to keep in mind though:

- protobufs are raw binary data, and they can contain NULL bytes. In
Vinay's message above, GDB is probably treating the serialized protos as
text strings, and so will stop printing at the first NULL byte. You need to
get GDB to print the entire binary proto, which will probably require
having an explicit length for the string and passing that to GDB's printing
routine somehow.

- protoc --decode_raw expects its input in binary format. So passing it
input like 0?)\224\002\177 won't do the right thing -- that is a text
rendering of binary data. You could use printf to convert this to real
binary data:

$ printf "0?)\224\002\177" > binary_data

But that proto still fails to parse with --decode_raw, probably because it
is incomplete due to a NULL byte in the middle of the proto.

Hope this helps,
Josh

Post by tarun sharma
Hi Vinay
I am facing the same problem, I am reading protobuf data from a coredump
and I need to decode it to see what values it has, and I am getting the
same error ("Failed to parse input."). Did you find out a way out?
Thanks!
Tarun

Post by Vinay Wagh
Hi,
I am debugging a core file in gdb and I have the raw encoded data of a
message. I want to decode this message to see the values. I tried using
--decode_raw with no success. Here is what I am doing
(gdb) p $12->send_ctx_.parts[0]
$23 = {_ = "0?)\224\002\177", '\000' <repeats 25 times>, "\200"}
(gdb) p $12->send_ctx_.parts[1]
$24 = {_ = "\000M0\224\002\177", '\000' <repeats 25 times>, "\200"}
parts is an array of serialized messages. Now I try
0?)\224\002\177
Failed to parse input.
I tried adding that data in a file and tried giving that as input
protoc --decode_raw < ~/data.txt
Failed to parse input.
I thought maybe I need to \000 repeated 25 times followed by a \200 so I
tried with that as well and still no luck. I then tried to use the decode
instead of decode_raw option by specifying the proto path and the proto
file and the message name but it still did not work.
I then tried to encode some data and decode it back to see if it works. I
entered the values in text form in a file
error:OUT_OF_MEMORY,object_id:123,size:4096
these are all valid values. I then ran the --encode option specifying the
proto path and file
protoc --proto_path=/path_to_proto_dir
/path_to_proto_dir/proto_file.proto --encode=MessageName < ~/proto.txt
?{?
I copied the text as is in to a file and rand --decode on it and it fails
with 'Failed to parse input". What am I missing ? How do I use the
decode_raw or decode option ? Is there a format the data has to be in for
it to be decoded ?
Thanks,
Vinay