Denys Yurchenko | 3 Dec 00:26 2012
Picon

Re: Multiple messages using std::streams

Brilliant post!


I had spent a lot of time to solve a problem of getting messages from a boost::asio::ip::tcp::iostream without closing it.

assert(messageSize > 0) NOT good for me. I use condition messageSize == 0 in order to know if boost::asio::ip::tcp::iostream A.K.A TCP/IP connection has been closed.

On Saturday, February 5, 2011 5:54:06 AM UTC+2, Gabriel Becedillas wrote:
I had a hard time trying to serialize multiple messages in a
std::ostream, and then trying to parse them one at a time from an
std::istream, so I'm sharing my code for two reasons:
- It may be useful for someone else.
- To receive comments from more experienced people.

template <typename TMessage>
bool serialize_delimited(std::ostream& stream, TMessage& message)
{
        assert(message.IsInitialized());

        google::protobuf::io::OstreamOutputStream ostreamWrapper(&stream);
        google::protobuf::io::CodedOutputStream
codedOStream(&ostreamWrapper);

        // Write the message size first.
        int messageSize = message.ByteSize();
        assert(messageSize > 0);
        codedOStream.WriteLittleEndian32(messageSize);

        // Write the message.
        message.SerializeWithCachedSizes(&codedOStream);

        return stream.good();
}

template <typename TMessage>
bool parse_delimited(std::istream& stream, TMessage& message)
{
        uint32_t messageSize = 0;

        // Read the message size.
        {
                google::protobuf::io::IstreamInputStream istreamWrapper(&stream,
sizeof(uint32_t));
                google::protobuf::io::CodedInputStream
codedIStream(&istreamWrapper);

                // Don't consume more than sizeof(uint32_t) from the stream.
                google::protobuf::io::CodedInputStream::Limit oldLimit =
codedIStream.PushLimit(sizeof(uint32_t));
                codedIStream.ReadLittleEndian32(&messageSize);
                codedIStream.PopLimit(oldLimit);
                assert(messageSize > 0);
                assert(istreamWrapper.ByteCount() == sizeof(uint32_t));
        }

        // Read the message.
        {
                google::protobuf::io::IstreamInputStream istreamWrapper(&stream,
messageSize);
                google::protobuf::io::CodedInputStream
codedIStream(&istreamWrapper);

                // Read the message, but don't consume more than messageSize bytes
from the stream.
                google::protobuf::io::CodedInputStream::Limit oldLimit =
codedIStream.PushLimit(messageSize);
                message.ParseFromCodedStream(&codedIStream);
                codedIStream.PopLimit(oldLimit);
                assert(istreamWrapper.ByteCount() == messageSize);
        }

        return stream.good();
}

serialize_delimited is pretty simple. I write the message size first,
and then the message.

The tricky part was parsing from a stream. The problem that I was
facing was that after CodedInputStream got constructed, my stream was
consumed more than I wanted to. The first message was parsed
correctly, but for the rest the stream was not in the right position.
 The only way I found to avoid that was by setting the block_size
parameter while constructing IstreamInputStream instances, and that is
why I construct them twice (first to parse the size, then to parse the
message).

--
You received this message because you are subscribed to the Google Groups "Protocol Buffers" group.
To view this discussion on the web visit https://groups.google.com/d/msg/protobuf/-/Q1w-LooQPy4J.
To post to this group, send email to protobuf <at> googlegroups.com.
To unsubscribe from this group, send email to protobuf+unsubscribe <at> googlegroups.com.
For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.

Gmane