Re: [gnet-devel] gnet_conn_readn does not read the specified amount of bytes

[Fred Labrosse <ffl aber ac uk>]

On Tuesday 10 February 2009, Tim-Philipp Müller wrote:
> On Tue, 2009-02-10 at 14:55 +0000, Fred Labrosse wrote:
>
> Hi Fred,
>
> > I just realised that gnet_conn_readn does not necessarily read the
> > specified number of bytes. In one case (reproducible) it reads more
> > (everything available on the socket) than the specified amount and in
> > the following read, reads less than the specified amount (because some
> > had been taken by the previous readn).
>
> I'm not sure I follow. Does your callback get called with a sufficient
> number of bytes as data in each case, or not? FWIW, I'm not absolutely
> certain that reads are performed in the requested chunking in the layers
> below GConn (it's all a bit messy), so if you look at the reads with
> strace or so you might well find it does things differently.

What is happening is that I read data from a socket in chunks, the first chunk 
telling me how much I need to read for the second chunk.  Chunks are sent to 
me by a server on a more or less continuous flow.  The first chunk is for now 
2 bytes long and the second chunk is (for now too) fixed at 1 byte long.  In 
other words, I am supposed to have a succession of readn()'s that will read 2, 
then 1, then 2, then 1, etc., bytes.  In most situations, in between each pair 
of chunks (2 then 1 bytes) I send something to the server.  While this is 
happening, all is fine.  However, now and then, following user input, I need 
to not send anything to the server in between 2 chunks and therefore read 2, 
then 1, then 2, then 1 bytes (again without a write in the middle).  In this 
situation, all the data is available in the socket and when I read the first 1 
byte long chunk (with a readn(gconn 1)), I actually get 2 bytes according to 
the GConnEvent length field I receive in the callback function.  The following 
readn(gcon,2) only reads 1 byte (instead of 2).  At this point I realise that 
I am actually missing the final byte, but this is probably another problem.

I have checked the data I read and it is all there in the right order.  My 
problem is that the length of the various chunks is wrong.

>
> I'd be a bit surprised if this API didn't work right, since I've used it
> heavily in some of my apps and never had any issues reported in this
> respect (binary protocols), at least with normal linux desktop systems.

So you confirm that gnet_conn_readn() should indeed only read the specified 
number of bytes?

>
> > This is using gnet-2.0.7.  Is 2.0.8 likely to fix the problem? 
> > Unfortunately, installing 2.0.8 is not trivial (I would have to create
> > debian packages cross- compiled for an ARM-based platform).  So before
> > attempting it, I would prefer to have some confirmation that this would
> > help.
>
> I don't remember any fixes which might affect this, but it's a good idea
> to use 2.0.8 in any case - IIRC there were a few invalid memory accesses
> or race conditions in code paths which might be used in this case.
>
> Do you have a test program by any chance? Can you reproduce the issue on
> your desktop system?

I can't really do that because the whole thing uses an environment that is 
proper to the ARM environment.  However, all the tests I am doing at the 
moment run in a scratchbox environment on my laptop.

Reproducing the problem in another program is going to be quite some work...  
I'll try to do something.

Cheers,

Fred