net.Conn reading race

[maxsupermanhd]

So I implemented my proxy and I am trying to parse copies of packets that are passed through (to avoid modifying them by chance) in different gorutine. For some reason sometimes I get straight incorrect data from packets even though id is correct and packet seems to be correct too (not getting connection lost on connected client).

Here is example of how it is breaking itself: (parsing chunk and light packet)

Sources: https://github.com/maxsupermanhd/WebChunk/blob/master/proxy/proxy.go#L245

[maxsupermanhd]

I checked with -race flag and it seems like net.Conn.ReadPacket inside bot's HandleGame is still writing to the packet while it is being read by my gorutine that received it through channel in generic event handler, makes no sense for me, how ReadPacket is not blocking until writing operation finished?

[maxsupermanhd]

Instead of sending to channel packet structure I decided to reassemble it and make a new slice of bytes, looks like it is working now.

[Tnze]

All data is only available in the context which give the data to you. And the buffer will be reused in the next receiving.

You are exporting the data without copying the data, which is not allowed.

When introducing multithreading and asynchrony, it's your own responsibility to ensure that there is no data race.

[maxsupermanhd]

You are exporting the data without copying the data, which is not allowed.

I thought I am making full object copy, turns out actual slice buffer gets reused, only header is copied. I made sure that I specifically create new buffer with copied data in same thread and reassembled packet: maxsupermanhd/WebChunk@94ac5df

[Tnze]

Reusing buffers is a very important optimization method, and allocating memory is a huge overhead, especially for infrastructure such as network library, which needs to pay great attention to performance.

[Tnze]

I suggest you reconsider whether to introduce goroutine and channel. I don't think it's necessary for you.

[Tnze]

is it really necessary to copy the packet?

[maxsupermanhd]

The point is that I don't want to block all packet forwarding, in the proxy I am making I also cache packets and do stuff with them. I feel like it should be done in separate thread.

[Tnze]

You should always keep the sequence of packet

[maxsupermanhd]

They are not processed in parallel, still one processing thread but it must not block packet forwarding at any time. If forwarding will be delayed with processing then it will affect gameplay lag significantly.