Dynamically allocated buffer for Gio::InputStream

Currently, I’m using this method to read socket response:

In Rust, code example looks like:

connection.input_stream().read_all_async(
    vec![0; 0xfffff],
    Priority::DEFAULT,
    // ...

but it’s limited to 1Mb, how can I make this method working with minimum capacity on start but growable depending of content size? maybe some specialized methods exist in Glib for this task

If you’re not sure of how much content is going to be read, the safe way to go about things is to read in chunks in a loop. Otherwise you are providing a direct way for whoever’s in control of the other end of the socket to do a denial of service attack on your software by sending a very large response, which would trigger a very large allocation and memory exhaustion.

So, for example, call Gio.InputStream.read_async() in a loop until the end of the socket (or whatever other termination condition) is reached, reading somewhere between 8KB and 1MB each time depending on what kind of data you’d expect on average. You will then need to structure the code which handles this data to parse it / handle it in chunks.

Depending on the code which handles the data, you may want to restructure that code as a Gio.OutputStream. Then you can splice the socket code (Gio.InputStream) to the handler code (Gio.OutputStream) using Gio.OutputStream.splice_async() and that will implement the read and buffer loop for you.

1 Like

Thanks much for idea with chunks!

really, forgot about potential memory issues without incoming size validation. Will make it with Cursor if there is nothing for Gio/Glib yet.

From a quick glance, the GIO equivalent of Cursor is Gio.MemoryInputStream/Gio.MemoryOutputStream.

1 Like

Found also ByteArray that could be useful for similar purposes

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.