|
librsync
2.3.4
|
Manage librsync streams of IO. More...
Go to the source code of this file.
Functions | |
| rs_result | rs_tube_catchup (rs_job_t *job) |
| Put whatever will fit from the tube into the output of the stream. More... | |
| int | rs_tube_is_idle (rs_job_t const *job) |
| void | rs_tube_write (rs_job_t *job, void const *buf, size_t len) |
| Push some data into the tube for storage. More... | |
| void | rs_tube_copy (rs_job_t *job, size_t len) |
Queue up a request to copy through len bytes from the input to the output of the stream. More... | |
| void | rs_scoop_advance (rs_job_t *job, size_t len) |
Advance the input cursor forward len bytes. More... | |
| rs_result | rs_scoop_readahead (rs_job_t *job, size_t len, void **ptr) |
| Read from scoop without advancing. More... | |
| rs_result | rs_scoop_read (rs_job_t *job, size_t len, void **ptr) |
| Read LEN bytes if possible, and remove them from the input scoop. More... | |
| rs_result | rs_scoop_read_rest (rs_job_t *job, size_t *len, void **ptr) |
| Read whatever data remains in the input stream. More... | |
| static size_t | rs_scoop_avail (rs_job_t *job) |
| static bool | rs_scoop_eof (rs_job_t *job) |
| Test if the scoop has reached eof. More... | |
| static void * | rs_scoop_buf (rs_job_t *job) |
| Get a pointer to the next input in the scoop. More... | |
| static size_t | rs_scoop_len (rs_job_t *job) |
| Get the contiguous length of the next input in the scoop. More... | |
| static void * | rs_scoop_getbuf (rs_job_t *job, size_t *len) |
| Get the next contiguous buffer of data available in the scoop. More... | |
| static void * | rs_scoop_iterbuf (rs_job_t *job, size_t *len, size_t *ilen) |
| Iterate through and consume contiguous data buffers in the scoop. More... | |
| static void * | rs_scoop_nextbuf (rs_job_t *job, size_t *len, size_t *ilen) |
| Get the next iteration of contiguous data buffers from the scoop. More... | |
Manage librsync streams of IO.
See
OK, so I'll admit IO here is a little complex. The most important player here is the stream, which is an object for managing filter operations. It has both input and output sides, both of which is just a (pointer,len) pair into a buffer provided by the client. The code controlling the stream handles however much data it wants, and the client provides or accepts however much is convenient.
At the same time as being friendly to the client, we also try to be very friendly to the internal code. It wants to be able to ask for arbitrary amounts of input or output and get it without having to keep track of partial completion. So there are functions which either complete, or queue whatever was not sent and return RS_BLOCKED.
The output buffer is a little more clever than simply a data buffer. Instead it knows that we can send either literal data, or data copied through from the input of the stream.
In buf.c you will find functions that then map buffers onto stdio files.
On return from an encoding function, some input will have been consumed and/or some output produced, but either the input or the output or possibly both could have some remaining bytes available.
librsync never does IO or memory allocation for stream input, but relies on the caller. This is very nice for integration, but means that we have to be fairly flexible as to when we can ‘read’ or ‘write’ stuff internally.
librsync basically does two types of IO. It reads network integers of various lengths which encode command and control information such as versions and signatures. It also does bulk data transfer.
IO of network integers is internally buffered, because higher levels of the code need to see them transmitted atomically: it's no good to read half of a uint32. So there is a small and fixed length internal buffer which accumulates these. Unlike previous versions of the library, we don't require that the caller hold the start until the whole thing has arrived, which guarantees that we can always make progress.
On each call into a stream iterator, it should begin by trying to flush output. This may well use up all the remaining stream space, in which case nothing else can be done.
Definition in file scoop.h.
| void rs_tube_write | ( | rs_job_t * | job, |
| const void * | buf, | ||
| size_t | len | ||
| ) |
| void rs_tube_copy | ( | rs_job_t * | job, |
| size_t | len | ||
| ) |
Queue up a request to copy through len bytes from the input to the output of the stream.
The data is copied from the scoop (if there is anything there) or from the input, on the next call to rs_tube_write().
We can only accept this request if there is no copy command already pending.
| void rs_scoop_advance | ( | rs_job_t * | job, |
| size_t | len | ||
| ) |
Advance the input cursor forward len bytes.
This is used after doing readahead, when you decide you want to keep it. len must be no more than the amount of available data, so you can't cheat.
So when creating a delta, we require one block of readahead. But after examining that block, we might decide to advance over all of it (if there is a match), or just one byte (if not).
Read from scoop without advancing.
Ask for LEN bytes of input from the stream. If that much data is available, then return a pointer to it in PTR, advance the stream input pointer over the data, and return RS_DONE. If there's not enough data, then accept whatever is there into a buffer, advance over it, and return RS_BLOCKED.
The data is not actually removed from the input, so this function lets you do readahead. If you want to keep any of the data, you should also call rs_scoop_advance() to skip over it.
Read LEN bytes if possible, and remove them from the input scoop.
| *job | An rs_job_t pointer to the job instance. |
| len | The length of the data in the ptr buffer. |
| **ptr | will be updated to point to a read-only buffer holding the data, if enough is available. |
Read whatever data remains in the input stream.
| *job | The rs_job_t instance the job instance. |
| *len | will be updated to the length of the available data. |
| **ptr | will point at the available data. |
|
inlinestatic |
|
inlinestatic |
|
inlinestatic |
|
inlinestatic |
Get the next contiguous buffer of data available in the scoop.
This will return a pointer to the data and reduce len to the amount of contiguous data available at that position.
| *job | - the job instance to use. |
| *len | - the amount of data desired, updated to the amount available. |
|
inlinestatic |
Iterate through and consume contiguous data buffers in the scoop.
Example:
At each iteration buf and ilen are the data and its length for the current iteration, and len is the remaining data to iterate through including the current iteration. During an iteration you can change ilen to indicate only part of the buffer was processed and the next iteration will take this into account. Setting ilen = 0 to indicate blocking or errors will terminate iteration.
At the end of iteration buf will point at the next location in the scoop after the iterated data, len and ilen will be zero, or the remaining data and last ilen if iteration was terminated by setting ilen = 0.
| *job | - the job instance to use. |
| *len | - the size_t amount of data to iterate over. |
| *ilen | - the size_t amount of data in the current iteration. |
|
inlinestatic |
Get the next iteration of contiguous data buffers from the scoop.
This advances the scoop for the previous iteration, and gets the next iteration.
1.9.4