This file deals with readahead from caller-supplied buffers. More...

Include dependency graph for scoop.c:

Functions
static void	rs_scoop_input (rs_job_t *job, size_t len)
	Try to accept a from the input buffer to get LEN bytes in the scoop. More...

void	rs_scoop_advance (rs_job_t *job, size_t len)
	Advance the input cursor forward `len` bytes. More...

rs_result	rs_scoop_readahead (rs_job_t job, size_t len, void *ptr)
	Read from scoop without advancing. More...

rs_result	rs_scoop_read (rs_job_t job, size_t len, void *ptr)
	Read LEN bytes if possible, and remove them from the input scoop. More...

rs_result	rs_scoop_read_rest (rs_job_t job, size_t len, void **ptr)
	Read whatever data remains in the input stream. More...

Detailed Description

This file deals with readahead from caller-supplied buffers.

Many functions require a certain minimum amount of contiguous input data to do their processing. For example, to calculate a strong checksum of a block we need at least a block of input.

Since we put the buffers completely under the control of the caller, we can't count on ever getting this much data all in one go. We can't simply wait, because the caller might have a smaller buffer than we require and so we'll never get it.

Stream input data is used directly if there is sufficient data to satisfy the readhead requests, otherwise it is copied and accumulated into an internal buffer until there is enough. This means for large input buffers we can leave a "tail" of unprocessed data in the input buffer, and only consume all the data if it was too small and start accumulating into the internal buffer. Provided the input buffers always have enough data we avoid copying into the internal buffer at all.

Todo:: We probably know a maximum amount of data that can be scooped up, so we could just avoid dynamic allocation. However that can't be fixed at compile time, because when generating a delta it needs to be large enough to hold one full block. Perhaps we can set it up when the job is allocated? It would be kind of nice to not do any memory allocation after startup, as bzlib does this.

Definition in file scoop.c.

Function Documentation

◆ rs_scoop_input()

static void rs_scoop_input	(	rs_job_t *	job,
		size_t	len
	)

inlinestatic

Try to accept a from the input buffer to get LEN bytes in the scoop.

Definition at line 66 of file scoop.c.

◆ rs_scoop_advance()

void rs_scoop_advance	(	rs_job_t *	job,
		size_t	len
	)

Advance the input cursor forward len bytes.

This is used after doing readahead, when you decide you want to keep it. len must be no more than the amount of available data, so you can't cheat.

So when creating a delta, we require one block of readahead. But after examining that block, we might decide to advance over all of it (if there is a match), or just one byte (if not).

Definition at line 117 of file scoop.c.

◆ rs_scoop_readahead()

rs_result rs_scoop_readahead	(	rs_job_t *	job,
		size_t	len,
		void **	ptr
	)

Read from scoop without advancing.

Ask for LEN bytes of input from the stream. If that much data is available, then return a pointer to it in PTR, advance the stream input pointer over the data, and return RS_DONE. If there's not enough data, then accept whatever is there into a buffer, advance over it, and return RS_BLOCKED.

The data is not actually removed from the input, so this function lets you do readahead. If you want to keep any of the data, you should also call rs_scoop_advance() to skip over it.

Definition at line 148 of file scoop.c.

◆ rs_scoop_read()

rs_result rs_scoop_read	(	rs_job_t *	job,
		size_t	len,
		void **	ptr
	)

Read LEN bytes if possible, and remove them from the input scoop.

Parameters

*job	An rs_job_t pointer to the job instance.
len	The length of the data in the ptr buffer.
**ptr	will be updated to point to a read-only buffer holding the data, if enough is available.

Returns: RS_DONE if there was enough data, RS_BLOCKED if there was not enough data yet, or RS_INPUT_ENDED if there was not enough data and at EOF.

Definition at line 192 of file scoop.c.

◆ rs_scoop_read_rest()

rs_result rs_scoop_read_rest	(	rs_job_t *	job,
		size_t *	len,
		void **	ptr
	)

Read whatever data remains in the input stream.