parser_token_buffer

Defined in header: <orcus/detail/parser_token_buffer.hpp>

template<typename TokensT>
class parser_token_buffer

Class that manages synchronization of parser tokens used in multi-threaded parsers.

Public Functions

inline parser_token_buffer(size_t min_token_size, size_t max_token_size)

inline void check_and_notify(tokens_type &parser_tokens)

Check the size of the parser token buffer, and if it exceeds specified threshold, move it to the client buffer.

Call this from the parser thread.

Parameters:

parser_tokens – parser token buffer.

inline void notify_and_finish(tokens_type &parser_tokens)

Move the current parser token buffer to the client buffer, and signal the end of parsing.

Call this from the parser thread.

Parameters:

parser_tokens – parser token buffer.

inline void abort()

inline bool next_tokens(tokens_type &tokens)

Retrieve the tokens currently in the client token buffer.

Call this from the client (non-parser) thread.

Parameters:

tokens – place to move the tokens in the client token buffer to.

Returns:

true if the parsing is still in progress, therefore more tokens are expected, false if this is the last set of tokens.

inline size_t token_size_threshold() const

Return the current token size threshold. Call this only after the parsing has finished.

Returns:

current token size threshold.