protobuf-c-text
Library to generate & parse text format protobufs in C.
 All Data Structures Files Functions Variables Enumerations Enumerator Macros Groups
Data Structures | Macros | Enumerations | Functions
Routines related to lexing text format protobufs
Collaboration diagram for Routines related to lexing text format protobufs:

Data Structures

struct  Token
 A token and its value. More...
 
struct  Scanner
 Maintains state for successive calls to scan() . More...
 

Macros

#define CHUNK   4096
 Amount of data to read from a file each time. More...
 
#define RETURN(tt)   { t.id = tt; return t; }
 Return the token. More...
 
#define YYFILL(n)
 Retrieves more input if available. More...
 

Enumerations

enum  TokenId {
  TOK_EOF, TOK_BAREWORD, TOK_OBRACE, TOK_CBRACE,
  TOK_COLON, TOK_QUOTED, TOK_NUMBER, TOK_BOOLEAN,
  TOK_MALLOC_ERR
}
 Token types. More...
 

Functions

static const char * token2txt (Token *t)
 Converts a Token to a string based on its type. More...
 
static void token_free (Token *t, ProtobufCAllocator *allocator)
 Frees memory allocated in a Token instance. More...
 
static void scanner_init_file (Scanner *scanner, FILE *f)
 Initialise a Scanner from a FILE. More...
 
static void scanner_init_string (Scanner *scanner, char *buf)
 Initialise a Scanner from a string. More...
 
static void scanner_free (Scanner *scanner, ProtobufCAllocator *allocator)
 Free data internal to the Scanner instance. More...
 
static ProtobufCBinaryData * unesc_str (unsigned char *src, int len, ProtobufCAllocator *allocator)
 Unescape string. More...
 
static int fill (Scanner *scanner, ProtobufCAllocator *allocator)
 Function to request more data from input source in Scanner. More...
 
static Token scan (Scanner *scanner, ProtobufCAllocator *allocator)
 Generated lexer. More...
 

Detailed Description

Macro Definition Documentation

#define CHUNK   4096

Amount of data to read from a file each time.

Definition at line 326 of file parse.re.

Referenced by fill().

#define RETURN (   tt)    { t.id = tt; return t; }

Return the token.

Definition at line 378 of file parse.re.

#define YYFILL (   n)
Value:
{ fill_result = fill(scanner, allocator); \
if (fill_result <= 0) \
RETURN((fill_result == -1? TOK_MALLOC_ERR: TOK_EOF)); }
A memory allocation error occurred.
Definition: parse.re:100
#define RETURN(tt)
Return the token.
Definition: parse.re:378
static int fill(Scanner *scanner, ProtobufCAllocator *allocator)
Function to request more data from input source in Scanner.
Definition: parse.re:342
End of file.
Definition: parse.re:92

Retrieves more input if available.

Definition at line 380 of file parse.re.

Enumeration Type Documentation

enum TokenId

Token types.

Types of tokens found by scan(). These will be in the id field of Token .

Enumerator
TOK_EOF 

End of file.

TOK_BAREWORD 

A bare, unquoted single word.

TOK_OBRACE 

An opening brace.

TOK_CBRACE 

A closing brace.

TOK_COLON 

A colon.

TOK_QUOTED 

A quoted string.

TOK_NUMBER 

A number.

TOK_BOOLEAN 

The unquoted form of "true" and "false".

TOK_MALLOC_ERR 

A memory allocation error occurred.

Definition at line 91 of file parse.re.

Function Documentation

static int fill ( Scanner scanner,
ProtobufCAllocator *  allocator 
)
static

Function to request more data from input source in Scanner.

In the case of a string being the input source for Scanner, nothing happens. For a FILE backed Scanner, a CHUNK's worth of data is read from the FILE.

Parameters
[in,out]scannerThe state struct for the scanner.
[in]allocatorAllocator functions.
Returns
Returns the success of the function:
  • -1: Memory allocation failure.
  • 0: No more input added.
  • >0: Input added.

Definition at line 342 of file parse.re.

References Scanner::buffer, CHUNK, Scanner::cursor, Scanner::f, Scanner::limit, Scanner::marker, PBC_ALLOC, PBC_FREE, and Scanner::token.

static Token scan ( Scanner scanner,
ProtobufCAllocator *  allocator 
)
static

Generated lexer.

The guts of the parser generated by re2c.

Parameters
[in,out]scannerThe state struct for the scanner.
[in]allocatorAllocator functions.
Returns
Returns the next Token it finds.

Definition at line 393 of file parse.re.

References Scanner::cursor, and Scanner::token.

Referenced by protobuf_c_text_parse().

Here is the caller graph for this function:

static void scanner_free ( Scanner scanner,
ProtobufCAllocator *  allocator 
)
static

Free data internal to the Scanner instance.

Parameters
[in,out]scannerThe state struct for the scanner.
[in]allocatorAllocator functions.

Definition at line 233 of file parse.re.

References Scanner::buffer, Scanner::f, and PBC_FREE.

Referenced by protobuf_c_text_parse().

Here is the caller graph for this function:

static void scanner_init_file ( Scanner scanner,
FILE *  f 
)
static

Initialise a Scanner from a FILE.

The resulting Scanner will load input from a FILE.

Parameters
[in,out]scannerThe state struct for the scanner.
[in]fFILE to read input from.

Definition at line 202 of file parse.re.

References Scanner::f, and Scanner::line.

Referenced by protobuf_c_text_from_file().

Here is the caller graph for this function:

static void scanner_init_string ( Scanner scanner,
char *  buf 
)
static

Initialise a Scanner from a string.

The resulting Scanner will load input from a string.

Parameters
[in,out]scannerThe state struct for the scanner.
[in]bufString to get input from.

Definition at line 217 of file parse.re.

References Scanner::buffer, Scanner::cursor, Scanner::limit, Scanner::line, and Scanner::marker.

Referenced by protobuf_c_text_from_string().

Here is the caller graph for this function:

static const char* token2txt ( Token t)
static

Converts a Token to a string based on its type.

Provides a string summary of the type of token; used for error messages.

Parameters
[in]tThe token.
Returns
A string representation of the token. This is a char * on the stack. Do not modify or free it.

Definition at line 128 of file parse.re.

References Token::bareword, Token::boolean, Token::id, Token::number, TOK_BAREWORD, TOK_BOOLEAN, TOK_CBRACE, TOK_COLON, TOK_EOF, TOK_NUMBER, TOK_OBRACE, and TOK_QUOTED.

Referenced by state_assignment(), state_open(), and state_value().

Here is the caller graph for this function:

static void token_free ( Token t,
ProtobufCAllocator *  allocator 
)
static

Frees memory allocated in a Token instance.

Frees memory allocated in a Token instance, but not the Token itself.

Parameters
[in]tThe token.
[in]allocatorThe memory allocator functions.

Definition at line 161 of file parse.re.

References Token::bareword, Token::id, Token::number, PBC_FREE, Token::qs, TOK_BAREWORD, TOK_NUMBER, and TOK_QUOTED.

Referenced by protobuf_c_text_parse().

Here is the caller graph for this function:

static ProtobufCBinaryData* unesc_str ( unsigned char *  src,
int  len,
ProtobufCAllocator *  allocator 
)
static

Unescape string.

Remove escape sequences from a string and replace them with the actual characters.

Parameters
[in]srcString to unescape.
[in]lenLength of string to unescape.
[in]allocatorAllocator functions.
Returns
A ProtobufCBinaryData pointer with the unescaped data. Note this must be freed with the ProtobufCAllocator allocator you called this with.

Definition at line 253 of file parse.re.

References PBC_ALLOC, and PBC_FREE.