Reader classes

Reader classes

Functions

GArrowSchema * garrow_record_batch_reader_get_schema ()
GArrowRecordBatch * garrow_record_batch_reader_get_next_record_batch ()
GArrowRecordBatch * garrow_record_batch_reader_read_next_record_batch ()
GArrowRecordBatch * garrow_record_batch_reader_read_next ()
GArrowTableBatchReader * garrow_table_batch_reader_new ()
GArrowRecordBatchStreamReader * garrow_record_batch_stream_reader_new ()
GArrowRecordBatchFileReader * garrow_record_batch_file_reader_new ()
GArrowSchema * garrow_record_batch_file_reader_get_schema ()
guint garrow_record_batch_file_reader_get_n_record_batches ()
GArrowMetadataVersion garrow_record_batch_file_reader_get_version ()
GArrowRecordBatch * garrow_record_batch_file_reader_get_record_batch ()
GArrowRecordBatch * garrow_record_batch_file_reader_read_record_batch ()
GArrowFeatherFileReader * garrow_feather_file_reader_new ()
gchar * garrow_feather_file_reader_get_description ()
gboolean garrow_feather_file_reader_has_description ()
gint garrow_feather_file_reader_get_version ()
gint64 garrow_feather_file_reader_get_n_rows ()
gint64 garrow_feather_file_reader_get_n_columns ()
gchar * garrow_feather_file_reader_get_column_name ()
GArrowColumn * garrow_feather_file_reader_get_column ()
GList * garrow_feather_file_reader_get_columns ()
GArrowTable * garrow_feather_file_reader_read ()
GArrowTable * garrow_feather_file_reader_read_indices ()
GArrowTable * garrow_feather_file_reader_read_names ()
GArrowCSVReadOptions * garrow_csv_read_options_new ()
void garrow_csv_read_options_add_column_type ()
void garrow_csv_read_options_add_schema ()
GHashTable * garrow_csv_read_options_get_column_types ()
GArrowCSVReader * garrow_csv_reader_new ()
GArrowTable * garrow_csv_reader_read ()

Properties

gpointer record-batch-file-reader Write / Construct Only
gboolean allow-newlines-in-values Read / Write
gint block-size Read / Write
gboolean check-utf8 Read / Write
gchar delimiter Read / Write
gchar escape-character Read / Write
gboolean ignore-empty-lines Read / Write
gboolean is-double-quoted Read / Write
gboolean is-escaped Read / Write
gboolean is-quoted Read / Write
guint n-header-rows Read / Write
gpointer pool Write / Construct Only
gchar quote-character Read / Write
gboolean use-threads Read / Write
gpointer csv-table-reader Write / Construct Only
gpointer feather-table-reader Write / Construct Only
gpointer record-batch-reader Write / Construct Only

Types and Values

Object Hierarchy

    GObject
    ├── GArrowCSVReadOptions
    ├── GArrowCSVReader
    ├── GArrowFeatherFileReader
    ├── GArrowRecordBatchFileReader
    ╰── GArrowRecordBatchReader
        ├── GArrowRecordBatchStreamReader
        ├── GArrowTableBatchReader
        ╰── GArrowRecordBatchStreamReader

Includes

#include <arrow-glib/arrow-glib.h>

Description

GArrowRecordBatchReader is a base class for reading record batches in stream format from input.

GArrowRecordBatchStreamReader is a class for reading record batches in stream format from input synchronously.

GArrowRecordBatchFileReader is a class for reading record batches in file format from input.

GArrowFeatherFileReader is a class for reading columns in Feather file format from input.

GArrowCSVReader is a class for reading table in CSV format from input.

Functions

garrow_record_batch_reader_get_schema ()

GArrowSchema *
garrow_record_batch_reader_get_schema (GArrowRecordBatchReader *reader);

Parameters

Returns

The schema in the stream.

[transfer full]

Since: 0.4.0


garrow_record_batch_reader_get_next_record_batch ()

GArrowRecordBatch *
garrow_record_batch_reader_get_next_record_batch
                               (GArrowRecordBatchReader *reader,
                                GError **error);

garrow_record_batch_reader_get_next_record_batch has been deprecated since version 0.5.0 and should not be used in newly-written code.

Use garrow_record_batch_reader_read_next() instead.

Parameters

reader

A GArrowRecordBatchReader.

 

error

Return location for a GError or NULL.

[nullable]

Returns

The next record batch in the stream or NULL on end of stream.

[nullable][transfer full]

Since: 0.4.0


garrow_record_batch_reader_read_next_record_batch ()

GArrowRecordBatch *
garrow_record_batch_reader_read_next_record_batch
                               (GArrowRecordBatchReader *reader,
                                GError **error);

garrow_record_batch_reader_read_next_record_batch has been deprecated since version 0.8.0 and should not be used in newly-written code.

Use garrow_record_batch_reader_read_next() instead.

Parameters

reader

A GArrowRecordBatchReader.

 

error

Return location for a GError or NULL.

[nullable]

Returns

The next record batch in the stream or NULL on end of stream.

[nullable][transfer full]

Since: 0.5.0


garrow_record_batch_reader_read_next ()

GArrowRecordBatch *
garrow_record_batch_reader_read_next (GArrowRecordBatchReader *reader,
                                      GError **error);

Parameters

reader

A GArrowRecordBatchReader.

 

error

Return location for a GError or NULL.

[nullable]

Returns

The next record batch in the stream or NULL on end of stream.

[nullable][transfer full]

Since: 0.8.0


garrow_table_batch_reader_new ()

GArrowTableBatchReader *
garrow_table_batch_reader_new (GArrowTable *table);

Parameters

table

The table to be read.

 

Returns

A newly created GArrowTableBatchReader.

Since: 0.8.0


garrow_record_batch_stream_reader_new ()

GArrowRecordBatchStreamReader *
garrow_record_batch_stream_reader_new (GArrowInputStream *stream,
                                       GError **error);

Parameters

stream

The stream to be read.

 

error

Return location for a GError or NULL.

[nullable]

Returns

A newly created GArrowRecordBatchStreamReader or NULL on error.

[nullable]

Since: 0.4.0


garrow_record_batch_file_reader_new ()

GArrowRecordBatchFileReader *
garrow_record_batch_file_reader_new (GArrowSeekableInputStream *file,
                                     GError **error);

Parameters

file

The file to be read.

 

error

Return location for a GError or NULL.

[nullable]

Returns

A newly created GArrowRecordBatchFileReader or NULL on error.

[nullable]

Since: 0.4.0


garrow_record_batch_file_reader_get_schema ()

GArrowSchema *
garrow_record_batch_file_reader_get_schema
                               (GArrowRecordBatchFileReader *reader);

Parameters

Returns

The schema in the file.

[transfer full]

Since: 0.4.0


garrow_record_batch_file_reader_get_n_record_batches ()

guint
garrow_record_batch_file_reader_get_n_record_batches
                               (GArrowRecordBatchFileReader *reader);

Parameters

Returns

The number of record batches in the file.

Since: 0.4.0


garrow_record_batch_file_reader_get_version ()

GArrowMetadataVersion
garrow_record_batch_file_reader_get_version
                               (GArrowRecordBatchFileReader *reader);

Parameters

Returns

The format version in the file.

Since: 0.4.0


garrow_record_batch_file_reader_get_record_batch ()

GArrowRecordBatch *
garrow_record_batch_file_reader_get_record_batch
                               (GArrowRecordBatchFileReader *reader,
                                guint i,
                                GError **error);

garrow_record_batch_file_reader_get_record_batch has been deprecated since version 0.5.0 and should not be used in newly-written code.

Use garrow_record_batch_file_reader_read_record_batch() instead.

Parameters

reader

A GArrowRecordBatchFileReader.

 

i

The index of the target record batch.

 

error

Return location for a GError or NULL.

[nullable]

Returns

The i-th record batch in the file or NULL on error.

[nullable][transfer full]

Since: 0.4.0


garrow_record_batch_file_reader_read_record_batch ()

GArrowRecordBatch *
garrow_record_batch_file_reader_read_record_batch
                               (GArrowRecordBatchFileReader *reader,
                                guint i,
                                GError **error);

Parameters

reader

A GArrowRecordBatchFileReader.

 

i

The index of the target record batch.

 

error

Return location for a GError or NULL.

[nullable]

Returns

The i-th record batch in the file or NULL on error.

[nullable][transfer full]

Since: 0.5.0


garrow_feather_file_reader_new ()

GArrowFeatherFileReader *
garrow_feather_file_reader_new (GArrowSeekableInputStream *file,
                                GError **error);

Parameters

file

The file to be read.

 

error

Return location for a GError or NULL.

[nullable]

Returns

A newly created GArrowFeatherFileReader or NULL on error.

[nullable]

Since: 0.4.0


garrow_feather_file_reader_get_description ()

gchar *
garrow_feather_file_reader_get_description
                               (GArrowFeatherFileReader *reader);

Parameters

Returns

The description of the file if it exists, NULL otherwise. You can confirm whether description exists or not by garrow_feather_file_reader_has_description().

It should be freed with g_free() when no longer needed.

[nullable][transfer full]

Since: 0.4.0


garrow_feather_file_reader_has_description ()

gboolean
garrow_feather_file_reader_has_description
                               (GArrowFeatherFileReader *reader);

Parameters

Returns

Whether the file has description or not.

Since: 0.4.0


garrow_feather_file_reader_get_version ()

gint
garrow_feather_file_reader_get_version
                               (GArrowFeatherFileReader *reader);

Parameters

Returns

The format version of the file.

Since: 0.4.0


garrow_feather_file_reader_get_n_rows ()

gint64
garrow_feather_file_reader_get_n_rows (GArrowFeatherFileReader *reader);

Parameters

Returns

The number of rows in the file.

Since: 0.4.0


garrow_feather_file_reader_get_n_columns ()

gint64
garrow_feather_file_reader_get_n_columns
                               (GArrowFeatherFileReader *reader);

Parameters

Returns

The number of columns in the file.

Since: 0.4.0


garrow_feather_file_reader_get_column_name ()

gchar *
garrow_feather_file_reader_get_column_name
                               (GArrowFeatherFileReader *reader,
                                gint i);

Parameters

reader

A GArrowFeatherFileReader.

 

i

The index of the target column.

 

Returns

The i-th column name in the file.

It should be freed with g_free() when no longer needed.

[transfer full]

Since: 0.4.0


garrow_feather_file_reader_get_column ()

GArrowColumn *
garrow_feather_file_reader_get_column (GArrowFeatherFileReader *reader,
                                       gint i,
                                       GError **error);

Parameters

reader

A GArrowFeatherFileReader.

 

i

The index of the target column.

 

error

Return location for a GError or NULL.

[nullable]

Returns

The i-th column in the file or NULL on error.

[nullable][transfer full]

Since: 0.4.0


garrow_feather_file_reader_get_columns ()

GList *
garrow_feather_file_reader_get_columns
                               (GArrowFeatherFileReader *reader,
                                GError **error);

Parameters

reader

A GArrowFeatherFileReader.

 

error

Return location for a GError or NULL.

[nullable]

Returns

The columns in the file.

[element-type GArrowColumn][transfer full]

Since: 0.4.0


garrow_feather_file_reader_read ()

GArrowTable *
garrow_feather_file_reader_read (GArrowFeatherFileReader *reader,
                                 GError **error);

Parameters

reader

A GArrowFeatherFileReader.

 

error

Return location for a GError or NULL.

[nullable]

Returns

The table in the file that has all columns.

[transfer full]

Since: 0.12.0


garrow_feather_file_reader_read_indices ()

GArrowTable *
garrow_feather_file_reader_read_indices
                               (GArrowFeatherFileReader *reader,
                                const gint *indices,
                                guint n_indices,
                                GError **error);

Parameters

reader

A GArrowFeatherFileReader.

 

indices

The indices of column to be read.

[array length=n_indices]

n_indices

The number of indices.

 

error

Return location for a GError or NULL.

[nullable]

Returns

The table in the file that has only the specified columns.

[transfer full]

Since: 0.12.0


garrow_feather_file_reader_read_names ()

GArrowTable *
garrow_feather_file_reader_read_names (GArrowFeatherFileReader *reader,
                                       const gchar **names,
                                       guint n_names,
                                       GError **error);

Parameters

reader

A GArrowFeatherFileReader.

 

names

The names of column to be read.

[array length=n_names]

n_names

The number of names.

 

error

Return location for a GError or NULL.

[nullable]

Returns

The table in the file that has only the specified columns.

[transfer full]

Since: 0.12.0


garrow_csv_read_options_new ()

GArrowCSVReadOptions *
garrow_csv_read_options_new (void);

Returns

A newly created GArrowCSVReadOptions.

Since: 0.12.0


garrow_csv_read_options_add_column_type ()

void
garrow_csv_read_options_add_column_type
                               (GArrowCSVReadOptions *options,
                                const gchar *name,
                                GArrowDataType *data_type);

Add value type of a column.

Parameters

options

A GArrowCSVReadOptions.

 

name

The name of the target column.

 

data_type

The GArrowDataType for the column.

 

Since: 0.12.0


garrow_csv_read_options_add_schema ()

void
garrow_csv_read_options_add_schema (GArrowCSVReadOptions *options,
                                    GArrowSchema *schema);

Add value types for columns in the schema.

Parameters

options

A GArrowCSVReadOptions.

 

schema

The GArrowSchema that specifies columns and their types.

 

Since: 0.12.0


garrow_csv_read_options_get_column_types ()

GHashTable *
garrow_csv_read_options_get_column_types
                               (GArrowCSVReadOptions *options);

Parameters

options

A GArrowCSVReadOptions.

 

Returns

The column name and value type mapping of the options.

[transfer full][element-type gchar* GArrowDataType]

Since: 0.12.0


garrow_csv_reader_new ()

GArrowCSVReader *
garrow_csv_reader_new (GArrowInputStream *input,
                       GArrowCSVReadOptions *options,
                       GError **error);

Parameters

input

The input to be read.

 

options

A GArrowCSVReadOptions.

[nullable]

error

Return location for a GError or NULL.

[nullable]

Returns

A newly created GArrowCSVReader or NULL on error.

[nullable]

Since: 0.12.0


garrow_csv_reader_read ()

GArrowTable *
garrow_csv_reader_read (GArrowCSVReader *reader,
                        GError **error);

Parameters

reader

A GArrowCSVReader.

 

error

Return location for a GError or NULL.

[nullable]

Returns

A read GArrowTable or NULL on error.

[nullable][transfer full]

Since: 0.12.0

Types and Values

GARROW_TYPE_RECORD_BATCH_READER

#define GARROW_TYPE_RECORD_BATCH_READER (garrow_record_batch_reader_get_type())

struct GArrowRecordBatchReaderClass

struct GArrowRecordBatchReaderClass {
  GObjectClass parent_class;
};

GARROW_TYPE_TABLE_BATCH_READER

#define GARROW_TYPE_TABLE_BATCH_READER (garrow_table_batch_reader_get_type())

struct GArrowTableBatchReaderClass

struct GArrowTableBatchReaderClass {
  GArrowRecordBatchReaderClass parent_class;
};

struct GArrowRecordBatchStreamReader

struct GArrowRecordBatchStreamReader;

It wraps arrow::ipc::RecordBatchStreamReader.


struct GArrowRecordBatchFileReader

struct GArrowRecordBatchFileReader;

It wraps arrow::ipc::RecordBatchFileReader.


GARROW_TYPE_FEATHER_FILE_READER

#define GARROW_TYPE_FEATHER_FILE_READER (garrow_feather_file_reader_get_type())

struct GArrowFeatherFileReaderClass

struct GArrowFeatherFileReaderClass {
  GObjectClass parent_class;
};

GARROW_TYPE_CSV_READ_OPTIONS

#define GARROW_TYPE_CSV_READ_OPTIONS (garrow_csv_read_options_get_type())

struct GArrowCSVReadOptionsClass

struct GArrowCSVReadOptionsClass {
  GObjectClass parent_class;
};

GARROW_TYPE_CSV_READER

#define GARROW_TYPE_CSV_READER (garrow_csv_reader_get_type())

struct GArrowCSVReaderClass

struct GArrowCSVReaderClass {
  GObjectClass parent_class;
};

GArrowCSVReadOptions

typedef struct _GArrowCSVReadOptions GArrowCSVReadOptions;

GArrowCSVReader

typedef struct _GArrowCSVReader GArrowCSVReader;

GArrowFeatherFileReader

typedef struct _GArrowFeatherFileReader GArrowFeatherFileReader;

GArrowRecordBatchReader

typedef struct _GArrowRecordBatchReader GArrowRecordBatchReader;

GArrowTableBatchReader

typedef struct _GArrowTableBatchReader GArrowTableBatchReader;

Property Details

The “record-batch-file-reader” property

  “record-batch-file-reader” gpointer

The raw std::shared<arrow::ipc::RecordBatchFileReader> *.

Flags: Write / Construct Only


The “allow-newlines-in-values” property

  “allow-newlines-in-values” gboolean

Whether values are allowed to contain CR (0x0d) and LF (0x0a) characters.

Flags: Read / Write

Default value: FALSE

Since: 0.12.0


The “block-size” property

  “block-size”               gint

Block size we request from the IO layer; also determines the size of chunks when “use-threads” is TRUE.

Flags: Read / Write

Allowed values: >= 0

Default value: 1048576

Since: 0.12.0


The “check-utf8” property

  “check-utf8”               gboolean

Whether to check UTF8 validity of string columns.

Flags: Read / Write

Default value: TRUE

Since: 0.12.0


The “delimiter” property

  “delimiter”                gchar

Field delimiter character.

Flags: Read / Write

Allowed values: >= 0

Default value: 44

Since: 0.12.0


The “escape-character” property

  “escape-character”         gchar

Escaping character. This is used only when “is-escaped” is TRUE.

Flags: Read / Write

Allowed values: >= 0

Default value: 92

Since: 0.12.0


The “ignore-empty-lines” property

  “ignore-empty-lines”       gboolean

Whether empty lines are ignored. If FALSE, an empty line represents a simple empty value (assuming a one-column CSV file).

Flags: Read / Write

Default value: TRUE

Since: 0.12.0


The “is-double-quoted” property

  “is-double-quoted”         gboolean

Whether a quote inside a value is double quoted.

Flags: Read / Write

Default value: TRUE

Since: 0.12.0


The “is-escaped” property

  “is-escaped”               gboolean

Whether escaping is used.

Flags: Read / Write

Default value: FALSE

Since: 0.12.0


The “is-quoted” property

  “is-quoted”                gboolean

Whether quoting is used.

Flags: Read / Write

Default value: TRUE

Since: 0.12.0


The “n-header-rows” property

  “n-header-rows”            guint

The number of header rows to skip (including the first row containing column names)

Flags: Read / Write

Default value: 1

Since: 0.12.0


The “pool” property

  “pool”                     gpointer

The raw arrow::MemoryPool *.

Flags: Write / Construct Only


The “quote-character” property

  “quote-character”          gchar

Quoting character. This is used only when “is-quoted” is TRUE.

Flags: Read / Write

Allowed values: >= 0

Default value: 34

Since: 0.12.0


The “use-threads” property

  “use-threads”              gboolean

Whether to use the global CPU thread pool.

Flags: Read / Write

Default value: TRUE

Since: 0.12.0


The “csv-table-reader” property

  “csv-table-reader”         gpointer

The raw std::shared<arrow::csv::TableReader> *.

Flags: Write / Construct Only


The “feather-table-reader” property

  “feather-table-reader”     gpointer

The raw std::shared<arrow::ipc::feather::TableReader> *.

Flags: Write / Construct Only


The “record-batch-reader” property

  “record-batch-reader”      gpointer

The raw std::shared<arrow::ipc::RecordBatchRecordBatchReader> *.

Flags: Write / Construct Only