File variables

File variables define file-based data sources and destinations. They are primarily used for importing or exporting data, and support multiple file formats including CSV, Parquet, and ORC. Each file variable can reference one or more files.

Syntax

Syntax diagramSource

file_variable_definition

column_definitions

file_options

file_path

file_variable_definition ::=
    'FILE' file_variable '{' (column_definitions) ( ',' (column_definitions))* '}' '=' 'DATAFILE' '{' file_options '}'

column_definitions ::=
    column_name value_type

file_options ::=
    ('PATH' ':' file_path) ',' ('FORMAT' ':' ('"' 'CSV' '"'|'"' 'PARQUET' '"'|'"' 'ORC' '"')(',' additional_options)* )

file_path ::=
      '"' 'file://' path '"'
    | '"' 'hdfs://' hostname ':' port '/' path '"' 
    | '"' 's3://' (access_key ':' secret_key '@')? bucket '/' path ( '?' ( parameter_name '=' parameter_value) ('&' ( parameter_name '=' parameter_value))* )? '"'
    | '"' 'gs://' ((access_key ':' secret_key) | 'anonymous') '@' bucket '/' path ( '?' ( parameter_name '=' parameter_value) ('&' ( parameter_name '=' parameter_value))* )? '"'            

Parameters

file_variable

The name of the file variable being defined. file_variable can be a regular identifier or a delimited identifier enclosed in backticks (`). For more information about identifiers, see Identifier naming rules.

column_definitions

The structural definition of columns and their data types in the file. Each definition consists of a column_name and a value_type, separated by commas. Example: {id INT, name STRING}.

column_name

The name of a column in the file. column_name must be an identifier. For more information, see Identifier naming rules.

value_type

The data type of values stored in the column. Supported data types include:

• Numeric types, such as INT, UINT, FLOAT, DOUBLE, and DECIMAL.
• Character string types
• Boolean types
• Temporal types, such as DATE, LOCAL TIME, ZONED TIME, LOCAL DATETIME, and ZONED DATETIME.
• Vector types
• List types
• Set types
• Map types
• Spatial types

For more information, see Data types.

file_options

File configuration options, including PATH, FORMAT, and format-specific options. PATH and FORMAT are required; all other options are optional. Options can appear in any order.

Supported file formats: CSV, PARQUET, and ORC.

For optional configuration items, see the additional_options parameter below.

additional_options

Additional configuration options for file reading, parsing, conversion, and writing. These options vary by file format. Supported options include:

File type	Category	Option name	Data type	Default	Description
CSV	Read	SKIP_ROWS	Int32	0	Number of header rows to skip before reading columns.
	Read	SKIP_ROWS_AFTER_NAMES	Int32	0	Number of rows to skip after reading column names.
	Read	COLUMN_NAMES	Vector	[]	Explicit column names for the table. By default, column names are read automatically from the first row of the file.
	Read	AUTOGENERATE_COLUMN_NAMES	Bool	false	Whether to auto-generate column names (e.g., f0, f1) when none are provided.
	Parse	DELIMITER	Char	','	The delimiter character for columns.
	Parse	QUOTING	Bool	true	Whether quoting is enabled.
	Parse	QUOTE_CHAR	Char	'"'	The character used for quoting values.
	Parse	DOUBLE_QUOTE	Bool	true	Whether quotes within values are represented by double quotes.
	Parse	ESCAPING	Bool	false	Whether escaping is enabled.
	Parse	ESCAPE_CHAR	Char	'\'	The escape character for special characters.
	Parse	NEWLINES_IN_VALUES	Bool	false	Whether values may contain newline characters.
	Parse	IGNORE_EMPTY_LINES	Bool	true	Whether to ignore empty lines.
	Convert	CHECK_UTF8	Bool	true	Whether to verify UTF-8 encoding of string columns.
	Convert	NULL_VALUES	Vector	"", "#N/A", "#N/A N/A", "#NA", "-1.#IND", "-1.#QNAN", "-NaN", "-nan", "1.#IND", "1.#QNAN", "N/A", "NA", "NULL", "NaN", "n/a", "nan", "null"	Strings interpreted as Null values.
	Convert	TRUE_VALUES	Vector	{"1", "True", "TRUE", "true"}	Strings interpreted as the boolean value true.
	Convert	FALSE_VALUES	Vector	{"0", "False", "FALSE", "false"}	Strings interpreted as the boolean value false.
	Convert	STRINGS_CAN_BE_NULL	Bool	true	Whether string/binary columns may contain Null values.
	Convert	QUOTED_STRINGS_CAN_BE_NULL	Bool	false	Whether quoted string values can be treated as Null.
	Convert	INCLUDE_COLUMNS	Vector	[]	Columns to include in the output; unlisted columns are ignored. All columns are included by default.
	Convert	INCLUDE_MISSING_COLUMNS	Bool	false	Whether to create Null values for columns listed in INCLUDE_COLUMNS that are missing.
	Write	INCLUDE_HEADER	Bool	true	Whether to write a header row containing column names.
	Write	BATCH_SIZE	Int32	1024	Maximum number of rows to process per write batch.
	Write	DELIMITER	Char	','	The delimiter character for columns in output.
	Write	NULL_STRING	String	""	String representation of Null values. By default, Null values are replaced with an empty string.
	Write	EOL	String	"\n"	The end-of-line character sequence.
	Write	MAX_ROWS_PER_FILE	Int64	100000000	Maximum number of rows per output file.
Parquet	Read	BATCH_SIZE	Int64	1024	Maximum number of rows per record batch when reading.
	Write	MAX_ROWS_PER_FILE	Int64	100000000	Maximum number of rows per output file.
	Write	WRITE_BATCH_SIZE	Int64	1024	Batch size for writing Arrow values to Parquet.

Examples

The following examples are based on the example dataset.

Use a file variable to export query data to a file

'''
// Define an output file using a file variable
FILE output_file {id INT64, firstName STRING} = DATAFILE {
   PATH: "file:///tmp/output.csv",
   FORMAT: "CSV",
   INCLUDE_HEADER: true
}

// Export query data to the output file
USE movie
MATCH (v:Person)
EXPORT v.id, v.name INTO output_file
'''

Use a file variable to import data from a file

For more information, see Import data from files or file variables.