IMPORT statement¶
The IMPORT statement imports data into a temporary graph. It is typically used together with the USE clause, which specifies the target temporary graph. Data can come from files, NebulaGraph Database Developer Edition, or table variables.
Prerequisites¶
- A temporary graph has been created. For more information, see CREATE TEMPORARY GRAPH.
Notes¶
The following table summarizes the main restrictions for each import method:
| Data source and import method | Graph type or field mapping requirements | Edge type requirements | Other restrictions |
|---|---|---|---|
| Import an entire graph from NebulaGraph Database Developer Edition | The temporary graph type definition, including node types, edge types, and their properties, must match the source graph. | No additional requirement. | The source graph and target temporary graph must not run on the same Graph service. |
| Import selected node or edge types from NebulaGraph Database Developer Edition | The primary key structure of source node types and the topology of source edge types must correspond to the target definitions. | Edge types in the temporary graph must declare MULTIEDGE KEY(). |
The source graph and target temporary graph must not run on the same Graph service. |
| Import from files | File fields must be mapped correctly to target properties. Unmapped properties must have default values or allow NULL. |
Edge types in the temporary graph must declare MULTIEDGE KEY(). |
File formats and field types must be supported. |
| Import from a table variable | Table field types must be compatible with target property types. Unmapped properties must have default values or allow NULL. |
Edge types in the temporary graph must declare MULTIEDGE KEY(). |
The table variable must be defined before IMPORT. |
For example, when a multiedge key is required, define the edge type as EDGE Friend (Person)-[:Friend {since DATE, MULTIEDGE KEY()}]->(Person).
Syntax¶
- import_statement
- node_definition
- edge_definition
- file_source
- file_options
- file_path
- graph_source
- nebula_uri_string
- table_source
- data_option
import_statement ::=
'IMPORT' 'INTO' 'GRAPH' '{' ( ( (node_definition (file_source | graph_source | table_source) | edge_definition (file_source | graph_source | table_source)) (',' (node_definition (file_source | graph_source | table_source) | edge_definition (file_source | graph_source | table_source) ))* )| 'GRAPH' graph_source ) '}' ( 'OPTIONS' '{' data_option ( ',' data_option )* '}' )?
node_definition ::=
'NODE' '(' node_variable '@' node_type_name '{' (property_name ':' field_name) (',' (property_name ':' field_name))* '}' ')'
edge_definition ::=
'EDGE' '(' (primary_key_property_name ':' field_name) (',' (primary_key_property_name ':' field_name) )* ')' ( ( '-[' edge_variable '@' edge_type_name '{' ((property_name ':' field_name) (',' (property_name ':' field_name))*)? '}' ']->' )
| ('<-[' edge_variable '@' edge_type_name '{' ((property_name ':' field_name) (',' (property_name ':' field_name))*)? '}' ']-' )
| ('~[' edge_variable '@' edge_type_name '{' ((property_name ':' field_name) (',' (property_name ':' field_name))*)? '}' ']~' ) )
'(' (primary_key_property_name ':' field_name) (','(primary_key_property_name ':' field_name) )* ')'
file_source ::=
'FROM' ( file_variable | ( 'DATAFILE' '{' file_options '}' ) )
file_options ::=
('PATH' ':' file_path) ',' ('FORMAT' ':' ('"' 'CSV' '"'|'"' 'PARQUET' '"'|'"' 'ORC' '"')(',' additional_options)* )
file_path ::=
'"' 'file://' path '"'
| '"' 'hdfs://' hostname ':' port '/' path '"'
| '"' 's3://' (access_key ':' secret_key '@')? bucket '/' path ( '?' ( parameter_name '=' parameter_value) ('&' ( parameter_name '=' parameter_value))* )? '"'
| '"' 'gs://' ((access_key ':' secret_key) | 'anonymous') '@' bucket '/' path ( '?' ( parameter_name '=' parameter_value) ('&' ( parameter_name '=' parameter_value))* )? '"'
graph_source ::=
'FROM' 'NEBULA' '{' 'PATH' ':' '"' nebula_uri_string '"' ',' 'FORMAT' ':' ('"' 'NEBULA' '"') '}'
nebula_uri_string ::=
'nebula://' username ':' password '@' graph_address '?' ( 'graph' '=' graph_name) ('&' 'schema' '=' schema_path)? (('&' 'node_type' '=' source_node_type_name) | ('&' 'edge_type' '=' source_edge_type_name))? ('&' ( tls_parameter_name '=' tls_parameter_value))*
table_source ::=
'FROM' table_variable
data_option ::=
'PRIMARY_KEY_AS_NODE_ID' ':' ( 'true' | 'false' )
| 'SKIP_CONFLICT_PK_NODES' ':' ( 'true' | 'false' )
| 'SKIP_DANGLING_EDGES' ':' ( 'true' | 'false' )
Parameters¶
node_variable¶
A variable referring to a node in the temporary graph. For more information, see Variable value expression.
node_type_name¶
The name of the node type defined in the graph type referenced when creating the temporary graph.
property_name¶
The name of a property on the node type or edge type.
field_name¶
The name of a field in the file specified by file_source, or the name of a property in the graph specified by graph_source. The value of this parameter is used to populate the corresponding property in the temporary graph.
If no field is mapped for a property, the default value defined in the graph type is used. If the property is declared NOT NULL, has no default value, and is not mapped to a field, the import will fail.
primary_key_property_name¶
The name of a property that serves as a primary key on the node connected to the edge. If the primary key consists of multiple properties, all primary key properties must be listed.
edge_variable¶
A variable referring to an edge in the temporary graph. For more information, see Variable value expression.
edge_type_name¶
The name of the edge type defined in the graph type referenced when creating the temporary graph.
file_source¶
A file data source from which data is read to populate the temporary graph.
Note that file_source and graph_source cannot be used together in the same IMPORT statement.
file_variable¶
A file variable from which data is read to populate the temporary graph. This variable must be defined before the IMPORT statement is executed. For more information, see File variables.
When importing CSV files, if the type declared for a column in the file variable (value_type) is compatible with the actual data values in that column, the system parses the CSV text for that column according to the declared type. The types that are automatically recognized and parsed include: all integers, floating-point numbers, strings, and booleans. For information about data types, see Data types.
file_options¶
File configuration options, including PATH, FORMAT, and format-specific options. PATH and FORMAT are required; all other options are optional. Options can appear in any order.
Supported file formats: CSV, PARQUET, and ORC.
For optional configuration items, see the additional_options parameter below.
file_path¶
The path of the file from which data is read. Supported storage locations include the local file system, Hadoop Distributed File System (HDFS), and cloud storage services such as Amazon S3 and Google Cloud Storage. This parameter is required in file_source.
path¶
The absolute path of the data file on the storage service.
If the path points to a single file, only that file is read. If the path points to a directory, all files in the directory are read, and all such files must correspond to the same node type or edge type.
Note
When specifying a local path, ensure that the system has access permissions for that path.
hostname and port¶
The hostname or IP address and port used to access the storage service.
access_key and secret_key¶
The Access Key and Secret Key used to access cloud storage services.
When accessing Amazon S3, the Access Key and Secret Key can also be specified as parameters in the file path (parameter_name).
When accessing GCS, the Access Key and Secret Key may be omitted and anonymous can be specified instead to access public buckets.
bucket¶
The name of the bucket from which data is read on the cloud storage service.
parameter_name and parameter_value¶
Parameters used when reading files from cloud storage services.
When accessing Amazon S3, the following parameters are supported:
access_key: The Access Key for accessing the S3 bucket.secret_key: The Secret Key for accessing the S3 bucket.region: The region where the S3 bucket is located.scheme: The protocol for accessing the S3 bucket, e.g.,httporhttps.endpoint_override: A custom endpoint to override the default S3 endpoint.
When accessing GCS, the following parameters are supported:
scheme: The protocol for accessing the GCS bucket, e.g.,httporhttps.endpoint_override: A custom endpoint to override the default GCS endpoint.retry_limit_seconds: The maximum time (in seconds) to retry reading a file on failure. If the operation does not succeed within this time, an error is returned.
additional_options¶
Additional configuration options for file reading, parsing, and conversion. These options vary by file format. Supported options include:
| File type | Category | Option name | Data type | Description |
|---|---|---|---|---|
| CSV | Read | USE_THREADS |
Bool | Whether to use the global CPU thread pool for reading. |
| Read | BLOCK_SIZE |
Int32 | Block size for I/O operations; affects multi-threading granularity and record batch size. | |
| Read | SKIP_ROWS |
Int32 | Number of header rows to skip before reading columns. | |
| Read | SKIP_ROWS_AFTER_NAMES |
Int32 | Number of rows to skip after reading column names. | |
| Read | COLUMN_NAMES |
Vector |
Explicit column names for the table. | |
| Read | AUTOGENERATE_COLUMN_NAMES |
Bool | Whether to auto-generate column names (e.g., f0, f1) when none are provided. |
|
| Parse | DELIMITER |
Char | The delimiter character for columns. | |
| Parse | QUOTING |
Bool | Whether quoting is enabled. | |
| Parse | QUOTE_CHAR |
Char | The character used for quoting values. | |
| Parse | DOUBLE_QUOTE |
Bool | Whether quotes within values are represented by double quotes. | |
| Parse | ESCAPING |
Bool | Whether escaping is enabled. | |
| Parse | ESCAPE_CHAR |
Char | The escape character for special characters. | |
| Parse | NEWLINES_IN_VALUES |
Bool | Whether values may contain newline characters. | |
| Parse | IGNORE_EMPTY_LINES |
Bool | Whether to ignore empty lines. | |
| Convert | CHECK_UTF8 |
Bool | Whether to verify UTF-8 encoding of string columns. | |
| Convert | NULL_VALUES |
Vector |
Strings interpreted as null values. | |
| Convert | TRUE_VALUES |
Vector |
Strings interpreted as the boolean value true. |
|
| Convert | FALSE_VALUES |
Vector |
Strings interpreted as the boolean value false. |
|
| Convert | STRINGS_CAN_BE_NULL |
Bool | Whether string/binary columns may contain null values. | |
| Convert | QUOTED_STRINGS_CAN_BE_NULL |
Bool | Whether quoted string values can be treated as null. | |
| Convert | INCLUDE_COLUMNS |
Vector |
Columns to include in the output; unlisted columns are ignored. | |
| Convert | INCLUDE_MISSING_COLUMNS |
Bool | Whether to create null values for columns listed in INCLUDE_COLUMNS that are missing. |
|
| Parquet | Read | BUFFER_SIZE |
Int64 | The size of the buffered stream, in bytes. |
| Read | BUFFERED_STREAM |
Bool | Whether to enable buffered stream reading to control memory usage. | |
| Read | THRIFT_STRING_SIZE_LIMIT |
Int32 | Thrift string size limit to prevent excessive memory usage from large headers. | |
| Read | THRIFT_CONTAINER_SIZE_LIMIT |
Int32 | Thrift container size limit to prevent excessive memory usage from large headers. | |
| Read | BUFFERED_STREAM_ENABLED |
Bool | Whether buffered stream reading is enabled. | |
| Read | PAGE_CHECKSUM_VERIFICATION |
Bool | Whether to verify page checksums when reading. | |
| Read | READ_DENSE_FOR_NULLABLE |
Bool | Whether to optimize reading of nullable columns. | |
| Read | USE_THREADS |
Bool | Whether to use multiple threads for parallel column parsing. | |
| Read | BATCH_SIZE |
Int64 | Maximum number of rows per record batch when reading. | |
| Read | PRE_BUFFER |
Bool | Whether to coalesce reads by pre-buffering file regions (helpful for high-latency file systems). | |
| ORC | Read configuration is not supported. | - | - | - |
graph_source¶
A persistent graph data source from which data is read to populate the temporary graph.
Note:
file_sourceandgraph_sourcecannot be used together in the sameIMPORTstatement.- In a single
IMPORTstatement, data can only be imported from one graph data source. That is, thenebula_uri_stringspecified in eachgraph_sourcemust be exactly identical.
FORMAT and PATH¶
Required keywords in graph_source, used to define the format and path of the graph data respectively. They can appear in any order.
The supported format is NEBULA, which means NebulaGraph Database Developer Edition as the data source.
nebula_uri_string¶
The URI string used to connect to NebulaGraph Database Developer Edition. This parameter is required in graph_source.
username¶
The username for connecting to NebulaGraph Database Developer Edition.
password¶
The password for connecting to NebulaGraph Database Developer Edition.
graph_address¶
The address of the Graph service in NebulaGraph Database Developer Edition, in the format ip:port.
graph_name¶
The name of the graph to read from NebulaGraph Database Developer Edition. This graph must already exist in NebulaGraph Database Developer Edition.
The graph type of this graph must match the graph type specified by graph_type_name.
schema_path¶
The path of the schema to which the graph read from NebulaGraph Database Developer Edition belongs. This path must be an absolute path.
When &schema=schema_path is not specified, the default schema is used.
source_node_type_name¶
The name of the node type in NebulaGraph Database Developer Edition. This parameter is required when reading nodes via node_definition.
The primary key structure of the node type specified here must be consistent with the primary key structure of the node type defined in node_definition. That is, the number and order of columns must match. For example, if node_definition defines the node type Actor with PRIMARY KEY (id, name), then the node type here must also have a primary key consisting of two columns, such as PRIMARY KEY (id_, name_). Column names may differ (e.g., id maps to id_), but the number and order of columns must match to ensure correct primary key mapping.
source_edge_type_name¶
The name of the edge type in NebulaGraph Database Developer Edition. This parameter is required when reading edges via edge_definition.
The topological structure of the edge type specified here must be consistent with the edge type defined in edge_definition. That is, the source and destination node types of the edge must correspond respectively. For example, if the edge type in edge_definition is (Actor)-[:Act {MULTIEDGE KEY()}]->(Movie) and the edge type specified here is (Actor_)-[:Act_ {MULTIEDGE KEY()}]->(Movie_), then the node type definitions of Actor and Movie must map to Actor_ and Movie_ respectively.
tls_parameter_name¶
The name of additional configurable parameters when connecting to NebulaGraph Database Developer Edition. The following TLS-related parameters are supported:
| Parameter name | Data type | Required | Description | Valid values/Examples |
|---|---|---|---|---|
tls_enable |
Boolean | Yes | Whether to enable TLS. When the Graph service of NebulaGraph Database Developer Edition has TLS enabled, this parameter must be set to true. |
true, false, 1, 0 |
tls_cert |
String | Yes | Path to the client certificate file. Required only when tls_client_verify is set to true in the Graph service configuration file. |
/path/to/client-cert.pem |
tls_key |
String | Yes | Path to the client private key file. Required only when tls_client_verify is set to true in the Graph service configuration file. |
/path/to/client-key.pem |
tls_ca |
String | Yes | Path to the CA certificate file. | /path/to/ca-cert.pem |
tls_cadir |
String | No | Path to the CA certificate directory. Takes effect when tls_ca is not configured. |
/path/to/ca-certs/ |
tls_peer_name |
String | No | The CN or SAN field in the server certificate. If not specified, the hostname or IP address is used for verification. | server.example.com |
tls_peer_name_verify |
Boolean | No | Whether to verify the CN or SAN field in the server certificate. | true, false |
tls_passfile |
String | No | Path to the encrypted private key file. | /path/to/passfile |
tls_ciphers |
String | No | TLS 1.2 cipher suite. Configure only when tls_versions is set to TLSv1.2. |
ECDHE-RSA-AES256-GCM-SHA384 |
tls_ciphersuites |
String | No | TLS 1.3 cipher suite. Configure only when tls_versions is set to TLSv1.3. |
TLS_AES_256_GCM_SHA384 |
tls_versions |
String | No | Allowed TLS protocol versions, separated by commas. | TLSv1.2, TLSv1.3 |
tls_parameter_value¶
The value corresponding to the specified tls_parameter_name. See the table above for example values.
table_source¶
A table data source from which data is read to populate the temporary graph. A table is an in-memory table variable created during query execution.
Note that table_source and graph_source cannot be used together in the same IMPORT statement.
When importing from a table variable, the system performs strict type checking: the data type of each column in the table must match the corresponding property type in the graph type. Unlike file imports, implicit type conversion (e.g., STRING to INT) is not supported. If the types are incompatible, the import will fail due to a type mismatch.
Extra columns in the table variable that are not mapped to node or edge definition properties are ignored during import.
If a property defined in the graph type is not mapped to a table variable column, the default value defined in the graph type is used. If the property has no default value and is marked NOT NULL, the import will fail with an error.
table_variable¶
The name of a table variable defined within the procedure body. The table variable must be defined before it is referenced in an IMPORT statement. For more information, see Table variables.
data_option¶
The options for data processing when data is imported into the temporary graph. The following options are supported:
| Option | Type | Default value | Description |
|---|---|---|---|
PRIMARY_KEY_AS_NODE_ID |
Bool | - | Specifies whether to use the value of the property that serves as the primary key as the unique identifier of the node. This option takes effect only when the following conditions are met:
true or false to override the system behavior. Explicit settings apply only to integer primary keys. If set to true, the primary key value must fit within 6 bytes; otherwise the import fails. |
SKIP_CONFLICT_PK_NODES |
Bool | false |
Specifies whether to skip nodes with duplicate primary keys. This option takes effect only if the following conditions are met:
SKIP_CONFLICT_PK_NODES to true when nodes with duplicate primary keys exist in the data to be imported; otherwise, the import fails. |
SKIP_DANGLING_EDGES |
Bool | false |
Specifies whether to skip dangling edges. This option takes effect only if the temporary graph is created based on a graph type. Set SKIP_DANGLING_EDGES to true when dangling edges exist in the data to be imported; otherwise, the import fails. |
Examples¶
Import data from NebulaGraph Database Developer Edition¶
Import an entire graph from NebulaGraph Database Developer Edition¶
'''
// Create a graph type named `movie_type`
CREATE GRAPH TYPE IF NOT EXISTS movie_type AS {
NODE Actor (LABEL Person {id INT64, name STRING, birthDate DATE, PRIMARY KEY (id)}) ,
NODE Director (LABEL Person {id INT64, name STRING, birthDate DATE, PRIMARY KEY (id)}),
NODE `User` (LABEL Person {id INT64 PRIMARY KEY}),
NODE Movie (LABEL Movie {id INT64 PRIMARY KEY, name STRING}),
NODE Genre (LABEL Genre {id INT64 PRIMARY KEY, name STRING}),
EDGE Act (Actor)-[:Act]->(Movie),
EDGE Direct (Director)-[:Direct]->(Movie),
EDGE Watch (`User`)-[:Watch {rate FLOAT}]->(Movie),
EDGE WithGenre (Movie)-[:WithGenre]->(Genre)
}
'''
'''
// Create a temporary graph named `#movie_temporary_graph` based on graph type `movie_type`
CREATE TEMPORARY GRAPH IF NOT EXISTS #movie_temporary_graph TYPED movie_type
'''
'''
USE #movie_temporary_graph IMPORT INTO GRAPH {
GRAPH FROM NEBULA {
FORMAT: "NEBULA",
// Replace with actual NebulaGraph Database Developer Edition connection information
PATH: "nebula://root:<password>@<graph_host>:<graph_port>?graph=movie&schema=/default_schema"
}
}
'''
'''
// Verify the statistics of the temporary graph
USE #movie_temporary_graph SHOW STATS
'''
Return:
+------------+--------------+-----------+
| entry_name | element_type | total_num |
+------------+--------------+-----------+
| Node Total | Node | 4589 |
| Edge Total | Edge | 10175 |
+------------+--------------+-----------+
Import specific node and edge types from NebulaGraph Database Developer Edition¶
'''
// Create a graph type `movie_type` whose edge type declares MULTIEDGE KEY()
CREATE GRAPH TYPE IF NOT EXISTS movie_type AS {
NODE Actor (LABEL Person {id INT64, name STRING, birthDate DATE, PRIMARY KEY (id)}) ,
NODE Director (LABEL Person {id INT64, name STRING, birthDate DATE, PRIMARY KEY (id)}),
NODE `User` (LABEL Person {id INT64 PRIMARY KEY}),
NODE Movie (LABEL Movie {id INT64 PRIMARY KEY, name STRING}),
NODE Genre (LABEL Genre {id INT64 PRIMARY KEY, name STRING}),
EDGE Act (Actor)-[:Act {MULTIEDGE KEY()}]->(Movie)
}
'''
'''
// Create a temporary graph named `#movie_import` based on graph type `movie_type`
CREATE TEMPORARY GRAPH IF NOT EXISTS #movie_import TYPED movie_type
'''
'''
USE #movie_import IMPORT INTO GRAPH
{
NODE (v@Actor{id:id, name:name}) FROM NEBULA {
FORMAT: "NEBULA",
PATH: "nebula://root:<password>@<graph_host>:<graph_port>?graph=movie&node_type=Actor"
},
NODE (v@Movie{id:id, name:name}) FROM NEBULA {
FORMAT: "NEBULA",
PATH: "nebula://root:<password>@<graph_host>:<graph_port>?graph=movie&node_type=Movie"
},
EDGE (id:id)-[e@Act{}]->(id:id) FROM NEBULA {
FORMAT: "NEBULA",
PATH: "nebula://root:<password>@<graph_host>:<graph_port>?graph=movie&edge_type=Act"
}
}
'''
'''
// Verify the statistics of the temporary graph
USE #movie_import SHOW STATS
'''
Return:
+------------+--------------+-----------+
| entry_name | element_type | total_num |
+------------+--------------+-----------+
| Node Total | Node | 3557 |
| Edge Total | Edge | 3732 |
+------------+--------------+-----------+
Import data from files or file variables ¶
Import from CSV files¶
The following example uses pagerank_node.csv and pagerank_edge.csv as the source data.
u_id,u_name
1,Alice
2,Bob
3,Charlie
4,David
src_id,dst_id,src_name,dst_name
1,2,Alice,Bob
2,3,Bob,Charlie
1,4,Alice,David
3,4,Charlie,David
4,2,David,Bob
'''
// Create a graph type named `pagerank_graph_type`
CREATE GRAPH TYPE IF NOT EXISTS pagerank_graph_type AS {
NODE Person (LABEL Person {id INT, name STRING, PRIMARY KEY (id, name)}),
EDGE Knows (Person)-[LABEL Knows {MULTIEDGE KEY()}]->(Person)
}
'''
'''
// Create a temporary graph based on graph type `pagerank_graph_type`
CREATE TEMPORARY GRAPH IF NOT EXISTS #pagerank_graph TYPED pagerank_graph_type
'''
'''
// Import data directly from CSV files into temporary graph `#pagerank_graph`
USE #pagerank_graph IMPORT INTO GRAPH
{
NODE (v@Person {id: u_id, name: u_name}) FROM DATAFILE {
PATH: "file:///tmp/test/pagerank_node.csv",
FORMAT: "CSV",
DELIMITER: ','
},
EDGE (id: src_id, name: src_name)-[e@Knows{}]->(id: dst_id, name: dst_name) FROM DATAFILE {
PATH: "file:///tmp/test/pagerank_edge.csv",
FORMAT: "CSV",
DELIMITER: ','
}
} OPTIONS {PRIMARY_KEY_AS_NODE_ID:false}
'''
'''
// Verify the statistics of the temporary graph
USE #pagerank_graph SHOW STATS
'''
You can also import CSV files through file variables:
'''
FILE node_file { u_id INT, u_name STRING } = DATAFILE {
PATH: "file:///tmp/test/pagerank_node.csv",
FORMAT: "CSV",
DELIMITER: ','
}
FILE edge_file { src_id INT, dst_id INT, src_name STRING, dst_name STRING } = DATAFILE {
PATH: "file:///tmp/test/pagerank_edge.csv",
FORMAT: "CSV",
DELIMITER: ','
}
USE #pagerank_graph IMPORT INTO GRAPH {
NODE (v@Person {id: u_id, name: u_name}) FROM node_file,
EDGE (id: src_id, name: src_name)-[e@Knows{}]->(id: dst_id, name: dst_name) FROM edge_file
} OPTIONS {PRIMARY_KEY_AS_NODE_ID:false}
'''
Return:
+------------+--------------+-----------+
| entry_name | element_type | total_num |
+------------+--------------+-----------+
| Node Total | Node | 4 |
| Edge Total | Edge | 5 |
+------------+--------------+-----------+
Import from Parquet files¶
Apache Parquet is a columnar storage format widely used in big data analytics. The following example shows how to import data from Parquet files.
Assume the Parquet files have the following schemas and data:
-
pagerank_node.parquet: columnsu_id(INT) andu_name(STRING).u_id u_name 1 Alice 2 Bob 3 Carol 4 Dave
-
pagerank_edge.parquet: columnssrc_id(INT),dst_id(INT),src_name(STRING), anddst_name(STRING).src_id dst_id src_name dst_name 1 2 Alice Bob 1 3 Alice Carol 2 4 Bob Dave 3 4 Carol Dave 4 1 Dave Alice
'''
// Create a graph type
CREATE GRAPH TYPE IF NOT EXISTS pagerank_parquet_type AS {
NODE Person (LABEL Person {id INT, name STRING, PRIMARY KEY (id, name)}),
EDGE Knows (Person)-[LABEL Knows {MULTIEDGE KEY()}]->(Person)
}
'''
'''
// Create a temporary graph
CREATE TEMPORARY GRAPH IF NOT EXISTS #pagerank_parquet TYPED pagerank_parquet_type
'''
'''
// Import data from Parquet files
USE #pagerank_parquet IMPORT INTO GRAPH
{
NODE (v@Person {id: u_id, name: u_name}) FROM DATAFILE {
PATH: "file:///tmp/test/pagerank_node.parquet",
FORMAT: "PARQUET"
},
EDGE (id: src_id, name: src_name)-[e@Knows{}]->(id: dst_id, name: dst_name) FROM DATAFILE {
PATH: "file:///tmp/test/pagerank_edge.parquet",
FORMAT: "PARQUET"
}
} OPTIONS {PRIMARY_KEY_AS_NODE_ID:false}
'''
'''
// Verify the statistics of the temporary graph
USE #pagerank_parquet SHOW STATS
'''
You can also import Parquet files through file variables:
'''
FILE node_file { u_id INT, u_name STRING } = DATAFILE {
PATH: "file:///tmp/test/pagerank_node.parquet",
FORMAT: "PARQUET"
}
FILE edge_file { src_id INT, dst_id INT, src_name STRING, dst_name STRING } = DATAFILE {
PATH: "file:///tmp/test/pagerank_edge.parquet",
FORMAT: "PARQUET"
}
USE #pagerank_parquet IMPORT INTO GRAPH {
NODE (v@Person {id: u_id, name: u_name}) FROM node_file,
EDGE (id: src_id, name: src_name)-[e@Knows{}]->(id: dst_id, name: dst_name) FROM edge_file
} OPTIONS {PRIMARY_KEY_AS_NODE_ID:false}
'''
Return:
+------------+--------------+-----------+
| entry_name | element_type | total_num |
+------------+--------------+-----------+
| Node Total | Node | 4 |
| Edge Total | Edge | 5 |
+------------+--------------+-----------+
Import from ORC files¶
Apache ORC is a columnar storage format optimized for large-scale data processing. The following example shows how to import data from ORC files.
Assume the ORC files have the following schemas and data:
-
pagerank_node.orc: columnsu_id(INT) andu_name(STRING).u_id u_name 1 Alice 2 Bob 3 Carol 4 Dave
-
pagerank_edge.orc: columnssrc_id(INT),dst_id(INT),src_name(STRING), anddst_name(STRING).src_id dst_id src_name dst_name 1 2 Alice Bob 1 3 Alice Carol 2 4 Bob Dave 3 4 Carol Dave 4 1 Dave Alice
'''
// Create a graph type
CREATE GRAPH TYPE IF NOT EXISTS pagerank_orc_type AS {
NODE Person (LABEL Person {id INT, name STRING, PRIMARY KEY (id, name)}),
EDGE Knows (Person)-[LABEL Knows {MULTIEDGE KEY()}]->(Person)
}
'''
'''
// Create a temporary graph
CREATE TEMPORARY GRAPH IF NOT EXISTS #pagerank_orc TYPED pagerank_orc_type
'''
'''
// Import data from ORC files
USE #pagerank_orc IMPORT INTO GRAPH
{
NODE (v@Person {id: u_id, name: u_name}) FROM DATAFILE {
PATH: "file:///tmp/test/pagerank_node.orc",
FORMAT: "ORC"
},
EDGE (id: src_id, name: src_name)-[e@Knows{}]->(id: dst_id, name: dst_name) FROM DATAFILE {
PATH: "file:///tmp/test/pagerank_edge.orc",
FORMAT: "ORC"
}
} OPTIONS {PRIMARY_KEY_AS_NODE_ID:false}
'''
'''
// Verify the statistics of the temporary graph
USE #pagerank_orc SHOW STATS
'''
You can also import ORC files through file variables:
'''
FILE node_file { u_id INT, u_name STRING } = DATAFILE {
PATH: "file:///tmp/test/pagerank_node.orc",
FORMAT: "ORC"
}
FILE edge_file { src_id INT, dst_id INT, src_name STRING, dst_name STRING } = DATAFILE {
PATH: "file:///tmp/test/pagerank_edge.orc",
FORMAT: "ORC"
}
USE #pagerank_orc IMPORT INTO GRAPH {
NODE (v@Person {id: u_id, name: u_name}) FROM node_file,
EDGE (id: src_id, name: src_name)-[e@Knows{}]->(id: dst_id, name: dst_name) FROM edge_file
} OPTIONS {PRIMARY_KEY_AS_NODE_ID:false}
'''
Return:
+------------+--------------+-----------+
| entry_name | element_type | total_num |
+------------+--------------+-----------+
| Node Total | Node | 4 |
| Edge Total | Edge | 5 |
+------------+--------------+-----------+
Import data from table variables¶
You can import data from table variables into a temporary graph. This is useful when you want to build graph data directly from query results or literals without writing intermediate data to external files.
'''
// Create a graph type named movie_type_local_table, whose edge type declares MULTIEDGE KEY()
CREATE GRAPH TYPE IF NOT EXISTS movie_type_local_table AS {
NODE Actor (LABEL Person {id INT64, name STRING, birthDate DATE, PRIMARY KEY (id)}),
NODE Director (LABEL Person {id INT64, name STRING, birthDate DATE, PRIMARY KEY (id)}),
NODE `User` (LABEL Person {id INT64 PRIMARY KEY}),
NODE Movie (LABEL Movie {id INT64 PRIMARY KEY, name STRING}),
NODE Genre (LABEL Genre {id INT64 PRIMARY KEY, name STRING}),
EDGE Act (Actor)-[:Act {MULTIEDGE KEY()}]->(Movie)
}
'''
'''
// Create a temporary graph based on graph type movie_type_local_table
CREATE TEMPORARY GRAPH IF NOT EXISTS #movie_local_table TYPED movie_type_local_table
'''
'''
// Import data from table variables
USE #movie_local_table {
TABLE actors TYPED TABLE {id INT64, name STRING, birthDate DATE} =
{id: 1, name: "Tom Hanks", birthDate: date("1956-07-09")},
{id: 2, name: "Morgan Freeman", birthDate: date("1937-06-01")},
{id: 3, name: "Tim Robbins", birthDate: date("1958-10-16")}
TABLE directors TYPED TABLE {id INT64, name STRING, birthDate DATE} =
{id: 101, name: "Frank Darabont", birthDate: date("1959-01-28")},
{id: 102, name: "Robert Zemeckis", birthDate: date("1952-05-14")}
TABLE users TYPED TABLE {id INT64} =
{id: 1001},
{id: 1002}
TABLE movies TYPED TABLE {id INT64, name STRING} =
{id: 201, name: "The Shawshank Redemption"},
{id: 202, name: "Forrest Gump"},
{id: 203, name: "Se7en"}
TABLE genres TYPED TABLE {id INT64, name STRING} =
{id: 301, name: "Drama"},
{id: 302, name: "Crime"}
TABLE act_edges TYPED TABLE {src_id INT64, dst_id INT64} =
{src_id: 1, dst_id: 202},
{src_id: 2, dst_id: 201},
{src_id: 2, dst_id: 203},
{src_id: 3, dst_id: 201}
IMPORT INTO GRAPH {
NODE (a@Actor{ id: id, name: name, birthDate: birthDate }) FROM actors,
NODE (d@Director{ id: id, name: name, birthDate: birthDate }) FROM directors,
NODE (u@`User`{ id: id }) FROM users,
NODE (m@Movie{ id: id, name: name }) FROM movies,
NODE (g@Genre{ id: id, name: name }) FROM genres,
EDGE (id:src_id)-[e@Act{}]->(id:dst_id) FROM act_edges
}
}
'''
'''
// Verify the statistics of the temporary graph
USE #movie_local_table SHOW STATS
'''
Return:
+------------+--------------+-----------+
| entry_name | element_type | total_num |
+------------+--------------+-----------+
| Node Total | Node | 12 |
| Edge Total | Edge | 4 |
| Actor | Node | 3 |
| Director | Node | 2 |
| User | Node | 2 |
| Movie | Node | 3 |
| Genre | Node | 2 |
| Act | Edge | 4 |
+------------+--------------+-----------+