Import data with GQL templates¶

NebulaGraph Exchange supports importing data into NebulaGraph Database Developer Edition by using custom GQL templates. Unlike the standard workflow in Import data, this mode lets you write GQL statement templates directly. The system reads the source data row by row, fills placeholders in the template, and sends the generated statements to NebulaGraph Database Developer Edition for execution.

How this differs from the standard import workflow¶

	Standard import	Template-based GQL import
Task type (`taskType`)	`import`	`gqlTemplate`
Write mode	Choose from seven predefined modes (`INSERT`, `INSERT IGNORE`, `INSERT REPLACE`, `INSERT UPDATE`, `UPDATE`, `DELETE`, and `DETACH DELETE`)	Write any supported GQL statements directly
Node and edge mapping	Use `nodetypes` and `edgetypes` for mapping	Do not use `nodetypes` or `edgetypes`; define write logic directly in templates

Prerequisites¶

Deploy NebulaGraph Exchange.
The target NebulaGraph Database Developer Edition instance is running and reachable.
The target graph and graph type are already created, or are created automatically before import by using nebula.preHooks.
Source data files are ready.

Quick start¶

The following example shows a complete custom GQL template import configuration. It reads person data from a local CSV file and inserts nodes.

Prepare the source data files and the configuration file.

CSV file /tmp/person.csv:

id,name,age
1,Alice,30
2,Bob,25
3,Charlie,35

Configuration file:

Replace <current-password> with the current password for the specified account. If the password has changed, use the new password.

{
  # Task type: gqlTemplate
  taskType: gqlTemplate

  # Connection information for the target NebulaGraph Database Developer Edition instance
  nebula: {
    graphAddr: "127.0.0.1:9669"
    graphName: "demo_graph"
    user: "root"
    passwd: "<current-password>"
    preHooks:[
        """
        CREATE GRAPH TYPE IF NOT EXISTS demo_graph_type AS {
            NODE Person (LABEL Person {id INT PRIMARY KEY, name STRING, age INT}),
            EDGE Follow (Person)-[LABEL Follow {degree INT, MULTIEDGE KEY()}]->(Person)
        }
        """,
        "CREATE GRAPH IF NOT EXISTS movie TYPED movie_type"
        ]
  }

  # Error handling: write failed records to this directory
  error: {
    path: "/tmp/errors"
    maxRecords: 100
  }

  # Checkpoint configurations
  checkPoint: {
    path: "/tmp/checkpoint"
    resume: true
  }

  # Shared source defaults for all templates
  sourcesCommon: {
    type: "local"
    format: "csv"
    header: true
    separator: ","
  }

  # GQL template list, each defines one task
  gqlTemplates: [
    {
      name: "insert_person"
      template: "USE demo_graph INSERT (@Person{id: $${id}, name: '$${name}', age: $${age}})"
      source: {
        path: "file:///tmp/person.csv"
      }
    }
  ]
}

The system reads the CSV file row by row and replaces placeholders in the form $${field_name} with actual values. For example, the first row generates the following GQL statement:

USE demo_graph INSERT (@Person{id: 1, name: "Alice", age: 30})

Run the import command:
```
java -jar cli-5.3.0.jar -f <path_of_config_file> -n 5 -p 2
```
For information about the command, see Import data with NebulaGraph Exchange.

Configuration reference¶

The structure of a gqlTemplate configuration file is as follows:

{
    taskType: gqlTemplate

    nebula: {}

    error: {}

    checkPoint: {}

    sourcesCommon: {}

    gqlTemplates: []
}

The configuration items in nebula, error, checkPoint, and sourcesCommon are the same as those in the standard import workflow. For details, see the corresponding sections in Data import configurations:

nebula: see NebulaGraph Database Developer Edition configurations.
error: see Error configurations.
checkPoint: see Checkpoint file configurations.
sourcesCommon: see Common source configurations.

Note

The gqlTemplate mode does not use the writeCommon or sources configuration sections.

GQL template configurations (`gqlTemplates`)¶

gqlTemplates is an array. Each item defines one GQL template task.

Parameter	Type	Required	Description
name	String	Yes	Template name. Used to identify the task and error log directory.
template	String	Yes	GQL statement template. Use `$${field_name}` placeholders to reference source fields.
source	Object	No	Source configuration for this template. Fields with the same names override those in `sourcesCommon`. For supported fields and formats of different data source types, see the Data source configurations for each data source type.

Placeholder usage

Use $${field_name} (double dollar signs), not ${field_name}.
The generated statements are sent directly to NebulaGraph Database Developer Edition for execution, so any supported GQL statements can be used in templates.

Priority between `sourcesCommon` and `gqlTemplates[].source`¶

Data source configurations can be specified in two places: sourcesCommon and each template's gqlTemplates[].source. sourcesCommon provides global defaults, and gqlTemplates[].source overrides fields with the same names. The resolution priority is gqlTemplates[].source > sourcesCommon > system defaults.

Examples¶

The following examples are based on the graph type and graph below. If they do not exist in the target instance, you can create them automatically before import by using nebula.preHooks.

"""
-- Graph type
CREATE GRAPH TYPE IF NOT EXISTS demo_graph_type AS {
    NODE Person (LABEL Person {id INT PRIMARY KEY, name STRING, age INT}),
    EDGE Follow (Person)-[LABEL Follow {degree INT, MULTIEDGE KEY()}]->(Person)
}
"""

"""
-- Graph
CREATE GRAPH IF NOT EXISTS demo_graph TYPED demo_graph_type
"""

Import nodes¶

This example reads person data from a CSV file and inserts nodes through a GQL template.

CSV file /tmp/person.csv:

id,name,age
1,Alice,30
2,Bob,25
3,Charlie,35

Configuration file:

{
  taskType: gqlTemplate

  nebula: {
    graphAddr: "127.0.0.1:9669"
    graphName: "demo_graph"
    user: "root"
    passwd: "<current-password>"
  }

  sourcesCommon: {
    type: "local"
    format: "csv"
    header: true
    separator: ","
  }

  gqlTemplates: [
    {
      name: "insert_person"
      template: "USE demo_graph INSERT (@Person{id: $${id}, name: '$${name}', age: $${age}})"
      source: {
        path: "file:///tmp/person.csv"
      }
    }
  ]
}

Import edges¶

This example reads relationship data from a CSV file and inserts edges through a GQL template.

CSV file /tmp/follow.csv:

src,dst,degree
1,2,90
2,3,80
3,1,85

Configuration file:

{
  taskType: gqlTemplate

  nebula: {
    graphAddr: "127.0.0.1:9669"
    graphName: "demo_graph"
    user: "root"
    passwd: "<current-password>"
  }

  sourcesCommon: {
    type: "local"
    format: "csv"
    header: true
    separator: ","
  }

  gqlTemplates: [
    {
      name: "insert_follow"
      template: "USE demo_graph MATCH (s@Person), (t@Person) WHERE s.id = $${src} AND t.id = $${dst} INSERT (s)-[@Follow{degree: $${degree}}]->(t)"
      source: {
        path: "file:///tmp/follow.csv"
      }
    }
  ]
}

Import with multiple templates¶

This example defines multiple templates in one configuration file and imports both nodes and edges.

{
  taskType: gqlTemplate

  nebula: {
    graphAddr: "127.0.0.1:9669"
    graphName: "demo_graph"
    user: "root"
    passwd: "<current-password>"
  }

  sourcesCommon: {
    type: "local"
    format: "csv"
    header: true
    separator: ","
  }

  gqlTemplates: [
    {
      name: "insert_person"
      template: "USE demo_graph INSERT (@Person{id: $${id}, name: '$${name}', age: $${age}})"
      source: {
        path: "file:///tmp/person.csv"
      }
    },
    {
      name: "insert_follow"
      template: "USE demo_graph MATCH (s@Person), (t@Person) WHERE s.id = $${src} AND t.id = $${dst} INSERT (s)-[@Follow{degree: $${degree}}]->(t)"
      source: {
        path: "file:///tmp/follow.csv"
      }
    }
  ]
}

Import data with GQL templates¶

How this differs from the standard import workflow¶

Prerequisites¶

Quick start¶

Configuration reference¶

GQL template configurations (gqlTemplates)¶

Priority between sourcesCommon and gqlTemplates[].source¶

Examples¶

Import nodes¶

Import edges¶

Import with multiple templates¶

GQL template configurations (`gqlTemplates`)¶

Priority between `sourcesCommon` and `gqlTemplates[].source`¶