Export data with NebulaGraph Exchange

Use NebulaGraph Exchange to export a graph to files or to another graph. The source and destination graphs may be in the same NebulaGraph Database Developer Edition cluster or in different clusters.

Prerequisites

• Deploy NebulaGraph Exchange.
• Start the source NebulaGraph Database Developer Edition service group. See Quick start.
• If exporting to another cluster, start its destination service group as well.

Run an export

1. In the nebula-graph-exchange/ directory, start Spark:

   ./start.sh
   

   The script prints the process IDs of the Spark master and workers when they start successfully.

2. In the same directory, run export.sh:

   ./export.sh [ -f | -g ] -c "<export_parameters>"
   

   Use -f to export to files or -g to export to a graph. Both modes require -c with the relevant export parameters.

   Option	Description	Required	Default
   -f	Export to files. Required if -g is omitted.	No	None
   -g	Export to NebulaGraph Database Developer Edition. Required if -f is omitted.	No	None
   -c	Export parameters; see the sections below.	Yes	None
   -j	Path to the NebulaGraph Exchange JAR.	No	nebula-exchange-5.3.0.jar
   -m	Maximum memory per job.	No	1G
   -n	Number of export jobs.	No	20
   -p	CPU cores used in parallel per job.	No	5
   -P	Spark server port.	No	1077
   -h	Print help.	No	None
   -s	Spark home directory, if you use your own Spark installation.	No	None
   -o	Spark master mode when using your own Spark installation.	No	None

   Set n × p lower than c × i, where c and i are the values set during NebulaGraph Exchange deployment.

3. Wait for the export to finish. For a file export, the output includes a log path and a completion message:

   start to export data to file...
   logging to /home/user/tmp/nebula-graph-exchange/logs/export_to_file.log
   25/05/06 18:21:37 INFO Nebula2File$: >>>>> finished the export! cost: 15.743s.
   

   For an export to another graph, the corresponding log is export_to_nebula.log:

   start to export data to nebula...
   logging to /home/user/tmp/nebula-graph-exchange/logs/export_to_nebula.log
   25/05/06 18:21:37 INFO Nebula2Nebula$: >>>>> finished the sync! cost: 15.743s.
   

4. Check the result. For files, inspect the location specified by --writePath. For a graph export, query the destination graph at --targetGraphAddress. If the result is unexpected, inspect the log named above for the chosen mode.

Export parameter reference

Pass export parameters as a double-quoted value of -c.

Export to files

For example:

Replace <source-password> and <target-password> with the current passwords for the source and target accounts, respectively. If a password has changed, use the new password.

./export.sh -f -c "-sourceGraph 127.0.0.1:9669 -sourceGraphName movie -sourceGraphType movie_type -sourcePasswd <source-password> -path file:///tmp/export1/"

Option	Alias	Description	Required	Default
--excludeEdges	-excludeEdges	Edge types to exclude, separated by commas.	No	None
--excludeNodes	-excludeNodes	Node types to exclude, separated by commas.	No	None
--format	-f	File format: csv, txt, or json.	No	csv
--header	-h	Write a header row in CSV files: true or false.	No	true
--includeNode	-includeNode	Export only this node type, useful for testing.	No	None
--writePath	-path	Destination path. Use hdfs:// for HDFS or file:/// for a local file system.	Yes	None
--readBatchSize	-readBatchSize	Batch size for read requests.	No	None
--sep	-s	Field separator in CSV files.	No	Comma
--sourceGraphAddress	-sourceGraph	Source Graph service addresses, separated by commas.	Yes	None
--sourceGraphName	-sourceGraphName	Source graph name.	Yes	None
--sourceGraphType	-sourceGraphType	Source graph type.	Yes	None
--sourceGraphPassword	-sourcePasswd	Password for the source Graph service.	Yes	None
--sourceSchema	-sourceSchema	Source Schema path.	No	None
--sourceGraphUser	-sourceUser	Username for the source Graph service.	No	root
--timeout	-timeout	Client timeout in seconds. Requests exceeding it are cancelled.	No	5

Export to a graph

For example:

./export.sh -g -c "-sourceGraph 127.0.0.1:9669 -sourceGraphName movie -sourceGraphType movie_type -sourcePasswd <source-password> -targetGraph 127.0.0.1:9669 -targetGraphName movie1 -targetPasswd <target-password>"

Option	Alias	Description	Required	Default
--excludeEdges	-excludeEdges	Edge types to exclude, separated by commas.	No	None
--excludeNodes	-excludeNodes	Node types to exclude, separated by commas.	No	None
--includeNode	-includeNode	Export only this node type, useful for testing.	No	None
--mode	-m	Write mode. All modes insert nodes and edges that do not exist. For existing nodes and edges, insert fails the export and reports an error; insert_ignore ignores the write and keeps existing data unchanged without reporting an error; insert_replace replaces them with the submitted data; insert_update updates them with the submitted property values. See INSERT.	No	insert_ignore
--parallel	-p	Write parallelism. -1 matches the NebulaGraph Database Developer Edition partition parallelism; see SHOW PARTITIONS.	No	-1
--readBatchSize	-readBatchSize	Batch size for read requests.	No	None
--sourceGraphAddress	-sourceGraph	Source Graph service addresses, separated by commas.	Yes	None
--sourceGraphName	-sourceGraphName	Source graph name.	Yes	None
--sourceGraphType	-sourceGraphType	Source graph type.	Yes	None
--sourceGraphPassword	-sourcePasswd	Password for the source Graph service.	Yes	None
--sourceSchema	-sourceSchema	Source Schema path.	No	None
--sourceGraphUser	-sourceUser	Username for the source Graph service.	No	root
--targetGraphAddress	-targetGraph	Destination Graph service addresses, separated by commas.	Yes	None
--targetGraphName	-targetGraphName	Destination graph name; if omitted, uses the source graph name.	No	None
--targetGraphType	-targetGraphType	Destination graph type; if omitted, uses the source graph type.	No	None
--targetGraphPassword	-targetPasswd	Password for the destination Graph service.	Yes	None
--targetSchema	-targetSchema	Destination Schema path.	No	None
--targetUser	-targetUser	Username for the destination Graph service.	No	root
--timeout	-timeout	Client timeout in seconds. Requests exceeding it are cancelled.	No	5
--writeBatchSize	-writeBatchSize	Batch size for write requests.	No	None