Reading from NebulaGraph Database Developer Edition¶
The Spark connector allows you to read data from nodes, edges, or the result of a GQL query and then load it into a Spark DataFrame. The examples below use reader options and configuration objects.
Reading nodes¶
The following samples read nodes of node type Movie from the graph movie and load the node properties into a Spark DataFrame.
By using reader options¶
Replace <current-password> with the current password for the specified account. If the password has changed, use the new password.
val df = spark
.read
.format("com.vesoft.nebula.spark.connector.NebulaDataSource")
.option("type", "node")
.option("operate_type", "read")
.option("graph_address", "127.0.0.1:9669")
.option("label", "User")
.option("user", "root")
.option("passwd", "<current-password>")
.option("graph_name", "movie")
.option("batch_size", "10")
.load()
By using configuration objects¶
val connectionConfig: NebulaConnectionConfig = NebulaConnectionConfig
.builder()
.withGraphAddress("127.0.0.1:9669")
.withUser("root")
.withPasswd("<current-password>")
.build()
val nebulaNodeReadConfig: ReadNebulaConfig = ReadNebulaConfig
.builder()
.withGraphName("movie")
.withTypeName("Movie")
.withReturnCols(List("id"))
.withBatchSize(10)
.withPartitionNum(1)
.build()
val df = spark.read.nebula(connectionConfig, nebulaNodeReadConfig).loadNode()
Reading edges¶
The following samples read edges of edge type Act from the graph movie and load the edge properties into a Spark DataFrame.
By using reader options¶
val df = spark.read
.format("com.vesoft.nebula.spark.connector.NebulaDataSource")
.option("type", "edge")
.option("operate_type", "read")
.option("graph_address", "127.0.0.1:9669")
.option("user", "root")
.option("passwd", "<current-password>")
.option("graph_name", "movie")
.option("label", "Watch")
.option("batch_size", "10")
.load()
By using configuration objects¶
val connectionConfig: NebulaConnectionConfig = NebulaConnectionConfig
.builder()
.withGraphAddress("127.0.0.1:9669")
.withUser("root")
.withPasswd("<current-password>")
.build()
val nebulaReadEdgeConfig: ReadNebulaConfig = ReadNebulaConfig
.builder()
.withGraphName("movie")
.withTypeName("Act")
.withReturnCols(null)
.withBatchSize(1000)
.withPartitionNum(1)
.build()
val df = spark.read.nebula(connectionConfig, nebulaReadEdgeConfig).loadEdge()
Reading query results¶
The following samples read the results of a GQL query from the graph movie and load the results into a Spark DataFrame.
By using reader options¶
val df = spark
.read
.format("com.vesoft.nebula.spark.connector.NebulaDataSource")
.option("type", "gql")
.option("operate_type", "read")
.option("graph_address", "127.0.0.1:9669")
.option("user", "root")
.option("passwd", "<current-password>")
.option("gql","use movie match(v:Movie) return v.id, v.name")
.load()
By using configuration objects¶
val connectionConfig: NebulaConnectionConfig = NebulaConnectionConfig
.builder()
.withGraphAddress("127.0.0.1:9669")
.withUser("root")
.withPasswd("<current-password>")
.build()
val nebulaGqlConfig:GqlNebulaConfig = GqlNebulaConfig
.builder()
.withGql("USE movie MATCH(v:Movie) RETURN v.id, v.name")
.build()
val df = spark.read.gql(connectionConfig, nebulaGqlConfig).load()