Kafka connector configurations¶
Configure the NebulaGraph Database Developer Edition Kafka connector with the options below.
Configuration parameters¶
| Parameter | Description | Required | Default value | Data type |
|---|---|---|---|---|
name |
The name of the connector. | Yes | N/A | String |
connector.class |
The entry class of the connector. Set the value to com.vesoft.nebula.connector.sink.NebulaSinkConnector. |
Yes | N/A | String |
tasks.max |
The maximum number of tasks that can be created for the connector. | Yes | N/A | Integer |
topics |
The Kafka topics from which the connector sinks data into NebulaGraph Database Developer Edition. Separate multiple topics with commas. | Yes | N/A | String |
nebula.graph.servers |
The IPs and ports of Graph services. Separate multiple addresses with commas. | Yes | N/A | String |
nebula.user |
The username of the Graph service. | Yes | N/A | String |
nebula.passwd |
The password of the user. Required if nebula.authOptions is not specified. |
No | N/A | String |
nebula.authOptions |
The authentication information of the user. For password authentication, the authentication information must be in the format {password:<password>}. Required if nebula.passwd is not specified. |
No | N/A | String |
nebula.schema |
The schema into which the connector sinks data. If not specified, the home schema in NebulaGraph Database Developer Edition is used. | No | N/A | String |
nebula.date.format |
The date format to be set in the session. If not specified, the date format set in NebulaGraph Database Developer Edition is used. | No | N/A | String |
nebula.zonedDatetime.format |
The zoned datetime format to be set in the session. If not specified, the zoned datetime format set in NebulaGraph Database Developer Edition is used. | No | N/A | String |
nebula.localDatetime.format |
The local datetime format to be set in the session. If not specified, the local datetime format set in NebulaGraph Database Developer Edition is used. | No | N/A | String |
nebula.zonedTime.format |
The zoned time format to be set in the session. If not specified, the zoned time format set in NebulaGraph Database Developer Edition is used. | No | N/A | String |
nebula.localTime.format |
The local time format to be set in the session. If not specified, the local time format set in NebulaGraph Database Developer Edition is used. | No | N/A | String |
nebula.enable.tls |
Whether to enable TLS encryption. | No | false |
Boolean |
nebula.disable.verify.cert |
Whether to disable server certificate verification. | No | false |
Boolean |
nebula.tls.ca.path |
The path to the CA (Certificate Authority) certificate file, which is used to verify the server's certificate. Required if nebula.enable.tls is set to true and nebula.disable.verify.cert is set to false. |
No | N/A | String |
nebula.tls.cert.path |
The path to the certificate file of the client. Required if tls_client_verify is set to true in the configuration file of NebulaGraph Database Developer Edition's Graph service. |
No | N/A | String |
nebula.tls.key.path |
The path to the private key file of the client. Required if tls_client_verify is set to true in the configuration file of NebulaGraph Database Developer Edition's Graph service. |
No | N/A | String |
nebula.connect.timeout |
The timeout in milliseconds for establishing a connection to the Graph service. | No | 3000 |
Integer |
nebula.request.timeout |
The timeout in milliseconds for each request to the Graph service. | No | 3000 |
Integer |
nebula.sink.partitions |
The number of partitions for the connector. | No | 10 |
Integer |
nebula.sink.interval.time.mills |
The time interval in milliseconds between retries when the connector fails to sink data into NebulaGraph Database Developer Edition. | No | 0 |
Integer |
nebula.sink.retry.times |
The maximum number of retries when the connector fails to sink data into NebulaGraph Database Developer Edition. | No | 0 |
Integer |
nebula.graph.name |
The name of the graph into which the connector sinks data. | Yes | N/A | String |
nebula.data.type |
The type of graph element in the graph. Valid values are NODE, EDGE, and BOTH. |
No | NODE |
String |
nebula.node.typeName |
The name of the node type into which the connector sinks data. Required if nebula.data.type is set to NODE or BOTH. |
No | N/A | String |
nebula.node.primaryKeys |
The properties that constitute the primary key of the node type. Required if nebula.data.type is set to NODE or BOTH and the primary key consists of multiple properties. |
No | N/A | LIST |
kafka.node.primaryKeys |
The fields in the Kafka topic that are written as the properties of the primary key of the node type. The order of the fields must match the order of the properties in nebula.node.primaryKeys. Required if nebula.data.type is set to NODE or BOTH and nebula.sink.mode is set to UPDATE or DELETE. |
No | N/A | LIST |
nebula.node.property.names |
The properties of the node type into which the connector sinks data. Required if nebula.data.type is set to NODE or BOTH. |
No | N/A | LIST |
kafka.node.property.names |
The fields in the Kafka topic that are written as the properties of the node type. The order of the fields must match the order of the properties in nebula.node.property.names. Required if nebula.data.type is set to NODE or BOTH. |
No | N/A | LIST |
nebula.edge.typeName |
The name of the edge type into which the connector sinks data. Required if nebula.data.type is set to EDGE or BOTH. |
No | N/A | String |
nebula.edge.srcPks |
The properties that constitute the primary key of the source node type of the edge type. Required if nebula.data.type is set to EDGE or BOTH and the primary key consists of multiple properties. |
No | N/A | LIST |
kafka.edge.srcPks |
The fields in the Kafka topic that are written as the properties of the primary key of the source node type of the edge type. The order of the fields must match the order of the properties in nebula.edge.srcPks. Required if nebula.data.type is set to EDGE or BOTH. |
No | N/A | LIST |
nebula.edge.dstPks |
The properties that constitute the primary key of the destination node type of the edge type. Required if nebula.data.type is set to EDGE or BOTH and the primary key consists of multiple properties. |
No | N/A | LIST |
kafka.edge.dstPks |
The fields in the Kafka topic that are written as the properties of the primary key of the destination node type of the edge type. The order of the fields must match the order of the properties in nebula.edge.dstPks. Required if nebula.data.type is set to EDGE or BOTH. |
No | N/A | LIST |
nebula.edge.property.names |
The properties of the edge type into which the connector sinks data. Required if nebula.data.type is set to EDGE or BOTH. |
No | N/A | LIST |
kafka.edge.property.names |
The fields in the Kafka topic that are written as the properties of the edge type. The order of the fields must match the order of the properties in nebula.edge.property.names. Required if nebula.data.type is set to EDGE or BOTH. |
No | N/A | LIST |
nebula.batchSize |
The total number of nodes and edges processed in each request when sinking data into NebulaGraph Database Developer Edition. | No | 2000 |
Integer |
nebula.sink.mode |
The data manipulation mode when sinking data into NebulaGraph Database Developer Edition. Valid value are INSERT, INSERTIGNORE, INSERTREPLACE, UPDATE, DELETE, and DETACHDELETE. |
No | INSERTIGNORE |
String |
kafka.null.value |
The field value in the Kafka topic that is treated as null. |
No | \N |
String |
key.converter |
The converter class that overrides the default key converter class set by the Kafka worker. | No | org.apache.kafka.connect.storage.StringConverter |
String |
value.converter |
The converter class that overrides the default value converter class set by the Kafka worker. | No | org.apache.kafka.connect.storage.StringConverter |
String |