Tencent Cloud configurations

This topic introduces the configuration options for importing data from a Tencent Cloud data source. For information about the complete configuration file and options, see Data import configurations.

Data source configurations

The following table lists the configuration options specific to the Tencent Cloud data source.

Option	Description	Type	Required	Default value
type	The type of the data source. In this case, set the value to "oss".	String	Yes	N/A
format	The file format of the source data. Valid values are: "csv", "json", "orc",and "parquet"	String	No	"csv"
region	The region for the bucket that needs to be accessed. For more information, see Regions and Access Endpoints.	String	Yes	N/A
path	The path to the source data in Tencent cloud. The path can be specified as a directory or a file using the prefix cosn:// with an absolute path.	String	Yes	N/A
header	Whether the CSV file contains a header. This option takes effect only when format is set to "csv".	Bool	No	false
separator	The separator used in the CSV file. This option takes effect only when format is set to "csv".	String	No	","
secretId	The SecretId of Tencent Cloud OSS (Object Storage Service).	String	Yes	N/A
secretKey	The SecretKey of Tencent Cloud OSS.	String	Yes	N/A
endpointSuffix	The domain suffix set for Tencent Cloud OSS. For more information, see Setting Access Domain Names (CDN/Global Acceleration).	String	No	N/A
credentialsProvider	The method to obtain the Tencent Cloud OSS SecretId and SecretKey. For more information, see the configuration note of the attribute key fs.cosn.credentials.provider	String	No	N/A
useHttps	Whether to use HTTPS as the protocol with the Tencent Cloud object storage server.	Bool	No	false
clientEncryptionEnabled	Whether to enable encryption for the client.	String	No	false
clientEncryptionPublicKeyPath	The path for the client public key.	String	No	N/A
clientEncryptionPrivateKeyPath	The path for the client private key.	String	No	N/A

Common source configurations

The following configuration options can also be specified as global settings in the section sourcesCommon of the configuration file:

• type
• format
• region
• header
• separator
• secretId
• secretKey
• endpointSuffix
• credentialsProvider
• useHttps
• clientEncryptionEnabled
• clientEncryptionPublicKeyPath
• clientEncryptionPrivateKeyPath

Note that the configurations on specific data sources override the global settings.

Data source configuration example

Tencent Cloud source configurations

sources: [
{
      type:"oss"
      format: "csv"
      region:"ap-guangzhou"
      secretId:"xxxx"
      secretKey:"xxx"
      path: "cosn://import_movie/player.csv"
      header: true
      separator: ","
//    nodetypes: []
//    edgetypes: []
    }
] 

Complete configuration example

Replace <current-password> with the current password for the specified account. If the password has changed, use the new password.

{
  taskType: import
  nebula: {
    graphAddr: "127.0.0.1:9669"
    graphName: "movie"
    user: "root"
    passwd: "<current-password>"
  }

  error: {
    path: "/tmp/errors"
    maxRecords: 100
  }

  checkPoint: {
    path: "/tmp/checkpoint"
    resume: false
  } 

  sourcesCommon:{
    type:"oss"
    format: "csv"
    region:"ap-guangzhou"
    secretId:"xxxx"
    secretKey:"xxx"    
    header: true
    separator: ","
  }

   writeCommon:{
    batchSize: 2000
    enableRealTimeQPSLog: true
    mode: "INSERT IGNORE"
  } 

  sources: [
    # node type Actor
    {   
      path: "cosn://import_movie/sub_actor.csv"

      pre_processes: {
          filter: ["birth_date!='\\N'"]
          concatConstant: [
            {
              field: "name"
              prefixValue: "test_"
              suffixValue: ""
            }
          ]
      }

      nodetypes: [
        {
          name: "Actor"
          sourceFields: [people_id,name,birth_date]
          nebulaFields: [id,name,birthDate]
        }
      ]
    }

    # node type Movie
    {    
      path: "cosn://import_movie/sub_movie.csv"
      nodetypes: [
        {
          name: "Movie"
          sourceFields: [movie_id,name]
          nebulaFields: [id,name]
        }
      ]
    }

   # edge type Act
    {   
      path: "cosn://import_movie/sub_actor_act_movie.csv"
      edgetypes: [
        {
          name: "Act"
          sourceKeys: [people_id]
          targetKeys: [movie_id]
          sourceFields: []
          nebulaFields: []
        }
      ]
    }

  ]
}