Source: https://dev-docs.nebula-graph.io/v5.3.2/utilities/exchange/config/sources/s3/
Documentation version: v5.3.2

# Amazon S3 configurations

This topic introduces the configuration options for importing data from an Amazon S3 data source. For information about the complete configuration file and options, see [Data import configurations](https://dev-docs.nebula-graph.io/v5.3.2/utilities/exchange/config/).

## Data source configurations

The following table lists the configuration options specific to the Amazon S3 data source.

 | Option | Description | Type | Required | Default value
 | `type` | The type of the data source. In this case, set the value to `"s3"`. | String | Yes | N/A
 | `format` | The file format of the source data. Valid values are: `"csv"`, `"json"`, `"orc"`,and `"parquet"` | String | No | `"csv"`
 | `path` | The path to the source data in Amazon S3. The path can be specified as a directory or a file using the prefix `s3a://` with an absolute path. | String | Yes | N/A
 | `endpoint` | The service endpoint of the Amazon S3 server in the format of `s3.<region-code>.amazonaws.com`. Replace `<region-code>` with the region code of your AWS account. For more information, see [Region codes](https://docs.aws.amazon.com/general/latest/gr/rande.html#regional-endpoints). | String | Yes | N/A
 | `accessKey` | The access key for accessing the Amazon S3 bucket. | String | Yes | N/A
 | `secretKey` | The secret key for accessing the Amazon S3 bucket. | String | Yes | N/A
 | `header` | Whether the CSV file contains a header. This option takes effect only when `format` is set to `"csv"`. If the CSV file has no header, use the format `_cX` to refer to columns when you [configure the mapping](https://dev-docs.nebula-graph.io/v5.3.2/utilities/exchange/config/sources/mapping/) between Amazon S3 data and graph elements. For example, `_c0` represents the first column, `_c1` represents the second column, and so on. | Bool | No | `false`
 | `separator` | The separator used in the CSV file. This option takes effect only when `format` is set to `"csv"`. | String | No | `","`

## Common source configurations

The following configuration options can also be specified as global settings in the section `sourcesCommon` of the configuration file:

- `type`
- `format`
- `header`
- `separator`
- `endpoint`
- `accessKey`
- `secretKey`

Note that the configurations on specific data sources override the global settings.

## Data source configuration example

Amazon S3 source configurations

````
sources: [
{
      type:"s3"
      format: "csv"
      path: "s3a://bucket/path/player.csv"
      endpoint: "s3.us-west-2.amazonaws.com"
      accessKey: "xxx"
      secretKey: "xxxx"
      header: true
      separator: ","
//      nodetypes: []
//      edgetypes: []
    }
]  

````

## Complete configuration example

Replace `<current-password>` with the current password for the specified account. If the password has changed, use the new password.

````
{
  taskType: import
  nebula: {
    graphAddr: "127.0.0.1:9669"
    graphName: "movie"
    user: "root"
    passwd: "<current-password>"
  }

  error: {
    path: "/tmp/errors"
    maxRecords: 100
  }

  checkPoint: {
    path: "/tmp/checkpoint"
    resume: false
  } 

  sourcesCommon:{
    type:"s3"
    format: "csv"
    endpoint: "s3.us-west-2.amazonaws.com"
    accessKey: "xxx"
    secretKey: "xxxx"
    header: true
    separator: ","
  }

   writeCommon:{
    batchSize: 2000
    enableRealTimeQPSLog: true
    mode: "INSERT IGNORE"
  } 

  sources: [
    # node type Actor
    {
      path: "s3a://nebula-data-trans-test-us-west-2/import_movie/sub_actor.csv"

      pre_processes: {
          filter: ["birth_date!='\\N'"]
          concatConstant: [
            {
              field: "name"
              prefixValue: "test_"
              suffixValue: ""
            }
          ]
      }

      nodetypes: [
        {
          name: "Actor"
          sourceFields: [people_id,name,birth_date]
          nebulaFields: [id,name,birthDate]
        }
      ]
    }

    # node type Movie
    {
      path: "s3a://nebula-data-trans-test-us-west-2/import_movie/sub_movie.csv"
      nodetypes: [
        {
          name: "Movie"
          sourceFields: [movie_id,name]
          nebulaFields: [id,name]
        }
      ]
    }

   # edge type Act
    {
      path: "s3a://nebula-data-trans-test-us-west-2/import_movie/sub_actor_act_movie.csv"
      edgetypes: [
        {
          name: "Act"
          sourceKeys: [people_id]
          targetKeys: [movie_id]
          sourceFields: []
          nebulaFields: []
        }
      ]
    }

  ]
}

````