ETL: Neo4jCsvRelationships
Table of Contents
Description
Loads data from CSV into Neo4j. Data from CSV are used to insert or update relationships. Driver creates "LOAD CSV" Cypher in a background to load data.
This driver is useful in scenario where there are many columns in CSV and all columns need to be loaded into Neo4j relationships and relationship attribute name is equal to CSV column name. This is also useful when columns in CSV are changed from time to time because this driver loads all columns automatically.
Driver can be configured to insert all data from CSV or update data.
Connection
Attributes
Name | Description | Required |
---|---|---|
url | url to Neo4j server, Bolt or REST url | yes |
user | username for authorization at Neo4j | no |
password | password for authorization at Neo4j | no |
Parameters
Name | Description | Required | Default |
---|---|---|---|
source_node_labels | list of labels to match source node (separated by comma) | yes | |
target_node_labels | list of labels to match target node (separated by comma) | yes | |
relationship_type | type of relationship to create or update | yes | |
source_node_match_columns | pair - name of attribute in Neo4j : name of column in CSV. Which CSV column is used to match Neo4j attribute to match source node | no | |
target_node_match_columns | pair - name of attribute in Neo4j : name of column in CSV. Which CSV column is used to match Neo4j attribute to match target node | no | |
source_node_match_dbId_column | name of column in CSV which contains database id to match source node | no | |
target_node_match_dbId_column | name of column in CSV which contains database id to match target node | no | |
relationship_merge_columns | name of columns which are used to merge relationship (separated by comma) | no | |
relationship_merge_dbId_column | name of column which contains database id. This column is used to match relationship by database id. | no | |
timestamp_properties | names of attributes which will be set to current timestamp (separated by comma) | no | |
empty_string_is_null | true / false. If true and CSV column contains empty string then null is set into Neo4j attribute. If false and CSV column contains empty string then empty string is set into Neo4j attribute. | no | false |
commit_size | size of commit. If it is used then "USING PERIODIC COMMIT" is added to Cypher and data are committed on the fly. If it is not used data are committed at the end. | no | |
csv_path | path to CSV file with relationships | yes | |
csv_delimiter | delimiter of columns in CSV file | no | ; |
Query
Not used.
Script
Load all data from CSV and create or update relationships in Neo4j.
Create
If driver is configured to create data then driver takes all columns from CSV and create new relationships in Neo4j with attributes equal to columns.
Merge
If driver is configured to merge data with Neo4j relationships then driver takes data from CSV and create merge statement with merge columns which are configured.
Data are merged in this way:
- if there is a relationship in CSV which matches relationship in Neo4j:
- if CSV contains column and Neo4j contains same attribute then update this attribute
- if CSV contains column and Neo4j doesn't contain attribute with this name then create this attribute
- if CSV doesn't contain column with name of some Neo4j attribute then attribute in Neo4j is not changed
- if there is a relationship in CSV which is not matched in Neo4j: create new relationship in Neo4j
- if there isn't a relationship in CSV which is in Neo4j: relationship in Neo4j is not changed
Examples
Sample CSV data with relationships:
sourceLogicalName;targetLogicalName;relationshipName;type;subtype
App1;Server1;App1-Server1;Physical;Uses
App2;Server2;App2-Server2;Physical;Uses
Create data (no merge). Match source and target node on columnName. Use column sourceLogicalName to match with attribute logicalName for source node. Use column targetLogicalName to match with attribute logicalName for target node.
<!DOCTYPE etl SYSTEM "
https://scriptella.org/dtd/etl.dtd
">
<
etl
>
<
description
>Relationships from CSV</
description
>
<
connection
id
=
"relationshipImport"
driver
=
"neo4jCsvRelationships"
url
=
"bolt://localhost:7687"
user
=
"neo4j"
password
=
"admin"
>
source_node_labels=Ci
target_node_labels=Ci
relationship_type=RELATED
source_node_match_columns=logicalName:sourceLogicalName
target_node_match_columns=logicalName:targetLogicalName
timestamp_properties=_created,_updated
csv_path=/rels.csv
</
connection
>
<
script
connection-id
=
"relationshipImport"
/>
</
etl
>
Merge relationships based on column name. Match source and target node on columnName. Use column sourceLogicalName to match with attribute logicalName for source node. Use column targetLogicalName to match with attribute logicalName for target node.
<!DOCTYPE etl SYSTEM "
https://scriptella.org/dtd/etl.dtd
">
<
etl
>
<
description
>Relationships from CSV</
description
>
<
connection
id
=
"relationshipImport"
driver
=
"neo4jCsvRelationships"
url
=
"bolt://localhost:7687"
user
=
"neo4j"
password
=
"admin"
>
source_node_labels=Ci
target_node_labels=Ci
relationship_type=RELATED
source_node_match_columns=logicalName:sourceLogicalName
target_node_match_columns=logicalName:targetLogicalName
relationship_merge_columns=relationshipName
timestamp_properties=_created,_updated
csv_path=/rels.csv
</
connection
>
<
script
connection-id
=
"relationshipImport"
/>
</
etl
>