Feature #2602
openTypes import by CSV - examples and hierarchy
Description
I look in manual and in https://redmine.openatlas.eu/issues/2416, but, very sorry I don't find the way. My specific needs are:
- One example (with data) of csv for importing types
- I have to relate each type to one of existing types , eg: 'Actor function', 'Actor relation', etc
- I need to put some types in hierarchies of types.
and check manual http://localhost:8080/static/manual/admin/import.html#origin-types-import concepts parent_id and openatlas_parent_id if also applies to Types (not only Places)
Task:
Extent the manual with examples and what columns are needed for which class.
Files
Updated by Bernhard Koschiček-Krombholz 4 months ago
- File type_example.csv type_example.csv added
- Tracker changed from Feature to Question
- Category set to Data import/export
- Status changed from New to Acknowledged
- Assignee set to Bernhard Koschiček-Krombholz
- Target version set to 8.15.0
Dear Enric,
Thank you for reaching out. I'll take this as a reason to update the manual regarding imports. It has grown in the last year, and while I tried to keep it small and comprehensible, I should extend it with multiple examples.
I created an example (also attached as a file to this ticket):
id,name,description,reference_ids,reference_system_wikidata,origin_reference_ids,parent_id,openatlas_parent_id type_1,New type attached to hierarchy,,,Q152419;close_match,,,29 type_2,Sub type ,,,,,type_1, type_3,Sub type 2,,,,,type_1, type_4,Sub sub type,,,,,type_3, type_5,Type attached to existing type,,,,,,4067 type_6,Sub type of type_5,,,,,type_5,
First, you cannot create a new hierarchy through the import. Each entry must have either a parent_id or an openatlas_parent_id. The required columns are: id, name, parent_id, and openatlas_parent_id.
You can attach it directly to a hierarchy (e.g., Actor function) or to an existing type (e.g., Kinship). If you want to create a more nested structure, you can use parent_id (this is the ID you give the imported entries, e.g., type_1) to specify which imported type is the super type.
So in my example:
- type_1 is connected to a hierarchy, e.g., Actor relation.
- type_2 and type_3 are connected to the newly created type_1.
- type_4 is connected to the newly created type_3.
- type_5 is connected to an existing type, e.g., Kinship.
- type_6 is connected to the newly created type_5.
The order in which you enter it in the CSV should not be important, as long as each type has a parent (either through parent_id or openatlas_parent_id, not both, but this will be checked during import).
Regarding SQL, we removed this function due to security concerns. I'm sure you have already read that.
Updated by Bernhard Koschiček-Krombholz 4 months ago
- Status changed from Acknowledged to Assigned
Updated by Enric Rodellas 4 months ago
Thanks very much Bernhard. It works perfectly . Just a last question: how I get the import ID of 'head' types like: 'Actor function', 'Actor relation' .... In the UI I cannot afford it. All my 'types' will be put under these 'head' types.
Updated by Alexander Watzinger 4 months ago
Thank you Enric for reporting and Bernhard for taking care of it.
About finding out the ids of the type roots. You can find it out through a sub type.
E.g. at the demo entry for Abbess at https://demo.openatlas.eu/entity/316 you can see the root Actor function in the breadcrumb and if you hover over it it will show: https://demo.openatlas.eu/entity/10, so the id of the Actor function root type would be 10.
Of course now that I'm writing this I realize how cumbersome that may be so I wrote an issue (see, #2605) to display it more conveniently.
Updated by Alexander Watzinger 4 months ago
- Related to Feature #2605: Display the root type id added
Updated by Enric Rodellas 4 months ago
Thanks Alexander and Bernhard
About that question I think that for type roots would be better to have possibility to put string like 'Actor function', ... 'Artifact' and import process could find associated 'id'. I said that because id number of type roots can be different in DBs. For instance, 'Actor relation' in Bernhard example is '29'. In my DB (only executed initdb in docker) is '14'. And 'Actor function' in your example, Alex, is '10' and mine is '14'
- initdb - Where DB is initialized with just main and root data needed for CIDOC/Openatlas model
- populate_with_example - Where DB is populated with data that helps newcomers to understand how it works
When I would get more expertise, only I would execute initdb step.
Said that, I hope some day we could import and operate (read and write) with APIs.. or import a CIDOC rdf
Updated by Bernhard Koschiček-Krombholz 4 months ago
The problem is with names, there can be duplicates. You can even make a custom type hierarchy named "Artifact" and the import script has absolutely no chance to know which one you mean.
And the installation of the db is in multiple steps:
- 1_structure.sql
- 2_data_model.sql
- 3_data_web.sql
- 4_data_type.sql
We just summarized it into one command
cat 1_structure.sql 2_data_model.sql 3_data_web.sql 4_data_type.sql | psql -d openatlas -f -
because of convenience.
The step 4_data_type.sql populates the database with the example data. But, you never can be sure, which ID Postgres assign to the data and if the script changes due to development, the order also might change. So older OpenAtlas databases has for example a "higher" ID for e.g. Artifact, because this was added in a later version of the system.
And to your last comment, yes, this would be very comfortable. I would work on it, but currently I have no time for this (and the #1962 Markdown format is also still waiting to be done). But I see there is no issue for that, maybe you can create one.
Updated by Bernhard Koschiček-Krombholz 4 months ago
- Tracker changed from Question to Feature
Updated by Bernhard Koschiček-Krombholz about 1 month ago
- Target version changed from 8.15.0 to 9.2.0
Updated by Bernhard Koschiček-Krombholz about 1 month ago
- Status changed from Assigned to Acknowledged
- Assignee deleted (
Bernhard Koschiček-Krombholz)