Feature #1567
closed
Added by Stefan Eichert over 3 years ago.
Updated 7 months ago.
Category:
Data import/export
Description
Idea:
create a generic import functionality that imports all kind of data with their relation from discipline specific CSV templates into openatlas
Prerequisite:
discuss structure of generic import functionality
Stefan's approach:
1. Fill in thanados specific template like this one
https://docs.google.com/spreadsheets/d/1IIPGKvSRNi6Qeir6070VS4u7uvHdlpYenEhRkCaGJcI/edit?usp=sharing
2. Get the data as CSV to OpenAtlas
3. Map the CSV to the generic import format (Name, Alias, Geometry, Types, Dates, Reference Systems etc.)
4. Witchcraft and Magic -> all is imported to OpenAtlas
To-do:
- Alias* ✅
- Geometry as WKT* ✅
- Existing External Reference System with ID including identifier* ✅
- Value types* ✅
- Administrative Unit* ✅
- Historical Place* ✅
- Existing References with ID including page number -> #2244
- OpenAtlas Class name to know which level the place is -> #2244
- Parent import ID -> #2244
(* These will be added to all import options)
- Better warnings (showing where warning appears)✅
- Remove
no cover
by manipulating CSV in tests ✅
- What if duplicate reference systems are given ✅
- Handle error if multiple type ids from the same single type ✅
- Translations ✅
- Manual entry ✅
- Category changed from Backend to Data import/export
- Status changed from New to Acknowledged
- Status changed from Acknowledged to Assigned
- Assignee set to Bernhard Koschiček-Krombholz
- Target version changed from Wishlist to 8.7.0
- Target version changed from 8.7.0 to 8.4.0
- Description updated (diff)
By comparing the example and the current import, following things needs to be added:
- Alias*
- Geometry as WKT (Should multiple geometries be possible)*
- Existing References with ID including page number
- Existing External Reference System with ID including identifier*
- Value types*
- OpenAtlas Class name to know which level the place is
- Parent import ID
(* These can also be added to the current import)
For this, I will make a new button like "Place hierarchy".
First steps will be to develop the things, which can be added to the current import actions.
- Description updated (diff)
- Description updated (diff)
- Description updated (diff)
Took some time, but I replaced northing/easting with WKT and therefore also support for Polygons and Linestrings.
- Description updated (diff)
- Description updated (diff)
- Description updated (diff)
Added external reference system with id;identifier;match
Not sure if ; is the optimal delimiter, but it is a start and can be easily be changed.
external_reference_system |
wikidata |
geonames |
156149;vienna;exact_match 6666;stadt-wien;close_match |
Q152419;close_match |
2761369;exact_match |
- Description updated (diff)
- Description updated (diff)
Included import of value types:
value_type_ids |
666;12.34 777;987 |
Again delimiter is ;
for type_id;value and space for each pair.
- Description updated (diff)
- Description updated (diff)
Added administrative unit and historical place:
administrative_unit |
historical_place |
87 |
102 |
Alex pointed out, at the developer meeting, that the implementation of external reference systems doesn't look nice. So I refactored it and each reference system has a own row with reference_system_*
where *
is the full name of the external reference system (spaces has to be transformed to _
and upper/lower case doesn't matter).
First value is the identifier and second value is the match.
reference_system_getty_aat |
reference_system_wikidata |
reference_system_geonames |
300251645;exact_match |
Q152419;close_match |
2761369;exact_match |
- Description updated (diff)
- Description updated (diff)
- Description updated (diff)
- Description updated (diff)
- Description updated (diff)
- Description updated (diff)
- Target version changed from 8.4.0 to 8.3.0
- Description updated (diff)
- Status changed from Assigned to Closed
Closing this issue and move place hierarchy related tasks to #2244.
Also available in: Atom
PDF