Feature #1567
closedEnhanced Import
Description
Idea:
create a generic import functionality that imports all kind of data with their relation from discipline specific CSV templates into openatlas
Prerequisite:
discuss structure of generic import functionality
Stefan's approach:
1. Fill in thanados specific template like this one
https://docs.google.com/spreadsheets/d/1IIPGKvSRNi6Qeir6070VS4u7uvHdlpYenEhRkCaGJcI/edit?usp=sharing
2. Get the data as CSV to OpenAtlas
3. Map the CSV to the generic import format (Name, Alias, Geometry, Types, Dates, Reference Systems etc.)
4. Witchcraft and Magic -> all is imported to OpenAtlas
- Alias* ✅
- Geometry as WKT* ✅
- Existing External Reference System with ID including identifier* ✅
- Value types* ✅
- Administrative Unit* ✅
- Historical Place* ✅
- Existing References with ID including page number -> #2244
- OpenAtlas Class name to know which level the place is -> #2244
- Parent import ID -> #2244
(* These will be added to all import options)
- Better warnings (showing where warning appears)✅
- Remove
no cover
by manipulating CSV in tests ✅ - What if duplicate reference systems are given ✅
- Handle error if multiple type ids from the same single type ✅
- Translations ✅
- Manual entry ✅
Updated by Alexander Watzinger about 3 years ago
- Category changed from Backend to Data import/export
- Status changed from New to Acknowledged
Updated by Bernhard Koschiček-Krombholz about 3 years ago
- Related to Feature #1047: Enter WKT geometries added
Updated by Bernhard Koschiček-Krombholz 9 months ago
- Status changed from Acknowledged to Assigned
- Assignee set to Bernhard Koschiček-Krombholz
- Target version changed from Wishlist to 8.7.0
Updated by Bernhard Koschiček-Krombholz 9 months ago
- Target version changed from 8.7.0 to 8.4.0
Updated by Bernhard Koschiček-Krombholz 9 months ago
- Related to Feature #2193: Import artifacts added
Updated by Bernhard Koschiček-Krombholz 9 months ago
- Description updated (diff)
By comparing the example and the current import, following things needs to be added:
- Alias*
- Geometry as WKT (Should multiple geometries be possible)*
- Existing References with ID including page number
- Existing External Reference System with ID including identifier*
- Value types*
- OpenAtlas Class name to know which level the place is
- Parent import ID
(* These can also be added to the current import)
For this, I will make a new button like "Place hierarchy".
First steps will be to develop the things, which can be added to the current import actions.
Updated by Bernhard Koschiček-Krombholz 8 months ago
- Description updated (diff)
Took some time, but I replaced northing/easting with WKT and therefore also support for Polygons and Linestrings.
Updated by Bernhard Koschiček-Krombholz 8 months ago
- Description updated (diff)
Updated by Bernhard Koschiček-Krombholz 8 months ago
- Description updated (diff)
Updated by Bernhard Koschiček-Krombholz 8 months ago
- Description updated (diff)
Added external reference system with id;identifier;match
Not sure if ; is the optimal delimiter, but it is a start and can be easily be changed.
external_reference_system | wikidata | geonames |
---|---|---|
156149;vienna;exact_match 6666;stadt-wien;close_match | Q152419;close_match | 2761369;exact_match |
Updated by Bernhard Koschiček-Krombholz 8 months ago
- Description updated (diff)
Updated by Bernhard Koschiček-Krombholz 8 months ago
- Description updated (diff)
value_type_ids |
---|
666;12.34 777;987 |
Again delimiter is ;
for type_id;value and space for each pair.
Updated by Bernhard Koschiček-Krombholz 8 months ago
- Description updated (diff)
Updated by Bernhard Koschiček-Krombholz 8 months ago
- Description updated (diff)
administrative_unit | historical_place |
---|---|
87 | 102 |
Updated by Bernhard Koschiček-Krombholz 8 months ago
Alex pointed out, at the developer meeting, that the implementation of external reference systems doesn't look nice. So I refactored it and each reference system has a own row with reference_system_*
where *
is the full name of the external reference system (spaces has to be transformed to _
and upper/lower case doesn't matter).
First value is the identifier and second value is the match.
reference_system_getty_aat | reference_system_wikidata | reference_system_geonames |
---|---|---|
300251645;exact_match | Q152419;close_match | 2761369;exact_match |
Updated by Bernhard Koschiček-Krombholz 8 months ago
- Description updated (diff)
Updated by Bernhard Koschiček-Krombholz 8 months ago
- Description updated (diff)
Updated by Bernhard Koschiček-Krombholz 8 months ago
- Description updated (diff)
Updated by Bernhard Koschiček-Krombholz 8 months ago
- Description updated (diff)
Updated by Bernhard Koschiček-Krombholz 8 months ago
- Description updated (diff)
Updated by Bernhard Koschiček-Krombholz 8 months ago
- Description updated (diff)
- Target version changed from 8.4.0 to 8.3.0
Updated by Bernhard Koschiček-Krombholz 8 months ago
- Precedes Feature #2244: Import of place hierarchy added