Project

General

Profile

Feature #1206

API: also show entities of subtypes

Added by Stefan Eichert 3 months ago. Updated 6 days ago.

Status:
Assigned
Priority:
Low
Category:
API
Target version:
Start date:
2020-06-07
Estimated time:
(Total: 0.00 h)

Description

By now it is possible to see all entities of a certain type (e55) if this type is selected via the api.

The entities that have this type are linked via: crm:p2_has_type.

We should discuss if we include all entities that have subtypes of this very type (recursively)

Actually each entity that has a subtype of this one also has the superior one...

This also occurs to other entitites e.g places. For example "europe". Not only the places (in our case E18-places and their locations) that are linked directly to "europe" (via has current or former location P53) are in Europe but also the ones that are in subunits (countries, provinces, towns etc.) of Europe.

This is my point for now and I would like to open the discussion. Looking forward to feedback!
Best, Stefan


Subtasks

Question #1270: Administrative Unit and Historical Place get not covered with links or relationshipClosedAlexander WatzingerActions

History

#1

Updated by Alexander Watzinger 2 months ago

  • Target version set to API
#2

Updated by Bernhard Koschicek 2 months ago

Stefan Eichert wrote:

By now it is possible to see all entities of a certain type (e55) if this type is selected via the api.

The entities that have this type are linked via: crm:p2_has_type.

We should discuss if we include all entities that have subtypes of this very type (recursively)

Actually each entity that has a subtype of this one also has the superior one...

This also occurs to other entitites e.g places. For example "europe". Not only the places (in our case E18-places and their locations) that are linked directly to "europe" (via has current or former location P53) are in Europe but also the ones that are in subunits (countries, provinces, towns etc.) of Europe.

This is my point for now and I would like to open the discussion. Looking forward to feedback!
Best, Stefan

So you want every subtypes of a type under the relation tag? The question is, how do we build the tree? It can look like this:

relations: [
{
  label: "Kärnten",
  relationTo: "http://thanados.openatlas.eu/api/0.1/entity/89",
  relationType: "crm:P89_falls_within",
  subtype: [
       {
      label: "Hermagor",
      relationTo: "http://thanados.openatlas.eu/api/0.1/entity/27072",
      relationType: "crm:P89_falls_within",
      subtype: [
      {
        label: "DELLACH",
        relationTo: "http://thanados.openatlas.eu/api/0.1/entity/27111",
        relationType: "crm:P89_falls_within",
        },
        {
        label: "GITSCHTAL",
        relationTo: "http://thanados.openatlas.eu/api/0.1/entity/27088",
        relationType: "crm:P89_falls_within",
        },
        {
        label: "HERMAGOR - PRESSEGGER SEE",
        relationTo: "http://thanados.openatlas.eu/api/0.1/entity/27091",
        relationType: "crm:P89_falls_within",
        },
        {
        label: "KIRCHBACH",
        relationTo: "http://thanados.openatlas.eu/api/0.1/entity/27083",
        relationType: "crm:P89_falls_within",
        },
        {
        label: "KÖTSCHACH - MAUTHEN",
        relationTo: "http://thanados.openatlas.eu/api/0.1/entity/27073",
        relationType: "crm:P89_falls_within",
          subtype: [
          {
            label: "KÖTSCHACH",
            relationTo: "http://thanados.openatlas.eu/api/0.1/entity/27074",
            relationType: "crm:P89_falls_within",
            },
            {
            label: "MAUTHEN",
            relationTo: "http://thanados.openatlas.eu/api/0.1/entity/27075",
            relationType: "crm:P89_falls_within",
          }
          ]
        }
        ]
    },
    {
      label: "Klagenfurt",
      relationTo: "http://thanados.openatlas.eu/api/0.1/entity/26747",
      relationType: "crm:P89_falls_within",
      subtype: [
        {
        label: "KLAGENFURT",
        relationTo: "http://thanados.openatlas.eu/api/0.1/entity/26748",
        relationType: "crm:P89_falls_within",
        }
      ]
    }
  ]
}

]

But the question is, why do we need show everything? As far as I understand APIs, the goal is, that the client can navigate with links through the data, and not that one API page gives you all information. I also think, there can be a issue with the performance, if we always include all the subtypes.

I suppose, you want this data prepered for the front end?

#3

Updated by Stefan Eichert 2 months ago

Hi everyone!
Thanks Berni for the discussion contribution. I think the issue is more a conceptual one that also regards classes etc.
Its not so much about the recursive tree of types or classes but about the entities that are classified/categorised by them.
Per definitionem each entity of a subclass (e.g. of E21 Person) of a certain class (e.g. E39 Actor) is, respectively inherits, all super classes. So each person is at the same time an actor as well as all the superior classes (up to E1). So we should discuss if we show all entitites of all subclasses as well if the API is asked for an entity of a superior class. Same would need to be discussed for types too. If an entity is linked to a certain type, that has a broader term (p127) this entity inherits the broader term too.

From my point of view there are several possible solutions. Each of them should be implemented somehow. For the beginning we should discuss the following:
1. one exact query that only gets the specific results
2. one query that delivers the entitites of the requested class/type/etc. and all entities that are classified via subtypes/subclasses/etc.

I am not sure about performance but if we stick to the idea of the CRM i think version 2 should be default and version one somehow optional.

One example to make it a little bit clearer:
If I want to get all burial sites (and entities of their subtypes) from the api, I would need to
1. ask the API for the json-ld of https://thanados.openatlas.eu/entity/77
2. parse all narrower term relations and get their p2 relations
3. recursively parse them too and their relations via p2
4. then retrieve all the data from the p2 relations and build my data object from them.

So to cut a short story long ;) why not show entities of recursive subunits of types, classes, places, periods etc. in the API relations as they inherit everything that all their superior classes/types are defined of?

best, Stefan

#4

Updated by Alexander Watzinger 2 months ago

Just a little input about performance: if we use what is already implemented:

  • We can access all nodes (types and hierarchical places) from the g object (g.nodes)
  • Every node "knows" the ids of all their supers recursively (node.root)
  • Every node "knows" the id of all sub nodes (node.subs, not recursively)

With classes it works similar (g.classes) except that they only know their direct super. It's a little different because CIDOC CRM is not as strict hierarchical as with our nodes where every node can only have one super.

However, since we already have access to all nodes and all classes in g.nodes and g.classes it's not a performance issue to access them even with recursive functions.

Next step would be to get all related entities. Entity.get_by_class(code) takes only one class as argument. I can rewrite this (make a ticket if you need this) to optional take a list so that we only need one database query to get entities of multiple classes. Of course if there are tens of thousands in the result we will run into performance problems eventually. Christoph said something about paging the results which sounds useful.

Also, I already experimented in OpenAtlas a while ago to load everything (every entity and link) in one go - it didn't go well performance like and I guess that extreme can't be the solution so we have to find some middle ground.

#5

Updated by Alexander Watzinger 2 months ago

  • Status changed from New to Assigned

I renamed and refactored get entities functions at model/entity for easier use in the API. It's already in the development branch on GitHub.

get_by_class_code() now takes a string (e.g. 'E21') or a list of strings
get_by_menu_item() takes one of these strings: actor, event, place, reference, source

Difference is that the first one "blindly" gets all entities with specified class code(s). The other one returns results like you see in the UI overview tables. E.g. with sources you get all 'E33' with the system type 'source content'.
I hope that helps to make these functions easier to use and understand.

#6

Updated by Bernhard Koschicek 30 days ago

Sorry, that this takes quite long. We have to talk about this again at the API meeting.

I have done it for normal entities. But for the Types I encountered the bug #1270. I need to talk with Alex for more insight of the Entity and Link structure.

Also available in: Atom PDF