Project

General

Profile

Question #1696

API: Bug in generated client code

Added by Eugen Hotwagner 7 months ago. Updated 13 days ago.

Status:
Closed
Priority:
Normal
Category:
API
Target version:
-
Start date:
2022-04-19
Estimated time:

Description

I generated a fsharp client using two different tools and have noticed a few empty results for certain methods. Fe:

type GetApi03Classes =

| OK of payload: list<ClassMappingModel>
    | NotFound

is ok since it has a payload but:

type GetApi03CidocClassByCidocClass =

| OK
    | NotFound

does not. I assumed that this was due to some bug in the fsharp tools but looking into other generated clients on swaggerhub (csharp, java, scala) have the same problem (i think). There is an online tool to test swagger apis here: https://apitools.dev/swagger-parser/online/
that results for the OA api:

@This API is valid, but it cannot be shown because it contains circular references

TypeError: Converting circular structure to JSON
--> starting at object with constructor 'Object' | property 'properties' -> object with constructor 'Object' | property 'children' > object with constructor 'Object'
--
property 'items' closes the circle
at JSON.stringify (<anonymous>)
at toText (https://apitools.dev/swagger-parser/online/js/bundle.min.js:17:226677)
at Function.editors.addResult (https://apitools.dev/swagger-parser/online/js/bundle.min.js:17:227813)
at Function.editors.showResult (https://apitools.dev/swagger-parser/online/js/bundle.min.js:17:227094)
at https://apitools.dev/swagger-parser/online/js/bundle.min.js:17:231618
at Array.forEach (<anonymous>)
at https://apitools.dev/swagger-parser/online/js/bundle.min.js:17:231598@

which is a likely reason that many code generation tools fail.


Related issues

Related to OpenAtlas - Feature #1749: API: Testsystems for API Acknowledged2022-06-14Actions

History

#1

Updated by Bernhard Koschiček-Krombholz 7 months ago

  • Status changed from New to Assigned

Thank you for posting this issue. I think I fixed the circular structure.
Please retry it with https://app.swaggerhub.com/apis/ctot-nondef/OpenAtlas/0.3/

If I have time, I will also try these generated clients.

Thank you again for reporting this!

#2

Updated by Alexander Watzinger 7 months ago

  • Subject changed from Bug in generated Client Code to Bug in generated client code
  • Found in version changed from to 7.2.0
#3

Updated by Bernhard Koschiček-Krombholz 7 months ago

  • Assignee changed from Bernhard Koschiček-Krombholz to Eugen Hotwagner

Eugen, I assign this ticket to you. If you have test everything and found more problems, please assign it back to me with your feedback.
Newest version in swaggerhub: https://app.swaggerhub.com/apis/ctot-nondef/OpenAtlas/0.3/

#4

Updated by Eugen Hotwagner 7 months ago

  • Assignee changed from Eugen Hotwagner to Bernhard Koschiček-Krombholz

Bernhard Koschiček-Krombholz wrote:

Eugen, I assign this ticket to you. If you have test everything and found more problems, please assign it back to me with your feedback.
Newest version in swaggerhub: https://app.swaggerhub.com/apis/ctot-nondef/OpenAtlas/0.3/

I only had time to do a preliminery check but the problem seems to persist. Did you check if the data payload conforms to the api using any of those tools https://openapi.tools/#data-validators ?

If yes, then the problem is most likely on my side and i will investigate further.

#5

Updated by Eugen Hotwagner 7 months ago

Eugen Hotwagner wrote:

Bernhard Koschiček-Krombholz wrote:

Eugen, I assign this ticket to you. If you have test everything and found more problems, please assign it back to me with your feedback.
Newest version in swaggerhub: https://app.swaggerhub.com/apis/ctot-nondef/OpenAtlas/0.3/

I only had time to do a preliminery check but the problem seems to persist. Did you check if the data payload conforms to the api using any of those tools https://openapi.tools/#data-validators ?

If yes, then the problem is most likely on my side and i will investigate further.

Upon reading a bit more, i think that many code generators are not able to deal with "oneof". Might be the issue.

#6

Updated by Bernhard Koschiček-Krombholz 7 months ago

  • Tracker changed from Bug to Question
  • Assignee changed from Bernhard Koschiček-Krombholz to Eugen Hotwagner
  • Found in version deleted (7.2.0)

Thank you for this inside. I don't know, if I can work without "oneof", but I can gladly create a new swagger without "oneof": https://app.swaggerhub.com/apis/ctot-nondef/OpenAtlas/0.3-without-oneof

Now only the main schema (LPF) is available. Please try it again.

I change the issue from bug to question. Since it seems that the problem lies within the generated code, this issue is not a bug within OpenAtlas.

#7

Updated by Eugen Hotwagner 7 months ago

  • File bug1.png added
  • File bug2.png added
#8

Updated by Eugen Hotwagner 7 months ago

Interesting i can delete a post but uploaded files are still there :)

Without oneof i indeed get responses with payloads. I think that you should use oneofs. It is not that much work to modify the client to fix that. Maybe mentioning that in the api documentation is sufficient. It is unfortunate that swaggerhug generates brokent clients that fail silently on these methods.

I also encountered some deserialization errors that might hint that the json payload does not conform to the api. But maybe thats just the no oneof fork.

#9

Updated by Bernhard Koschiček-Krombholz 7 months ago

You are right, we will keep the oneofs, but for debugging reasons, I will keep the additional swagger documentation.

I think I found the bug from the deserialization errors: the key of the TimespansModel was wrong ("first" not "start") This was a relic of an older version. Please try it again.

I'm happy that we are moving forward, thank you again for the constructive feedback!

#10

Updated by Alexander Watzinger 7 months ago

@ Eugen Hotwagner:
Completely unrelated to your discussion, but about file deletion: you should be able to delete them not in the note but at the bottom of the yellow information area above in the Files section with a little trashcan icon.
If not, please tell me which you want to be deleted (both?) and I can do it for you.

#11

Updated by Alexander Watzinger 7 months ago

  • File deleted (bug1.png)
#12

Updated by Alexander Watzinger 7 months ago

  • File deleted (bug2.png)
#13

Updated by Alexander Watzinger 6 months ago

Is it working now as expected? If yes can you (Eugen) please close the issue?

#14

Updated by Eugen Hotwagner 6 months ago

Sorry for the long delay. I still encounter deserialization errors.
@"Error: Newtonsoft.Json.JsonSerializationException: Cannot deserialize the current JSON object (e.g. {"name":"value"}) into type 'Microsoft.FSharp.Collections.FSharpList`1[OAAPI.Types.WhenModel]' because the type requires a JSON array (e.g. [1,2,3]) to deserialize correctly.
To fix this error either change the JSON to a JSON array (e.g. [1,2,3]) or change the deserialized type so that it is a normal .NET type (e.g. not a primitive type like integer, not a collection type like an array or List<T>) that can be deserialized from a JSON object. JsonObjectAttribute can also be added to the type to force it to deserialize from a JSON object.

@ "when": {
"timespans": [ {
"start": {
"earliest": "None",
"latest": "None"
},
"end": {
"earliest": "None",
"latest": "None"
}
}
]
}@

The problem seems to be here, but not sure why?

WhenModel:
properties:
timespans:
items:
$ref: '#/components/schemas/TimespansModel'
type: array
type: object
TimespansModel:
properties:
end:
$ref: '#/components/schemas/TimeDetailModel'
start:
$ref: '#/components/schemas/TimeDetailModel'
type: object
#15

Updated by Eugen Hotwagner 6 months ago

Not sure if relevant but swaggerhub gives this:

@ "when": [ {
"timespans": [ {
"end": {
"earliest": "string",
"latest": "string"
},
"start": {
"earliest": "string",
"latest": "string"
}
}
]@

as a possible example for the api which looks like an array of an array.

#16

Updated by Alexander Watzinger 6 months ago

  • Subject changed from Bug in generated client code to API: Bug in generated client code
  • Status changed from Assigned to In Progress
  • Assignee changed from Eugen Hotwagner to Bernhard Koschiček-Krombholz

Thank you for the update and new information, reassigning this to our OpenAtlas API expert Bernhard, to take a look at it.

#17

Updated by Bernhard Koschiček-Krombholz 6 months ago

Changed the models to:

    TimeDetailModel:
      properties:
        earliest:
          type: string
        latest:
          type: string
      type: object
    TimeStartEndModel:
      properties:
        end:
          $ref: '#/components/schemas/TimeDetailModel'
        start:
          $ref: '#/components/schemas/TimeDetailModel'
      type: object
    TimespansModel:
      items:
        $ref: '#/components/schemas/TimeStartEndModel'
      type: array
    WhenModel:
      properties:
        timespans:
          $ref: '#/components/schemas/TimespansModel'
      type: object

This should be working. I will try to get another look at the tools you presented. I couldn't run the python tools properly.

#18

Updated by Eugen Hotwagner 6 months ago

Thanks, i will try the new version later.

@data validation tools (https://openapi.tools/#data-validators). I ran https://github.com/KissPeter/APIFuzzer which is very interesting but it was not very useful. The problem is that it concentrates on the request space and finds many "problems" that are right now not handled in that part of the api but by error handling. For example some input should only be an integer but is defined as number in the api so the fuzzer will test floats and get back an defined error. Or another is defined as a general string, where it can only be some well defined possibilities and reports errors if something else is used. Now, OpenApi has tools to tighten types much more and i have seen some examples in the API but i do not know how clever and powerful APIFuzzer is. Trying to tighten (put constraints on) the types of input requests might make this tool quite useful. Making partial functions into total functions and illegal states unrepresentable.

Of more immediate usefulness should be https://github.com/p1c2u/openapi-core where the above error should have been caught with a test by comparing the response with the spec.

#19

Updated by Eugen Hotwagner 6 months ago

Still persists. I think the problem lies here:

    FeatureGeoJSON:
      properties:
...
        when:
          items:
            $ref: '#/components/schemas/WhenModel'
          type: array

which is why the specs define an array:

 "when": [
        {
          "timespans": [
            {
              "end": {
                "earliest": "string",
                "latest": "string" 
              },
              "start": {
                "earliest": "string",
                "latest": "string" 
              }
            }
          ]

while the actual response is:
 "when": {
            "timespans": [
              {
                "start": {
                  "earliest": "None",
                  "latest": "None" 
                },
                "end": {
                  "earliest": "None",
                  "latest": "None" 
                }
              }
            ]
          }

I also found two other possible issues:

The without oneof branch uses a oneof here:

    GeometryModel:
      properties:
        coordinates:
          items:
            oneOf:
              - type: integer
              - items:
                  type: integer
                type: array

and you specify some responses as dictionaries

            application/ld+json:
              schema:
                $ref: '#/components/schemas/OutputModelLPF'
          description: A dictionary with a result dictionary and pagination information


but as far as i can tell dictionaries are defined by the additionalProperties keyword https://swagger.io/docs/specification/data-models/dictionaries/

that you dont use.

#20

Updated by Eugen Hotwagner 6 months ago

Hmm, the problem persists when i change the specs to

          "when": {

              "$ref": "#/components/schemas/WhenModel" 

          }

but no real clue if i am doing it right.

#21

Updated by Eugen Hotwagner 6 months ago

Got it, there was a second instance of the issue:

RelationModel:
      properties:
        label:
          type: string
        relationDescription:
          type: string
        relationSystemClass:
          type: string
        relationTo:
          type: string
        relationType:
          type: string
        type:
          type: string
        when:
          items:
            $ref: '#/components/schemas/WhenModel'
          type: array
      type: object

Works fine if the array is removed.

#22

Updated by Eugen Hotwagner 6 months ago

The same problem seem to exist with features/geometry. Removing the array fixes it.

#23

Updated by Bernhard Koschiček-Krombholz 6 months ago

Thank you, is fixed in swaggerhub.

#24

Updated by Bernhard Koschiček-Krombholz 6 months ago

#25

Updated by Alexander Watzinger 3 months ago

  • Target version deleted (208)

Moving question issues out of roadmap.

#26

Updated by Alexander Watzinger about 2 months ago

What is the status of this question? Can we close it?

#27

Updated by Bernhard Koschiček-Krombholz 13 days ago

  • Status changed from In Progress to Closed

A new swagger file will be written. We have to test it then.
Thank you, Eugen, for this question and I hope we will figure it out.

Also available in: Atom PDF