[REQ][python] Option to treat custom content-types as plain text

[nponsard]

Is your feature request related to a problem? Please describe.

I want to set the content type as application/x-pem-file in a request but an exception is raised: NotImplementedError('Serialization has not yet been implemented for application/x-pem-file').

No serialization is needed, this is just a text file.

Describe the solution you'd like

Add an option in the endpoint functions to disable serialization when calling a endpoint. Something like skip_serialization=True.

Describe alternatives you've considered

Doing the request manually using ApiClient.rest_client.request(), this is kinda against the idea of an API code generator.

[spacether]

Can you please provide the description of the request and response? Should text or bytes be sent?

[nponsard]

Here's an extract of a request using the x-pem-file type :

...
    put:
      responses:
        "201":
          description: Successful import of a certificate.
        "401":
          description: Authentication required but none provided.
        "403":
          description: Access denied.
        "406":
          description: Content type in Accept header not supported.
        "409":
          description: Conflict, certificate already exists for this KeyID.
        "412":
          description: Precondition failed (NetHSM was not *Operational*).
      description: |
        Store a certificate.
        Maximum size 1MB.
        The content-type header provides the media type.
        Only application/json, application/x-pem-file, application/x-x509-ca-cert,
        application/octet-stream, text/plain and application/pgp-keys is allowed.
      requestBody:
        content:
          application/x-pem-file:
            schema:
              $ref: "#/components/schemas/PemCert"
        required: true
      security:
        - basic: []
components:
  schemas:
    PemCert:
      example: |
        -----BEGIN CERTIFICATE-----
        MIIEvDCCAqQCCQCZ9VKHZSKydDANBgkqhkiG9w0BAQsFADAgMQ4wDAYDVQQIDAVo
        ZWxsbzEOMAwGA1UEBwwFaGVsbG8wHhcNMTgxMDMwMTI0MDQ3WhcNMTkxMDMwMTI0
        MDQ3WjAgMQ4wDAYDVQQIDAVoZWxsbzEOMAwGA1UEBwwFaGVsbG8wggIiMA0GCSqG
        SIb3DQEBAQUAA4ICDwAwggIKAoICAQC8WRImgeu4U6Yg5q0/SvoieQL5JwPHaCrN
        6Cc0mj8pnzTBu/f0QYNentcnu0EclD99kla6/gdiJnt0b/IT23o37+t2n0BvEgVQ
        MJxJ/BjIqSZCykq4SWHKia1OS7tJKE2Q/hgGq/RQJ0zecWKA9SuafXDt0plw0saz
        P8ZosFyToMYnPB54SzXj4qeD5t2L2dOamTTyziw/GJDGdwssImEDfeWyMRRyHcIa
        u9HNGaTxjCdBGzCR+PEX3ZHhGzl5iMlgtZUICgXMgAvFZHHEYNtyCW9ZzUB2ZKE2
        nMbu8hbl7GXRCdJaAqlJfKiGp/WvbmrLhCzcwqhu4UnqGXOxeEQryrkPAbiLafqb
        fPLHB7dr8OEGlUzp1SCHR1JxwdBVohuO2lL+O5nMD3ZVdVnh2F5Et6I9lO4w9+5n
        aI6pmO4oy9PsqKvUjxVnEeo1Ee1n9o/4T6fe6XBIQLPqEmy2IXynZP0eJn1Yu9SF
        TkTSmJ+jRiU7EMaagQrM4Ds5seVNVWKOvUjDQCV+QRZaaPcrkMoFtjLCVMhLgy7M
        /Dfl63qCcdVkPwdbI/apjfG8qMrez4k3qAZDYgS3fujyuq9wv7xbq2+o0Ck4Ono2
        mMAA6JDSZr8XwR3ac/UK1KGv+weh1fOVO+c6mSN9M+W1MtkJS+i4/Aj2yDCJzVk3
        GZpd5/E6WwIDAQABMA0GCSqGSIb3DQEBCwUAA4ICAQAB+8+1dk20P6R+RvDRs1mo
        e1CegaPH2bl9t1WVuXJIUkrIGDceVr5f19pcaNQRYwZuoQd8VQXCZUpWbo//TWxl
        ItS3prQgcrfcTi7gCzp1+q1bZk6H21tzESDJTXPFp019/ypdUB4Jsty7qQL73sE3
        nj67DH+srPCknAhfFag0dPOOjAY19IrIomtBMlvYNpb5XupEUAQFCjk6VdLVAzzl
        M5Swj+kmLA0OUHTF2zAwj2k+B9RAkbetu/lXLwUiiQNG00j8YP03lErxDd29m2jl
        NFlSn8YYsgzmihF5fUFTJXP5tS7lF4FnLNbw9pfdeVZBT1uTOcckmFv6n6edttZS
        FhC70hBB8tFFtemGFquZ0N7w8O7DmIYI936n283rzxyuWdi+JqOUQ0KzpGbFOP3t
        AOdVcx/JW8kjDdWRcRDr85M4MnkG6HEzW4uzGBtxBplL7KH0L8LX+lx6ySjIrbRD
        QtwBxwZGtHZJAqskO6vobzzybCKuVUaQwkPrQsK5iLA6Nn1Yxu6XmtlvoPeNIc63
        Pa3ix8vNFbBTtS6iBdbOPse5qgRV0DC94ocedzKw4QzQRa5REIniJ2X8hVoVAcBh
        3nQKyZApl2I/pZMlbZ0Cg5yxjhBaxKwci7jyoMerMMo2EdbZN87gP+2x6tKuY1MO
        35mqzHn+FjWq0eAtpzhO5Q==
        -----END CERTIFICATE-----
      type: string

A curl request would look like this :

curl -X 'PUT' \                                                                                                                          
  'https://nethsmdemo.nitrokey.com/api/v1/keys/aaa/cert' \
  -H 'accept: */*' \
  -H 'Content-Type: application/x-pem-file' \
  -d "-----BEGIN CERTIFICATE-----
MIIEvDCCAqQCCQCZ9VKHZSKydDANBgkqhkiG9w0BAQsFADAgMQ4wDAYDVQQIDAVo
ZWxsbzEOMAwGA1UEBwwFaGVsbG8wHhcNMTgxMDMwMTI0MDQ3WhcNMTkxMDMwMTI0
MDQ3WjAgMQ4wDAYDVQQIDAVoZWxsbzEOMAwGA1UEBwwFaGVsbG8wggIiMA0GCSqG
SIb3DQEBAQUAA4ICDwAwggIKAoICAQC8WRImgeu4U6Yg5q0/SvoieQL5JwPHaCrN
6Cc0mj8pnzTBu/f0QYNentcnu0EclD99kla6/gdiJnt0b/IT23o37+t2n0BvEgVQ
MJxJ/BjIqSZCykq4SWHKia1OS7tJKE2Q/hgGq/RQJ0zecWKA9SuafXDt0plw0saz
P8ZosFyToMYnPB54SzXj4qeD5t2L2dOamTTyziw/GJDGdwssImEDfeWyMRRyHcIa
u9HNGaTxjCdBGzCR+PEX3ZHhGzl5iMlgtZUICgXMgAvFZHHEYNtyCW9ZzUB2ZKE2
nMbu8hbl7GXRCdJaAqlJfKiGp/WvbmrLhCzcwqhu4UnqGXOxeEQryrkPAbiLafqb
fPLHB7dr8OEGlUzp1SCHR1JxwdBVohuO2lL+O5nMD3ZVdVnh2F5Et6I9lO4w9+5n
aI6pmO4oy9PsqKvUjxVnEeo1Ee1n9o/4T6fe6XBIQLPqEmy2IXynZP0eJn1Yu9SF
TkTSmJ+jRiU7EMaagQrM4Ds5seVNVWKOvUjDQCV+QRZaaPcrkMoFtjLCVMhLgy7M
/Dfl63qCcdVkPwdbI/apjfG8qMrez4k3qAZDYgS3fujyuq9wv7xbq2+o0Ck4Ono2
mMAA6JDSZr8XwR3ac/UK1KGv+weh1fOVO+c6mSN9M+W1MtkJS+i4/Aj2yDCJzVk3
GZpd5/E6WwIDAQABMA0GCSqGSIb3DQEBCwUAA4ICAQAB+8+1dk20P6R+RvDRs1mo
e1CegaPH2bl9t1WVuXJIUkrIGDceVr5f19pcaNQRYwZuoQd8VQXCZUpWbo//TWxl
ItS3prQgcrfcTi7gCzp1+q1bZk6H21tzESDJTXPFp019/ypdUB4Jsty7qQL73sE3
nj67DH+srPCknAhfFag0dPOOjAY19IrIomtBMlvYNpb5XupEUAQFCjk6VdLVAzzl
M5Swj+kmLA0OUHTF2zAwj2k+B9RAkbetu/lXLwUiiQNG00j8YP03lErxDd29m2jl
NFlSn8YYsgzmihF5fUFTJXP5tS7lF4FnLNbw9pfdeVZBT1uTOcckmFv6n6edttZS
FhC70hBB8tFFtemGFquZ0N7w8O7DmIYI936n283rzxyuWdi+JqOUQ0KzpGbFOP3t
AOdVcx/JW8kjDdWRcRDr85M4MnkG6HEzW4uzGBtxBplL7KH0L8LX+lx6ySjIrbRD
QtwBxwZGtHZJAqskO6vobzzybCKuVUaQwkPrQsK5iLA6Nn1Yxu6XmtlvoPeNIc63
Pa3ix8vNFbBTtS6iBdbOPse5qgRV0DC94ocedzKw4QzQRa5REIniJ2X8hVoVAcBh
3nQKyZApl2I/pZMlbZ0Cg5yxjhBaxKwci7jyoMerMMo2EdbZN87gP+2x6tKuY1MO
35mqzHn+FjWq0eAtpzhO5Q==
-----END CERTIFICATE-----"

[spacether]

Hi there. I am not seeing the content-type application/x-pem-file registered as media-type in iana.org.
If specific handling is added for this content type we risk a server implementing this content-type differently for example with bytes.

How about:

1. defining your content-type as text-plain (then sending a string will work)
2. or how about defining it as pem-certificate-chain which is a registered media-type that stores text (I would need to add handling for this media-type)

Note:

• x-x509-ca-cert is registered per https://www.rfc-editor.org/rfc/rfc8894.html
• registered text media-types: https://www.iana.org/assignments/media-types/media-types.xhtml#text

[nponsard]

In Nitrokey we are thinking about changing to the standard pem-certificate-chain.

I still think it would be useful to have an option to manually override the serialization with an option, so this project doesn't have to add support for every content-type possible.

[ansiwen]

Hi there. I am not seeing the content-type application/x-pem-file registered as media-type in iana.org. If specific handling is added for this content type we risk a server implementing this content-type differently for example with bytes.

It is not registered but a very common format, see here for example: "x-pem-file and the .pem extension stem from a predecessor of S/MIME: Privacy Enhanced Mail."

It is defined in https://datatracker.ietf.org/doc/html/rfc1421.

1. defining your content-type as text-plain (then sending a string will work)

Our endpoints support several formats and therefore need the info in order to know how to process the body.

2. or how about defining it as pem-certificate-chain which is a registered media-type that stores text (I would need to add handling for this media-type)

We also use the PEM format for keys and CSRs. And in general there are many other unregistered mime types in use. (Actually I always thought, all mime types that start with x- are unregistered.)

So wouldn't it be sensible to default to a raw interface for unknown mime types and leave the body encoding to the SDK user? Or at least offer an additional raw interface for these cases? (IIRC all other OpenAPI SDKs that I worked that offered that.)

I also saw mime types with an additional format specifier like `

• x-x509-ca-cert is registered per https://www.rfc-editor.org/rfc/rfc8894.html

This is a nice example that registration doesn't necessarily help: some implementations expect a PEM encoding, some expect a pure binary DER encoding. I saw these kind content types to solve this: application/x-x509-ca-cert; format=pem, but I have no idea if that is standardized either. So it's registered, but still not unambiguous. x-pem-file OTOH is not registered but unambiguous.

[spacether]

Custom content-types could require any type of input and serialization so I don't feel that defaulting unknown ones to text/plain is a helpful solution for all unknown cases.

Some possible solutions are:

1. you modify the template for the api_client.hbs and add code that knows that application/x-pem-file is a text content type
2. the project provides a method that can be called where input body is any type. This sounds complicated, how would body serialization be handled, how would _get_fields_and_body work? Maybe adding a (body) content_type_to_serialization (used_val, serialization_val)
3. a generate command line option is added to the python generator, something like request_body_plain_text_content_types which adds passed in content types to the ones known about in class RequestBody in api_client.hbs
4. write a PR adding application/x-pem-file as a known plain text content type in class RequestBody in api_client.hbs

What do you think about these possible solutions?

[spacether]

@nponsard @ansiwen if I don't hear from you in a couple of weeks I will close this as stale/inactive because a month will have passed since hearing back.
Per my above response, this can be made to work today by using a custom template and adding handling for this content type.

[robin-nitrokey]

I started looking into option 4. I still have to figure out which content types we need. I’ll try to prepare a PR with the required changes soon.

[spacether]

Thanks!