Currently, optional array properties are required in the generated struct, but skipped if serializing. This is incorrect behavior, since an empty Vec is not the same as not supplying a Vec.

This breaks server-side apply-ing new resources to remove things from the Vec, as well as cases where a property is required in a one-of, but you actually want to send the empty Vec as one of those properties.

We should instead make it an Option<Vec<T>> with #[serde(default, skip_serializing_if = "Option::is_none")].

Example:

              podSelector:
                description: Selects pods in the same namespace.
                oneOf:
                - required:
                  - matchExpressions
                - required:
                  - matchLabels
                properties:
                  matchExpressions:
                    items:
                      properties:
                        key:
                          type: string
                        operator:
                          enum:
                          - In
                          - NotIn
                          - Exists
                          - DoesNotExist
                          type: string
                        values:
                          items:
                            type: string
                          type: array
                      required:
                      - key
                      - operator
                      type: object
                    type: array
                  matchLabels:
                    type: object
                    x-kubernetes-preserve-unknown-fields: true
                type: object

This generates:

/// Selects pods in the same namespace.
#[derive(Serialize, Deserialize, Clone, Debug)]
pub struct ServerPodSelector {
    #[serde(default, skip_serializing_if = "Vec::is_empty")]
    pub matchExpressions: Vec<ServerPodSelectorMatchExpressions>,
    pub matchLabels: Option<ServerPodSelectorMatchLabels>,
}

#[derive(Serialize, Deserialize, Clone, Debug)]
pub struct ServerPodSelectorMatchExpressions {
    pub key: String,
    pub operator: String,
    #[serde(default, skip_serializing_if = "Vec::is_empty")]
    pub values: Vec<String>,
}

#[derive(Serialize, Deserialize, Clone, Debug)]
pub struct ServerPodSelectorMatchLabels {
}

Complete CRD https://gist.github.com/alex-hunt-materialize/2743b1e2e58a49c4df0a11ecb39f46ab

What we should get:

/// Selects pods in the same namespace.
#[derive(Serialize, Deserialize, Clone, Debug, Default)]
pub struct ServerPodSelector {
    #[serde(default, skip_serializing_if = "Option::is_none")]
    pub matchExpressions: Option<Vec<ServerPodSelectorMatchExpressions>>,
    pub matchLabels: Option<ServerPodSelectorMatchLabels>,
}

#[derive(Serialize, Deserialize, Clone, Debug)]
pub struct ServerPodSelectorMatchExpressions {
    pub key: String,
    pub operator: String,
    #[serde(default, skip_serializing_if = "Option::is_none")]
    pub values: Option<Vec<String>>,
}

#[derive(Serialize, Deserialize, Clone, Debug)]
pub struct ServerPodSelectorMatchLabels {
}

I am running kopium against cluster-api CRDs and generated structs appear incorrect

Here is what Rust code for top-level object clusters.cluster.x-k8s.io

#[derive(CustomResource, Serialize, Deserialize, Clone, Debug)]
#[kube(group = "cluster.x-k8s.io", version = "v1beta1", kind = "Cluster", plural = "clusters")]
#[kube(namespaced)]
#[kube(schema = "disabled")]
pub struct ClusterSpec {
    pub clusterNetwork: Option<Cluster>,
    pub controlPlaneEndpoint: Option<Cluster>,
    pub controlPlaneRef: Option<Cluster>,
    pub infrastructureRef: Option<Cluster>,
    pub paused: Option<bool>,
    pub topology: Option<Cluster>,
}
...

This is definitely not correct as the struct cannot reference itself

Here is what I think it should be producing (ignore field names, types are important)

#[derive(Clone, Debug, Serialize, Deserialize, CustomResource)]
#[serde(rename_all = "camelCase")]
#[skip_serializing_none]
#[kube(
    group = "cluster.x-k8s.io",
    version = "v1beta1",
    kind = "Cluster",
    plural = "clusters",
)]
#[kube(namespaced)]
#[kube(schema = "disabled")]
pub struct ClusterSpec {
    pub cluster_network: Option<ClusterNetwork>,
    pub control_plane_endpoint: Option<ApiEndpoint>,
    pub control_plane_ref: Option<core::v1::ObjectReference>,
    pub infrastructure_ref: Option<core::v1::ObjectReference>,
    pub paused: Option<bool>,
    pub topology: Option<Topology>,
}
...

Related: #92 and #93

Initial draft that simply replaces invalid enum variants with auto generated names.

Things to discuss:

• [ ] What to do with with empty "" variants, we could call them Empty (maybe a separate issue)
• [ ] How do we name generated Variants?
• [ ] Should there be a duplicate check for auto generated variants (what if Variant{i} is a valid variant defined by the CRD (unlikely))

I also wasn't sure where to do the renaming, some of the renaming is done in Container::rename and some is done in main and there is also a little bit of logic in analyze_enum_properties.

This currently also changes integer enum generation:

#[derive(Serialize, Deserialize, Clone, Debug, JsonSchema)]
pub enum HTTPRouteRulesBackendRefsFiltersRequestRedirectStatusCode {
    #[serde(rename = "301")]
    r#_301,
    #[serde(rename = "302")]
    r#_302,
}

But I think int enums need to be generated differently anyways using serde_repr e.g.:

#[derive(Serialize_repr, Deserialize_repr, Clone, Debug, JsonSchema)]
#[repr(u64)]
pub enum HTTPRouteRulesBackendRefsFiltersRequestRedirectStatusCode {
    r#_301 = 301,
    r#_302 = 302,
}

If the object is empty and it has s-kubernetes-preserve-unknown-fields set, do not create an empty struct. Instead, use a BTreeMap<String, serde_json::Value>.

I was generating the rust code for this CRD and noticed that the code generated is using Array instead of Vec for some fields.

use kube::CustomResource;
use serde::{Serialize, Deserialize};
use std::collections::BTreeMap;

#[derive(CustomResource, Serialize, Deserialize, Clone, Debug)]
#[kube(group = "agent-install.openshift.io", version = "v1beta1", kind = "Agent", plural = "agents")]
#[kube(namespaced)]
#[kube(schema = "disabled")]
pub struct AgentSpec {
    pub approved: bool,
    pub clusterDeploymentName: Option<AgentClusterDeploymentName>,
    pub hostname: Option<String>,
    pub ignitionConfigOverrides: Option<String>,
    pub installation_disk_id: Option<String>,
    pub installerArgs: Option<String>,
    pub machineConfigPool: Option<String>,
    pub role: String,
}

#[derive(Serialize, Deserialize, Clone, Debug)]
pub struct AgentClusterDeploymentName {
    pub name: Option<String>,
    pub namespace: Option<String>,
}

#[derive(Serialize, Deserialize, Clone, Debug)]
pub struct AgentStatus {
    pub bootstrap: Option<bool>,
    #[serde(default, skip_serializing_if = "Vec::is_empty")]
    pub conditions: Vec<AgentStatusConditions>,
    pub debugInfo: Option<AgentStatusDebugInfo>,
    pub hostname: Option<String>,
    pub inventory: Option<AgentStatusInventory>,
    #[serde(default, skip_serializing_if = "Vec::is_empty")]
    pub ntpSources: Vec<AgentStatusNtpSources>,
    pub progress: Option<AgentStatusProgress>,
    pub role: Option<String>,
    #[serde(default, skip_serializing_if = "BTreeMap::is_empty")]
    pub validationsInfo: BTreeMap<String, Array>,
}

#[derive(Serialize, Deserialize, Clone, Debug)]
pub struct AgentStatusConditions {
    pub lastHeartbeatTime: Option<String>,
    pub lastTransitionTime: Option<String>,
    pub message: Option<String>,
    pub reason: Option<String>,
    pub status: String,
    pub r#type: String,
}

#[derive(Serialize, Deserialize, Clone, Debug)]
pub struct AgentStatusDebugInfo {
    pub eventsURL: Option<String>,
    pub state: Option<String>,
    pub stateInfo: Option<String>,
}

#[derive(Serialize, Deserialize, Clone, Debug)]
pub struct AgentStatusInventory {
    pub bmcAddress: Option<String>,
    pub bmcV6Address: Option<String>,
    pub boot: Option<AgentStatusInventoryBoot>,
    pub cpu: Option<AgentStatusInventoryCpu>,
    #[serde(default, skip_serializing_if = "Vec::is_empty")]
    pub disks: Vec<AgentStatusInventoryDisks>,
    pub hostname: Option<String>,
    #[serde(default, skip_serializing_if = "Vec::is_empty")]
    pub interfaces: Vec<AgentStatusInventoryInterfaces>,
    pub memory: Option<AgentStatusInventoryMemory>,
    pub reportTime: Option<String>,
    pub systemVendor: Option<AgentStatusInventorySystemVendor>,
}

#[derive(Serialize, Deserialize, Clone, Debug)]
pub struct AgentStatusInventoryBoot {
    pub currentBootMode: Option<String>,
    pub pxeInterface: Option<String>,
}

#[derive(Serialize, Deserialize, Clone, Debug)]
pub struct AgentStatusInventoryCpu {
    pub architecture: Option<String>,
    pub clockMegahertz: Option<i64>,
    pub count: Option<i64>,
    #[serde(default, skip_serializing_if = "Vec::is_empty")]
    pub flags: Vec<String>,
    pub modelName: Option<String>,
}

#[derive(Serialize, Deserialize, Clone, Debug)]
pub struct AgentStatusInventoryDisks {
    pub bootable: Option<bool>,
    pub byID: Option<String>,
    pub byPath: Option<String>,
    pub driveType: Option<String>,
    pub hctl: Option<String>,
    pub id: String,
    pub installationEligibility: Option<AgentStatusInventoryDisksInstallationEligibility>,
    pub ioPerf: Option<AgentStatusInventoryDisksIoPerf>,
    pub model: Option<String>,
    pub name: Option<String>,
    pub path: Option<String>,
    pub serial: Option<String>,
    pub sizeBytes: Option<i64>,
    pub smart: Option<String>,
    pub vendor: Option<String>,
    pub wwn: Option<String>,
}

#[derive(Serialize, Deserialize, Clone, Debug)]
pub struct AgentStatusInventoryDisksInstallationEligibility {
    pub eligible: Option<bool>,
    pub notEligibleReasons: Vec<String>,
}

#[derive(Serialize, Deserialize, Clone, Debug)]
pub struct AgentStatusInventoryDisksIoPerf {
    pub syncDurationMilliseconds: Option<i64>,
}

#[derive(Serialize, Deserialize, Clone, Debug)]
pub struct AgentStatusInventoryInterfaces {
    pub biosDevName: Option<String>,
    pub clientID: Option<String>,
    pub flags: Vec<String>,
    pub hasCarrier: Option<bool>,
    pub ipV4Addresses: Vec<String>,
    pub ipV6Addresses: Vec<String>,
    pub macAddress: Option<String>,
    pub mtu: Option<i64>,
    pub name: Option<String>,
    pub product: Option<String>,
    pub speedMbps: Option<i64>,
    pub vendor: Option<String>,
}

#[derive(Serialize, Deserialize, Clone, Debug)]
pub struct AgentStatusInventoryMemory {
    pub physicalBytes: Option<i64>,
    pub usableBytes: Option<i64>,
}

#[derive(Serialize, Deserialize, Clone, Debug)]
pub struct AgentStatusInventorySystemVendor {
    pub manufacturer: Option<String>,
    pub productName: Option<String>,
    pub serialNumber: Option<String>,
    pub virtual: Option<bool>,
}

#[derive(Serialize, Deserialize, Clone, Debug)]
pub struct AgentStatusNtpSources {
    pub sourceName: Option<String>,
    pub sourceState: Option<String>,
}

#[derive(Serialize, Deserialize, Clone, Debug)]
pub struct AgentStatusProgress {
    pub currentStage: Option<String>,
    pub progressInfo: Option<String>,
    pub stageStartTime: Option<String>,
    pub stageUpdateTime: Option<String>,
}

Tried to run kopium on:

kopium applications.argoproj.io --docs > argoapp.rs

Get the following due to some of the CRD's being self-referential:

error[E0072]: recursive type `ApplicationStatusSyncComparedToSourceDirectory` has infinite size
    --> src/argoapp.rs:1330:1
     |
1330 | pub struct ApplicationStatusSyncComparedToSourceDirectory {
     | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ recursive type has infinite size
...
1336 |     pub jsonnet: Option<ApplicationStatusSyncComparedToSourceDirectory>,
     |                  ------------------------------------------------------ recursive without indirection
     |
help: insert some indirection (e.g., a `Box`, `Rc`, or `&`) to make `ApplicationStatusSyncComparedToSourceDirectory` representable
     |
1336 |     pub jsonnet: Box<Option<ApplicationStatusSyncComparedToSourceDirectory>>,
     |                  ++++                                                      +

I think I can fixup these myself by Boxing them, any other ideas?

When multiple versions are described in the CRD and no --api-version flag is given it may be better to use the one that is specified in storedVersion of the status, rather than just the first one.

Hi :wave:

When using kopium to generate Kubernetes Gateway API resources, I found that it's not capable of generating one of the APIs, HTTPRoute specifically:

$ curl -sSL 'https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/main/config/crd/standard/gateway.networking.k8s.io_httproutes.yaml?ref=v0.5.1' | kopium -Af -
// WARNING: generated by kopium - manual changes will be overwritten
Error: not handling non-string enum

I found the line here. Looks like there's just more TODO here to add support? Is there already work tracking this TODO?

I have quite a lot of positive experience with srtuctopt, and clap 3 is essentially it, so I rewrote arg parsing to use Parser trait. Hope you don't mind.

Selecting required version - please have a look, I believe I got it right but would love to get feedback.

And the last one - macro_use is so Rust 2015, just couldn't help it :)

Add extra flags to clap args and handle them:

• [x] --hide-prelude bool (in case people need custom crate names)
• [x] --api-version 1 optional setter (otherwise we should pick .first version as is) - #15
• [x] --derive PartialEq optional (multi) setter (appends to derive list per struct)

the enums generated by 0.12 (#65) don't label which variant is the default, so if you try using --derive Default, you will end up with a lot of errors like

error: no default declared
    --> foo/bar/baz.rs:1116:48
     |
1116 | #[derive(Serialize, Deserialize, Clone, Debug, Default)]
     |                                                ^^^^^^^
     |
     = help: make a unit variant default by placing `#[default]` above it
     = note: this error originates in the derive macro `Default` (in Nightly builds, run with -Z macro-backtrace for more info)

• [x] Add discriminant to integer enums
• [x] Improve integration test for httproute so that it uses serde logic for the statusCode field
• [ ] Add serde_repr
• [ ] Drop #[serde(rename)] attrs on serde_repr enums

Last thing I want to fix before next release.

A rare case that we haven't seen, but should support because kube supports it, and it makes unit testing easier (no need to wrap enums in structs to check their output). Came out of #100

There is a test for this top_level_enum_with_integers which is currently #[ignore]d because of this bug.

We should handle enum members at the top level like we do for struct members.

In particular;

• HasStatus
• HasSpec

and maybe more like HasConditions once we get going with https://github.com/kube-rs/k8s-pb/issues/4

This is not really actionable yet since the traits need to find a common place to live first, but making a note of it.

Sometimes when writing auxiliary tools/controllers for existing crds, we are sometimes only necessary to generate the spec object for a type. In these cases it would be OK to omit the status objects of the resources.

We should add a --hide-status flag that hides the status object and everything only used by the status object.

This can be done by pruning the schema yaml under:

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
spec:
  versions:
  - name: v1beta1
    schema:
      openAPIV3Schema:
        properties:
          status: <- remove this key for every spec.versions

before running the analyzer.

Leaving this here as a help wanted for now. It might be an interesting place to play around with the schema. We haven't done this type of short-circuiting before, so it might need its own file.

Currently kopium generates BTreeMap for all map types except arbitrary json (stored in HashMap).

While this is OK on the surface level, I think we should probably move all containers to HashMap as the cheapest default, and let users do a big global --btreemap flag to switch all of them over to BTreeMap when users need deterministic ordering.

runtime.RawExtensions serves as a blob in the k8s CRD afaik and is reflected as an object in the openapi spec, like this.

The specific objects are origin and target, which are defined in go as following

// TransformInputSpec struct
type TransformInputSpec struct {
	Origin runtime.RawExtension `json:"origin,omitempty"`
	Target runtime.RawExtension `json:"target,omitempty"`
}

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  annotations:
    controller-gen.kubebuilder.io/version: v0.9.0
  creationTimestamp: null
  name: transforminputs.app.yndd.io
spec:
  group: app.example.io
  names:
    categories:
    - app
    kind: TransformInput
    listKind: TransformInputList
    plural: transforminputs
    singular: transforminput
  scope: Namespaced
  versions:
  - name: v1alpha1
    schema:
      openAPIV3Schema:
        description: TransformInput is the Schema for the TransformInput API
        properties:
          apiVersion:
            description: 'APIVersion defines the versioned schema of this representation
              of an object. Servers should convert recognized schemas to the latest
              internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
            type: string
          kind:
            description: 'Kind is a string value representing the REST resource this
              object represents. Servers may infer this from the endpoint the client
              submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
            type: string
          metadata:
            type: object
          spec:
            description: TransformInputSpec struct
            properties:
              origin:
                type: object
              target:
                type: object
            type: object
        type: object
    served: true
    storage: true

when I use kopium to generate the rust code I get this.

use kube::CustomResource;
use schemars::JsonSchema;
use serde::{Serialize, Deserialize};

/// TransformInputSpec struct
#[derive(CustomResource, Serialize, Deserialize, Clone, Debug, JsonSchema)]
#[kube(
    group = "app.example.io",
    version = "v1alpha1",
    kind = "TransformInput",
    plural = "transforminputs"
)]
#[kube(namespaced)]
pub struct TransformInputSpec {
    #[serde(default, skip_serializing_if = "Option::is_none")]
    pub origin: Option<TransformInputOrigin>,
    #[serde(default, skip_serializing_if = "Option::is_none")]
    pub target: Option<TransformInputTarget>,
}

#[derive(Serialize, Deserialize, Clone, Debug, JsonSchema)]
pub struct TransformInputOrigin {
}

#[derive(Serialize, Deserialize, Clone, Debug, JsonSchema)]
pub struct TransformInputTarget {
}

As you can see the struct TransformInputOrigin and TransformInputTarget are empty. The issue is that when you serializer/deserializerl it in json/yaml the data behind these object is no longer accessible. When I change it to the following manually the marshal/unmarshal function works as the serde_json::Value is seen as a generic object in the serializer/deserializer afaik.

use kube::CustomResource;
use schemars::JsonSchema;
use serde::{Deserialize, Serialize};
use serde_json::Value;

/// TransformInputSpec struct
#[derive(CustomResource, Serialize, Deserialize, Clone, Debug, JsonSchema)]
#[kube(
    group = "app.example.io",
    version = "v1alpha1",
    kind = "TransformInput",
    plural = "transforminputs"
)]
#[kube(namespaced)]
pub struct TransformInputSpec {
    #[serde(default, skip_serializing_if = "Option::is_none")]
    pub origin: Option<Value>,
    #[serde(default, skip_serializing_if = "Option::is_none")]
    pub target: Option<Value>,
}

Can this be accommodated?