9608 drf spectacular2

[arthanson]

Fixes: #9608

Removes drf-yasg and replaces it with drf-spectacular which allows us to go from Swagger 2 to OpenAPI 3.

Warning
This will be a breaking change to people who rely on the swagger files.

The documentation will also need to be updated for this change.

There are still a few warnings around IPAddressField but it defaults to string which is correct. You can generate the file and see any warnings via:

netbox/manage.py spectacular --file schema.yaml --validate --fail-on-warn 

Implemenation Notes:

1. had to add allow_null=True to many serializers. This should be correct as for example provider_network if you look at the class it is a nullable field.

2. code for NetBoxAutoSchema is to fix the bulk operations.

3. The 3 generated spec specify all(?) readonly fields as required. e.g. Prefix which list as required: [ "_depth", "children", "created" "display", "family", "id", "last_updated", "prefix", "url"]. This is different then the 2 spec where the readonly fields are not required. The 3 spec is more correct as the field are marked as readonly, however it is possible some client generators may have difficult with this, this can be controlled via a settings for SPECTACULAR_SETTINGS:

# Aid client generator targets that have trouble with read-only properties. 'COMPONENT_NO_READ_ONLY_REQUIRED': True, 

4. Had to add some get_serializer_class to some ViewSets - these views had different serializers for get/post but didn't define a base serializer which caused issues with drf-spectacular. First put a base serializer in with serializer_class =, however this caused side-issues of it being a class within NetBoxAutoSchema so used the get_serializer_class.

5. There is difference in the paths in the schema old (api is prefixed): "/circuits/circuit-terminations/" new: "/api/circuits/circuit-terminations/" not sure if this would have any effects on clients.

6. The Writable classes have "Request" post-pended to the name via spectacular, again don't think this will have any major effect on clients: "$ref: '#/definitions/WritableCircuitTermination'" -> "$ref: '#/components/schemas/WritableCircuitTerminationRequest'"

7. Descriptions for the classes in the schema are not good, usually defaulting to a base item, however the descriptions are not present in the 2.0 schema file, so not something broken just has never been added. Better descriptions to the serializers can be added in the future. For example the WritableCircuitRequest has the description "description: Adds support for custom fields and tags."

Merging Info

2.0 Compatibility: We should wait to merge this to just before Beta (all other changes in) then generate the 2.0 yaml and json schema files and include them in the repo (or make the available to people) then merge this. This will give end-users more time to adapt to this as the 2.0 schema files will still be available.

Warnings

Currently has the following warnings - these default to string, or auto resolve, can look at going through and cleaning these up so no warnings later.

Not sure why IPAddressField is causing warnings - tried to override it in several places to prevent the warning without success, again defaults to string which is correct.

/Users/ahanson/dev/work/netbox/netbox/ipam/api/nested_serializers.py: Warning [DeviceViewSet > DeviceWithConfigContextSerializer > NestedIPAddressSerializer]: model field "IPAddressField" has no mapping in ModelSerializer. It may be a deprecated field. Defaulting to "string"
/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/rest_framework/serializers.py: Warning [DeviceViewSet > WritableDeviceWithConfigContextSerializer]: unable to resolve type hint for function "primary_ip". Consider using a type hint or @extend_schema_field. Defaulting to string.
/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/rest_framework/serializers.py: Warning [InterfaceViewSet > WritableInterfaceSerializer]: unable to resolve type hint for function "l2vpn_termination". Consider using a type hint or @extend_schema_field. Defaulting to string.
/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/rest_framework/serializers.py: Warning [VirtualDeviceContextViewSet > WritableVirtualDeviceContextSerializer]: unable to resolve type hint for function "primary_ip". Consider using a type hint or @extend_schema_field. Defaulting to string.
/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/rest_framework/serializers.py: Warning [AggregateViewSet > WritableAggregateSerializer]: unable to resolve type hint for function "family". Consider using a type hint or @extend_schema_field. Defaulting to string.
/Users/ahanson/dev/work/netbox/netbox/ipam/api/serializers.py: Warning [IPAddressViewSet > IPAddressSerializer]: model field "IPAddressField" has no mapping in ModelSerializer. It may be a deprecated field. Defaulting to "string"
/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/rest_framework/serializers.py: Warning [IPAddressViewSet > WritableIPAddressSerializer]: unable to resolve type hint for function "family". Consider using a type hint or @extend_schema_field. Defaulting to string.
/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/rest_framework/serializers.py: Warning [IPAddressViewSet > WritableIPAddressSerializer]: model field "IPAddressField" has no mapping in ModelSerializer. It may be a deprecated field. Defaulting to "string"
/Users/ahanson/dev/work/netbox/netbox/ipam/api/serializers.py: Warning [IPRangeViewSet > IPRangeSerializer]: model field "IPAddressField" has no mapping in ModelSerializer. It may be a deprecated field. Defaulting to "string"
/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/rest_framework/serializers.py: Warning [IPRangeViewSet > WritableIPRangeSerializer]: unable to resolve type hint for function "family". Consider using a type hint or @extend_schema_field. Defaulting to string.
/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/rest_framework/serializers.py: Warning [IPRangeViewSet > WritableIPRangeSerializer]: model field "IPAddressField" has no mapping in ModelSerializer. It may be a deprecated field. Defaulting to "string"
/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/rest_framework/serializers.py: Warning [PrefixViewSet > WritablePrefixSerializer]: unable to resolve type hint for function "family". Consider using a type hint or @extend_schema_field. Defaulting to string.
/Users/ahanson/dev/work/netbox/netbox/ipam/filtersets.py: Warning [ServiceViewSet > ServiceFilterSet]: model field "IPAddressField" has no mapping in ModelSerializer. It may be a deprecated field. Defaulting to "string"
/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/rest_framework/serializers.py: Warning [AvailableVLANsView > WritableVLANSerializer]: unable to resolve type hint for function "l2vpn_termination". Consider using a type hint or @extend_schema_field. Defaulting to string.
/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/rest_framework/serializers.py: Warning [VLANViewSet > WritableVLANSerializer]: unable to resolve type hint for function "l2vpn_termination". Consider using a type hint or @extend_schema_field. Defaulting to string.
/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/rest_framework/serializers.py: Warning [VMInterfaceViewSet > WritableVMInterfaceSerializer]: unable to resolve type hint for function "l2vpn_termination". Consider using a type hint or @extend_schema_field. Defaulting to string.
/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/rest_framework/serializers.py: Warning [VirtualMachineViewSet > WritableVirtualMachineWithConfigContextSerializer]: unable to resolve type hint for function "primary_ip". Consider using a type hint or @extend_schema_field. Defaulting to string.
Warning: enum naming encountered a non-optimally resolvable collision for fields named "type". The same name has been used for multiple choice sets in multiple components. The collision was resolved with "TypeDdfEnum". add an entry to ENUM_NAME_OVERRIDES to fix the naming.
Warning: enum naming encountered a non-optimally resolvable collision for fields named "type". The same name has been used for multiple choice sets in multiple components. The collision was resolved with "Type261Enum". add an entry to ENUM_NAME_OVERRIDES to fix the naming.
Warning: enum naming encountered a non-optimally resolvable collision for fields named "type". The same name has been used for multiple choice sets in multiple components. The collision was resolved with "Type666Enum". add an entry to ENUM_NAME_OVERRIDES to fix the naming.
Warning: enum naming encountered a non-optimally resolvable collision for fields named "type". The same name has been used for multiple choice sets in multiple components. The collision was resolved with "Type6e5Enum". add an entry to ENUM_NAME_OVERRIDES to fix the naming.
Warning: enum naming encountered a non-optimally resolvable collision for fields named "type". The same name has been used for multiple choice sets in multiple components. The collision was resolved with "Type2fcEnum". add an entry to ENUM_NAME_OVERRIDES to fix the naming.
Warning: enum naming encountered a non-optimally resolvable collision for fields named "type". The same name has been used for multiple choice sets in multiple components. The collision was resolved with "Type209Enum". add an entry to ENUM_NAME_OVERRIDES to fix the naming.
Warning: enum naming encountered a non-optimally resolvable collision for fields named "type". The same name has been used for multiple choice sets in multiple components. The collision was resolved with "Type653Enum". add an entry to ENUM_NAME_OVERRIDES to fix the naming.
Warning: enum naming encountered a non-optimally resolvable collision for fields named "protocol". The same name has been used for multiple choice sets in multiple components. The collision was resolved with "Protocol766Enum". add an entry to ENUM_NAME_OVERRIDES to fix the naming.
Warning: enum naming encountered a non-optimally resolvable collision for fields named "protocol". The same name has been used for multiple choice sets in multiple components. The collision was resolved with "ProtocolFe4Enum". add an entry to ENUM_NAME_OVERRIDES to fix the naming.
Warning: enum naming encountered a non-optimally resolvable collision for fields named "auth_type". The same name has been used for multiple choice sets in multiple components. The collision was resolved with "AuthTypeF47Enum". add an entry to ENUM_NAME_OVERRIDES to fix the naming.
Warning: enum naming encountered a non-optimally resolvable collision for fields named "status". The same name has been used for multiple choice sets in multiple components. The collision was resolved with "Status565Enum". add an entry to ENUM_NAME_OVERRIDES to fix the naming.
Warning: enum naming encountered a non-optimally resolvable collision for fields named "status". The same name has been used for multiple choice sets in multiple components. The collision was resolved with "Status6a0Enum". add an entry to ENUM_NAME_OVERRIDES to fix the naming.
Warning: enum naming encountered a non-optimally resolvable collision for fields named "status". The same name has been used for multiple choice sets in multiple components. The collision was resolved with "Status62cEnum". add an entry to ENUM_NAME_OVERRIDES to fix the naming.
Warning: enum naming encountered a non-optimally resolvable collision for fields named "status". The same name has been used for multiple choice sets in multiple components. The collision was resolved with "Status71fEnum". add an entry to ENUM_NAME_OVERRIDES to fix the naming.
Warning: enum naming encountered a non-optimally resolvable collision for fields named "status". The same name has been used for multiple choice sets in multiple components. The collision was resolved with "StatusB66Enum". add an entry to ENUM_NAME_OVERRIDES to fix the naming.