Namespaces on top of Kafka Broker, Kafka Connect and Schema Registry
Project moved to https://github.com/michelin/ns4kafka
- Desired state API
- Mimics K8S principles
- Easy to use
- View : GET /api/namespaces/<ns-name>/topics/<topic-name>
- List : GET /api/namespaces/<ns-name>/topics
- Create : POST /api/namespaces/<ns-name>/topics
- Self-service
- Topics
- Schemas, Connects, KafkaUsers
- Configuration validation that can differ for each namespace
- Enforce any configuration for any resource
- min.insync.replica = 2
- partitions between 3 and 10
- Multiple validators
- Enforce any configuration for any resource
- Isolation between Kafka Users within a cluster
- Security model
- Disk Quota management
- Enforce limits per namespace
- Provide usage metrics
- Cross Namespace ACLs
- Multi cluster
Example namespace:
{ "name": "namespace-project1", "cluster": "kafka-dev", "defaulKafkatUser": "u_project1", "topicValidator": { "validationConstraints": { "replication.factor": { "validation-type": "Range", "min": 1, "max": 1 }, "partitions": { "validation-type": "Range", "min": 3, "max": 10 }, "cleanup.policy": { "validation-type": "ValidList", "validStrings": [ "delete", "compact" ] }, "retention.ms": { "validation-type": "Range", "min": 1000, "max": 3600000 } } } }{ "apiVersion": "v1", "kind": "AccessControlEntry", "metadata": { "name": "namespace-project1-6b062011", "namespace": "namespace-project1", "cluster": "kafka-dev", "labels": { "grantor": "admin" } }, "spec": { "resourceType": "TOPIC", "resource": "project1.", "resourcePatternType": "PREFIXED", "permission": "OWNER", "grantedTo": "namespace-project1" } }POST /api/namespaces/namespace-project1/topics HTTP/1.1 Host: localhost:8080 X-Gitlab-Token: xxxxxxxxxx Content-Type: application/json Content-Length: 250 { "metadata": { "name": "project1.topic1" }, "spec": { "replicationFactor": 1, "partitions": 3, "configs": { "cleanup.policy": "delete", "min.insync.replicas": "1", "retention.ms": "10000" } } } POST /api/namespaces/namespace-project1/acls HTTP/1.1 Host: localhost:8080 X-Gitlab-Token: xxxxxxxxxx Content-Type: application/json Content-Length: 195 { "spec": { "resourceType": "TOPIC", "resource": "project1.topic1", "resourcePatternType": "LITERAL", "permission": "READ", "grantedTo": "namespace-anotherproject" } }