Skip to content

Unable to create ReplicaSet with "configsrv" clusterRole #192

Closed
@jlpedrosa

Description

@jlpedrosa

What did you do to encounter the bug?
In a kube cluster create a CR that configures the configsvr clusterRole:

apiVersion: mongodbcommunity.mongodb.com/v1 kind: MongoDBCommunity metadata: name: experiment spec: members: 3 type: ReplicaSet version: "8.0.10" additionalMongodConfig: sharding: clusterRole: "configsvr"

What did you expect?
The ReplicaSet to be up and ready.

What happened instead?
The Replicaset stays on pending, and authentication is not possible.

kubectl get mdbc -o wide NAME PHASE VERSION experiment Pending 

But pods are actually booted and ready.

kubectl get pods NAME READY STATUS RESTARTS AGE experiment-0 2/2 Running 0 13m experiment-1 2/2 Running 0 12m experiment-2 2/2 Running 0 11m 
  1. I think the error is isolated to the mms automation.
    when I exec into the agent containter in the pod, I see in the logs (in a repeat loop)
    (process experiment-0.experiment-svc.mongodb-operator.svc.cluster.local:27017 doesn't have the automation user).
[2025-06-14T13:28:53.716+0000] [.info] [src/config/config.go:ReadClusterConfig:447] [13:28:53.716] Retrieving cluster config from /var/lib/automation/config/cluster-config.json... [2025-06-14T13:28:53.716+0000] [.info] [main/components/agent.go:LoadClusterConfig:276] [13:28:53.716] clusterConfig unchanged [2025-06-14T13:28:54.164+0000] [.info] [src/mongoclientservice/mongoclientservice.go:func1:1737] [13:28:54.164] Testing auth with username __system db=local to experiment-0.experiment-svc.mongodb-operator.svc.cluster.local:27017 (local=false,router=false) connectMode=SingleConnect ipversion=0 tls=false [2025-06-14T13:28:54.168+0000] [.info] [src/mongoctl/processctl.go:GetKeyHashes:2148] <experiment-0> [13:28:54.168] Able to successfully auth to experiment-0.experiment-svc.mongodb-operator.svc.cluster.local:27017 (local=false,router=false) using desired auth key [2025-06-14T13:28:54.177+0000] [.info] [state2/currentstate/auth.go:EvaluateUsers:124] <experiment-0> [13:28:54.177] AreUsersRight returning false because 1 users need to be adjusted ([my-user@admin]) and 0 users need to be deleted ([]). currentUsers=[] desired auth user info = authconfig.ProcessAuthInfoUsersWanted=[AuthUser(my-user@admin,roles=AuthRoles(4:{"role":"clusterAdmin","db":"admin","minFcv":""},{"role":"userAdminAnyDatabase","db":"admin","minFcv":""},{"role":"clusterAdmin","db":"config","minFcv":""},{"role":"clusterAdmin","db":"local","minFcv":""}),restrictions=[])]UsersDeleted=[]Roles=[]DesiredKey=[ZpnXOsjiRNY-REDACTED-G04AKp5vLX0]DesiredNewKey=[ZpnXOsjiRNY-REDACTED-G04AKp5vLX0]DesiredKeyHash=[58 144 182 249 191 92 4 9 230 184 238 254 225 65 157 53 86 253 195 148 117 148 230 121 245 218 1 18 161 29 27 60]DesiredNewKeyHash=[]KeyfileHashes=[[58 144 182 249 191 92 4 9 230 184 238 254 225 65 157 53 86 253 195 148 117 148 230 121 245 218 1 18 161 29 27 60]]UsingAuth=true [2025-06-14T13:28:54.185+0000] [.info] [src/mongoctl/processctl.go:Update:3890] <experiment-0> [13:28:54.185] <DB_WRITE> Updated with query map[] and update [{$set [{agentFeatures [StateCache]} {nextVersion 1}]}] and upsert=true on local.clustermanager [2025-06-14T13:28:54.194+0000] [.info] [src/director/director.go:computePlan:283] <experiment-0> [13:28:54.194] ... process has a plan : WaitHasCorrectAutomationCredentials,AdjustUsers [2025-06-14T13:28:54.194+0000] [.info] [src/director/director.go:tracef:830] <experiment-0> [13:28:54.194] Running step: 'WaitHasCorrectAutomationCredentials' of move 'WaitHasCorrectAutomationCredentials' [2025-06-14T13:28:54.194+0000] [.info] [src/director/director.go:tracef:830] <experiment-0> [13:28:54.194] because  [All the following are true:   [All the following are true:   ['currentState.CanUpdateAutomationCredentials' = false]  ['currentState.Up' = true]  ['currentState.IsAuthSpecified' = true]  [All the following are false:   [<current and desired keyfiles contain common key with hash=[58 144 182 249 191 92 4 9 230 184 238 254 225 65 157 53 86 253 195 148 117 148 230 121 245 218 1 18 161 29 27 60]>]  ]  [All the following are true:   ['currentState.ReplSetConf.Id' = 'desiredState.ReplSetConf.Id'; both are (experiment)]  ['currentState.ReplSetConf.Members' = 'desiredState.ReplSetConf.Members'; both are ([{"_id":0,"arbiterOnly":false,"buildIndexes":true,"hidden":false,"host":"experiment-0.experiment-svc.mongodb-operator.svc.cluster.local:27017","priority":1,"secondaryDelaySecs":0,"tags":{},"votes":1},{"_id":1,"arbiterOnly":false,"buildIndexes":true,"hidden":false,"host":"experiment-1.experiment-svc.mongodb-operator.svc.cluster.local:27017","priority":1,"secondaryDelaySecs":0,"tags":{},"votes":1},{"_id":2,"arbiterOnly":false,"buildIndexes":true,"hidden":false,"host":"experiment-2.experiment-svc.mongodb-operator.svc.cluster.local:27017","priority":1,"secondaryDelaySecs":0,"tags":{},"votes":1}])]  ['currentState.ReplSetConf.ProtocolVersion' = 'desiredState.ReplSetConf.ProtocolVersion'; both are (1)]  [<No WriteConcernMajorityJournalDefault set.>]  ['desiredState.ReplSetConf.Version' = 0]  ]  ['currentState.BackupRestoreCorrect' = true]  ['current Auth mode' = 'desired Auth mode'; both are (authOn)]  ]  ['currentState.ProcessHasCorrectAutomationCredentials' = false] ] [2025-06-14T13:28:54.194+0000] [.info] [src/director/director.go:planAndExecute:588] <experiment-0> [13:28:54.194] Step=WaitHasCorrectAutomationCredentials as part of Move=WaitHasCorrectAutomationCredentials in plan failed : <experiment-0> [13:28:54.194] Postcondition not yet met for step WaitHasCorrectAutomationCredentials because  ['currentState.ProcessHasCorrectAutomationCredentials' = false] (process experiment-0.experiment-svc.mongodb-operator.svc.cluster.local:27017 doesn't have the automation user).  Recomputing a plan... [2025-06-14T13:28:54.752+0000] [.info] [src/config/config.go:ReadClusterConfig:447] [13:28:54.752] Retrieving cluster config from /var/lib/automation/config/cluster-config.json... [2025-06-14T13:28:54.752+0000] [.info] [main/components/agent.go:LoadClusterConfig:276] [13:28:54.752] clusterConfig unchanged

Operator Information

  • Operator Version: quay.io/mongodb/mongodb-kubernetes:1.1.0
  • MongoDB Image used 8.0.10

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions