diff options
| -rw-r--r-- | README | 193 | ||||
| -rw-r--r-- | config.yaml | 136 | ||||
| -rwxr-xr-x | hooks/config-changed | 25 | ||||
| -rwxr-xr-x | hooks/database-relation-joined | 13 | ||||
| -rwxr-xr-x | hooks/install | 83 | ||||
| -rwxr-xr-x | hooks/replica-set-relation-changed | 93 | ||||
| -rwxr-xr-x | hooks/replica-set-relation-joined | 28 | ||||
| -rwxr-xr-x | hooks/start | 6 | ||||
| -rwxr-xr-x | hooks/stop | 10 | ||||
| -rw-r--r-- | metadata.yaml | 23 |
10 files changed, 334 insertions, 276 deletions
@@ -40,20 +40,150 @@ The MongoDB charm allows for certain values to be configurable via the config.ya A sample of the default settings of the config.yaml file at the time of writing are as follows: options: + dbpath: + default: "/var/lib/mongodb" + type: string + description: The path where the data files will be kept. + logpath: + default: "/var/log/mongodb/mongodb.log" + type: string + description: The path where to send log data. + logappend: + default: True + type: boolean + description: Append log entries to existing log file + bind_ip: + default: "all" + type: string + description: IP address that mongodb should listen for connections. + port: + default: 27017 + type: int + description: Default MongoDB port + journal: + default: True + type: boolean + description: Enable journaling, http://www.mongodb.org/display/DOCS/Journaling + cpu: + default: False + type: boolean + description: Enables periodic logging of CPU utilization and I/O wait + auth: + default: False + type: boolean + description: Turn on/off security + verbose: + default: False + type: boolean + description: Verbose logging output + objcheck: + default: False + type: boolean + description: Inspect all client data for validity on receipt (useful for developing drivers) + quota: + default: False + type: boolean + description: Enable db quota management + diaglog: + default: 0 + type: int + description: Set oplogging level where n is 0=off (default), 1=W, 2=R, 3=both, 7=W+some reads + nocursors: + default: False + type: boolean + description: Diagnostic/debugging option + nohints: + default: False + type: boolean + description: Ignore query hints + noscripting: + default: False + type: boolean + description: Turns off server-side scripting. This will result in greatly limited functionality + notablescan: + default: False + type: boolean + description: Turns off table scans. Any query that would do a table scan fails + noprealloc: + default: False + type: boolean + description: Disable data file preallocation + nssize: + default: "default" + type: string + description: Specify .ns file size for new databases + mms-token: + default: "disabled" + type: string + description: Accout token for Mongo monitoring server + mms-name: + default: "disabled" + type: string + description: Server name for Mongo monitoring server + mms-interval: + default: "disabled" + type: string + description: Ping interval for Mongo monitoring server ( in number of seconds ) + autoresync: + default: False + type: boolean + description: Automatically resync if slave data is stale + oplogSize: + default: "default" + type: string + description: Custom size for replication operation log + opIdMem: + default: "default" + type: string + description: Size limit for in-memory storage of op ids replicaset: default: myset type: string description: Name of the replica set web_admin_ui: - default: yes - type: string - description: Replica Set Admin UI ( yes / no ) + default: True + type: boolean + description: Replica Set Admin UI ( accessible via default_port + 1000 ) replicaset_master: default: auto type: string - description: Replica Set master ( optional ). Possible values are 'auto' for automatic - detection based on install time or 'host:port' to connect to 'host' on - 'port' and register as a member. + description: Replica Set master ( optional ). Possible values are 'auto' for automatic detection based on install time or 'host:port' to connect to 'host' on 'port' and register as a member. + master: + default: "self" + type: string + description: Who is the master DB. If not "self", put the Master DB here as "host:port" + config_server_port: + default: 27019 + type: int + description: Port number to use for the config-server + config_server_dbpath: + default: "/mnt/var/lib/mongodb/configsvr" + type: string + description: The path where the config server data files will be kept. + config_server_logpath: + default: "/mnt/var/log/mongodb/configsvr.log" + type: string + description: The path where to send config server log data. + arbiter: + default: "disabled" + type: string + description: Enable arbiter mode. Possible values are 'disabled' for no arbiter, 'enable' to become an arbiter or 'host:port' to declare another host as an arbiter. replicaset_master must be set for this option to work. + mongos_logpath: + default: "/mnt/var/log/mongodb/mongos.log" + type: string + description: The path where to send log data from the mongo router. + mongos_port: + default: 27021 + type: int + description: Port number to use for the mongo router + extra_config_options: + default: "none" + type: string + description: Extra options ( comma separated ) to be included ( at the end ) in the mongodb.conf file. + extra_daemon_options: + default: "none" + type: string + description: Extra options ( exactly as you would type them in the command line ) to be added via the command line to the mongodb daemon Where: @@ -74,6 +204,8 @@ Where: - ie: hostname ( will connect to hostname on the default port of 27017 ) - ie: hostname:port ( will connect to hostname on port number <port> ) +Most of the options in config.yaml have been modeled after the default configuration file for mongodb (normally in /etc/mongodb.conf) and should be familiar to most mongodb admins. Each option in this charm have a brief description of what it does. + Deployment ========== @@ -149,6 +281,55 @@ Add multiple nodes to your replicaset We now have a working MongoDB replica-set. +Sharding +-------- + +According the the mongodb documentation found on their website (http://docs.mongodb.org/manual/tutorial/deploy-shard-cluster/), one way of deploying a Shard Cluster is as follows: + +- deploy config servers +- deploy a mongo shell (mongos) +- deploy shards +- connect the config servers to the mongo shell +- add the shards to the mongo shell + + +Using Juju we can deploy a sharded cluster using the following commands: + +- Bootstrap the environment +--- juju bootstrap + +- Config Servers ( we'll deploy 3 of them ) +--- juju deploy mongodb configsvr + +- Mongo Shell ( We just deploy one for now ) +--- juju deploy mongodb mongos + +- Shards ( We'll deploy three replica-sets ) +--- juju deploy mongodb shard1 -n3 +--- juju deploy mongodb shard2 -n3 +--- juju deploy mongodb shard3 -n3 + +- Connect the Config Servers to the Mongo shell (mongos) +--- juju add-relation mongos:mongos configsvr:configsvr + +- Connect each Shard to the Mongo shell (mongos) +--- juju add-realtion mongos:mongos shard1:database +--- juju add-realtion mongos:mongos shard2:database +--- juju add-realtion mongos:mongos shard3:database + +With the above commands, we should now have a three replica-set sharded cluster running. +Using the default configuration, here are some details of our sharded cluster: +- mongos is running on port 27021 +- configsvr is running on port 27019 +- the shards are running on the default mongodb port of 27017 +- The web admin is turned on by default and accessible with your browser on port 28017 on each of the shards. + +To verify that your sharded cluster is running, connect to the mongo shell and run sh.status(): +- mongo --host <mongos_host>:<mongos_port> +- run sh.status() +You should see your the hosts for your shards in the status output. + + Troubleshooting =============== diff --git a/config.yaml b/config.yaml index 56fd97c..969d45e 100644 --- a/config.yaml +++ b/config.yaml @@ -1,17 +1,145 @@ options: - default_port: + dbpath: + default: "/var/lib/mongodb" + type: string + description: The path where the data files will be kept. + logpath: + default: "/var/log/mongodb/mongodb.log" + type: string + description: The path where to send log data. + logappend: + default: True + type: boolean + description: Append log entries to existing log file + bind_ip: + default: "all" + type: string + description: IP address that mongodb should listen for connections. + port: default: 27017 type: int description: Default MongoDB port + journal: + default: True + type: boolean + description: Enable journaling, http://www.mongodb.org/display/DOCS/Journaling + cpu: + default: False + type: boolean + description: Enables periodic logging of CPU utilization and I/O wait + auth: + default: False + type: boolean + description: Turn on/off security + verbose: + default: False + type: boolean + description: Verbose logging output + objcheck: + default: False + type: boolean + description: Inspect all client data for validity on receipt (useful for developing drivers) + quota: + default: False + type: boolean + description: Enable db quota management + diaglog: + default: 0 + type: int + description: Set oplogging level where n is 0=off (default), 1=W, 2=R, 3=both, 7=W+some reads + nocursors: + default: False + type: boolean + description: Diagnostic/debugging option + nohints: + default: False + type: boolean + description: Ignore query hints + noscripting: + default: False + type: boolean + description: Turns off server-side scripting. This will result in greatly limited functionality + notablescan: + default: False + type: boolean + description: Turns off table scans. Any query that would do a table scan fails + noprealloc: + default: False + type: boolean + description: Disable data file preallocation + nssize: + default: "default" + type: string + description: Specify .ns file size for new databases + mms-token: + default: "disabled" + type: string + description: Accout token for Mongo monitoring server + mms-name: + default: "disabled" + type: string + description: Server name for Mongo monitoring server + mms-interval: + default: "disabled" + type: string + description: Ping interval for Mongo monitoring server ( in number of seconds ) + autoresync: + default: False + type: boolean + description: Automatically resync if slave data is stale + oplogSize: + default: "default" + type: string + description: Custom size for replication operation log + opIdMem: + default: "default" + type: string + description: Size limit for in-memory storage of op ids replicaset: default: myset type: string description: Name of the replica set web_admin_ui: - default: "yes" - type: string - description: Replica Set Admin UI ( yes / no ) + default: True + type: boolean + description: Replica Set Admin UI ( accessible via default_port + 1000 ) replicaset_master: default: auto type: string description: Replica Set master ( optional ). Possible values are 'auto' for automatic detection based on install time or 'host:port' to connect to 'host' on 'port' and register as a member. + master: + default: "self" + type: string + description: Who is the master DB. If not "self", put the Master DB here as "host:port" + config_server_port: + default: 27019 + type: int + description: Port number to use for the config-server + config_server_dbpath: + default: "/mnt/var/lib/mongodb/configsvr" + type: string + description: The path where the config server data files will be kept. + config_server_logpath: + default: "/mnt/var/log/mongodb/configsvr.log" + type: string + description: The path where to send config server log data. + arbiter: + default: "disabled" + type: string + description: Enable arbiter mode. Possible values are 'disabled' for no arbiter, 'enable' to become an arbiter or 'host:port' to declare another host as an arbiter. replicaset_master must be set for this option to work. + mongos_logpath: + default: "/mnt/var/log/mongodb/mongos.log" + type: string + description: The path where to send log data from the mongo router. + mongos_port: + default: 27021 + type: int + description: Port number to use for the mongo router + extra_config_options: + default: "none" + type: string + description: Extra options ( comma separated ) to be included ( at the end ) in the mongodb.conf file. + extra_daemon_options: + default: "none" + type: string + description: Extra options ( exactly as you would type them in the command line ) to be added via the command line to the mongodb daemon diff --git a/hooks/config-changed b/hooks/config-changed deleted file mode 100755 index df92ffb..0000000 --- a/hooks/config-changed +++ /dev/null @@ -1,25 +0,0 @@ -#!/bin/bash -# This must be renamed to the name of the relation. The goal here is to -# affect any change needed by relationships being formed -# This script should be idempotent. - -set -ux - -DEFAULT_REPLSET_NAME=`config-get replicaset` -REPLICASET_MASTER=`config-get replicaset_master` -HOSTNAME=`unit-get public-address` - -############################################################################################################ -# Are we connecting to an existing replica set? -############################################################################################################ -if [ "${REPLICASET_MASTER}" != "auto" ]; then - grep "${DEFAULT_REPLSET_NAME}" /etc/init/mongodb.conf - if [ $? -ne 0 ];then - sed -i -e "s/ -- / -- --replSet ${DEFAULT_REPLSET_NAME} /" /etc/init/mongodb.conf - service mongodb stop - rm -f /var/lib/mongodb/mongod.lock - service mongodb start - fi - mongo --host ${REPLICASET_MASTER} --eval "rs.add(\""${HOSTNAME}"\")" -fi - diff --git a/hooks/database-relation-joined b/hooks/database-relation-joined deleted file mode 100755 index 23ead04..0000000 --- a/hooks/database-relation-joined +++ /dev/null @@ -1,13 +0,0 @@ -#!/bin/bash -# This must be renamed to the name of the relation. The goal here is to -# affect any change needed by relationships being formed -# This script should be idempotent. - -set -ux - -HOSTNAME=`unit-get public-address` -REPLICASET=`config-get replicaset` - -relation-set hostname=${HOSTNAME} replset=${REPLICASET} - -echo $JUJU_REMOTE_UNIT joined diff --git a/hooks/install b/hooks/install deleted file mode 100755 index b0bf7c2..0000000 --- a/hooks/install +++ /dev/null @@ -1,83 +0,0 @@ -#!/bin/bash -# Here do anything needed to install the service -# i.e. apt-get install -y foo or bzr branch http://myserver/mycode /srv/webroot - -set -ux - -############################################################################################################ -# Set some variables that we'll need for later -############################################################################################################ -DEFAULT_REPLSET_NAME=`config-get replicaset` -#HOSTNAME=`hostname -f` -HOSTNAME=`unit-get public-address` -EPOCH=`date +%s` -INSTALL_ORDER=$(echo ${JUJU_UNIT_NAME} | awk -F/ '{ print $2 }') -WEB_ADMIN_UI=`config-get web_admin_ui` -REPLICASET_MASTER=`config-get replicaset_master` -DEFAULT_PORT=`config-get default_port` - - -############################################################################################################ -# Install mongodb -############################################################################################################ -DEBIAN_FRONTEND=noninteractive apt-get install -y mongodb - - -############################################################################################################ -# Change the default mongodb configuration -############################################################################################################ -sed -e "s/#master = true/master = true/" \ - -e "s/bind_ip/#bind_ip/" \ - -e "s/#port = 27017/port = ${DEFAULT_PORT}/" \ - -i /etc/mongodb.conf - - -############################################################################################################ -# Reconfigure the upstart script to include the replica-set option. -# We'll need this so, when we add nodes, they can all talk to each other. -# Replica sets can only talk to each other if they all belong to the same -# set. In our case, we have defaulted to "myset". -############################################################################################################ -# Web Admin UI -if [ "$WEB_ADMIN_UI" == "yes" ]; then - sed -i -e "s/ -- / -- --rest /" /etc/init/mongodb.conf -fi - -if [ "${REPLICASET_MASTER}" != "auto" ]; then - sed -i -e "s/ -- / -- --replSet ${DEFAULT_REPLSET_NAME} /" /etc/init/mongodb.conf -fi - - -############################################################################################################ -# stop then start ( *** not restart **** ) mongodb so we can finish the configuration -############################################################################################################ -service mongodb stop -# There is a bug in the upstart script that leaves a lock file orphaned.... Let's wipe that file out -rm -f /var/lib/mongodb/mongod.lock -service mongodb start - - -############################################################################################################ -# Are we connecting to an existing replica set? -############################################################################################################ -if [ "${REPLICASET_MASTER}" != "auto" ]; then - grep "${DEFAULT_REPLSET_NAME}" /etc/init/mongodb.conf - if [ $? -ne 0 ];then - sed -i -e "s/ -- / -- --replSet ${DEFAULT_REPLSET_NAME} /" /etc/init/mongodb.conf - service mongodb stop - rm -f /var/lib/mongodb/mongod.lock - service mongodb start - fi - mongo --host ${REPLICASET_MASTER} --eval "rs.add(\""${HOSTNAME}"\")" -fi - - -############################################################################################################ -# Register the port -############################################################################################################ -[ -x /usr/bin/open-port ] && open-port ${DEFAULT_PORT}/TCP -if [ "$WEB_ADMIN_UI" == "yes" ]; then - [ -x /usr/bin/open-port ] && open-port $((${DEFAULT_PORT} + 1000))/TCP -fi - -exit 0 diff --git a/hooks/replica-set-relation-changed b/hooks/replica-set-relation-changed deleted file mode 100755 index d0b41bc..0000000 --- a/hooks/replica-set-relation-changed +++ /dev/null @@ -1,93 +0,0 @@ -#!/bin/bash -# This must be renamed to the name of the relation. The goal here is to -# affect any change needed by relationships being formed, modified, or broken -# This script should be idempotent. - -############################################################################################################ -# Set debugging information -############################################################################################################ -set -ux - -############################################################################################################ -# Set some global variables -############################################################################################################ -MY_HOSTNAME=`unit-get public-address` -MY_REPLSET=`config-get replicaset` -MY_INSTALL_ORDER=$(echo ${JUJU_UNIT_NAME} | awk -F/ '{ print $2 }') -MY_REPLICASET_MASTER=`config-get replicaset_master` - - -############################################################################################################ -# If we are joining an existing replicaset cluster, just join and leave. -############################################################################################################ -if [ "${MY_REPLICASET_MASTER}" != "auto" ]; then - mongo --host ${MY_REPLICASET_MASTER} --eval "rs.add(\""${MY_HOSTNAME}"\")" - exit $? -fi - -MASTER_HOSTNAME=${MY_HOSTNAME} -MASTER_REPLSET=${MY_REPLSET} -MASTER_INSTALL_ORDER=${MY_INSTALL_ORDER} - -echo "My hosntmae: ${MY_HOSTNAME}" -echo "My ReplSet: ${MY_REPLSET}" -echo "My install order: ${MY_INSTALL_ORDER}" - -############################################################################################################ -# Here we need to find out which is the first node ( we record the install order ). -# The one with the lowest order time is the master. -# Initialize the master node. -# Add the other nodes to the master's replica set. -############################################################################################################ -# Find the master ( lowest install order ) -for MEMBER in `relation-list` -do - HOSTNAME=`relation-get hostname ${MEMBER}` - REPLSET=`relation-get replset ${MEMBER}` - INSTALL_ORDER=`relation-get install-order ${MEMBER}` - if [ ${INSTALL_ORDER} -lt ${MASTER_INSTALL_ORDER} ];then - MASTER_HOSTNAME=${HOSTNAME} - MASTER_REPLSET=${REPLSET} - MASTER_INSTALL_ORDER=${INSTALL_ORDER} - fi -done - -echo "Master Hostname: ${MASTER_HOSTNAME}" -echo "Master ReplSet: ${MASTER_REPLSET}" -echo "Master install order: ${MASTER_INSTALL_ORDER}" - -# We should now have all the information about the master node. -# If the node has already been initialized, it will just inform you -# about it with no other consequence. -if [ "${MASTER_HOSTNAME}" == "${MY_HOSTNAME}" ]; then - mongo --eval "rs.initiate()" -else - mongo --host ${MASTER_HOSTNAME} --eval "rs.initiate()" -fi - -# Now we need to add the rest of nodes to the replica set -for MEMBER in `relation-list` -do - HOSTNAME=`relation-get hostname ${MEMBER}` - REPLSET=`relation-get replset ${MEMBER}` - INSTALL_ORDER=`relation-get install-order ${MEMBER}` - if [ "${MASTER_HOSTNAME}" != "${HOSTNAME}" ]; then - if [ "${HOSTNAME}" == "${MY_HOSTNAME}" ]; then - mongo --eval "rs.add(\""${HOSTNAME}"\")" - else - mongo --host ${MASTER_HOSTNAME} --eval "rs.add(\""${HOSTNAME}"\")" - fi - fi -done - -# Add myself to the replicaset if needed -if [ "${MASTER_HOSTNAME}" != "${MY_HOSTNAME}" ]; then - mongo --host ${MASTER_HOSTNAME} --eval "rs.add(\""${MY_HOSTNAME}"\")" -fi - - -echo $JUJU_REMOTE_UNIT modified its settings -echo Relation settings: -relation-get -echo Relation members: -relation-list diff --git a/hooks/replica-set-relation-joined b/hooks/replica-set-relation-joined deleted file mode 100755 index ecf1e71..0000000 --- a/hooks/replica-set-relation-joined +++ /dev/null @@ -1,28 +0,0 @@ -#!/bin/bash -# This must be renamed to the name of the relation. The goal here is to -# affect any change needed by relationships being formed -# This script should be idempotent. - -set -ux - -DEFAULT_REPLSET_NAME=`config-get replicaset` -INSTALL_ORDER=$(echo ${JUJU_UNIT_NAME} | awk -F/ '{ print $2 }') -HOSTNAME=`unit-get public-address` - -############################################################################################################ -# Reconfigure the upstart script to include the replica-set option. -# We'll need this so, when we add nodes, they can all talk to each other. -# Replica sets can only talk to each other if they all belong to the same -# set. In our case, we have defaulted to "myset". -############################################################################################################ -grep "${DEFAULT_REPLSET_NAME}" /etc/init/mongodb.conf -if [ $? -ne 0 ];then - sed -i -e "s/ -- / -- --replSet ${DEFAULT_REPLSET_NAME} /" /etc/init/mongodb.conf - service mongodb stop - rm -f /var/lib/mongodb/mongod.lock - service mongodb start -fi - -relation-set hostname=${HOSTNAME} replset=${DEFAULT_REPLSET_NAME} install-order=${INSTALL_ORDER} - -echo $JUJU_REMOTE_UNIT joined diff --git a/hooks/start b/hooks/start deleted file mode 100755 index cd918d6..0000000 --- a/hooks/start +++ /dev/null @@ -1,6 +0,0 @@ -#!/bin/bash -# Here put anything that is needed to start the service. -# Note that currently this is run directly after install -# i.e. 'service apache2 start' - -service mongodb status && service mongodb restart || service mongodb start diff --git a/hooks/stop b/hooks/stop deleted file mode 100755 index 5175b72..0000000 --- a/hooks/stop +++ /dev/null @@ -1,10 +0,0 @@ -#!/bin/bash -# This will be run when the service is being torn down, allowing you to disable -# it in various ways.. -# For example, if your web app uses a text file to signal to the load balancer -# that it is live... you could remove it and sleep for a bit to allow the load -# balancer to stop sending traffic. -# rm /srv/webroot/server-live.txt && sleep 30 - -service mongodb stop -rm -f /var/lib/mongodb/mongod.lock diff --git a/metadata.yaml b/metadata.yaml index eea0a59..c0966dc 100644 --- a/metadata.yaml +++ b/metadata.yaml @@ -1,20 +1,27 @@ -name: mongodb +name: mongodb-redone maintainer: Juan Negron <juan.negron@canonical.com> summary: An object/document-oriented database (metapackage) description: | MongoDB is a high-performance, open source, schema-free document- oriented data store that's easy to deploy, manage and use. It's network accessible, written in C++ and offers the following features : - * Collection oriented storage - easy storage of object- style data - * Full index support, including on inner objects * Query profiling - * Replication and fail-over support * Efficient storage of binary - data including large objects (e.g. videos) * Auto-sharding for - cloud-level scalability (Q209) High performance, scalability, and - reasonable depth of functionality are the goals for the project. This - is a metapackage that depends on all the mongodb parts. + * Collection oriented storage - easy storage of object-style data + * Full index support, including on inner objects + * Query profiling + * Replication and fail-over support + * Efficient storage of binary data including large + objects (e.g. videos) + * Auto-sharding for cloud-level scalability (Q209) + High performance, scalability, and reasonable depth of functionality + are the goals for the project. provides: database: interface: mongodb + configsvr: + interface: mongodb +requires: + mongos: + interface: mongodb peers: replica-set: interface: mongodb-replica-set |