Derby example

Please note that the version you need to specify is not only the version of Apache Hive which you want to roll out, but has to be amended with a Stackable version as shown. This Stackable version is the version of the underlying container image which is used to execute the processes. For a list of available versions please check our image registry. It should generally be safe to simply use the latest image version that is available.

Create a single node Apache Hive Metastore cluster using Derby:

--- apiVersion: hive.stackable.tech/v1alpha1 kind: HiveCluster metadata: name: simple-hive-derby spec: image: productVersion: 3.1.3 clusterConfig: database: connString: jdbc:derby:;databaseName=/tmp/metastore_db;create=true user: APP password: mine dbType: derby metastore: roleGroups: default: replicas: 1

You should not use the Derby database in production. Derby stores data locally which does not work in high availability setups (multiple replicas) and all data is lost after Pod restarts.

To create a single node Apache Hive Metastore (v3.1.3) cluster with derby and S3 access, deploy a minio (or use any available S3 bucket):

helm install minio \ minio \ --repo https://charts.bitnami.com/bitnami \ --set auth.rootUser=minio-access-key \ --set auth.rootPassword=minio-secret-key

In order to upload data to minio we need a port-forward to access the web ui.

kubectl port-forward service/minio 9001

Then, connect to localhost:9001 and login with the user minio-access-key and password minio-secret-key. Create a bucket and upload data.

Deploy the hive cluster:

--- apiVersion: hive.stackable.tech/v1alpha1 kind: HiveCluster metadata: name: simple-hive-derby spec: image: productVersion: 3.1.3 clusterConfig: database: connString: jdbc:derby:;databaseName=/stackable/metastore_db;create=true user: APP password: mine dbType: derby s3: inline: host: minio port: 9000 accessStyle: Path credentials: secretClass: simple-hive-s3-secret-class metastore: roleGroups: default: replicas: 1 --- apiVersion: secrets.stackable.tech/v1alpha1 kind: SecretClass metadata: name: simple-hive-s3-secret-class spec: backend: k8sSearch: searchNamespace: pod: {} --- apiVersion: v1 kind: Secret metadata: name: simple-hive-s3-secret labels: secrets.stackable.tech/class: simple-hive-s3-secret-class stringData: accessKey: minio-access-key secretKey: minio-secret-key

To create a single node Apache Hive Metastore using PostgreSQL, deploy a PostgreSQL instance via helm.

PostgreSQL introduced a new way to encrypt its passwords in version 10. This is called scram-sha-256 and has been the default as of PostgreSQL 14. Unfortunately, Hive up until the latest 3.3.x version ships with JDBC drivers that do not support this method. You might see an error message like this: The authentication type 10 is not supported. If this is the case please either use an older PostgreSQL version or change its password_encryption setting to md5.

This installs PostgreSQL in version 10 to work around the issue mentioned above:

helm install hive bitnami/postgresql --version=12.1.5 \ --set postgresqlUsername=hive \ --set postgresqlPassword=hive \ --set postgresqlDatabase=hive

Create Hive Metastore using a PostgreSQL database

apiVersion: hive.stackable.tech/v1alpha1 kind: HiveCluster metadata: name: simple-hive-postgres spec: image: productVersion: 3.1.3 clusterConfig: database: connString: jdbc:postgresql://hive-postgresql.default.svc.cluster.local:5432/hive user: hive password: hive dbType: postgres metastore: roleGroups: default: replicas: 1