Installation

On this page you will install the Stackable Operator for Apache Hive and all required dependencies. For the installation of the dependencies and operators you can use Helm or stackablectl.

The stackablectl command line tool is the recommended way to interact with operators and dependencies. Follow the installation steps for your platform if you choose to work with stackablectl.

Dependencies

First you need to install MinIO and PostgreSQL instances for the Hive metastore.

There are 2 ways to install the dependencies:

Using stackablectl
Using Helm

stackablectl

In order to install MinIO and PostgreSQL, you can use the Stacks feature of stackablectl. This is for demonstration purposes, a simpler method via Helm is shown below.

stackablectl \
--additional-stacks-file stackablectl-postgres-minio-stack.yaml \
--additional-releases-file release.yaml \
stack install minio-postgres

For this to work you need to create two yaml files.

The stackablectl-hive-postgres-s3-stack.yaml:

---
stacks:
  minio-postgres:
    stackableRelease: empty
    description: Stack containing MinIO and PostgreSQL
    labels:
      - minio
      - postgresql
    manifests:
      - helmChart:
          releaseName: minio
          name: minio
          repo:
            name: minio
            url: https://charts.min.io/
          version: 4.0.2
          options:
            rootUser: root
            rootPassword: rootroot
            mode: standalone
            users:
              - accessKey: hive
                secretKey: hivehive
                policy: readwrite
            buckets:
              - name: hive
                policy: public
            resources:
              requests:
                memory: 2Gi
            service:
              type: NodePort
              nodePort: null
            consoleService:
              type: NodePort
              nodePort: null
      - helmChart:
          releaseName: postgresql
          name: postgresql
          repo:
            name: bitnami
            url: https://charts.bitnami.com/bitnami/
          version: 10.16.2
          options:
            # Old version (10) of helm-charts has old way of setting credentials
            postgresqlUsername: hive
            postgresqlPassword: hive
            postgresqlDatabase: hive

The release.yaml:

---
releases:
  empty:
    releaseDate: 2022-08-12
    description: Empty release for testing / doc purposes
    products: {}

Consult the Quickstart to learn more about how to use stackablectl.

Helm

In order to install the MinIO and PostgreSQL dependencies via Helm, you have to deploy two charts.

Minio

helm install minio \
--namespace default \
--version 4.0.2 \
--set mode=standalone \
--set replicas=1 \
--set persistence.enabled=false \
--set buckets[0].name=hive,buckets[0].policy=none \
--set users[0].accessKey=hive,users[0].secretKey=hivehive,users[0].policy=readwrite \
--set resources.requests.memory=1Gi \
--set service.type=NodePort,service.nodePort=null \
--set consoleService.type=NodePort,consoleService.nodePort=null \
--repo https://charts.min.io/ minio

PostgresSQL

helm install postgresql \
--version=10 \
--namespace default \
--set postgresqlUsername=hive \
--set postgresqlPassword=hive \
--set postgresqlDatabase=hive \
--repo https://charts.bitnami.com/bitnami postgresql

After the dependencies are deployed, you can start to install the operators.

Stackable Operators

There are 2 ways to run Stackable Operators:

Using stackablectl
Using Helm

stackablectl

Run the following command to install all operators necessary for Apache Hive:

stackablectl operator install \
  commons=0.4.0 \
  secret=0.6.0 \
  hive=0.8.0

The tool will show

[INFO ] Installing commons operator in version 0.4.0
[INFO ] Installing secret operator in version 0.6.0
[INFO ] Installing hive operator in version 0.8.0

Helm

Run the following commands Helm to install the operators via Helm

Add the Stackable Helm repository:

helm repo add stackable-stable https://repo.stackable.tech/repository/helm-stable/

Then install the Stackable operators:

helm install --wait commons-operator stackable-stable/commons-operator --version 0.4.0
helm install --wait secret-operator stackable-stable/secret-operator --version 0.6.0
helm install --wait hive-operator stackable-stable/hive-operator --version 0.8.0

Helm will deploy the operators in a Kubernetes Deployment and apply the CRDs for the Apache Hive service (as well as the CRDs for the required operators). You are now ready to deploy the Apache Hive metastore in Kubernetes.

What’s next

Set up an Apache Hive cluster