Skip to content

Commit 42fe8d5

Browse files
author
Jonathan S. Katz
committed
Commence tutorial
This adds a tutorial section to the documentation, which is a relative rehash of other sections but aims to tell an improved story on how to use the Operator. This is still a work in progress and more effort will be put into it prior to the 4.5.0 GA, but it is good enough to get started.
1 parent 686ecab commit 42fe8d5

File tree

6 files changed

+355
-2
lines changed

6 files changed

+355
-2
lines changed

‎docs/content/installation/prerequisites.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,8 @@ The PostgreSQL Operator is tested in the following environments:
1515

1616
* Kubernetes v1.13+
1717
* Red Hat OpenShift v3.11+
18-
* Red Hat OpenShift v4.3+
18+
* Red Hat OpenShift v4.4+
19+
* Amazon EKS
1920
* VMWare Enterprise PKS 1.3+
2021
* IBM Cloud Pak Data
2122

‎docs/content/quickstart/_index.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -243,7 +243,7 @@ PGPASSWORD=datalake psql -h localhost -p 5432 -U testuser hippo
243243
You should then be greeted with the PostgreSQL prompt:
244244

245245
```
246-
psql (12.4)
246+
psql ({{< param postgresVersion >}})
247247
Type "help" for help.
248248
249249
hippo=>

‎docs/content/tutorial/_index.md‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
---
2+
title: "Tutorial"
3+
draft: false
4+
weight: 15
5+
---
6+
7+
The PostgreSQL Operator provides functionality that lets you run your own database-as-a-service: from deploying PostgreSQL clusters with [high availability]({{< relref "architecture/high-availability/_index.md" >}}), to a [full stack monitoring]({{< relref "architecture/high-availability/_index.md" >}}) solution, essential [disaster recovery and backup tools]({{< relref "architecture/disaster-recovery.md" >}}), the ability to secure your cluster with TLS, and much more!
8+
9+
What's more, you can manage your PostgreSQL clusters with the convenient [`pgo` client]({{< relref "pgo-client/_index.md" >}}) or by interfacing directly with the PostgreSQL Operator [custom resources]({{< relref "custom-resources/_index.md" >}}).
10+
11+
Given the robustness of the PostgreSQL Operator, we think it's helpful to break down the functionality in this step-by-step tutorial. The tutorial covers the essential functions the PostgreSQL Operator can perform and covers many common basic and advanced use cases.
12+
13+
So what are you waiting for? Let's [get started]({{< relref "tutorial/getting-started.md" >}})!

‎docs/content/tutorial/connect-cluster.md‎

Lines changed: 149 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,149 @@
1+
---
2+
title: "Connect to a Postgres Cluster"
3+
draft: false
4+
weight: 120
5+
---
6+
7+
Naturally, once the [PostgreSQL cluster is created]({{< relref "tutorial/create-cluster.md" >}}), you may want to connect to it. You can get the credentials of the users of the cluster using the [`pgo show user`]({{< relref "pgo-client/reference/pgo_show_user.md" >}}) command, i.e.:
8+
9+
```
10+
pgo show user hippo
11+
```
12+
13+
yields output similar to:
14+
15+
```
16+
CLUSTER USERNAME PASSWORD EXPIRES STATUS ERROR
17+
------- -------- -------------------------------- ------- ------ -----
18+
hippo testuser securerandomlygeneratedpassword never ok
19+
```
20+
21+
If you need to get the password of one of the system or privileged accounts, you will need to use the `--show-system-accounts` flag, i.e.:
22+
23+
```
24+
pgo show user hippo --show-system-accounts
25+
```
26+
27+
```
28+
CLUSTER USERNAME PASSWORD EXPIRES STATUS ERROR
29+
------- ----------- -------------------------------- ------- ------ -----
30+
hippo postgres B>xy}9+7wTVp)gkntf}X|H@N never ok
31+
hippo primaryuser ^zULckQy-\KPws:2UoC+szXl never ok
32+
hippo testuser securerandomlygeneratedpassword never ok
33+
```
34+
35+
Let's look at three different ways we can connect to the PostgreSQL cluster.
36+
37+
## Connecting via `psql`
38+
39+
Let's see how we can connect to `hippo` using [`psql`](https://www.postgresql.org/docs/current/app-psql.html), the command-line tool for accessing PostgreSQL. Ensure you have [installed the `psql` client](https://www.crunchydata.com/developers/download-postgres/binaries/postgresql12).
40+
41+
The PostgreSQL Operator creates a service with the same name as the cluster. See for yourself! Get a list of all of the Services available in the `pgo` namespace:
42+
43+
```
44+
kubectl -n pgo get svc
45+
46+
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
47+
hippo ClusterIP 10.96.218.63 <none> 2022/TCP,5432/TCP 59m
48+
hippo-backrest-shared-repo ClusterIP 10.96.75.175 <none> 2022/TCP 59m
49+
postgres-operator ClusterIP 10.96.121.246 <none> 8443/TCP,4171/TCP,4150/TCP 71m
50+
```
51+
52+
Let's connect the `hippo` cluster. First, in a different console window, set up a port forward to the `hippo` service:
53+
54+
```
55+
kubectl -n pgo port-forward svc/hippo 5432:5432
56+
```
57+
58+
You can connect to the database with the following command, substituting `datalake` for your actual password:
59+
60+
```
61+
PGPASSWORD=datalake psql -h localhost -p 5432 -U testuser hippo
62+
```
63+
64+
You should then be greeted with the PostgreSQL prompt:
65+
66+
```
67+
psql ({{< param postgresVersion >}})
68+
Type "help" for help.
69+
70+
hippo=>
71+
```
72+
73+
## Connecting via [pgAdmin 4]({{< relref "architecture/pgadmin4.md" >}})
74+
75+
[pgAdmin 4]({{< relref "architecture/pgadmin4.md" >}}) is a graphical tool that can be used to manage and query a PostgreSQL database from a web browser. The PostgreSQL Operator provides a convenient integration with pgAdmin 4 for managing how users can log into the database.
76+
77+
To add pgAdmin 4 to `hippo`, you can execute the following command:
78+
79+
```
80+
pgo create pgadmin -n pgo hippo
81+
```
82+
83+
It will take a few moments to create the pgAdmin 4 instance. The PostgreSQL Operator also creates a pgAdmin 4 service. See for yourself! Get a list of all of the Services available in the `pgo` namespace:
84+
85+
```
86+
kubectl -n pgo get svc
87+
88+
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
89+
hippo ClusterIP 10.96.218.63 <none> 2022/TCP,5432/TCP 59m
90+
hippo-backrest-shared-repo ClusterIP 10.96.75.175 <none> 2022/TCP 59m
91+
hippo-pgadmin ClusterIP 10.96.165.27 <none> 5050/TCP 5m1s
92+
postgres-operator ClusterIP 10.96.121.246 <none> 8443/TCP,4171/TCP,4150/TCP 71m
93+
```
94+
95+
Let's connect to our `hippo` cluster via pgAdmin 4! In a different terminal, set up a port forward to pgAdmin 4:
96+
97+
```
98+
kubectl -n pgo port-forward svc/hippo-pgadmin 5050:5050
99+
```
100+
101+
Navigate your browser to http://localhost:5050 and use your database username (`testuser`) and password (e.g. `datalake`) to log in. Though the prompt says “email address”, using your PostgreSQL username will work:
102+
103+
![pgAdmin 4 Login Page](/images/pgadmin4-login2.png)
104+
105+
(There are occasions where the initial credentials do not properly get set in pgAdmin 4. If you have trouble logging in, try running the command `pgo update user -n pgo hippo --username=testuser --password=datalake`).
106+
107+
Once logged into pgAdmin 4, you will be automatically connected to your database. Explore pgAdmin 4 and run some queries!
108+
109+
## Connecting from a Kubernetes Application
110+
111+
### Within a Kubernetes Cluster
112+
113+
Connecting a Kubernetes application that is within the same cluster that your PostgreSQL cluster is deployed in is as simple as understanding the default [Kubernetes DNS system](https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/#what-things-get-dns-names). A cluster created by the PostgreSQL Operator automatically creates a Service of the same name (e.g. `hippo`).
114+
115+
Following the example we've created, the hostname for our PostgreSQL cluster is `hippo.pgo` (or `hippo.pgo.svc.cluster.local`). To get your exact [DNS resolution rules](https://kubernetes.io/docs/tasks/administer-cluster/dns-debugging-resolution/), you may need to consult with your Kubernetes administrator.
116+
117+
Knowing this, we can construct a [Postgres URI](https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING) that contains all of the connection info:
118+
119+
`postgres://testuser:securerandomlygeneratedpassword@hippo.jkatz.svc.cluster.local:5432/hippo`
120+
121+
which breaks down as such:
122+
123+
- `postgres`: the scheme, i.e. a Postgres URI
124+
- `testuser`: the name of the PostgreSQL user
125+
- `securerandomlygeneratedpassword`: the password for `testuser`
126+
- `hippo.jkatz.svc.cluster.local`: the hostname
127+
- `5432`: the port
128+
- `hippo`: the database you want to connect to
129+
130+
If your application or connection driver cannot use the Postgres URI, the above should allow for you to break down the connection string into its appropriate components.
131+
132+
### Outside a Kubernetes Cluster
133+
134+
To connect to a database from an application that is outside a Kubernetes cluster, you will need to set one of the following:
135+
136+
- A Service type of [`LoadBalancer`](https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer) or [`NodePort`](https://kubernetes.io/docs/concepts/services-networking/service/#nodeport)
137+
- An [Ingress](https://kubernetes.io/docs/concepts/services-networking/ingress/). The PostgreSQL Operator does not provide any management for Ingress types.
138+
139+
To have the PostgreSQL Operator create a Service that is of type `LoadBalancer` or `NodePort`, you can use the `--service-type` flag as part of creating a PostgreSQL cluster, e.g.:
140+
141+
```
142+
pgo create cluster hippo --service-type=LoadBalancer
143+
```
144+
145+
You can also set the `ServiceType` attribute of the [PostgreSQL Operator configuration]({{< relref "configuration/pgo-yaml-configuration.md" >}}) to provide a default Service type for all PostgreSQL clusters that are created.
146+
147+
## Next Steps
148+
149+
We've created a cluster and we've connected to it! Now, let's learn what customizations we can make as part of the cluster creation process.

‎docs/content/tutorial/create-cluster.md‎

Lines changed: 115 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,115 @@
1+
---
2+
title: "Create a Postgres Cluster"
3+
draft: false
4+
weight: 110
5+
---
6+
7+
If you came here through the [quickstart]({{< relref "quickstart/_index.md" >}}), you may have already [created a cluster]({{< relref "quickstart/_index.md" >}}#create-a-postgresql-cluster), in which case, feel free to skip ahead, or read onward for a more in depth look into cluster creation!
8+
9+
## Create a PostgreSQL Cluster
10+
11+
Creating a cluster is simple with the [`pgo create cluster`]({{< relref "pgo-client/reference/pgo_create_cluster.md" >}}) command:
12+
13+
```
14+
pgo create cluster hippo
15+
```
16+
17+
with output similar to:
18+
19+
```
20+
created cluster: hippo
21+
workflow id: 25c870a0-5d27-42c2-be00-92f0ba8768e7
22+
database name: hippo
23+
users:
24+
username: testuser password: securerandomlygeneratedpassword
25+
```
26+
27+
This creates a new PostgreSQL cluster named `hippo` with a database in it named `hippo`. This operation may take a few moments to complete. Note the name of the database user (`testuser`) and password (`securerandomlygeneratedpassword`) for when we connect to the PostgreSQL cluster.
28+
29+
To make it easier to copy and paste statements used throughout this tutorial, you can set the password of `testuser` as part of creating the PostgreSQL cluster:
30+
31+
```
32+
pgo create cluster hippo --password=securerandomlygeneratedpassword
33+
```
34+
35+
You can check on the status of the cluster creation using the [`pgo test`]({{< relref "pgo-client/reference/pgo_test.md" >}}) command. The `pgo test` command checks to see if the Kubernetes Services and the Pods that comprise the PostgreSQL cluster are available to receive connections. This includes:
36+
37+
- Testing that the Kubernetes Endpoints are available and able to route requests to healthy Pods.
38+
- Testing that each PostgreSQL instance is available and ready to accept client connections by performing a connectivity check similar to the one performed by [`pg_isready`](https://www.postgresql.org/docs/current/app-pg-isready.html).
39+
40+
For example, when the `hippo` cluster is ready,
41+
42+
```
43+
pgo test hippo
44+
```
45+
46+
will yield output similar to:
47+
48+
```
49+
cluster : hippo
50+
Services
51+
primary (10.96.179.126:5432): UP
52+
Instances
53+
primary (hippo-57675d4f8f-wwx64): UP
54+
```
55+
56+
57+
### The Create Cluster Process
58+
59+
So what just happened? Let's break down what occurs during the create cluster process.
60+
61+
1. First, `pgo` client creates an entry in the PostgreSQL Operator [pgcluster custom resource definition]({{< relref "custom-resources/_index.md" >}}) with the attributes desired to create the cluster. In the case above, this fills in the name of the cluster (`hippo`) and leverages a lot of defaults from the [PostgreSQL Operator configuration]({{< relref "configuration/pgo-yaml-configuration.md" >}}). We'll discuss more about the PostgreSQL Operator configuration later in the tutorial.
62+
63+
2. Once the custom resource is added, the PostgreSQL Operator begins provisioning the PostgreSQL instace and a pgBackRest repository which is used to store backups. The following actions occur as part of this process:
64+
65+
- Creating [persistent volume claims](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) (PVCs) for the PostgreSQL instance and the pgBackRest repository.
66+
- Creating [services](https://kubernetes.io/docs/concepts/services-networking/service/) that provide a stable network interface for connecting to the PostgreSQL instance and pgBackRest repository.
67+
- Creating [deployments](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/) that house each PostgreSQL instance and pgBackRest repository. Each of these is responsible for one Pod.
68+
- The PostgreSQL Pod, when it is started, provisions a PostgreSQL database and performs other bootstrapping functions, such as creating `testuer`.
69+
- The pgBackRest Pod, when it is started, initializes a pgBackRest repository. Note that the pgBackRest repository is not yet ready to start taking backups, but will be after the next step!
70+
71+
3. When the PostgreSQL Operator detects that the PostgreSQL and pgBackRest deployments are up and running, it creates a Kubenretes Job to create a pgBackRest stanza. This is necessary as part of intializing the pgBackRest repository to accept backups from our PostgreSQL cluster.
72+
73+
4. When the PostgreSQL Operator detects that the stanza creation is completed, it will take an initial backup of the cluster.
74+
75+
In order for a PostgreSQL cluster to be considered successfully created, all of these steps need to succeed. You can connect to the PostgreSQL cluster after step two completes, but note for the cluster to be considered "healthy", you need for pgBackRest to finish initializig.
76+
77+
You may ask yourself, "wait, why do I need for the pgBackRest repository to be initialized for a cluster to be successfully created?" That is a good question! The reason is that pgBackRest plays a fundamental role in both the [disaster recovery]({{< relref "architecture/disaster-recovery.md" >}}) AND [high availability]({{< relref "architecture/high-availability/_index.md" >}}) system with the PostgreSQL Operator, particularly around self-healing.
78+
79+
### What Is Created?
80+
81+
There are several Kubernetes objects that are created as part of the `pgo create cluster` command, including:
82+
83+
- A Deployment representing the primary PostgreSQL instance
84+
- A PVC that persists the data of this instance
85+
- A Service that can connect to this instance
86+
- A Deployment representing the pgBackRest repository
87+
- A PVC that persists the data of this repository
88+
- A Service that can connect to this repository
89+
- [Secrets](https://kubernetes.io/docs/concepts/configuration/secret/) representing the following three user accounts:
90+
- `postgres`: the database superuser for the PostgreSQL cluster. This is in a secret called `hippo-postgres-secret`.
91+
- `primaryuser`: the replication user. This is used for copying data between PostgreSQL instance. You should not need to login as this user. This is in a secret called `hippo-primaryuser-secret`.
92+
- `testuser`: the regular user account. This user has access to log into the `hippo` database that is created. This is the account you want to give out to your user / application. In a later section, we will see how we can change the default user that is created. This is in a secret called `hippo-testuser-secret`, where `testuser` can be substituted for the name of the user account.
93+
- [ConfigMaps](https://kubernetes.io/docs/concepts/configuration/configmap/), including:
94+
- `hippo-pgha-config`, which allows you to [customize the configuration of your PostgreSQL cluster]({{< relref "advanced/custom-configuration.md">}}). We will cover more about this topic in later sections.
95+
- `hippo-config` and `hippo-leader`, which are used by the high availability system. You should not modify these ConfigMaps.
96+
97+
Each deployment contains a single Pod. **Do not scale the deployments!**: further into the tutorial, we will cover some commands that let you scale your PostgreSQL cluster.
98+
99+
Some Job artifacts may be left around after the cluster creation process completes, including the stanza creation job (`hippo-stanza-create`) and initial backup job (`backrest-backup-hippo`). If the jobs completed successfully, you can safely delete these objects.
100+
101+
## Troubleshooting
102+
103+
### PostgreSQL / pgBackRest Pods Stuck in `Pending` Phase
104+
105+
The most common occurrence of this is due to PVCs not being bound. Ensure that you have configure your [storage options]({{< relref "installation/configuration.md" >}}#storage-settings) correctly for your Kubernetes environment, if for some reason you cannot use your default storage class or it is unavailable.
106+
107+
Also ensure that you have enough persistent volumes available: your Kubernetes administrator may need to provision more.
108+
109+
### `stanza-create` Job Never Finishes
110+
111+
The most common occurrence of this is due to the Kubernetes network blocking SSH connections between Pods. Ensure that your Kubernetes networking layer allows for SSH connections over port 2022 in the Namespace that you are deploying your PostgreSQL clusters into.
112+
113+
## Next Steps
114+
115+
Once your cluster is created, the next step is to [connect to your PostgreSQL cluster]({{< relref "tutorial/connect-cluster.md" >}}).

0 commit comments

Comments
 (0)