Skip to content

Commit d40b632

Browse files
adwk67razvan
andcommitted
hdfs bundle for 23.4 (#350)
# Description Bundle files for the 23.4 release of the hdfs operator. Tests when deployed from bundle:on 4.11: ``` --- PASS: kuttl (5784.18s) --- PASS: kuttl/harness (0.00s) --- PASS: kuttl/harness/cluster-operation_hadoop-latest-3.3.4-stackable23.4.0_zookeeper-latest-3.8.0-stackable23.4.0 (284.03s) --- PASS: kuttl/harness/smoke_hadoop-3.2.2-stackable23.4.0_zookeeper-3.8.0-stackable23.4.0_number-of-datanodes-1_datanode-pvcs-2hdd-1ssd_listener-class-cluster-internal (169.42s) --- PASS: kuttl/harness/logging_hadoop-3.3.4-stackable23.4.0_zookeeper-latest-3.8.0-stackable23.4.0 (294.27s) --- PASS: kuttl/harness/logging_hadoop-3.3.3-stackable23.4.0_zookeeper-latest-3.8.0-stackable23.4.0 (303.57s) --- PASS: kuttl/harness/logging_hadoop-3.2.2-stackable23.4.0_zookeeper-latest-3.8.0-stackable23.4.0 (254.94s) --- PASS: kuttl/harness/orphaned-resources_hadoop-latest-3.3.4-stackable23.4.0_zookeeper-latest-3.8.0-stackable23.4.0 (155.49s) --- PASS: kuttl/harness/smoke_hadoop-3.3.4-stackable23.4.0_zookeeper-3.8.0-stackable23.4.0_number-of-datanodes-2_datanode-pvcs-default_listener-class-external-unstable (203.30s) --- PASS: kuttl/harness/smoke_hadoop-3.3.4-stackable23.4.0_zookeeper-3.8.0-stackable23.4.0_number-of-datanodes-2_datanode-pvcs-default_listener-class-cluster-internal (223.24s) --- PASS: kuttl/harness/smoke_hadoop-3.3.3-stackable23.4.0_zookeeper-3.8.0-stackable23.4.0_number-of-datanodes-1_datanode-pvcs-default_listener-class-cluster-internal (175.53s) --- PASS: kuttl/harness/smoke_hadoop-3.3.3-stackable23.4.0_zookeeper-3.8.0-stackable23.4.0_number-of-datanodes-1_datanode-pvcs-2hdd-1ssd_listener-class-external-unstable (155.69s) --- PASS: kuttl/harness/smoke_hadoop-3.3.3-stackable23.4.0_zookeeper-3.8.0-stackable23.4.0_number-of-datanodes-1_datanode-pvcs-2hdd-1ssd_listener-class-cluster-internal (175.98s) --- PASS: kuttl/harness/smoke_hadoop-3.2.2-stackable23.4.0_zookeeper-3.8.0-stackable23.4.0_number-of-datanodes-2_datanode-pvcs-default_listener-class-external-unstable (169.14s) --- PASS: kuttl/harness/smoke_hadoop-3.2.2-stackable23.4.0_zookeeper-3.8.0-stackable23.4.0_number-of-datanodes-2_datanode-pvcs-default_listener-class-cluster-internal (200.51s) --- PASS: kuttl/harness/smoke_hadoop-3.2.2-stackable23.4.0_zookeeper-3.8.0-stackable23.4.0_number-of-datanodes-2_datanode-pvcs-2hdd-1ssd_listener-class-external-unstable (207.47s) --- PASS: kuttl/harness/smoke_hadoop-3.2.2-stackable23.4.0_zookeeper-3.8.0-stackable23.4.0_number-of-datanodes-2_datanode-pvcs-2hdd-1ssd_listener-class-cluster-internal (197.76s) --- PASS: kuttl/harness/smoke_hadoop-3.2.2-stackable23.4.0_zookeeper-3.8.0-stackable23.4.0_number-of-datanodes-1_datanode-pvcs-default_listener-class-external-unstable (161.26s) --- PASS: kuttl/harness/smoke_hadoop-3.2.2-stackable23.4.0_zookeeper-3.8.0-stackable23.4.0_number-of-datanodes-1_datanode-pvcs-default_listener-class-cluster-internal (181.51s) --- PASS: kuttl/harness/smoke_hadoop-3.2.2-stackable23.4.0_zookeeper-3.8.0-stackable23.4.0_number-of-datanodes-1_datanode-pvcs-2hdd-1ssd_listener-class-external-unstable (165.62s) --- PASS: kuttl/harness/smoke_hadoop-3.3.4-stackable23.4.0_zookeeper-3.8.0-stackable23.4.0_number-of-datanodes-1_datanode-pvcs-2hdd-1ssd_listener-class-external-unstable (160.48s) --- PASS: kuttl/harness/smoke_hadoop-3.3.4-stackable23.4.0_zookeeper-3.8.0-stackable23.4.0_number-of-datanodes-2_datanode-pvcs-2hdd-1ssd_listener-class-external-unstable (207.90s) --- PASS: kuttl/harness/smoke_hadoop-3.3.4-stackable23.4.0_zookeeper-3.8.0-stackable23.4.0_number-of-datanodes-2_datanode-pvcs-2hdd-1ssd_listener-class-cluster-internal (219.98s) --- PASS: kuttl/harness/smoke_hadoop-3.3.4-stackable23.4.0_zookeeper-3.8.0-stackable23.4.0_number-of-datanodes-1_datanode-pvcs-default_listener-class-external-unstable (155.46s) --- PASS: kuttl/harness/smoke_hadoop-3.3.3-stackable23.4.0_zookeeper-3.8.0-stackable23.4.0_number-of-datanodes-1_datanode-pvcs-default_listener-class-external-unstable (156.55s) --- PASS: kuttl/harness/smoke_hadoop-3.3.4-stackable23.4.0_zookeeper-3.8.0-stackable23.4.0_number-of-datanodes-1_datanode-pvcs-2hdd-1ssd_listener-class-cluster-internal (174.96s) --- PASS: kuttl/harness/smoke_hadoop-3.3.4-stackable23.4.0_zookeeper-3.8.0-stackable23.4.0_number-of-datanodes-1_datanode-pvcs-default_listener-class-cluster-internal (149.32s) --- PASS: kuttl/harness/smoke_hadoop-3.3.3-stackable23.4.0_zookeeper-3.8.0-stackable23.4.0_number-of-datanodes-2_datanode-pvcs-2hdd-1ssd_listener-class-external-unstable (210.23s) --- PASS: kuttl/harness/smoke_hadoop-3.3.3-stackable23.4.0_zookeeper-3.8.0-stackable23.4.0_number-of-datanodes-2_datanode-pvcs-default_listener-class-cluster-internal (185.76s) --- PASS: kuttl/harness/smoke_hadoop-3.3.3-stackable23.4.0_zookeeper-3.8.0-stackable23.4.0_number-of-datanodes-2_datanode-pvcs-2hdd-1ssd_listener-class-cluster-internal (244.39s) --- PASS: kuttl/harness/smoke_hadoop-3.3.3-stackable23.4.0_zookeeper-3.8.0-stackable23.4.0_number-of-datanodes-2_datanode-pvcs-default_listener-class-external-unstable (230.97s) ``` Co-authored-by: Razvan-Daniel Mihai <84674+razvan@users.noreply.github.com>
1 parent f8c4b04 commit d40b632

File tree

8 files changed

+4740
-1
lines changed

8 files changed

+4740
-1
lines changed

.gitignore

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,4 +14,7 @@ crate-hashes.json
1414
result
1515
image.tar
1616

17-
tilt_options.json
17+
tilt_options.json
18+
19+
**/bundle/
20+
**/bundle.Dockerfile

CHANGELOG.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,13 +4,18 @@ All notable changes to this project will be documented in this file.
44

55
## [Unreleased]
66

7+
### Added
8+
9+
- Generate OLM bundle for Release 23.4.0 ([#350]).
10+
711
### Changed
812

913
- Operator-rs: `0.40.2` -> `0.41.0` ([#349]).
1014
- Use 0.0.0-dev product images for testing ([#351])
1115
- Use testing-tools 0.2.0 ([#351])
1216

1317
[#349]: https://github.com/stackabletech/hdfs-operator/pull/349
18+
[#350]: https://github.com/stackabletech/hdfs-operator/pull/350
1419
[#351]: https://github.com/stackabletech/hdfs-operator/pull/351
1520

1621
## [23.4.0] - 2023-04-17
Lines changed: 177 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,177 @@
1+
---
2+
apiVersion: v1
3+
data:
4+
properties.yaml: |
5+
version: 0.1.0
6+
spec:
7+
units:
8+
- unit: &unitUri
9+
name: "uri"
10+
regex: "^(([^:/?#]+):)?(//([^/?#]*))?([^?#]*)(\\?([^#]*))?(#(.*))?"
11+
examples:
12+
- "ldap://[2001:db8::7]/c=GB?objectClass?one"
13+
comment: "Specified in https://tools.ietf.org/html/rfc3986#appendix-B"
14+
15+
properties:
16+
- property:
17+
propertyNames:
18+
- name: "dfs.namenode.name.dir"
19+
kind:
20+
type: "file"
21+
file: "hdfs-site.xml"
22+
datatype:
23+
type: "string"
24+
unit: *unitUri
25+
roles:
26+
- name: "namenode"
27+
required: false
28+
- name: "datanode"
29+
required: false
30+
asOfVersion: "0.0.0"
31+
description: "Determines where on the local filesystem the DFS name node should store the name table(fsimage). If this is a comma-delimited list of directories then the name table is replicated in all of the directories, for redundancy."
32+
33+
- property:
34+
propertyNames:
35+
- name: "dfs.datanode.data.dir"
36+
kind:
37+
type: "file"
38+
file: "hdfs-site.xml"
39+
datatype:
40+
type: "string"
41+
unit: *unitUri
42+
roles:
43+
- name: "namenode"
44+
required: false
45+
- name: "datanode"
46+
required: false
47+
asOfVersion: "0.0.0"
48+
description: "Determines where on the local filesystem an DFS data node should store its blocks. If this is a comma-delimited list of directories, then data will be stored in all named directories, typically on different devices. Directories that do not exist are ignored."
49+
50+
- property:
51+
propertyNames:
52+
- name: "dfs.replication"
53+
kind:
54+
type: "file"
55+
file: "hdfs-site.xml"
56+
datatype:
57+
type: "integer"
58+
min: "1"
59+
max: "512"
60+
defaultValues:
61+
- fromVersion: "0.0.0"
62+
value: "3"
63+
roles:
64+
- name: "namenode"
65+
required: true
66+
- name: "datanode"
67+
required: true
68+
asOfVersion: "0.0.0"
69+
description: "Default block replication. The actual number of replications can be specified when the file is created. The default is used if replication is not specified in create time."
70+
71+
- property:
72+
propertyNames:
73+
- name: "fs.defaultFS"
74+
kind:
75+
type: "file"
76+
file: "core-site.xml"
77+
datatype:
78+
type: "string"
79+
unit: *unitUri
80+
roles:
81+
- name: "namenode"
82+
required: false
83+
- name: "datanode"
84+
required: false
85+
asOfVersion: "0.0.0"
86+
description: "The name of the default file system. A URI whose scheme and authority determine the FileSystem implementation. The uri's scheme determines the config property (fs.SCHEME.impl) naming the FileSystem implementation class. The uri's authority is used to determine the host, port, etc. for a filesystem."
87+
88+
- property:
89+
propertyNames:
90+
- name: "dfs.namenode.http-address"
91+
kind:
92+
type: "file"
93+
file: "hdfs-site.xml"
94+
datatype:
95+
type: "string"
96+
roles:
97+
- name: "namenode"
98+
required: false
99+
asOfVersion: "0.0.0"
100+
description: "The address and the base port where the dfs namenode web ui will listen on."
101+
102+
- property:
103+
propertyNames:
104+
- name: "dfs.datanode.ipc.address"
105+
kind:
106+
type: "file"
107+
file: "hdfs-site.xml"
108+
datatype:
109+
type: "string"
110+
roles:
111+
- name: "datanode"
112+
required: false
113+
asOfVersion: "0.0.0"
114+
description: "The datanode ipc server address and port."
115+
116+
- property:
117+
propertyNames:
118+
- name: "dfs.datanode.address"
119+
kind:
120+
type: "file"
121+
file: "hdfs-site.xml"
122+
datatype:
123+
type: "string"
124+
roles:
125+
- name: "datanode"
126+
required: false
127+
asOfVersion: "0.0.0"
128+
description: "The datanode server address and port for data transfer."
129+
130+
- property:
131+
propertyNames:
132+
- name: "dfs.datanode.http.address"
133+
kind:
134+
type: "file"
135+
file: "hdfs-site.xml"
136+
datatype:
137+
type: "string"
138+
roles:
139+
- name: "datanode"
140+
required: false
141+
asOfVersion: "0.0.0"
142+
description: "The datanode http server address and port."
143+
144+
- property:
145+
propertyNames:
146+
- name: "dfs.journalnode.http-address"
147+
kind:
148+
type: "file"
149+
file: "hdfs-site.xml"
150+
datatype:
151+
type: "string"
152+
roles:
153+
- name: "journalnode"
154+
required: false
155+
asOfVersion: "0.0.0"
156+
description: "The address and port the JournalNode HTTP server listens on. If the port is 0 then the server will start on a free port."
157+
158+
- property:
159+
propertyNames:
160+
- name: "dfs.journalnode.https-address"
161+
kind:
162+
type: "file"
163+
file: "hdfs-site.xml"
164+
datatype:
165+
type: "string"
166+
roles:
167+
- name: "journalnode"
168+
required: false
169+
asOfVersion: "0.0.0"
170+
description: "The address and port the JournalNode HTTPS server listens on. If the port is 0 then the server will start on a free port."
171+
kind: ConfigMap
172+
metadata:
173+
name: hdfs-operator-configmap
174+
labels:
175+
app.kubernetes.io/name: hdfs-operator
176+
app.kubernetes.io/instance: hdfs-operator
177+
app.kubernetes.io/version: "23.4.0"

0 commit comments

Comments
 (0)