Good morning, I am trying to install Kylin-3.1.1 on a remote linux server, I made sure it had all software requirments and I installed already the following programms: apache-hive-3.1.2-bin , kylin-3.1.1-bin-hadoop3 , kafka_2.12-2.5.0 , hadoop-3.3.0 , and hbase-2.3.4. But something is going wrong. I have checked the environment with $KYLIN_HOME/bin/check-env.sh and no error apperaed, the checking was completed succesfully. But whenever I try to start kylin by $KYLIN_HOME/bin/kylin.sh start it gives me the following error:
2021-02-15 07:35:22,542 INFO [ReadOnlyZKClient-localhost:2181@0x5bf22f18] zookeeper.ZooKeeper:109 : Client environment:java.library.path=/home/cuquerella/hadoop-3.3.0/lib/native 2021-02-15 07:35:22,542 INFO [ReadOnlyZKClient-localhost:2181@0x5bf22f18] zookeeper.ZooKeeper:109 : Client environment:java.io.tmpdir=/tmp 2021-02-15 07:35:22,542 INFO [ReadOnlyZKClient-localhost:2181@0x5bf22f18] zookeeper.ZooKeeper:109 : Client environment:java.compiler=<NA> 2021-02-15 07:35:22,542 INFO [ReadOnlyZKClient-localhost:2181@0x5bf22f18] zookeeper.ZooKeeper:109 : Client environment:os.name=Linux 2021-02-15 07:35:22,545 INFO [ReadOnlyZKClient-localhost:2181@0x5bf22f18] zookeeper.ZooKeeper:109 : Client environment:os.arch=amd64 2021-02-15 07:35:22,546 INFO [ReadOnlyZKClient-localhost:2181@0x5bf22f18] zookeeper.ZooKeeper:109 : Client environment:os.version=5.4.0-65-generic 2021-02-15 07:35:22,546 INFO [ReadOnlyZKClient-localhost:2181@0x5bf22f18] zookeeper.ZooKeeper:109 : Client environment:user.name=cuquerella 2021-02-15 07:35:22,546 INFO [ReadOnlyZKClient-localhost:2181@0x5bf22f18] zookeeper.ZooKeeper:109 : Client environment:user.home=/home/cuquerella 2021-02-15 07:35:22,546 INFO [ReadOnlyZKClient-localhost:2181@0x5bf22f18] zookeeper.ZooKeeper:109 : Client environment:user.dir=/home/cuquerella/kylin-3.1.1-bin-hadoop3 2021-02-15 07:35:22,546 INFO [ReadOnlyZKClient-localhost:2181@0x5bf22f18] zookeeper.ZooKeeper:109 : Client environment:os.memory.free=793MB 2021-02-15 07:35:22,546 INFO [ReadOnlyZKClient-localhost:2181@0x5bf22f18] zookeeper.ZooKeeper:109 : Client environment:os.memory.max=4062MB 2021-02-15 07:35:22,546 INFO [ReadOnlyZKClient-localhost:2181@0x5bf22f18] zookeeper.ZooKeeper:109 : Client environment:os.memory.total=990MB 2021-02-15 07:35:22,555 INFO [ReadOnlyZKClient-localhost:2181@0x5bf22f18] zookeeper.ZooKeeper:868 : Initiating client connection, connectString=localhost:2181 sessionTimeout=90000 watcher=org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$14/900792976@1dda2073 2021-02-15 07:35:22,567 INFO [ReadOnlyZKClient-localhost:2181@0x5bf22f18] common.X509Util:79 : Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation 2021-02-15 07:35:22,583 INFO [ReadOnlyZKClient-localhost:2181@0x5bf22f18] zookeeper.ClientCnxnSocket:237 : jute.maxbuffer value is 4194304 Bytes 2021-02-15 07:35:22,598 INFO [ReadOnlyZKClient-localhost:2181@0x5bf22f18] zookeeper.ClientCnxn:1653 : zookeeper.request.timeout value is 0. feature enabled= 2021-02-15 07:35:22,615 INFO [ReadOnlyZKClient-localhost:2181@0x5bf22f18-SendThread(localhost:2181)] zookeeper.ClientCnxn:1112 : Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) 2021-02-15 07:35:22,625 INFO [ReadOnlyZKClient-localhost:2181@0x5bf22f18-SendThread(localhost:2181)] zookeeper.ClientCnxn:959 : Socket connection established, initiating session, client: /127.0.0.1:46350, server: localhost/127.0.0.1:2181 2021-02-15 07:35:22,637 INFO [ReadOnlyZKClient-localhost:2181@0x5bf22f18-SendThread(localhost:2181)] zookeeper.ClientCnxn:1394 : Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x10000255ea6000b, negotiated timeout = 90000 Exception in thread "main" java.lang.IllegalArgumentException: Failed to find metadata store by url: kylin_metadata@hbase at org.apache.kylin.common.persistence.ResourceStore.createResourceStore(ResourceStore.java:101) at org.apache.kylin.common.persistence.ResourceStore.getStore(ResourceStore.java:113) at org.apache.kylin.rest.service.AclTableMigrationTool.checkIfNeedMigrate(AclTableMigrationTool.java:99) at org.apache.kylin.tool.AclTableMigrationCLI.main(AclTableMigrationCLI.java:43) Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.kylin.common.persistence.ResourceStore.createResourceStore(ResourceStore.java:94) ... 3 more Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=2, exceptions: 2021-02-15T07:35:23.302Z, RpcRetryingCaller{globalStartTime=1613374523078, pause=100, maxAttempts=2}, org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=2, exceptions: 2021-02-15T07:35:23.159Z, RpcRetryingCaller{globalStartTime=1613374523130, pause=100, maxAttempts=2}, java.io.IOException: No meta znode available 2021-02-15T07:35:23.263Z, RpcRetryingCaller{globalStartTime=1613374523130, pause=100, maxAttempts=2}, java.io.IOException: No meta znode available 2021-02-15T07:35:23.506Z, RpcRetryingCaller{globalStartTime=1613374523078, pause=100, maxAttempts=2}, org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=2, exceptions: 2021-02-15T07:35:23.404Z, RpcRetryingCaller{globalStartTime=1613374523402, pause=100, maxAttempts=2}, java.io.IOException: No meta znode available 2021-02-15T07:35:23.506Z, RpcRetryingCaller{globalStartTime=1613374523402, pause=100, maxAttempts=2}, java.io.IOException: No meta znode available at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:145) at org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3014) at org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3006) at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:464) at org.apache.kylin.storage.hbase.HBaseConnection.tableExists(HBaseConnection.java:291) at org.apache.kylin.storage.hbase.HBaseConnection.createHTableIfNeeded(HBaseConnection.java:316) at org.apache.kylin.storage.hbase.HBaseResourceStore.createHTableIfNeeded(HBaseResourceStore.java:119) at org.apache.kylin.storage.hbase.HBaseResourceStore.<init>(HBaseResourceStore.java:89) ... 8 more Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=2, exceptions: 2021-02-15T07:35:23.404Z, RpcRetryingCaller{globalStartTime=1613374523402, pause=100, maxAttempts=2}, java.io.IOException: No meta znode available 2021-02-15T07:35:23.506Z, RpcRetryingCaller{globalStartTime=1613374523402, pause=100, maxAttempts=2}, java.io.IOException: No meta znode available at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:145) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:383) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:357) at org.apache.hadoop.hbase.MetaTableAccessor.getTableState(MetaTableAccessor.java:1166) at org.apache.hadoop.hbase.MetaTableAccessor.tableExists(MetaTableAccessor.java:463) at org.apache.hadoop.hbase.client.HBaseAdmin$6.rpcCall(HBaseAdmin.java:467) at org.apache.hadoop.hbase.client.HBaseAdmin$6.rpcCall(HBaseAdmin.java:464) at org.apache.hadoop.hbase.client.RpcRetryingCallable.call(RpcRetryingCallable.java:58) at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:107) ... 15 more Caused by: java.io.IOException: No meta znode available at org.apache.hadoop.hbase.client.ZKConnectionRegistry.getMetaRegionLocation(ZKConnectionRegistry.java:139) at org.apache.hadoop.hbase.client.ZKConnectionRegistry.lambda$getMetaRegionLocations$5(ZKConnectionRegistry.java:205) at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) at org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$ZKTask$1.exec(ReadOnlyZKClient.java:174) at org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:342) at java.lang.Thread.run(Thread.java:748) 2021-02-15 07:35:23,578 INFO [close-hbase-conn] hbase.HBaseConnection:138 : Closing HBase connections... ERROR: Unknown error. Please check full log. If I run it with a sudo command the answer is the following:
/home/cuquerella/kylin-3.1.1-bin-hadoop3/bin/set-java-home.sh: line 32: [: : integer expression expected Retrieving hadoop conf dir... /home/cuquerella/kylin-3.1.1-bin-hadoop3/bin/find-hadoop-conf-dir.sh: line 33: hbase: command not found ...................................................[PASS] KYLIN_HOME is set to /home/cuquerella/kylin-3.1.1-bin-hadoop3 Checking HBase ...................................................[FAIL] Please make sure the user has the privilege to run hbase shell I tend to think it has to do with some problem when connecting to HBase cause whenever I ran a list it gives me the following error:
hbase(main):001:0> list TABLE ERROR: org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2921) at org.apache.hadoop.hbase.master.MasterRpcServices.isMasterRunning(MasterRpcServices.java:1182) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:393) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:133) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:338) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:318) When I search in log of HBase I get the following outcome:
$ less hbase-2.3.4/logs/hbase-cuquerella-master-tfg.log 2021-02-13 18:12:42,107 ERROR [master/tfg:16000:becomeActiveMaster] master.HMaster: Failed to become active master java.io.IOException: cannot get log writer at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(AsyncFSWALProvider.java:126) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:651) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:128) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:797) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:531) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.init(AbstractFSWAL.java:472) at org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:158) at org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:63) at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:288) at org.apache.hadoop.hbase.master.region.MasterRegion.createWAL(MasterRegion.java:193) at org.apache.hadoop.hbase.master.region.MasterRegion.open(MasterRegion.java:266) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:309) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:948) I would greatly appreciate if anybody could shed some light upon the problem. I am sorry for any mistakes I have done while asking this question cause I am a real beginner at this.
