• Last Published: 2020-10-22|
• Version: 3.8

Use Cli to operate Hadoop Unit

Hadoop-unit can be used with common tools such as:

• zookeeper command
• hdfs command
• kafka-console command
• ksql-console command
• hbase shell
• hive shell
• hive beeline
• cassandra shell
• alluxio shell
• pulsar shell

Zookeeper command

• Download and unzip zookeeper
• From directory ZK_HOME/bin, execute command:

./zkCli.sh -server localhost:22010

HDFS command

• From directory HADOOP_HOME/bin, execute command:

hdfs dfs -ls hdfs://localhost:20112/

For windows user, you could have some issue like -classpath is not known. The cause of these errors are because your JAVA_HOME has space into. If your JAVA_HOME is linked to C:\Program File\Java\... then declared it as C:\Progra~1\Java\...

Kafka-console command

• Download and unzip kafka
• From directory KAFKA_HOME/bin (or KAFKA_HOME/bin/windows for windows), execute command:

kafka-console-consumer --zookeeper localhost:22010 --topic topic

Ksql command

• Download and unzip confluent
• From directory CONFLUENT_HOME/bin, execute command:

./ksql http://localhost:8083

HBase Shell

• Download and unzip HBase
• set variable HBASE_HOME
• edit file HBASE_HOME/conf/hbase-site.xml:

<configuration>
	<property>
		<name>hbase.zookeeper.quorum</name>
		<value>127.0.0.1:22010</value>
	</property>
	<property>
		<name>zookeeper.znode.parent</name>
		<value>/hbase-unsecure</value>
	</property>
</configuration>

• From directory HBASE_HOME/bin, execute command:

hbase shell

Hive Shell

• Download and unzip Hive
• Edit file HIVE_HOME/conf/hive-site.xml:

<configuration>
	<property>
		<name>hive.metastore.uris</name>
		<value>thrift://127.0.0.1:20102</value>
	</property>
</configuration>

• From directory HIVE_HOME/bin, execute command:

hive

Hive beeline

For linux/MacOS user only

• Download and unzip Hive
• From directory HIVE_HOME/bin, execute command:

beeline

• When prompted, fill:

!connect jdbc:hive2://localhost:20103 user password

Cassandra Shell

• Download and unzip cassandra
• From directory CASSANDRA_HOME/bin, execute command:

./cqlsh localhost 13433

Alluxio Shell

• Download and unzip alluxio
• edit file ALLUXIO_HOME/conf/alluxio-env.sh:

ALLUXIO_MASTER_HOSTNAME=localhost

• edit file ALLUXIO_HOME/conf/alluxio-site.properties:

alluxio.master.port=14001

• From directory ALLUXIO_HOME/bin, execute command:

./alluxio fs ls <path>

Pulsar Shell

• Download and unzip pulsar
• From directory PULSAR_HOME/bin, execute command:

./pulsar-client --url pulsar://localhost:22022/ consume -s subscriptionName topic
./pulsar-client --url pulsar://localhost:22022/ produce -m message topic