Let us get into the details related to cluster topology in a typical production cluster and also details about the servers we are going to use to learn the setup process. A typical production cluster will contain 100s of nodes, out of which Few servers which will contain databases to store data for different services …
Let us validate HBase installation by running few commands using HBase Shell. On the gateway node of the hbase cluster run hbase shell help command provides list of commands in different categories Namespace – group of tables (similar to schema or database) create – create_namespace ‘training’ list – list_namespace ‘training’ list tables – list_namespace_tables ‘training’ Table …
HBase is a distributed data storage that comes as part of the Hadoop ecosystem with most of the distributions such as Cloudera, Hortonworks etc. HBase is master-slave architecture with HBase Master Process and Region Servers as slaves. Key Components Here are key components of HBase. HBase Master HBase Region Servers WAL – Write Ahead log …
Let us go ahead and validate Kafka by creating a topic and then publishing messages to a topic as well as consuming messages from a topic. Create Kafka Topic kafka-topics –create –zookeeper bigdataserver-2:2181,bigdataserver-3:2181,bigdataserver-4:2181 –replication-factor 2 –partitions 3 –topic testing Produce the message to the topic kafka-console-producer –broker-list bigdataserver-5:9092,bigdataserver-6:9092 –topic testing Consume the message from the topic kafka-console-consumer …
Let us see how we can add Kafka to the cluster using Cloudera Manager. We will also see how to review components, properties as well as log files with respect to Kafka. Go to Hosts -> Parcels If you want to add any other version of Kafka Click on Configuration Add a “Remote Parcel Repository …
Let us look into the overview of Apache Kafka before getting into details about configuring and validating it. Apache Kafka is an open-source stream-processing software platform developed by the Apache Software Foundation, written in Scala and Java. It was initially developed as an internal product at LinkedIn and was open-sourced and adopted by apache foundation. …
