Cloudera Manager HipChat Chatbot

Introduction

This chatbot forwards information provided by a Cloudera Manager Server on the managed clusters. The information can be requested via messages in a HipChat chatroom. There is a list of valid messages the chatbot can process. It is necessary for the hipchat.com service to be able to access the endpoint the chatbot is running on. As well as the chatbot needs to be able to access the Cloudera Manager Server.

Setup

How to Build

• make sure java (at least 1.7) is installed on your machine
• make sure gradle is installed on your machine
• clone the project
• optional: you are free to now already edit the properties file /src/main/resources/application.properties to match your setup. Be aware, that you cannot edit this file after building the project. Though as this is a Spring Boot application it is possible to add an application.properties file to a /config subdirectory after building as well (section 'How to Deploy'). This new file will hide the one in the .jar. Have a look at http://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-external-config.html § 24.3 Application property files for more information.
• open bash
• go to containing folder
• run ./gradlew assemble

Creating a Cloudera Manager User

To access the information the manager provides the chatbot can log in as any user already existing. However it is recommended to create an extra account only used by the application. To create a new user

• open Cloudera Manager UI
• log in with administrative rights
• go to tab Administration
• select Add User
• choose a user name and password
• select role Read-Only and press ok.

Setting up a HipChat Chat Room

In addition to many already build in integrations Hipchat provides the opportunity to build your own integration. That is creating a room that delivers messages starting with a slash command of your choice directly to the chatbot application. Have a look at the following link for more information on how to build your own integration.

Link: https://blog.hipchat.com/2015/02/11/build-your-own-integration-with-hipchat/

How to Deploy

• copy build/libs/cloudera-manager-hipchat-chatbot-[version].jar to wherever you want the program be executed
  • make sure java is installed on this machine as well
• if you didn't edit the initial properties file before the build or if you want to use different properties
  • create a subdirectory /config next to your .jar file
  • create a file named application.properties within /config
  • copy the following skeleton into the file an insert the values you configured in the sections 'Creating a Cloudera Manager User' and 'Setting up a HipChat Chat Room'

#application.properties
cm.password=<cm login password>
cm.user=<cm user name>
cm.hostname=<Cloudera Manager hostname>
cm.port=<Cloudera Manager server port, normally 7180>
server.port=<port to listen on>
webhook.room.id=<room id>
webhook.notificationUrl =<your notification url>

• start application by running path/to/java-version/bin/java -jar cloudera-manager-hipchat-chatbot-[version].jar from your bash

You can also use supervisord to start it as a deamon)

(instructions taken from http://stackoverflow.com/questions/21503883/spring-boot-application-as-a-service as the 3rd party libs forces us to stay on spring boot 1.2.5)

#/etc/supervisor/conf.d/cloudera-manager-hipchat-chatbot.conf
[program:cmchatbot]
command=/usr/bin/java -jar /path/to/application.jar
user=usertorun
autostart=true
autorestart=true
startsecs=10
startretries=3
stdout_logfile=/var/log/cmchatbot-stdout.log
stderr_logfile=/var/log/cmchatbot-stderr.log

and then use

# sudo supervisorctl
cmchatbot             RUNNING   pid 123123, uptime 1 day, 15:00:00
supervisor> stop cmchatbot
supervisor> start cmchatbot

to start/stop your instance.

Communicate with the Chatbot

Creating the chatroom you chose a slash command for the communication. For example /cm. This implies that you need to precede each command with /cm. If you are not sure which commands are available start with /cm help. The chatbot will respond listing all available commands.

Available Commands

• current load: current cpu load across hosts
• current storage: current DFS capacity
• current io network: current byte rates across network interfaces
• current io hdfs: current byte rates across datanodes
• current io disk: current byte rates across disks
• current pool cores:
• current pool memory:
• hosts: list of all hosts
• hosts decommisioned: list of all hosts with status decommissioned
• hosts decommissioning: list of all hosts with status decommissioning
• hosts commisioned: list of all hosts with status commissioned
• hosts bad: list of all hosts with status bad health
• hosts good: list of all hosts with status good health
• hosts concerning: list of all hosts with status concerning health
• services: list of all services
• services bad: list of all services with status bad health
• services good: list of all services with status good health
• services concerning: list of all services with status concerning health
• status: information about the general health of the cluster
• status service <service>: health status of the given service (e.g. hdfs, yarn, hue...)