The Power of the IDH Rest API … Integrating Big Data in to your Management Platform of Choice

Joe Natoli is the Director of ISV Applications as part of Intel’s Data Center Software division.  Joe leads technology development and go-to-market programs which combine Intel technology with partner platforms to create new and innovative solutions.  In his over a decade of work at Intel, Joe has held many roles including architect, engineering manager, and program manager.  When not working Joe enjoys his family, his dogs, martial arts, and the great outdoors of Arizona.


When you're planning a deployment of the Intel® Distribution of Apache Hadoop* (IDH) as an end user or an integration as a technology partner, you'll want to know that you can integrate IDH into your existing management & monitoring environment rather than having to change what you already have in place. For example, you've already established services for performance monitoring, network security and many administrative tasks. You'd like IDH to fit right in to what you've built alongside everything you are already managing.

The IDH REST API delivers just that magic. It allows you to create scripts for the deployment, configuration and administration of a Hadoop cluster. Your existing administrative infrastructure continues to operate as it always has, but now IDH is running right alongside. You have a comprehensive view of your entire operation.

Intel® Manager for IDH simplifies and automates Hadoop deployment and administration. The IDH REST API gives you additional tools: the ability to write scripts for creating and configuring a Hadoop cluster that fits into your existing environment.

For example, the API allows you to:

  • Create a cluster with a user-defined name.
  • Add and remove nodes from the cluster.
  • Install and delete services from the cluster.
  • Assign service roles to a node, e.g., you can specify a particular node as a datanode.
  • Add and delete racks.
  • Propagate service property changes across the cluster.
  • Format HDFS.
  • Obtain a list of nodes installed in the cluster, the service roles assigned to those nodes, and state of each Apache Hadoop* service.

How It Works

Calls to the REST API are sent to the Intel® Manager (IM) server using an HTTP client such as curl. (Red Hat Enterprise Linux 6.3 uses curl 7.19.7, for example). The curl commands use option k, notated as – k, to establish an SSL connection between the client and the IM server. The IM server sends a self-signed certificate during the SSL transaction. Therefore, the client need not (should not) attempt to verify the certificate.

REST API calls are made using JSON. All of the resources defined within the API conform to standard CRUD (Create-Read-Update-Delete) format. That is, an HTTP request path determines the entity to be acted upon, while the HTTP method is the action (Get, Post, Put, Delete) performed. 

For example:











Get a list of nodes in the cluster

Add nodes to the cluster




Get role assigned to user


Change password or role assigned to user

Delete user

The API provides calls that interact with HDFS, MapReduce, Hbase, ZooKeeper, Hive, Oozie, users, nodes and others. As noted, a call to Hadoop nodes can add nodes to a Hadoop cluster. It can  get a list of all the nodes in the cluster. That list includes the name of each (typically the FQDN); its IP address; the name of its rack and its cluster.

To make a nodes call specify the:

  • Path: /restapi/intelcloud/api/v1/cluster/{clusterName}/nodes
  • URL: https://hadoop1:9443/restapil/intelcloud/api/v1/cluster/[user-supplied clusterName}/nodes

The REST API lets you fully integrate Intel® Distribution of Apache Hadoop* (IDH) into your business. You'll find complete details in the IDH REST API documentation at the IDH web site under Resources.

Bye for now,