We need to add HTTPFS Role Instance to the cluster in order to use HDFS commands via http. Let us go ahead and setup on our gateway (bigdataserver-1)
To add HTTPFS role
- Click HDFS -> Add Role Instance -> Select “HTTPFS”
- Select the host to install HTTPFS gateway daemon
- Click on Install.
- Once services are restarted, we need to open the port if the server is behind firewall.
Here are some of the examples of using HTTPFS.
https://gist.github.com/dgadiraju/d6bfce4f7d0668c48ff493486cafe60e
For more details on Accessing HDFS via HTTP Rest API, visit the link.