Both YARN and MR2 have command line interface. YARN CLI can be used to submit YARN applications and manage them Map Reduce CLI can be used to submit jobs using job files and also to manage them There are hand full of commands which Hadoop Administrator should be aware of. Type yarn or mapred and …
Now let us review the web interfaces and also log files to troubleshoot any YARN or Map Reduce specific issues. Web UIs Resource Manager Web UI Job History Server Web UI Using Cloudera Manager to troubleshoot the issues of Resource Manager components. Using Resource Manager Web UI to troubleshoot the jobs Resource Manager Web UI …
As we have changed the properties with respect to node manager capacity, let us run randomtextwriter again and see how long it take. We can override individual properties at runtime using -D and multiple properties using -conf and xml file similar to yarn-site.xml or mapred-site.xml. https://gist.github.com/dgadiraju/f2852840916b1e79f4fb6830d93c8b22 Now let us run word count program from hadoop examples …
configuring files and important properties – Running Jobs Read More »
https://gist.github.com/dgadiraju/0ff21e302882e6565ca02aeb068fd287
https://gist.github.com/dgadiraju/e8b8687e00a8b43087348068e536c18d
Unlike plain vanilla distribution and other vendor distributions, Cloudera manages configuration files a bit different. Typically configuration files will be in /etc/hadoop/conf. But when it comes to Cloudera, /etc/hadoop/conf will only have templates. Actual properties files are managed under /var/run/cloudera-scm-agent/process on each node. hadoop-env.sh – for memory settings of Resource Manager, Node Manager etc. core-site.xml …
Configuration files and Important Properties – Overview Read More »
