Now let us review the web interfaces and also log files to troubleshoot any YARN or Map Reduce specific issues.
Web UIs
- Resource Manager Web UI
- Job History Server Web UI
- Using Cloudera Manager to troubleshoot the issues of Resource Manager components.
- Using Resource Manager Web UI to troubleshoot the jobs
- Resource Manager Web UI acts as a proxy to get details about running or completed jobs.
- Running Jobs details are served by per application Application Master
- Completed Jobs are served by History Servers.
- Typically logs related to completed jobs will be stored in HDFS and History Servers provide UI for them.
Log files
There are service level logs as well as application or job level logs.
- Service level logs can be accessed using Cloudera Manager Web UI or command prompt on each of the servers.
- We need to login to the respective server to access logs via command prompt. For example, if we want to look into Resource Manager logs we need to login to the node where Resource Manager is running.
- Job level logs are primarily available through Resource Manager Web UI via Proxy.
- While job is running, logs and progress is provided by web service with in Application Master.
- Once job is completed logs will be copied to HDFS and served by History Servers of the underlying framework (Map Reduce in this case)