yarn监控---获取任务的状态信息

版权声明：原创文章 欢迎参考 请勿抄袭 https://blog.csdn.net/aA518189/article/details/88570211

简介

         生产环境大多数任务都会提交到yarn上去运行，所有如何获取任务在yarn上的状态以及信息，非常重要。如果能获取任务在yarn上的状态和信息，那么我们就能及时发现任务出现的问题。我们可以通过java或者python去获取yarn任务的信息。

java版

第一步：把yarn-site.xml文件导入到监控系统（meavn工程）的resources文件下

第二步：编写监控代码

public class YarnUtil {
//获取任务的applicationId
    public static String getAppId(String jobName) {
        YarnClient client = YarnClient.createYarnClient();
        Configuration conf = new Configuration();
        client.init(conf);
        client.start();
        EnumSet<YarnApplicationState> appStates = EnumSet.noneOf(YarnApplicationState.class);
        if (appStates.isEmpty()) {
            appStates.add(YarnApplicationState.RUNNING);
            appStates.add(YarnApplicationState.ACCEPTED);
            appStates.add(YarnApplicationState.SUBMITTED);
        }
        List<ApplicationReport> appsReport = null;
        try {
       //返回EnumSet<YarnApplicationState>中个人任务状态的所有任务
            appsReport = client.getApplications(appStates);
        } catch (YarnException | IOException e) {
            e.printStackTrace();
        }
        assert appsReport != null;
        for (ApplicationReport appReport : appsReport) {
           //获取任务名
            String jn = appReport.getName();
            String applicationType = appReport.getApplicationType();
            if (jn.equals(jobName) && "Apache Flink".equals(applicationType)) {
                try {
                    client.close();
                } catch (IOException e) {
                    e.printStackTrace();
                }
                return appReport.getApplicationId().toString();
            }
        }
        try {
            client.close();
        } catch (IOException e) {
            e.printStackTrace();
        }
        return null;
    }
//根据任务的applicationId去获取任务的状态
    public static YarnApplicationState getState(String appId) {
        YarnClient client = YarnClient.createYarnClient();
        Configuration conf = new Configuration();
        client.init(conf);
        client.start();
        ApplicationId applicationId = ApplicationId.fromString(appId);
        YarnApplicationState yarnApplicationState = null;
        try {
            ApplicationReport applicationReport = client.getApplicationReport(applicationId);
            yarnApplicationState = applicationReport.getYarnApplicationState();
        } catch (YarnException | IOException e) {
            e.printStackTrace();
        }
        try {
            client.close();
        } catch (IOException e) {
            e.printStackTrace();
        }
        return yarnApplicationState;
    }
    public static void main(String[] args) throws IOException, InterruptedException {
        YarnApplicationState state = getState("application_id4244262");
        System.out.println(state);
        System.out.println(state == YarnApplicationState.RUNNING);
    }
}

yarn上的任务有如下几个状态：在源码的YarnApplicationState枚举的类中定义，我们可以更具获取任务的applicationId去获取任务的状态。

注意事项

1.client.getApplications(appStates)：是为了返回appStates中所包含状态的所有任务，可以设置只返回哪几种状态的任务列表

2. ApplicationReport appReport：是返回的任务 ，我们可以通过任务实例appReport获取任务的所有信息，也就是yarn界面上任务的所有信息，队列 ，applicationId,任务名。。。。。