目标:使用iotop&top&jstack命令定位 io高的应用以及线程信息
实现:load过高但是cpu占用率低的模拟与验证
一般load高,说明系统大量的线程在等待执行
cpu us sy 显示都不高,并且wa 占用高,说明系统在等待大量的io操作
1、安装iotop
sudo yum install iotop
2、top命令显示
从top命令可以看出,系统load 6.43,对于4 cpu的系统来说,已经是超载,另外 wa 60%,可以确定系统的线程在等待io操作
其中 线程 2492 比较可疑
top -H -p 2492
3、iostat
iostat ALL :可以查阅 那个磁盘读写频繁
4、sudo iotop
可以查看哪些线程在 做读写io
FileReadWrite 应用:三个线程,每个线程都在每秒36M,42M,75M的速度读写io
试验测试代码为:
import java.io.File; import java.io.FileInputStream; import java.io.FileOutputStream; public class FileReadWrite { public static void main(String[] args) { for(int i =0;i<4;i++){ new T().start(); } } public static class T extends Thread{ @Override public void run() { while (true){ try { exec(); } catch (Exception e) { e.printStackTrace(); } } } } private static void exec() throws Exception{ long start = System.currentTimeMillis(); String fileName = "A"+System.currentTimeMillis(); File f = new File(fileName); if(!f.exists()){ System.out.println(fileName + " created "); f.createNewFile(); } FileOutputStream fi = new FileOutputStream(f); int i = 0; while(true){ i++; byte[] data = new byte[1024*1024*5]; fi.write(data); if(i>50){ break; } } fi.close(); FileInputStream f2 = new FileInputStream(f); byte[] data = new byte[1024*1024*1]; int ret = f2.read(data); while(ret != -1){ data = new byte[1024*1024*1]; ret = f2.read(data); } boolean rt = f.delete(); System.out.println(fileName + " deleted " + rt + " time :" + (System.currentTimeMillis() - start)); } }
jstack 可以看大量的线程阻塞在了read 和write上:
"DestroyJavaVM" prio=10 tid=0x00007fabb4008800 nid=0x1774 waiting on condition [0x0000000000000000] java.lang.Thread.State: RUNNABLE "Thread-3" prio=10 tid=0x00007fabb40d6800 nid=0x1784 runnable [0x00007faba7af9000] java.lang.Thread.State: RUNNABLE at java.io.FileOutputStream.writeBytes(Native Method) at java.io.FileOutputStream.write(FileOutputStream.java:325) at FileReadWrite.exec(FileReadWrite.java:42) at FileReadWrite.access$000(FileReadWrite.java:6) at FileReadWrite$T.run(FileReadWrite.java:19) "Thread-2" prio=10 tid=0x00007fabb40d4800 nid=0x1783 runnable [0x00007faba7bfa000] java.lang.Thread.State: RUNNABLE at java.io.FileInputStream.readBytes(Native Method) at java.io.FileInputStream.read(FileInputStream.java:243) at FileReadWrite.exec(FileReadWrite.java:54) at FileReadWrite.access$000(FileReadWrite.java:6) at FileReadWrite$T.run(FileReadWrite.java:19) "Thread-1" prio=10 tid=0x00007fabb40d2800 nid=0x1782 runnable [0x00007faba7cfb000] java.lang.Thread.State: RUNNABLE at java.io.FileInputStream.readBytes(Native Method) at java.io.FileInputStream.read(FileInputStream.java:243) at FileReadWrite.exec(FileReadWrite.java:54) at FileReadWrite.access$000(FileReadWrite.java:6) at FileReadWrite$T.run(FileReadWrite.java:19) "Thread-0" prio=10 tid=0x00007fabb40d0800 nid=0x1781 waiting on condition [0x00007faba7dfc000] java.lang.Thread.State: RUNNABLE at FileReadWrite.exec(FileReadWrite.java:53) at FileReadWrite.access$000(FileReadWrite.java:6) at FileReadWrite$T.run(FileReadWrite.java:19)