[TOC]
一、介绍:cactus:
Arthas 是Alibaba开源的Java诊断工具，深受开发者喜爱。
当你遇到以下类似问题而束手无策时，Arthas可以帮助你解决：
-
这个类从哪个 jar 包加载的？为什么会报各种类相关的 Exception？
-
我改的代码为什么没有执行到？难道是我没 commit？分支搞错了？
-
遇到问题无法在线上 debug，难道只能通过加日志再重新发布吗？
-
线上遇到某个用户的数据处理有问题，但线上同样无法 debug，线下无法重现！
-
是否有一个全局视角来查看系统的运行状况？
有什么办法可以监控到JVM的实时运行状态？
Arthas支持JDK 6+，支持Linux/Mac/Winodws，采用命令行交互模式，同时提供丰富的 Tab 自动补全功能，进一步方便进行问题的定位和诊断。
二、 实践教程 :cake:
wget https://code.aliyun.com/middleware-container/handsonLabExternedFiles/raw/master/demo-arthas-spring-boot.jar;java -jar demo-arthas-spring-boot.jar
复制代码1.1 启动arthas-boot
wget https://arthas.aliyun.com/arthas-boot.jar;java -jar arthas-boot.jar --target-ip 0.0.0.0
复制代码arthas-boot是Arthas的启动程序，它启动后，会列出所有的Java进程，用户可以选择需要诊断的目标进程。
选择第一个进程，输入 1 ，再Enter/回车：
[INFO] arthas-boot version: 3.5.5
[INFO] Found existing java process, please choose one and input the serial number of the process, eg : 1. Then hit ENTER.
* [1]: 1265 demo-arthas-spring-boot.jar
复制代码1
复制代码Attach成功之后，会打印Arthas LOGO。输入 help 可以获取到更多的帮助信息。
三、查看JVM信息:factory:
下面介绍Arthas里查看JVM信息的命令。
1.1 sysprop
sysprop 可以打印所有的System Properties信息。
$ sysprop
KEY VALUE
-----------------------------------------------------------------------------------------------------------------------------
awt.toolkit sun.awt.X11.XToolkit
file.encoding.pkg sun.io
java.specification.vers 1.8
ion
sun.cpu.isalist
sun.jnu.encoding UTF-8
java.class.path demo-arthas-spring-boot.jar
java.vm.vendor Private Build
sun.arch.data.model 64
java.vendor.url http://java.oracle.com/
catalina.useNaming false
user.timezone Asia/Shanghai
org.jboss.logging.provi slf4j
der
os.name Linux
java.vm.specification.v 1.8
ersion
user.country US
sun.java.launcher SUN_STANDARD
sun.boot.library.path /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64
sun.java.command demo-arthas-spring-boot.jar
sun.cpu.endian little
user.home /home/shell
user.language en
java.specification.vend Oracle Corporation
or
java.home /usr/lib/jvm/java-8-openjdk-amd64/jre
file.separator /
line.separator
java.vm.specification.v Oracle Corporation
endor
java.specification.name Java Platform API Specification
java.awt.graphicsenv sun.awt.X11GraphicsEnvironment
java.awt.headless true
sun.boot.class.path /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/resources.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/
rt.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/sunrsasign.jar:/usr/lib/jvm/java-8-openjdk-amd64/
jre/lib/jsse.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/jce.jar:/usr/lib/jvm/java-8-openjdk-amd
64/jre/lib/charsets.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/jfr.jar:/usr/lib/jvm/java-8-open
jdk-amd64/jre/classes
java.protocol.handler.p org.springframework.boot.loader
kgs
sun.management.compiler HotSpot 64-Bit Tiered Compilers
java.runtime.version 1.8.0_292-8u292-b10-0ubuntu1~18.04-b10
user.name shell
path.separator :
os.version 3.10.0-957.21.3.el7.x86_64
java.endorsed.dirs /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/endorsed
java.runtime.name OpenJDK Runtime Environment
file.encoding UTF-8
spring.beaninfo.ignore true
sun.nio.ch.bugLevel
java.vm.name OpenJDK 64-Bit Server VM
java.vendor.url.bug http://bugreport.sun.com/bugreport/
java.io.tmpdir /tmp
catalina.home /tmp/tomcat.8558577891545713433.61000
java.version 1.8.0_292
user.dir /home/shell
os.arch amd64
PID 1265
java.vm.specification.n Java Virtual Machine Specification
ame
java.awt.printerjob sun.print.PSPrinterJob
sun.os.patch.level unknown
catalina.base /tmp/tomcat.8558577891545713433.61000
java.library.path /usr/java/packages/lib/amd64:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-l
inux-gnu:/usr/lib/jni:/lib:/usr/lib
java.vm.info mixed mode
java.vendor Private Build
java.vm.version 25.292-b10
java.ext.dirs /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext:/usr/java/packages/lib/ext
sun.io.unicode.encoding UnicodeLittle
java.class.version 52.0
复制代码也可以指定单个key： sysprop java.version
$ sysprop java.version
KEY VALUE
-----------------------------------------------------------------------------------------------------------------------------
java.version 1.8.0_292
复制代码也可以通过grep来过滤： sysprop | grep user
$ sysprop | grep user
user.timezone Asia/Shanghai
user.country US
user.home /home/shell
user.language en
user.name shell
user.dir /home/shell
复制代码可以设置新的value： sysprop testKey testValue
$ sysprop testKey testValue
Successfully changed the system property.
KEY VALUE
-----------------------------------------------------------------------------------------------------------------------------
testKey testValue
复制代码1.2 sysenv
sysenv 命令可以获取到环境变量。和sysprop命令类似。
sysenv
KEY VALUE
-----------------------------------------------------------------------------------------------------------------------------
PATH /usr/local/nvm/versions/node/v12.13.1/bin:/home/shell/.local/bin:/home/shell/bin:/usr/shell/bin:/u
sr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
USE_CLOUD_STORAGE false
ACCOUNT_ID 1030655505651579
TZ Asia/Shanghai
ALIBABA_CLOUD_VENDOR CloudShell/0.2
CLOUD_SHELL_ROOT_KEY_ID LTAI4G8C8WqQihqKWH6L54Cr
ALIYUN_ACCESS_KEY_SECRE J4TnqPfEhKWLpmuAk5ohhfqqNVHhPPP4YMwUmgXWCqfy
T
USER_ID 1030655505651579
ALIBABA_CLOUD_ACCESS_KE J4TnqPfEhKWLpmuAk5ohhfqqNVHhPPP4YMwUmgXWCqfy
Y_SECRET
GOCACHE /go/.cache/go-build
DEFAULT_REGION cn-hangzhou
ALICLOUD_REGION cn-hangzhou
LOGNAME shell
CLOUDSHELL_CONNECT_KEY 7zgfkrzvimgbwpun
PWD /home/shell
NVM_CD_FLAGS
CHANNEL_ID OFFICIAL
ALICLOUD_ACCOUNT_ID 1030655505651579
ACCESS_KEY_ID TMP.3KfX6ZiSosuHVWXdnmkZsJfhE7YtEQpzzhES4zuLz4BfQh1jAWHjaqE9gtygQycXhj9qyhSWVhgwdsh2hm7Rs4BaUTu4Ez
GOPATH /go
OLDPWD /
ALIYUN_ACCESS_KEY_ID TMP.3KfX6ZiSosuHVWXdnmkZsJfhE7YtEQpzzhES4zuLz4BfQh1jAWHjaqE9gtygQycXhj9qyhSWVhgwdsh2hm7Rs4BaUTu4Ez
GEM_HOME /home/shell/.gems
REGION cn-hangzhou
CLOUDSHELL_TENANT_CERT -----BEGIN CERTIFICATE-----\nMIIDazCCAlOgAwIBAgIQWntjiFQ5tgjmatJi2Ol7iDANBgkqhkiG9w0BAQsFADBC\nMRY
wFAYDVQQKDA1jbG91ZHNoZWxsIENBMRMwEQYDVQQLDApjbG91ZHNoZWxsMRMw\nEQYDVQQDDApjbG91ZHNoZWxsMB4XDTIxMTA
yNjAwNDgwOVoXDTMxMTAyNjAwNDgw\nOVowQjEWMBQGA1UECgwNY2xvdWRzaGVsbCBDQTETMBEGA1UECwwKY2xvdWRzaGVs\nb
DETMBEGA1UEAwwKY2xvdWRzaGVsbDCCASIwDQYJKoZIhvcNAQEBBQADggEPADCC\nAQoCggEBAKz3d8k3+Ak9+c81EDbRTI2Ls
AWhXnrArh2E1c2dFAYQk5UyndT80wUv\nNt1uvqk23KOVE4rQOZ2KZ6mQczxlH62H0nX6gSMeZsCqGNtjRtb7j3PNDXGXoYTF\
n2WxXhYLgzszcUaJ+/iAiedQx0uIc9J5+sG3Do5LjOsFYGc8UYYBA+UpRJFq71oYs\nZXPJiNxLcOfF3k0JnKMqgNXmIoqEZFm
OizpcVGMQZfycgqgid1yDqRVHJRwQHwHv\nclnUzxYTYVa5DsqYDr+OXmx++uHmQDtdli5eIA7rATqKt6BhFY/6VbdSklA8m5s
Y\nt1xOZcE/tVSfNyAtsNpwirqAF2U6EeECAwEAAaNdMFswHwYDVR0jBBgwFoAUOC1+\nGQ6yPqW7TzKjy2IBHaqj190wHQYDV
R0OBBYEFDgtfhkOsj6lu08yo8tiAR2qo9fd\nMAsGA1UdDwQEAwICBDAMBgNVHRMEBTADAQH/MA0GCSqGSIb3DQEBCwUAA4IBA
QAb\nI0APX3asjhpxGkGxQrrBrtAzdyFuQNSBPP6uIdhK+BskYaK3VLh2RwRxiHdoUGO1\nzXK9VFvW2jb/1Xkcdlbm8ZkUefa
P03D7E8+1HYuIx1uNo24TcOLZgOCeNvd3sFkz\nLvbluvF4PC44q9M067qlQOk5XvlkhJILYDXdfZLUKY1kC5o6u4f0sv0f6yO
yRY55\nUE9WAhZdq1qNvaWD2bqR21x1Gc7fCIXv6rkHmETzcjdrEAaclShs6DmsYi3p148H\nXWfHNEqbIYzZJtzErvjqcgMOC
xOHQ8mo6FmripT7w+9YhwbjOfIukIifPg3XdUj3\nUd3M38VSncE805iPPkGM\n-----END CERTIFICATE-----\n
ALIBABA_CLOUD_ACCOUNT_I 1030655505651579
D
ALIBABA_CLOUD_ACCESS_KE TMP.3KfX6ZiSosuHVWXdnmkZsJfhE7YtEQpzzhES4zuLz4BfQh1jAWHjaqE9gtygQycXhj9qyhSWVhgwdsh2hm7Rs4BaUTu4Ez
Y_ID
LS_COLORS rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:
su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31
:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:
*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;
31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31
:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.
alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm
=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.b
mp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tif
f=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpe
g=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vo
b=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=0
1;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35
:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*
.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.o
pus=00;36:*.spx=00;36:*.xspf=00;36:
CLOUD_SHELL_ROOT_KEY_SE cf4x2LqNAyQXdSqzR5dLUkDrNQQ5VZ
CRET
SHLVL 3
CLOUDSHELL_PROXY_ACCESS 96924942-e575-484f-8fc5-a1cbee03a437
_KEY
default_serverless_devs {"AccountID":"1030655505651579","AccessKeyID":"TMP.3KfX6ZiSosuHVWXdnmkZsJfhE7YtEQpzzhES4zuLz4BfQh1
_access jAWHjaqE9gtygQycXhj9qyhSWVhgwdsh2hm7Rs4BaUTu4Ez","AccessKeySecret":"J4TnqPfEhKWLpmuAk5ohhfqqNVHhPP
P4YMwUmgXWCqfy"}
ALIYUN_DEFAULT_REGION cn-hangzhou
TERM xterm-256color
LANG en_US.UTF-8
GOPROXY https://mirrors.aliyun.com/goproxy/
ALICLOUD_SECRET_KEY J4TnqPfEhKWLpmuAk5ohhfqqNVHhPPP4YMwUmgXWCqfy
ACCESS_KEY_SECRET J4TnqPfEhKWLpmuAk5ohhfqqNVHhPPP4YMwUmgXWCqfy
CLOUD_SHELL_LANG zh
_ /usr/bin/java
ALIBABA_CLOUD_DEFAULT_R cn-hangzhou
EGION
CLOUDSHELL_DOWNLOAD_MAX 5368709120
_SIZE
NVM_DIR /usr/local/nvm
CLOUD_SHELL true
FC_NO_ISOLATION y
USER shell
GEM_PATH /home/shell/.gems:/var/lib/gems/2.5.0
ALICLOUD_ACCESS_KEY TMP.3KfX6ZiSosuHVWXdnmkZsJfhE7YtEQpzzhES4zuLz4BfQh1jAWHjaqE9gtygQycXhj9qyhSWVhgwdsh2hm7Rs4BaUTu4Ez
ALIYUN_USER_AGENT CloudShell/0.2
HOSTNAME 68ea4d9747b3
MOUNT_FAILED false
NVM_BIN /usr/local/nvm/versions/node/v12.13.1/bin
TF_PLUGIN_CACHE_DIR /home/shell/.terraform.d/plugin-cache
CLOUDSHELL_PROXY_ACCESS c2b141e2-0c63-4959-a267-2faf34d2b8be
_SECRET
HOME /home/shell
复制代码1.3 jvm
jvm 命令会打印出JVM的各种详细信息。
$ jvm
RUNTIME
-----------------------------------------------------------------------------------------------------------------------------
MACHINE-NAME 1265@68ea4d9747b3
JVM-START-TIME 2022-01-27 09:39:11
MANAGEMENT-SPEC-VERSION 1.2
SPEC-NAME Java Virtual Machine Specification
SPEC-VENDOR Oracle Corporation
SPEC-VERSION 1.8
VM-NAME OpenJDK 64-Bit Server VM
VM-VENDOR Private Build
VM-VERSION 25.292-b10
INPUT-ARGUMENTS []
CLASS-PATH demo-arthas-spring-boot.jar
BOOT-CLASS-PATH /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/resources.jar:/usr/lib/jvm/java-8-openjdk-amd
64/jre/lib/rt.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/sunrsasign.jar:/usr/lib/jvm
/java-8-openjdk-amd64/jre/lib/jsse.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/jce.ja
r:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/charsets.jar:/usr/lib/jvm/java-8-openjdk-am
d64/jre/lib/jfr.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/classes
LIBRARY-PATH /usr/java/packages/lib/amd64:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/l
ib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib
-----------------------------------------------------------------------------------------------------------------------------
CLASS-LOADING
-----------------------------------------------------------------------------------------------------------------------------
LOADED-CLASS-COUNT 9071
TOTAL-LOADED-CLASS-COUNT 9071
UNLOADED-CLASS-COUNT 0
IS-VERBOSE false
-----------------------------------------------------------------------------------------------------------------------------
COMPILATION
-----------------------------------------------------------------------------------------------------------------------------
NAME HotSpot 64-Bit Tiered Compilers
TOTAL-COMPILE-TIME 13882
[time (ms)]
-----------------------------------------------------------------------------------------------------------------------------
GARBAGE-COLLECTORS
-----------------------------------------------------------------------------------------------------------------------------
Copy name : Copy
[count/time (ms)] collectionCount : 97
collectionTime : 602
MarkSweepCompact name : MarkSweepCompact
[count/time (ms)] collectionCount : 3
collectionTime : 208
-----------------------------------------------------------------------------------------------------------------------------
MEMORY-MANAGERS
-----------------------------------------------------------------------------------------------------------------------------
CodeCacheManager Code Cache
Metaspace Manager Metaspace
Compressed Class Space
Copy Eden Space
Survivor Space
MarkSweepCompact Eden Space
Survivor Space
Tenured Gen
-----------------------------------------------------------------------------------------------------------------------------
MEMORY
-----------------------------------------------------------------------------------------------------------------------------
HEAP-MEMORY-USAGE init : 31457280(30.0 MiB)
[memory in bytes] used : 42087040(40.1 MiB)
committed : 81309696(77.5 MiB)
max : 466288640(444.7 MiB)
NO-HEAP-MEMORY-USAGE init : 2555904(2.4 MiB)
[memory in bytes] used : 67432512(64.3 MiB)
committed : 68878336(65.7 MiB)
max : -1(-1 B)
PENDING-FINALIZE-COUNT 0
-----------------------------------------------------------------------------------------------------------------------------
OPERATING-SYSTEM
-----------------------------------------------------------------------------------------------------------------------------
OS Linux
ARCH amd64
PROCESSORS-COUNT 1
LOAD-AVERAGE 0.02
VERSION 3.10.0-957.21.3.el7.x86_64
-----------------------------------------------------------------------------------------------------------------------------
THREAD
-----------------------------------------------------------------------------------------------------------------------------
COUNT 30
DAEMON-COUNT 28
PEAK-COUNT 30
STARTED-COUNT 38
DEADLOCK-COUNT 0
-----------------------------------------------------------------------------------------------------------------------------
FILE-DESCRIPTOR
-----------------------------------------------------------------------------------------------------------------------------
MAX-FILE-DESCRIPTOR-COUNT 1048576
OPEN-FILE-DESCRIPTOR-COUNT 67
复制代码1.4 dashboard
dashboard 命令可以查看当前系统的实时数据面板。
输入 q 或者 Ctrl+C 可以退出dashboard命令。
四、 Arthas使用技巧:dart:
1.1 help
Arthas里每一个命令都有详细的帮助信息。可以用-h来查看。帮助信息里有EXAMPLES和WIKI链接。
sysprop -h
复制代码1.2 自动补全
Arthas支持丰富的自动补全功能，在使用有疑惑时，可以输入Tab来获取更多信息。
比如输入 sysprop java. 之后，再输入Tab，会补全出对应的key：
复制代码$ sysprop java.
java.runtime.name java.protocol.handler.pkgs java.vm.version
java.vm.vendor java.vendor.url java.vm.name
...
复制代码1.3 readline的快捷键支持
Arthas支持常见的命令行快捷键，比如Ctrl + A跳转行首，Ctrl + E跳转行尾。
更多的快捷键可以用 keymap 命令查看。
[arthas@1265]$ keymap
Shortcut Description Name
-----------------------------------------------------------------------------------------------------------------------------
"\C-a" Ctrl + a beginning-of-line
"\C-e" Ctrl + e end-of-line
"\C-f" Ctrl + f forward-word
"\C-b" Ctrl + b backward-word
"\e[D" Left arrow backward-char
"\e[C" Right arrow forward-char
"\e[A" Up arrow history-search-backward
"\e[B" Down arrow history-search-forward
"\C-h" Ctrl + h backward-delete-char
"\C-?" Ctrl + ? backward-delete-char
"\C-u" Ctrl + u undo
"\C-d" Ctrl + d delete-char
"\C-k" Ctrl + k kill-line
"\C-i" Ctrl + i complete
"\C-j" Ctrl + j accept-line
"\C-m" Ctrl + m accept-line
"\C-w" Ctrl + w backward-delete-word
"\C-x\e[3~" "\C-x\e[3~" backward-kill-line
"\e\C-?" "\e\C-?" backward-kill-word
"\e[1~" "\e[1~" beginning-of-line
"\e[4~" "\e[4~" end-of-line
"\e[5~" "\e[5~" beginning-of-history
"\e[6~" "\e[6~" end-of-history
"\e[3~" "\e[3~" delete-char
"\e[2~" "\e[2~" quoted-insert
"\e[7~" "\e[7~" beginning-of-line
"\e[8~" "\e[8~" end-of-line
"\eOH" "\eOH" beginning-of-line
"\eOF" "\eOF" end-of-line
"\e[H" "\e[H" beginning-of-line
"\e[F" "\e[F" end-of-line
复制代码1.4 历史命令的补全
如果想再执行之前的命令，可以在输入一半时，按Up/↑ 或者 Ddown/↓，来匹配到之前的命令。
比如之前执行过sysprop java.version，那么在输入sysprop ja之后，可以输入Up/↑，就会自动补全为sysprop java.version。
如果想查看所有的历史命令，也可以通过 history 命令查看到。
history
复制代码1.5 pipeline
Arthas支持在pipeline之后，执行一些简单的命令，比如：
sysprop | grep java
复制代码sysprop | wc -l
复制代码五、 sc/sm 查看已加载的类 :boxing_glove:
1.1 sc
sc 命令可以查找到所有JVM已经加载到的类。 如果搜索的是接口，还会搜索所有的实现类。比如查看所有的Filter实现类：
sc javax.servlet.Filter
复制代码通过-d参数，可以打印出类加载的具体信息，很方便查找类加载问题。
sc -d javax.servlet.Filter
复制代码sc支持通配，比如搜索所有的StringUtils：
sc *StringUtils
复制代码1.2 sm
sm命令则是查找类的具体函数。比如：
sm java.math.RoundingMode
复制代码通过-d参数可以打印函数的具体属性：
sm -d java.math.RoundingMode
复制代码也可以查找特定的函数，比如查找构造函数：
sm java.math.RoundingMode <init>
复制代码六、 Jad
可以通过 jad 命令来反编译代码：
[arthas@1268]$ jad com.example.demo.arthas.user.UserController
ClassLoader:
+-org.springframework.boot.loader.LaunchedURLClassLoader@19469ea2
+-sun.misc.Launcher$AppClassLoader@1b6d3586
+-sun.misc.Launcher$ExtClassLoader@7d0587f1
Location:
file:/home/shell/demo-arthas-spring-boot.jar!/BOOT-INF/classes!/
/*
* Decompiled with CFR.
*
* Could not load the following classes:
* com.example.demo.arthas.user.User
* org.slf4j.Logger
* org.slf4j.LoggerFactory
* org.springframework.web.bind.annotation.GetMapping
* org.springframework.web.bind.annotation.PathVariable
* org.springframework.web.bind.annotation.RestController
*/
package com.example.demo.arthas.user;
import com.example.demo.arthas.user.User;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RestController;
@RestController
public class UserController {
private static final Logger logger = LoggerFactory.getLogger(UserController.class);
@GetMapping(value={"/user/{id}"})
public User findUserById(@PathVariable Integer id) {
/*15*/ logger.info("id: {}", (Object)id);
/*17*/ if (id != null && id < 1) {
throw new IllegalArgumentException("id < 1");
}
return new User(id.intValue(), "name" + id);
}
}
Affect(row-cnt:1) cost in 1262 ms.
复制代码通过--source-only参数可以只打印出在反编译的源代码：
[arthas@1268]$ jad --source-only com.example.demo.arthas.user.UserController
/*
* Decompiled with CFR.
*
* Could not load the following classes:
* com.example.demo.arthas.user.User
* org.slf4j.Logger
* org.slf4j.LoggerFactory
* org.springframework.web.bind.annotation.GetMapping
* org.springframework.web.bind.annotation.PathVariable
* org.springframework.web.bind.annotation.RestController
*/
package com.example.demo.arthas.user;
import com.example.demo.arthas.user.User;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RestController;
@RestController
public class UserController {
private static final Logger logger = LoggerFactory.getLogger(UserController.class);
@GetMapping(value={"/user/{id}"})
public User findUserById(@PathVariable Integer id) {
/*15*/ logger.info("id: {}", (Object)id);
/*17*/ if (id != null && id < 1) {
throw new IllegalArgumentException("id < 1");
}
return new User(id.intValue(), "name" + id);
}
}
复制代码七、Ognl :kick_scooter:
在Arthas里，有一个单独的ognl命令，可以动态执行代码。
1.1 调用static函数
[arthas@1268]$ ognl '@[email protected]("hello ognl")'
复制代码可以检查Terminal 1里的进程输出，可以发现打印出了hello ognl。
1.2 查找UserController的ClassLoader
sc -d com.example.demo.arthas.user.UserController | grep classLoaderHash
复制代码注意hashcode是变化的，需要先查看当前的ClassLoader信息，提取对应ClassLoader的hashcode。
如果你使用-c，你需要手动输入hashcode：-c <hashcode>
八、案例: 排查函数调用异常 :rabbit:
1.1 现象
目前，访问 http://localhost:61000/user/0 ，会返回500异常：
curl http://localhost:61000/user/0
复制代码{"timestamp":1550223186170,"status":500,"error":"Internal Server Error","exception":"java.lang.IllegalArgumentException","message":"id < 1","path":"/user/0"}
复制代码但请求的具体参数，异常栈是什么呢？
1.2 查看UserController的 参数/异常
watch com.example.demo.arthas.user.UserController * '{params, throwExp}'
复制代码- 第一个参数是类名，支持通配
- 第二个参数是函数名，支持通配 访问
curl http://localhost:61000/user/0,watch命令会打印调用的参数和异常
curl http://localhost:61000/user/0
复制代码监控查看到异常结果：
[arthas@1268]$ watch com.example.demo.arthas.user.UserController * '{params, throwExp}'
Press Q or Ctrl+C to abort.
Affect(class count: 1 , method count: 2) cost in 227 ms, listenerId: 1
method=com.example.demo.arthas.user.UserController.findUserById location=AtExceptionExit
ts=2022-01-27 10:58:41; [cost=2.370685ms] result=@ArrayList[
@Object[][isEmpty=false;size=1],
@IllegalArgumentException[java.lang.IllegalArgumentException: id < 1],
]
复制代码可以看到实际抛出的异常是IllegalArgumentException。
可以输入 q 或者 Ctrl+C 退出watch命令。
如果想把获取到的结果展开，可以用-x参数：
[arthas@1268]$ watch com.example.demo.arthas.user.UserController * '{params, throwExp}' -x 2
Press Q or Ctrl+C to abort.
Affect(class count: 1 , method count: 2) cost in 59 ms, listenerId: 2
method=com.example.demo.arthas.user.UserController.findUserById location=AtExceptionExit
ts=2022-01-27 11:00:38; [cost=0.475683ms] result=@ArrayList[
@Object[][
@Integer[0],
],
java.lang.IllegalArgumentException: id < 1
at com.example.demo.arthas.user.UserController.findUserById(UserController.java:18)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133)
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738)
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)
at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:861)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:635)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:742)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:109)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:81)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:197)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:496)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:803)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:790)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1468)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
,
]
复制代码返回值表达式
在上面的例子里，第三个参数是返回值表达式，它实际上是一个ognl表达式，它支持一些内置对象：
- loader
- clazz
- method
- target
- params
- returnObj
- throwExp
- isBefore
- isThrow
- isReturn
你可以利用这些内置对象来组成不同的表达式。比如返回一个数组：
watch com.example.demo.arthas.user.UserController * '{params[0], target, returnObj}'
复制代码更多参考： arthas.aliyun.com/doc/advice-…
九、条件表达式:haircut_woman:
1.1 watch命令支持在第4个参数里写条件表达式，比如：
watch com.example.demo.arthas.user.UserController * returnObj 'params[0] > 100'
复制代码当访问 user/1 时，watch命令没有输出
当访问 user/101 时，watch会打印出结果。
$ watch com.example.demo.arthas.user.UserController * returnObj 'params[0] > 100'
Press Q or Ctrl+C to abort.
Affect(class-cnt:1 , method-cnt:2) cost in 47 ms.
ts=2019-02-13 19:42:12; [cost=0.821443ms] result=@User[
id=@Integer[101],
name=@String[name101],
]
复制代码1.2 当异常时捕获
watch命令支持-e选项，表示只捕获抛出异常时的请求：
watch com.example.demo.arthas.user.UserController * "{params[0],throwExp}" -e
复制代码1.3 按照耗时进行过滤
watch命令支持按请求耗时进行过滤，比如：
watch com.example.demo.arthas.user.UserController * '{params, returnObj}' '#cost>200'
复制代码十、 案例: 热更新代码 :fire:
下面介绍通过jad/mc/redefine 命令实现动态更新代码的功能。
目前，访问 http://localhost:61000/user/0 ，会返回500异常：
{"timestamp":1550223186170,"status":500,"error":"Internal Server Error","exception":"java.lang.IllegalArgumentException","message":"id < 1","path":"/user/0"}
复制代码下面通过热更新代码，修改这个逻辑。
1.1 jad反编译UserController
jad --source-only com.example.demo.arthas.user.UserController > /tmp/UserController.java
复制代码jad反编译的结果保存在 /tmp/UserController.java文件里了。
再打开一个Terminal 3，然后用vim来编辑/tmp/UserController.java：
## 新窗口打开
vim /tmp/UserController.java
复制代码比如当 user id 小于1时，也正常返回，不抛出异常：
@GetMapping(value={"/user/{id}"})
public User findUserById(@PathVariable Integer id) {
logger.info("id: {}", (Object)id);
if (id != null && id < 1) {
return new User(id, "name" + id);
// throw new IllegalArgumentException("id < 1");
}
return new User(id.intValue(), "name" + id);
}
复制代码1.2 sc查找加载UserController的ClassLoader
sc -d *UserController | grep classLoaderHash
classLoaderHash 19469ea2
复制代码可以发现是 spring boot LaunchedURLClassLoader@19469ea2 加载的。
请记下你的classLoaderHash，后面需要使用它。在这里，它是 19469ea2。
1.3 mc
保存好/tmp/UserController.java之后，使用mc(Memory Compiler)命令来编译，并且通过-c或者--classLoaderClass参数指定ClassLoader：
[arthas@2167]$ mc --classLoaderClass org.springframework.boot.loader.LaunchedURLClassLoader /tmp/UserController.java -d /tmp
Memory compiler output:
/tmp/com/example/demo/arthas/user/UserController.class
Affect(row-cnt:1) cost in 2972 ms.
复制代码也可以通过mc -c <classLoaderHash> /tmp/UserController.java -d /tmp，使用-c参数指定ClassLoaderHash:
$ mc -c 1be6f5c3 /tmp/UserController.java -d /tmp
复制代码1.4 redefine
再使用redefine命令重新加载新编译好的UserController.class：
[arthas@2167]$ redefine /tmp/com/example/demo/arthas/user/UserController.class
redefine success, size: 1, classes:
com.example.demo.arthas.user.UserController
复制代码redefine成功之后，再次访问 user/0 ，结果是：
{
"id": 0,
"name": "name0"
}
复制代码十一、案例: 动态更新应用Logger Level:hotdog:
在这个案例里，动态修改应用的Logger Level。
1.1 查找UserController的ClassLoader
[arthas@2167]$ sc -d com.example.demo.arthas.user.UserController | grep classLoaderHash
classLoaderHash 19469ea2
复制代码1.2 用ognl获取logger
[arthas@2167]$ ognl --classLoaderClass org.springframework.boot.loader.LaunchedURLClassLoader '@com.example.demo.arthas.user.UserController@logger'
@Logger[
serialVersionUID=@Long[5454405123156820674],
FQCN=@String[ch.qos.logback.classic.Logger],
name=@String[com.example.demo.arthas.user.UserController],
level=null,
effectiveLevelInt=@Integer[20000],
parent=@Logger[Logger[com.example.demo.arthas.user]],
childrenList=null,
aai=null,
additive=@Boolean[true],
loggerContext=@LoggerContext[ch.qos.logback.classic.LoggerContext[default]],
]
复制代码可以知道UserController@logger实际使用的是logback。可以看到level=null，则说明实际最终的level是从root logger里来的。
1.3 单独设置UserController的logger level
[arthas@2167]$ ognl --classLoaderClass org.springframework.boot.loader.LaunchedURLClassLoader '@[email protected](@ch.qos.logback.classic.Level@DEBUG)'
复制代码再次获取UserController@logger，可以发现已经是DEBUG了：
[arthas@2167]$ ognl --classLoaderClass org.springframework.boot.loader.LaunchedURLClassLoader '@com.example.demo.arthas.user.UserController@logger'
@Logger[
serialVersionUID=@Long[5454405123156820674],
FQCN=@String[ch.qos.logback.classic.Logger],
name=@String[com.example.demo.arthas.user.UserController],
level=@Level[DEBUG],
effectiveLevelInt=@Integer[10000],
parent=@Logger[Logger[com.example.demo.arthas.user]],
childrenList=null,
aai=null,
additive=@Boolean[true],
loggerContext=@LoggerContext[ch.qos.logback.classic.LoggerContext[default]],
]
复制代码1.4 修改logback的全局logger level
通过获取root logger，可以修改全局的logger level：
ognl --classLoaderClass org.springframework.boot.loader.LaunchedURLClassLoader '@org.slf4j.LoggerFactory@getLogger("root").setLevel(@ch.qos.logback.classic.Level@DEBUG)'
复制代码十二、案例: 排查logger冲突问题 :calendar:
在这个案例里，展示排查logger冲突的方法。
1.1 确认应用使用的logger系统
以UserController为例，它使用的是slf4j api，但实际使用到的logger系统是logback。
[arthas@2167]$ ognl --classLoaderClass org.springframework.boot.loader.LaunchedURLClassLoader '@com.example.demo.arthas.user.UserController@logger'
@Logger[
serialVersionUID=@Long[5454405123156820674],
FQCN=@String[ch.qos.logback.classic.Logger],
name=@String[com.example.demo.arthas.user.UserController],
level=@Level[DEBUG],
effectiveLevelInt=@Integer[10000],
parent=@Logger[Logger[com.example.demo.arthas.user]],
childrenList=null,
aai=null,
additive=@Boolean[true],
loggerContext=@LoggerContext[ch.qos.logback.classic.LoggerContext[default]],
]
复制代码1.2 获取logback实际加载的配置文件
[arthas@2167]$ ognl --classLoaderClass org.springframework.boot.loader.LaunchedURLClassLoader '#[email protected]@getLogger("root").loggerContext.objectMap, #map1.get("CONFIGURATION_WATCH_LIST")'
@ConfigurationWatchList[
mainURL=@URL[jar:file:/home/shell/demo-arthas-spring-boot.jar!/BOOT-INF/classes!/logback-spring.xml],
fileWatchList=@ArrayList[isEmpty=true;size=0],
lastModifiedList=@ArrayList[isEmpty=true;size=0],
noContextWarning=@Integer[0],
context=@LoggerContext[ch.qos.logback.classic.LoggerContext[default]],
declaredOrigin=@ConfigurationWatchList[ch.qos.logback.core.joran.spi.ConfigurationWatchList@3a34751e],
]
复制代码1.3 使用classloader命令查找可能存在的logger配置文件
[arthas@2167]$ classloader --classLoaderClass org.springframework.boot.loader.LaunchedURLClassLoader -r logback-spring.xml
jar:file:/home/shell/demo-arthas-spring-boot.jar!/BOOT-INF/classes!/logback-spring.xml
Affect(row-cnt:1) cost in 1 ms.
复制代码可以知道加载的配置的具体来源。
可以尝试加载容易冲突的文件：
[arthas@2167]$ classloader --classLoaderClass org.springframework.boot.loader.LaunchedURLClassLoader -r logback.xml
Affect(row-cnt:0) cost in 1 ms.
复制代码[arthas@2167]$ classloader --classLoaderClass org.springframework.boot.loader.LaunchedURLClassLoader -r log4j.properties
Affect(row-cnt:0) cost in 0 ms.
复制代码十三、案例: 获取Spring Context:kimono:
在这个案例里，展示获取spring context，再获取bean，然后调用函数。
1.1 使用tt命令获取到spring context
tt即 TimeTunnel，它可以记录下指定方法每次调用的入参和返回信息，并能对这些不同的时间下调用进行观测。
https://arthas.aliyun.com/doc/tt.html
复制代码访问：user/1
可以看到tt命令捕获到了一个请求：
[arthas@2167]$ tt -t org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter invokeHandlerMethod
Press Q or Ctrl+C to abort.
Affect(class count: 1 , method count: 1) cost in 115 ms, listenerId: 5
INDEX TIMESTAMP COST(ms) IS-RET IS-EX OBJECT CLASS METHOD
P
-----------------------------------------------------------------------------------------------------------------------------
1000 2022-01-27 11:26: 4.8384 true false 0x3b265475 RequestMappingHandlerAdapte invokeHandlerMethod
复制代码1.2使用tt命令从调用记录里获取到spring context
输入 q 或者 Ctrl + C 退出上面的 tt -t命令。
tt -i 1000 -w 'target.getApplicationContext()'
@AnnotationConfigEmbeddedWebApplicationContext[
reader=@AnnotatedBeanDefinitionReader[org.springframework.context.annotation.AnnotatedBeanDefinitionReader@2e457641],
scanner=@ClassPathBeanDefinitionScanner[org.springframework.context.annotation.ClassPathBeanDefinitionScanner@6eb38026],
annotatedClasses=null,
basePackages=null,
]
Affect(row-cnt:1) cost in 439 ms.
复制代码1.3 获取spring bean，并调用函数
[arthas@2167]$ tt -i 1000 -w 'target.getApplicationContext().getBean("helloWorldService").getHelloMessage()'
@String[Hello World]
Affect(row-cnt:1) cost in 7 ms.
复制代码十四、案例: 排查HTTP请求返回401:baby_chick:
在这个案例里，展示排查HTTP 401问题的技巧。
https://61000-dot-a5c4lpbtkr.shell.aliyuncs.com/admin
复制代码访问： admin
结果是：
Something went wrong: 401 Unauthorized
复制代码我们知道401通常是被权限管理的Filter拦截了，那么到底是哪个Filter处理了这个请求，返回了401？
1.1 跟踪所有的Filter函数
开始trace：
trace javax.servlet.Filter *
复制代码访问： admin
可以在调用树的最深层，找到AdminFilterConfig$AdminFilter返回了401：
+---[3.806273ms] javax.servlet.FilterChain:doFilter()
| `---[3.447472ms] com.example.demo.arthas.AdminFilterConfig$AdminFilter:doFilter()
| `---[0.17259ms] javax.servlet.http.HttpServletResponse:sendError()
复制代码1.2 通过stack获取调用栈
上面是通过trace命令来获取信息，从结果里，我们可以知道通过stack跟踪HttpServletResponse:sendError()，同样可以知道是哪个Filter返回了401
[arthas@2167]$ stack javax.servlet.http.HttpServletResponse sendError 'params[0]==401'
Press Q or Ctrl+C to abort.
Affect(class count: 3 , method count: 4) cost in 176 ms, listenerId: 7
ts=2022-01-27 11:33:39;thread_name=http-nio-61000-exec-6;id=15;is_daemon=true;priority=5;TCCL=org.springframework.boot.context.embedded.tomcat.TomcatEmbeddedWebappClassLoader@e2dab9b
@org.apache.catalina.connector.Response.sendError()
at org.apache.catalina.connector.ResponseFacade.sendError(ResponseFacade.java:462)
at com.example.demo.arthas.AdminFilterConfig$AdminFilter.doFilter(AdminFilterConfig.java:38)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:109)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:81)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:197)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:496)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:803)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:790)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1468)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
ts=2022-01-27 11:33:39;thread_name=http-nio-61000-exec-6;id=15;is_daemon=true;priority=5;TCCL=org.springframework.boot.context.embedded.tomcat.TomcatEmbeddedWebappClassLoader@e2dab9b
@org.apache.catalina.connector.ResponseFacade.sendError()
at com.example.demo.arthas.AdminFilterConfig$AdminFilter.doFilter(AdminFilterConfig.java:38)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:109)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:81)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:197)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:496)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:803)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:790)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1468)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
复制代码十四、案例: 排查HTTP请求返回404:honey_pot:
在这个案例里，展示排查HTTP 404问题的技巧。
访问： a.txt
结果是：
Something went wrong: 404 Not Found
复制代码那么到底是哪个Servlet处理了这个请求，返回了404？
1.1 跟踪所有的Servlet函数
trace javax.servlet.Servlet * > /tmp/servlet.txt
复制代码访问： a.txt
在Terminal 3里，查看/tmp/servlet.txt的内容：
cat /tmp/servlet.txt
复制代码/tmp/servlet.txt里的内容会比较多，需要耐心找到调用树里最长的地方。
可以发现请求最终是被freemarker处理的：
`---[13.974188ms] org.springframework.web.servlet.ViewResolver:resolveViewName(); +---[0.045561ms] javax.servlet.GenericServlet:<init>()
+---[min=0.045545ms,max=0.074342ms,total=0.119887ms,count=2] org.springframework.web.servlet.view.freemarker.FreeMarkerView$GenericServletAdapter:<init>()
+---[0.170895ms] javax.servlet.GenericServlet:init()
| `---[0.068578ms] javax.servlet.GenericServlet:init()
| `---[0.021793ms] javax.servlet.GenericServlet:init()
`---[0.164035ms] javax.servlet.GenericServlet:getServletContext()
复制代码十五、案例: 理解Spring Boot应用的ClassLoader结构:hourglass_flowing_sand:
下面介绍classloader命令的功能。
先访问一个jsp网页，触发jsp的加载： hello
1.1 列出所有ClassLoader
[arthas@2167]$ classloader -l
name loadedCount hash parent
BootstrapClassLoader 3748 null null
com.taobao.arthas.agent.ArthasClassloader@55948442 2628 55948442 sun.misc.Launcher$ExtClassLoader@7
d0587f1
java.net.FactoryURLClassLoader@7de451d6 841 7de451d6 sun.misc.Launcher$AppClassLoader@1
b6d3586
g.apache.jasper.servlet.JasperLoader@a9af914 1 a9af914 TomcatEmbeddedWebappClassLoader
context: ROOT
delegate: true
----------> Parent Classloader:
org.springframework.boot.loader.La
unchedURLClassLoader@19469ea2
0 e2dab9b org.springframework.boot.loader.La
unchedURLClassLoader@19469ea2
9ea2
org.springframework.boot.loader.LaunchedURLClassLoader@19469ea2 5325 19469ea2 sun.misc.Launcher$AppClassLoader@1
b6d3586
sun.misc.Launcher$AppClassLoader@1b6d3586 45 1b6d3586 sun.misc.Launcher$ExtClassLoader@7
d0587f1
sun.misc.Launcher$ExtClassLoader@7d0587f1 63 7d0587f1 null
Affect(row-cnt:8) cost in 14 ms.
复制代码- TomcatEmbeddedWebappClassLoader 加载的class数量是0，所以在spring boot embedded tomcat里，它只是一个空壳，所有的类加载都是
LaunchedURLClassLoader完成的
1.2 列出ClassLoader里加载的所有类
列出上面的org.apache.jasper.servlet.JasperLoader加载的类：
classloader -a --classLoaderClass apache.jasper.servlet.JasperLoader
复制代码classloader -a --classLoaderClass apache.jasper.servlet.JasperLoader
hash:1698045338, org.apache.jasper.servlet.JasperLoader@65361d9a
org.apache.jsp.jsp.hello_jsp
复制代码- 注：同ognl, 也可用
-c <hashcode>而不用--classLoaderClass指定
1.3 反编译jsp的代码
jad org.apache.jsp.jsp.hello_jsp
复制代码$ jad org.apache.jsp.jsp.hello_jsp
ClassLoader:
+-org.apache.jasper.servlet.JasperLoader@65361d9a
+-TomcatEmbeddedWebappClassLoader
context: ROOT
...
复制代码1.4 查看ClassLoader树
[arthas@1265]$ classloader -t
+-BootstrapClassLoader
+-sun.misc.Launcher$ExtClassLoader@7d0587f1
+-com.taobao.arthas.agent.ArthasClassloader@744a1259
+-sun.misc.Launcher$AppClassLoader@1b6d3586
+-org.springframework.boot.loader.LaunchedURLClassLoader@19469ea2
assLoader@19469ea2
+-org.apache.jasper.servlet.JasperLoader@7ca74939
Affect(row-cnt:7) cost in 20 ms.
复制代码注意：请使用你的classLoaderHash值覆盖 <classLoaderHash> ，然后手动执行下面相关命令
1.5 列出ClassLoader的urls
比如上面查看到的spring LaunchedURLClassLoader的 hashcode是19469ea2 ，可以通过-c或者--classLoaderClass参数来列出它的所有urls：
classloader --classLoaderClass org.springframework.boot.loader.LaunchedURLClassLoader
复制代码1.6 加载指定ClassLoader里的资源文件
查找指定的资源文件： classloader --classLoaderClass org.springframework.boot.loader.LaunchedURLClassLoader -r logback-spring.xml
[arthas@1265]$ classloader --classLoaderClass org.springframework.boot.loader.LaunchedURLClassLoader -r logback-spring.xml
jar:file:/home/shell/demo-arthas-spring-boot.jar!/BOOT-INF/classes!/logback-spring.xml
Affect(row-cnt:1) cost in 1 ms.
复制代码1.7尝试加载指定的类
比如用上面的spring LaunchedURLClassLoader 尝试加载 java.lang.String ：
[arthas@1265]$ classloader --classLoaderClass org.springframework.boot.loader.LaunchedURLClassLoader --load java.lang.String
load class success.
class-info java.lang.String
code-source
name java.lang.String
isInterface false
isAnnotation false
isEnum false
isAnonymousClass false
isArray false
isLocalClass false
isMemberClass false
isPrimitive false
isSynthetic false
simple-name String
modifier final,public
annotation
interfaces java.io.Serializable,java.lang.Comparable,java.lang.CharSequence
super-class +-java.lang.Object
class-loader
classLoaderHash null
复制代码十六、案例：查找Top N线程 :hammer_and_pick:
1.1 查看所有线程信息
[arthas@1265]$ thread
Threads Total: 35, NEW: 0, RUNNABLE: 11, BLOCKED: 0, WAITING: 14, TIMED_WAITING: 5, TERMINATED: 0, Internal threads: 5
ID NAME GROUP PRIORITY STATE %CPU DELTA_TIME TIME INTERRUPTE DAEMON
-1 C2 CompilerThread0 - -1 - 14.07 0.029 0:8.038 false true
-1 C1 CompilerThread1 - -1 - 1.02 0.002 0:2.718 false true
44 arthas-command-execute system 5 RUNNABLE 0.56 0.001 0:0.828 false true
-1 VM Periodic Task Thread - -1 - 0.11 0.000 0:0.432 false true
-1 VM Thread - -1 - 0.07 0.000 0:0.725 false true
28 http-nio-61000-AsyncTimeout main 5 TIMED_WAI 0.02 0.000 0:0.035 false true
2 Reference Handler system 10 WAITING 0.0 0.000 0:0.009 false true
3 Finalizer system 8 WAITING 0.0 0.000 0:0.018 false true
4 Signal Dispatcher system 9 RUNNABLE 0.0 0.000 0:0.001 false true
31 Attach Listener system 9 RUNNABLE 0.0 0.000 0:0.006 false true
33 arthas-timer system 9 WAITING 0.0 0.000 0:0.000 false true
36 arthas-NettyHttpTelnetBootstra system 5 RUNNABLE 0.0 0.000 0:0.024 false true
37 arthas-NettyWebsocketTtyBootst system 5 RUNNABLE 0.0 0.000 0:0.001 false true
38 arthas-NettyWebsocketTtyBootst system 5 RUNNABLE 0.0 0.000 0:0.001 false true
39 arthas-shell-server system 9 TIMED_WAI 0.0 0.000 0:0.002 false true
40 arthas-session-manager system 9 TIMED_WAI 0.0 0.000 0:0.001 false true
41 arthas-UserStat system 9 WAITING 0.0 0.000 0:0.000 false true
43 arthas-NettyHttpTelnetBootstra system 5 RUNNABLE 0.0 0.000 0:0.238 false true
13 ContainerBackgroundProcessor[S main 5 TIMED_WAI 0.0 0.000 0:0.007 false true
14 container-0 main 5 TIMED_WAI 0.0 0.000 0:0.002 false false
15 NioBlockingSelector.BlockPolle main 5 RUNNABLE 0.0 0.000 0:0.027 false true
16 http-nio-61000-exec-1 main 5 WAITING 0.0 0.000 0:1.055 false true
17 http-nio-61000-exec-2 main 5 WAITING 0.0 0.000 0:0.025 false true
18 http-nio-61000-exec-3 main 5 WAITING 0.0 0.000 0:0.000 false true
19 http-nio-61000-exec-4 main 5 WAITING 0.0 0.000 0:0.000 false true
20 http-nio-61000-exec-5 main 5 WAITING 0.0 0.000 0:0.000 false true
21 http-nio-61000-exec-6 main 5 WAITING 0.0 0.000 0:0.000 false true
22 http-nio-61000-exec-7 main 5 WAITING 0.0 0.000 0:0.000 false true
23 http-nio-61000-exec-8 main 5 WAITING 0.0 0.000 0:0.000 false true
24 http-nio-61000-exec-9 main 5 WAITING 0.0 0.000 0:0.000 false true
25 http-nio-61000-exec-10 main 5 WAITING 0.0 0.000 0:0.000 false true
26 http-nio-61000-ClientPoller-0 main 5 RUNNABLE 0.0 0.000 0:0.039 false true
27 http-nio-61000-Acceptor-0 main 5 RUNNABLE 0.0 0.000 0:0.006 false true
30 DestroyJavaVM main 5 RUNNABLE 0.0 0.000 0:3.569 false false
-1 Service Thread - -1 - 0.0 0.000 0:0.000 false true
复制代码1.2 查看具体线程的栈
查看线程ID 16的栈：
[arthas@1265]$ thread 16
"http-nio-61000-exec-1" Id=16 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@42ab2c2f
at sun.misc.Unsafe.park(Native Method)
- waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@42ab2c2f
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:103)
at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:31)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
复制代码1.3 查看CPU使用率top n线程的栈
[arthas@1265]$ thread -n 3
"C1 CompilerThread1" [Internal] cpuUsage=0.8% deltaTime=1ms time=2743ms
"C2 CompilerThread0" [Internal] cpuUsage=0.43% deltaTime=0ms time=8072ms
"arthas-command-execute" Id=44 cpuUsage=0.34% deltaTime=0ms time=839ms RUNNABLE
at sun.management.ThreadImpl.dumpThreads0(Native Method)
at sun.management.ThreadImpl.getThreadInfo(ThreadImpl.java:461)
at com.taobao.arthas.core.command.monitor200.ThreadCommand.processTopBusyThreads(ThreadCommand.java:206)
at com.taobao.arthas.core.command.monitor200.ThreadCommand.process(ThreadCommand.java:122)
at com.taobao.arthas.core.shell.command.impl.AnnotatedCommandImpl.process(AnnotatedCommandImpl.java:82)
at com.taobao.arthas.core.shell.command.impl.AnnotatedCommandImpl.access$100(AnnotatedCommandImpl.java:18)
at com.taobao.arthas.core.shell.command.impl.AnnotatedCommandImpl$ProcessHandler.handle(AnnotatedCommandImpl.java:111)
at com.taobao.arthas.core.shell.command.impl.AnnotatedCommandImpl$ProcessHandler.handle(AnnotatedCommandImpl.java:108)
at com.taobao.arthas.core.shell.system.impl.ProcessImpl$CommandProcessTask.run(ProcessImpl.java:385)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
复制代码1.4 查看5秒内的CPU使用率top n线程栈
[arthas@1265]$ thread -n 3 -i 5000
"C2 CompilerThread0" [Internal] cpuUsage=0.14% deltaTime=7ms time=8081ms
"VM Periodic Task Thread" [Internal] cpuUsage=0.08% deltaTime=4ms time=523ms
"C1 CompilerThread1" [Internal] cpuUsage=0.03% deltaTime=1ms time=2748ms
复制代码1.5 查找线程是否有阻塞
[arthas@1265]$ thread -b
No most blocking thread found!
复制代码十七、Web Console :hamster:
Arthas支持通过Web Socket来连接。
Arthas支持通过Web Socket来连接。
本地体验
当在本地启动时，可以访问 http://127.0.0.1:8563/ ，通过浏览器来使用Arthas。
十八、Exit/Stop :hamburger:
1.1 reset
Arthas在 watch/trace 等命令时，实际上是修改了应用的字节码，插入增强的代码。显式执行 reset 命令，可以清除掉这些增强代码。
reset
复制代码1.2 退出Arthas
用 exit 或者 quit 命令可以退出Arthas。
exit
复制代码Ctrl+C
退出Arthas之后，还可以再次用 java -jar arthas-boot.jar 来连接。
java -jar arthas-boot.jar
复制代码Ctrl+C
彻底退出Arthas
exit/quit命令只是退出当前session，arthas server还在目标进程中运行。
想完全退出Arthas，可以执行 stop 命令。
stop
复制代码十九、arthas-boot支持的参数:kiwi_fruit:
arthas-boot.jar 支持很多参数，可以执行 java -jar arthas-boot.jar -h 来查看。
java -jar arthas-boot.jar -h
复制代码1.1 允许外部访问
默认情况下， arthas server侦听的是 127.0.0.1 这个IP，如果希望远程可以访问，可以使用--target-ip的参数。
java -jar arthas-boot.jar --target-ip
复制代码1.2 列出所有的版本
java -jar arthas-boot.jar --versions
复制代码使用指定版本：
java -jar arthas-boot.jar --use-version 3.1.0
复制代码1.3 只侦听Telnet端口，不侦听HTTP端口
java -jar arthas-boot.jar --telnet-port 9999 --http-port -1
复制代码1.4 打印运行的详情
java -jar arthas-boot.jar -v
复制代码