1. Function introduction
In our daily development, if there is an abnormality in the online environment of the system, the developer cannot know it in time to repair it, which may cause heavy losses. Therefore, it is very necessary to add the function of abnormal alarm to the back-end service, and to develop a function A comprehensive abnormal alarm service may take a long period of time. Today, I will bring you a method to realize abnormal log alarm based on Slack.
Implementation logic: Under normal circumstances, the code will deal with the places where exceptions occur. The most basic thing is to print the log. In this article, when the log is printed, the exception information will be sent to the Slack channel at the same time. Development or operation and maintenance personnel Create a Slack account, join a channel, and receive real-time alerts of abnormal information.
2. Introduction to Slack
Slack is a web-based real-time communication tool available as a single app for desktop/laptop, mobile, as well as a web application. Basically, it's your private chat and collaboration room. For many companies, it has replaced email/private forums/chat rooms as the primary internal text-based communication channel. It can be understood that it is a chat group + large-scale tool integration + file integration + unified search . As of the end of 2014, Slack has integrated 65 tools and services such as email, SMS, Google Drives, Twitter, Trello, Asana, GitHub, etc., which can bring together various fragmented enterprise communication and collaboration. Several important concepts:
Workspaces : Rather go to workspaces, users can join or create different workspaces, many times the name and URL of the workspace will be the company name.
Channels : Channels can be divided into different teams or topics, and can also be understood as equivalent to WeChat, where members in a channel share information in the channel.
3. Preliminary preparation
slack configuration
- Create an account, log in, you can use the app or log in to the web version with a browser
- Create your own workspace and invite others to your workspace.
- Create a channel, invite colleagues to join, you can send information to the channel at this time, and everyone who joins the channel can see the information
- Add the application Incoming WebHook to the workspace , select the channel, save the Webhook URL, and then send messages to the channel through the Webhook implementation program.
pom.xml
<dependencies>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.5.2</version>
</dependency>
<dependency>
<groupId>com.alibaba</groupId>
<artifactId>fastjson</artifactId>
<version>1.2.83</version>
</dependency>
<dependency>
<groupId>commons-configuration</groupId>
<artifactId>commons-configuration</artifactId>
<version>1.10</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.10</version>
<scope>test</scope>
</dependency>
</dependencies>
复制代码Fourth, the specific implementation
1. Implement Slack to send messages
SlackUtil message tool class for Slack
package com.yy.operation;
import com.yy.common.CommonThreadFactory;
import com.yy.common.ConnUtil;
import org.apache.commons.lang.StringUtils;
import java.text.MessageFormat;
import java.text.SimpleDateFormat;
import java.util.concurrent.*;
import java.util.logging.Level;
import java.util.logging.Logger;
/**
* @author ：Max
* @date ：Created in 2022/8/26 下午12:54
* @description：
*/
public class SlackUtil {
private static final Logger logger = Logger.getLogger(SlackUtil.class.getCanonicalName());
private static SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
private static final String SEND_USER_NAME ="运维机器人";
private static int MAX_RETRY =3;
/**
* 线程池 抛弃策略DiscardPolicy：这种策略，会默默的把新来的这个任务给丢弃；不会得到通知
*/
private static ExecutorService executor = new ThreadPoolExecutor(10,30,60,TimeUnit.MILLISECONDS,new ArrayBlockingQueue<Runnable>(200),new CommonThreadFactory("Slack"), new ThreadPoolExecutor.DiscardPolicy());
private static String MSG_FORMAT ="payload='{'"channel": "{0}", "username": "{1}", "text": "{2}", "icon_emoji": ":ghost:"'}'" ;
/**
* 保存的Webhook URL ，需要初始化
*/
private static String WEBHOOK_URL ;
private static boolean SLACK_ABLE;
public static void setSlackConfig(String webhookUrl){
WEBHOOK_URL = webhookUrl;
SLACK_ABLE = true;
}
/**
* slack异步发消息，保证不能影响到主功能
* @param channel
* @param msg
*/
public static void send(final String channel, final String msg){
if(!SLACK_ABLE){
return;
}
if(StringUtils.isBlank(msg)){
return;
}
executor.submit(new Runnable() {
@Override
public void run() {
try {
SlackUtil.send(channel,sdf.format(System.currentTimeMillis())+" "+msg,MAX_RETRY);
} catch (Exception e) {
logger.log(Level.SEVERE, e.getMessage(), e);
}
}
});
}
/**
* 如果slask发消息失败，会最多尝试发三次，三次都失败，会打印异常信息
* @param channel
* @param msg
* @param retry
* @throws Exception
*/
public static void send(String channel, String msg, int retry) throws Exception {
if(msg.indexOf(""")>=0 ||msg.indexOf("{")>=0 ||msg.indexOf("}")>=0){
msg =msg.replace(""","'").replace("{","[").replace("}","]");
}
String payload = MessageFormat.format(MSG_FORMAT, channel,SEND_USER_NAME,msg);
String result = ConnUtil.getContentByPostWithUrlencode(WEBHOOK_URL,payload);
logger.info("result:"+result);
if(StringUtils.isEmpty(result) ||!result.startsWith("ok")){
--retry;
if(retry>0){
try {
TimeUnit.SECONDS.sleep(retry*5);
} catch (InterruptedException e) {
e.printStackTrace();
}
send(channel,msg,retry);
}else{
throw new Exception("Fail to send slack:"+result+"\nmsg:"+msg);
}
}
}
}
复制代码Make a request to the webhook through Urlencode
package com.yy.common;
import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.entity.StringEntity;
import org.apache.http.impl.client.HttpClientBuilder;
import java.io.BufferedReader;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.util.logging.Level;
import java.util.logging.Logger;
/**
* @author ：Max
* @date ：Created in 2022/8/26 下午1:44
* @description：
*/
public class ConnUtil {
private static final Logger logger = Logger.getLogger(ConnUtil.class.getCanonicalName());
public static String getContentByPostWithUrlencode(String url,String msg){
StringEntity entity = new StringEntity(msg, "UTF-8");
entity.setContentEncoding("UTF-8");
entity.setContentType(" application/x-www-form-urlencoded");
HttpClient httpClient = HttpClientBuilder.create().build();
HttpPost request = new HttpPost(url);
request.setEntity(entity);
HttpResponse response = null;
try {
response = httpClient.execute(request);
HttpEntity responseEntity = response.getEntity();
if (responseEntity != null) {
InputStream instream = responseEntity.getContent();
BufferedReader reader = new BufferedReader(new InputStreamReader(instream));
StringBuffer contents = new StringBuffer();
String line = null;
while ((line = reader.readLine()) != null) {
contents.append(line);
contents.append("\n");
}
return contents.toString();
}
} catch (Exception ex) {
logger.log(Level.SEVERE, ex.getMessage(), ex);
}
return null;
}
}
复制代码SlackUtil test
package com.yy.test;
import com.yy.common.SlackChannelEnum;
import com.yy.operation.SlackUtil;
import org.junit.Assert;
import org.junit.Test;
import java.util.concurrent.TimeUnit;
/**
* @author ：Max
* @date ：Created in 2022/8/28 下午2:37
* @description：
*/
public class SlackTest {
static {
SlackUtil.setSlackConfig("https://hooks.slack.com/services/*******");
}
@Test
public void test(){
SlackUtil.send(SlackChannelEnum.EXCEPTION.channel,"test ~");
try {
TimeUnit.MINUTES.sleep(1);
} catch (InterruptedException e) {
e.printStackTrace();
}
Assert.assertTrue(true);
}
}
复制代码Send successfully, you can see the information in the channel
2. Rewrite the print log class
Common exception log processing
public class LoggerTest {
private static final Logger logger = Logger.getLogger(LoggerTest.class.getCanonicalName());
@Test
public void test() {
try {
int i = 1 / 0;
} catch (Exception e) {
logger.log(Level.SEVERE, e.getMessage(), e);
}
}
}
复制代码Override the method of encapsulating the print log
package com.yy.operation;
import com.yy.common.SlackChannelEnum;
import org.apache.commons.lang.StringUtils;
import java.io.PrintWriter;
import java.io.StringWriter;
import java.net.Inet4Address;
import java.net.InetAddress;
import java.text.MessageFormat;
import java.util.logging.Level;
import java.util.logging.LogRecord;
import java.util.logging.Logger;
/**
* @author Max
* @date ：Created in 2022/8/4 下午5:14
* @description：
*/
public class CommonLogger {
private Logger logger;
private CommonLogger(String className) {
logger = Logger.getLogger(className);
}
private static String SERVER;
private static String EXCEPTION_ALARM_FORMAT = "EXCEPTION 发生异常！\n环境 :{0}\n信息 :{1}\n详情 :{2}";
private static String WARNING_ALARM_FORMAT = "WARNING 发生告警！\n环境 :{0}\n信息 :{1}";
private static String SEVERE_ALARM_FORMAT = "SEVERE 发生告警！\n环境 :{0}\n信息 :{1}";
private static String LOG_ALARM_FORMAT = "LOG 发生告警！\n环境 :{0}\n信息 :{1}";
private static String USER_BEHAVIOR_FORMAT = "CUSTOMER \n环境 :{0}\n信息 :{1}";
static {
try{
InetAddress ip4 = Inet4Address.getLocalHost();
SERVER = ip4.getHostAddress();
}catch (Exception e){
SERVER ="undefined server";
}
}
public static CommonLogger getLogger(String name) {
return new CommonLogger(name);
}
/**
* Print exception information, send slack
*
* @param level
* @param msg
* @param e
*/
public void log(Level level, String msg, Throwable e) {
if(StringUtils.isBlank(msg)){
return;
}
msg =dolog(level,msg, e);
msg = MessageFormat.format(EXCEPTION_ALARM_FORMAT, SERVER, formatMsg(msg), getErrmessage(e));
SlackUtil.send(SlackChannelEnum.EXCEPTION.channel, msg);
}
/**
* Print user behavior information, send slack
*
* @param msg
*/
public void userBehaviorInfo(String msg) {
if(StringUtils.isBlank(msg)){
return;
}
msg =dolog(Level.INFO,msg);
msg = MessageFormat.format(USER_BEHAVIOR_FORMAT, SERVER, formatMsg(msg));
SlackUtil.send(SlackChannelEnum.EXCEPTION.channel, msg);
}
public String formatMsg(String msg){
StringBuilder source =new StringBuilder(logger.getName());
msg=transferMsgSource(source,msg);
return source.toString()+" "+msg;
}
/**
* Print warning severe information, send slack
*
* @param msg
*/
public void severe(String msg) {
if(StringUtils.isBlank(msg)){
return;
}
msg = dolog(Level.SEVERE,msg);
msg = MessageFormat.format(SEVERE_ALARM_FORMAT, SERVER, formatMsg(msg));
SlackUtil.send(SlackChannelEnum.EXCEPTION.channel, msg);
}
/**
* Print warning severe information, send slack
*
* @param msg
*/
public void warning(String msg) {
if(StringUtils.isBlank(msg)){
return;
}
msg = dolog(Level.WARNING,msg);
msg = MessageFormat.format(WARNING_ALARM_FORMAT, SERVER, formatMsg(msg));
SlackUtil.send(SlackChannelEnum.EXCEPTION.channel, msg);
}
/**
* Print warning log information, send slack
*
* @param msg
*/
public void log(Level severe, String msg) {
if(StringUtils.isBlank(msg)){
return;
}
msg =dolog(severe,msg);
msg = MessageFormat.format(LOG_ALARM_FORMAT, SERVER, formatMsg(msg));
SlackUtil.send(SlackChannelEnum.EXCEPTION.channel, msg);
}
public static String getErrmessage(Throwable t) {
return getThrowable(t);
}
public void info(String msg) {
dolog(Level.INFO,msg);
}
public void fine(String msg) {
logger.fine(msg);
}
public void setLevel(Level level) {
logger.setLevel(level);
}
public String dolog(Level level, String msg) {
return dolog(level,msg,null);
}
/**
*
* @param level
* @param msg
* @param thrown
* @return msg="["+currentThread.getName()+"] "+a.getMethodName()+" "+msg;
*/
public String dolog(Level level, String msg, Throwable thrown) {
LogRecord lr = new LogRecord(level, msg);
lr.setLevel(level);
if(thrown!=null){
lr.setThrown(thrown);
}
Thread currentThread = Thread.currentThread();
StackTraceElement[] temp=currentThread.getStackTrace();
StackTraceElement a=(StackTraceElement)temp[3];
lr.setThreadID((int) currentThread.getId());
lr.setSourceClassName(logger.getName());
lr.setSourceMethodName(a.getMethodName());
lr.setLoggerName(logger.getName());
logger.log(lr);
return "["+currentThread.getName()+"] "+a.getMethodName()+" "+msg;
}
public static String getThrowable(Throwable e) {
String throwable = "";
if (e != null) {
StringWriter sw = new StringWriter();
PrintWriter pw = new PrintWriter(sw);
pw.println();
e.printStackTrace(pw);
pw.close();
throwable = sw.toString();
}
return throwable;
}
public static String transferMsgSource(StringBuilder source,String msg){
if(msg.indexOf(" ")>0){
String threadName = msg.substring(0,msg.indexOf(" "))+ " ";
msg=msg.substring(threadName.length());
source.insert(0,threadName);
if(msg.indexOf(" ")>0) {
String method = msg.substring(0, msg.indexOf(" "));
source.append( "." + method);
msg = msg.substring(method.length()+1);
}
}
return msg;
}
}
复制代码package com.yy.operation;
import java.text.MessageFormat;
import java.util.concurrent.ConcurrentHashMap;
import java.util.logging.Level;
import java.util.logging.Logger;
public class LoggerUtil {
private static Logger curLogger = Logger.getLogger(LoggerUtil.class.getCanonicalName());
private static ConcurrentHashMap<String, CommonLogger> loggers = new ConcurrentHashMap<String, CommonLogger>();
public static CommonLogger getLogger(Class<?> clazz) {
String className = clazz.getCanonicalName();
CommonLogger logger = loggers.get(className);
if (logger == null) {
logger = CommonLogger.getLogger(className);
curLogger.fine(MessageFormat.format("Register logger for {0}", className));
loggers.put(className, logger);
}
return logger;
}
}
复制代码test log class
When the log class is changed, the called code does not need to be changed, and the abnormal alarm function is integrated at a small cost.
public class LoggerTest {
private static final Logger logger = Logger.getLogger(LoggerTest.class.getCanonicalName());
@Test
public void test() {
try {
int i = 1 / 0;
} catch (Exception e) {
logger.log(Level.SEVERE, e.getMessage(), e);
}
}
}
复制代码Test results, abnormal information printed in the channel, which is convenient for development, operation and maintenance personnel to locate
Fifth, optimize the expansion idea
- It can not only print abnormal logs, but also print some key behaviors of users, such as recharge, etc. Multiple channels can be set to send messages with different topics
- Thread pool can be optimized
- If developers can't view slack in time, they can also integrate email. Mailclark application (charged separately) can be added to Slack. After configuration, the information in the activation channel can be automatically sent to any mailbox, and the recipient does not need to create a slack account. For specific configuration, please refer to the link .
Everyone is welcome to give optimization suggestions, and exchange a lot!
other code
package com.yy.common;
import java.util.concurrent.ThreadFactory;
import java.util.concurrent.atomic.AtomicInteger;
/**
* @author ：Max
* @date ：Created in 2022/8/26 下午1:51
* @description：
*/
public class CommonThreadFactory implements ThreadFactory {
private static final AtomicInteger poolNumber = new AtomicInteger(1);
private final ThreadGroup group;
private final AtomicInteger threadNumber = new AtomicInteger(1);
private final String threadNamePrefix;
private final String nameSpecific;
private final boolean isDaemon;
public CommonThreadFactory(String nameSpecific) {
this(nameSpecific, false);
}
public CommonThreadFactory(String nameSpecific, boolean isDaemon) {
SecurityManager s = System.getSecurityManager();
this.group = (s != null) ? s.getThreadGroup() :
Thread.currentThread().getThreadGroup();
this.threadNamePrefix = "eg-pool-" + poolNumber.getAndIncrement() + "-thread";
this.nameSpecific = nameSpecific;
this.isDaemon = isDaemon;
}
@Override
public Thread newThread(Runnable r) {
Thread t = new Thread(group, r, String.format("%s-%d-%s",
this.threadNamePrefix, threadNumber.getAndIncrement(), this.nameSpecific), 0);
t.setDaemon(isDaemon);
t.setPriority(Thread.NORM_PRIORITY);
return t;
}
}
复制代码public enum SlackChannelEnum {
EXCEPTION("#test-example");
public String channel;
SlackChannelEnum(String channel) {
this.channel = channel;
}
}
复制代码