2023 Dark Horse Headlines. Microservice Project. Learning Notes (4)

We media articles - automatic review

Today's content introduction

1. Automatic review process of self-media articles

article data flow

Process:

The process of automatic review is as follows:

1 After publishing the article from the media side, start reviewing the article

2 The review is mainly to review the content of the article (text content and pictures)

3 Review the text with the help of the interface provided by the third party

4 Use the interface provided by the third party to review the pictures. Since the pictures are stored in minIO, they need to be downloaded before they can be reviewed.

5 If the audit fails, you need to modify the status of the self-media article, status: 2 audit failed status: 3 go to manual audit

6 If the review is successful, you need to create the articles required by the app side in the article microservice

2. Content security third-party interface

2.1 Overview

Content security is an identification service that supports diversified scene detection for objects such as pictures, videos, texts, and voices, effectively reducing the risk of content violations.

At present, many platforms support content detection. For example, Alibaba Cloud, Tencent Cloud, Baidu AI, NetEase Cloud and other large domestic Internet companies have provided APIs to the outside world.

In terms of performance and fees, the Dark Horse Toutiao project uses the content security interface of Alibaba Cloud, which uses the review of pictures and texts.

Aliyun charging standard: https://www.aliyun.com/price/product/?spm=a2c4g.11186623.2.10.4146401eg5oeu8#/lvwang/detail

2.2 Preparations

Before using the content detection API, you need to register an Alibaba Cloud account, add an Access Key, and sign a contract with Cloud Shield Content Security.

Steps

1. Go to Alibaba Cloud official website to register an account. If you already have a registered account, please skip this step.

   After entering the Aliyun homepage, if you do not have an Aliyun account, you need to register first before you can log in. Since registration is relatively simple, courses and handouts are not reflected (registration can be done in a variety of ways, such as Taobao account, Alipay account, Weibo account, etc...).

   Real-name authentication and biometric authentication are required.

2. Open the cloud shield content security product trial page , click Activate now to officially activate the service.

Content Security Console

3. Manage your AccessKeyID and AccessKeySecret on the AccessKey management page .

Manage your own AccessKey, you can create and delete AccessKey

Check your AccessKey,

The AccessKey is hidden by default. You can save the AccessKey when applying for the first time, click to display it, and you can view it after verifying the mobile phone number.

2.3 Text content review interface

Text spam detection: https://help.aliyun.com/document_detail/70439.html?spm=a2c4g.11186623.6.659.35ac3db3l0wV5k

Text spam Java SDK: https://help.aliyun.com/document_detail/53427.html?spm=a2c4g.11186623.6.717.466d7544QbU8Lr

2.4 Picture review interface

Image spam detection: https://help.aliyun.com/document_detail/70292.html?spm=a2c4g.11186623.6.616.5d7d1e7f9vDRz4

Image Spam Java SDK: https://help.aliyun.com/document_detail/53424.html?spm=a2c4g.11186623.6.715.c8f69b12ey35j4

2.5 Project Integration

add dependencies

<dependency>
    <groupId>com.aliyun</groupId>
    <artifactId>aliyun-java-sdk-core</artifactId>
    <version>4.1.1</version>
</dependency>
<dependency>
    <groupId>com.aliyun</groupId>
    <artifactId>aliyun-java-sdk-green</artifactId>
    <version>3.6.6</version>
</dependency>
<dependency>
    <groupId>com.alibaba.fastjson2</groupId>
    <artifactId>fastjson2</artifactId>
    <version>2.0.9</version>
</dependency>
<dependency>
    <groupId>com.aliyun.oss</groupId>
    <artifactId>aliyun-sdk-oss</artifactId>
    <version>2.8.3</version>
</dependency>

①: Copy the class in the data folder to the common module and add it to the automatic configuration

Including GreenImageScan and GreenTextScan and corresponding tool classes

added to autoconfiguration

②: accessKeyId and secret (need to apply by yourself)

Add the following configuration to the nacos configuration center in heima-leadnews-wemedia:

aliyun:
 accessKeyId: LTAI5tCWHCcfvqQzu8k2oKmX
 secret: auoKUFsghimbfVQHpy7gtRyBkoR4vc
#aliyun.scenes=porn,terrorism,ad,qrcode,live,logo
 scenes: terrorism

③: Inject the bean of the audit text and picture into the test class in the self-media microservice for testing

package com.heima.wemedia;

import com.heima.common.aliyun.GreenImageScan;
import com.heima.common.aliyun.GreenTextScan;
import com.heima.file.service.FileStorageService;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.junit4.SpringRunner;

import java.util.Arrays;
import java.util.Map;

@SpringBootTest(classes = WemediaApplication.class)
@RunWith(SpringRunner.class)
public class AliyunTest {
    
    

    @Autowired
    private GreenTextScan greenTextScan;

    @Autowired
    private GreenImageScan greenImageScan;

    @Autowired
    private FileStorageService fileStorageService;

    @Test
    public void testScanText() throws Exception {
    
    
        Map map = greenTextScan.greeTextScan("我是一个好人,冰毒");
        System.out.println(map);
    }

    @Test
    public void testScanImage() throws Exception {
    
    
        byte[] bytes = fileStorageService.downLoadFile("http://192.168.200.130:9000/leadnews/2021/04/26/ef3cbe458db249f7bd6fb4339e593e55.jpg");
        Map map = greenImageScan.imageScan(Arrays.asList(bytes));
        System.out.println(map);
    }
}

3. App side article saving interface

3.1 Table Structure Description

ap_article article information table

ap_article_config article configuration table

ap_article_content Article content table

3.2 Distributed id

As the business grows, the article table may take up a lot of physical storage space. In order to solve this problem, the database fragmentation technology will be used later. Split a database and connect it through database middleware. If the ID auto-increment strategy is used for the table in the database, duplicate IDs may be generated. In this case, the distributed ID generation strategy should be used to generate IDs.

Distributed id-technology selection

Snowflake is Twitter's open-source distributed ID generation algorithm, and the result is a long-type ID. Its core idea is: use 41bit as the number of milliseconds, 10bit as the ID of the machine (5 bits for the data center, 5 bits for the machine ID), and 12bit as the serial number within milliseconds (meaning that each node can generate 4096 IDs), there is a sign bit at the end, which is always 0

The tables related to the article end use the snowflake algorithm to generate ids, including ap_article, ap_article_config, ap_article_content

mybatis-plus has integrated the snowflake algorithm, complete the following two steps to integrate the snowflake algorithm in the project

First: Add the following configuration to the id in the entity class, specifying the type as id_worker

@TableId(value = "id",type = IdType.ID_WORKER)
private Long id;

Second: Configure the data center id and machine id in the application.yml file

mybatis-plus:
  mapper-locations: classpath*:mapper/*.xml
  # 设置别名包扫描路径，通过该属性可以给包中的类注册别名
  type-aliases-package: com.heima.model.article.pojos
  global-config:
    datacenter-id: 1
    workerId: 1

datacenter-id: data center id (value range: 0-31)

workerId: machine id (value range: 0-31)

3.3 Thought Analysis

After the article review is successful, you need to add article data in the app's article library

1. Save the article information ap_article

2. Save the article configuration information ap_article_config

3. Save the article content ap_article_content

Implementation idea:

3.4 feign interface

	illustrate
interface path	/api/v1/article/save
request method	POST
parameter	ArticleDto
response result	ResponseResult

Proceed as follows:

ArticleDto

package com.heima.model.article.dtos;

import com.heima.model.article.pojos.ApArticle;
import lombok.Data;

@Data
public class ArticleDto  extends ApArticle {
    
    
    /**
     * 文章内容
     */
    private String content;
}

success:

{
    
    
  "code": 200,
  "errorMessage" : "操作成功",
  "data":"1302864436297442242"
 }

fail:

{
    
    
  "code":501,
  "errorMessage":"参数失效",
 }

{
    
    
  "code":501,
  "errorMessage":"文章没有找到",
 }

Function realization:

①: Add new interface in heima-leadnews-feign-api

First: first import feign's dependencies

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-openfeign</artifactId>
</dependency>

Second: Define the interface on the article side

package com.heima.apis.article;

import com.heima.model.article.dtos.ArticleDto;
import com.heima.model.common.dtos.ResponseResult;
import org.springframework.cloud.openfeign.FeignClient;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;

import java.io.IOException;


@FeignClient(value = "leadnews-article")
public interface IArticleClient {
    
    

    @PostMapping("/api/v1/article/save")
    public ResponseResult saveArticle(@RequestBody ArticleDto dto) ;
}

②: Implement this method in heima-leadnews-article

package com.heima.article.feign;

import com.heima.apis.article.IArticleClient;
import com.heima.article.service.ApArticleService;
import com.heima.model.article.dtos.ArticleDto;
import com.heima.model.common.dtos.ResponseResult;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.*;

import java.io.IOException;

@RestController
public class ArticleClient implements IArticleClient {
    
    

    @Autowired
    private ApArticleService apArticleService;

    @Override
    @PostMapping("/api/v1/article/save")
    public ResponseResult saveArticle(@RequestBody ArticleDto dto) {
    
    
        return apArticleService.saveArticle(dto);
    }

}

③: copy mapper

Copy the ApArticleConfigMapper class in the data folder to the mapper folder

At the same time, modify the ApArticleConfig class and add the following constructor

package com.heima.model.article.pojos;

import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableField;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
import lombok.Data;
import lombok.NoArgsConstructor;

import java.io.Serializable;

/**
 * <p>
 * APP已发布文章配置表
 * </p>
 *
 * @author itheima
 */

@Data
@NoArgsConstructor
@TableName("ap_article_config")
public class ApArticleConfig implements Serializable {
    
    


    public ApArticleConfig(Long articleId){
    
    
        this.articleId = articleId;
        this.isComment = true;
        this.isForward = true;
        this.isDelete = false;
        this.isDown = false;
    }

    @TableId(value = "id",type = IdType.ID_WORKER)
    private Long id;

    /**
     * 文章id
     */
    @TableField("article_id")
    private Long articleId;

    /**
     * 是否可评论
     * true: 可以评论   1
     * false: 不可评论  0
     */
    @TableField("is_comment")
    private Boolean isComment;

    /**
     * 是否转发
     * true: 可以转发   1
     * false: 不可转发  0
     */
    @TableField("is_forward")
    private Boolean isForward;

    /**
     * 是否下架
     * true: 下架   1
     * false: 没有下架  0
     */
    @TableField("is_down")
    private Boolean isDown;

    /**
     * 是否已删除
     * true: 删除   1
     * false: 没有删除  0
     */
    @TableField("is_delete")
    private Boolean isDelete;
}

④: Add a new method in ApArticleService

/**
     * 保存app端相关文章
     * @param dto
     * @return
     */
ResponseResult saveArticle(ArticleDto dto) ;

Implementation class:

@Autowired
private ApArticleConfigMapper apArticleConfigMapper;

@Autowired
private ApArticleContentMapper apArticleContentMapper;

/**
     * 保存app端相关文章
     * @param dto
     * @return
     */
@Override
public ResponseResult saveArticle(ArticleDto dto) {
    
    
    //1.检查参数
    if(dto == null){
    
    
        return ResponseResult.errorResult(AppHttpCodeEnum.PARAM_INVALID);
    }

    ApArticle apArticle = new ApArticle();
    BeanUtils.copyProperties(dto,apArticle);

    //2.判断是否存在id
    if(dto.getId() == null){
    
    
        //2.1 不存在id  保存  文章  文章配置  文章内容

        //保存文章
        save(apArticle);

        //保存配置
        ApArticleConfig apArticleConfig = new ApArticleConfig(apArticle.getId());
        apArticleConfigMapper.insert(apArticleConfig);

        //保存 文章内容
        ApArticleContent apArticleContent = new ApArticleContent();
        apArticleContent.setArticleId(apArticle.getId());
        apArticleContent.setContent(dto.getContent());
        apArticleContentMapper.insert(apArticleContent);

    }else {
    
    
        //2.2 存在id   修改  文章  文章内容

        //修改  文章
        updateById(apArticle);

        //修改文章内容
        ApArticleContent apArticleContent = apArticleContentMapper.selectOne(Wrappers.<ApArticleContent>lambdaQuery().eq(ApArticleContent::getArticleId, dto.getId()));
        apArticleContent.setContent(dto.getContent());
        apArticleContentMapper.updateById(apArticleContent);
    }


    //3.结果返回  文章的id
    return ResponseResult.okResult(apArticle.getId());
}

⑤: Test

Write junit unit tests, or use postman for testing

{
    
    
    "title":"黑马头条项目背景22222222222222",
    "authoId":1102,
    "layout":1,
    "labels":"黑马头条",
    "publishTime":"2028-03-14T11:35:49.000Z",
    "images": "http://192.168.200.130:9000/leadnews/2021/04/26/5ddbdb5c68094ce393b08a47860da275.jpg",
    "content":"22222222222222222黑马头条项目背景,黑马头条项目背景,黑马头条项目背景,黑马头条项目背景，黑马头条项目背景"
}

The test address is as follows: http://localhost:51802/api/v1/article/save
saved successfully

The database table ap_article was saved successfully

Let's try to modify it again

{
    
    
    "id":1676188422777188353,
    "title":"今日黑马头条项目",
    "authorId":1102,
    "layout":1,
    "labels":"黑马头条",
    "publishTime":"2023-07-04T11:35:49.000Z",
    "images": "http://192.168.200.130:9000/leadnews/2021/04/26/5ddbdb5c68094ce393b08a47860da275.jpg",
    "content":"22222222222222222黑马头条项目背景,黑马头条项目背景,黑马头条项目背景,黑马头条项目背景，黑马头条项目背景"
}

The modification is as follows:

we look at the table ap_article and the modification is successful

4. Implementation of the automatic review function of self-media articles

4.1 Table Structure Description

wm_news self-media article table

Status field: 0 Draft 1 Pending review 2 Review failed 3 Manual review 4 Manual review passed 8 Review passed (to be released) 9 Published

4.2 Implementation

New interface for service in heima-leadnews-wemedia

package com.heima.wemedia.service;

public interface WmNewsAutoScanService {
    
    

    /**
     * 自媒体文章审核
     * @param id  自媒体文章id
     */
    public void autoScanWmNews(Integer id);
}

Implementation class:

package com.heima.wemedia.service.impl;

import com.alibaba.fastjson.JSONArray;
import com.heima.apis.article.IArticleClient;
import com.heima.common.aliyun.GreenImageScan;
import com.heima.common.aliyun.GreenTextScan;
import com.heima.file.service.FileStorageService;
import com.heima.model.article.dtos.ArticleDto;
import com.heima.model.common.dtos.ResponseResult;
import com.heima.model.wemedia.pojos.WmChannel;
import com.heima.model.wemedia.pojos.WmNews;
import com.heima.model.wemedia.pojos.WmUser;
import com.heima.wemedia.mapper.WmChannelMapper;
import com.heima.wemedia.mapper.WmNewsMapper;
import com.heima.wemedia.mapper.WmUserMapper;
import com.heima.wemedia.service.WmNewsAutoScanService;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.lang3.StringUtils;
import org.springframework.beans.BeanUtils;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;

import java.util.*;
import java.util.stream.Collectors;


@Service
@Slf4j
@Transactional
public class WmNewsAutoScanServiceImpl implements WmNewsAutoScanService {
    
    

    @Autowired
    private WmNewsMapper wmNewsMapper;

    /**
     * 自媒体文章审核
     *
     * @param id 自媒体文章id
     */
    @Override
    public void autoScanWmNews(Integer id) {
    
    
        //1.查询自媒体文章
        WmNews wmNews = wmNewsMapper.selectById(id);
        if(wmNews == null){
    
    
            throw new RuntimeException("WmNewsAutoScanServiceImpl-文章不存在");
        }

        if(wmNews.getStatus().equals(WmNews.Status.SUBMIT.getCode())){
    
    
            //从内容中提取纯文本内容和图片
            Map<String,Object> textAndImages = handleTextAndImages(wmNews);

            //2.审核文本内容  阿里云接口
            boolean isTextScan = handleTextScan((String) textAndImages.get("content"),wmNews);
            if(!isTextScan)return;

            //3.审核图片  阿里云接口
            boolean isImageScan =  handleImageScan((List<String>) textAndImages.get("images"),wmNews);
            if(!isImageScan)return;

            //4.审核成功，保存app端的相关的文章数据
            ResponseResult responseResult = saveAppArticle(wmNews);
            if(!responseResult.getCode().equals(200)){
    
    
                throw new RuntimeException("WmNewsAutoScanServiceImpl-文章审核，保存app端相关文章数据失败");
            }
            //回填article_id
            wmNews.setArticleId((Long) responseResult.getData());
            updateWmNews(wmNews,(short) 9,"审核成功");

        }
    }

    @Autowired
    private IArticleClient articleClient;

    @Autowired
    private WmChannelMapper wmChannelMapper;

    @Autowired
    private WmUserMapper wmUserMapper;

    /**
     * 保存app端相关的文章数据
     * @param wmNews
     */
    private ResponseResult saveAppArticle(WmNews wmNews) {
    
    

        ArticleDto dto = new ArticleDto();
        //属性的拷贝
        BeanUtils.copyProperties(wmNews,dto);
        //文章的布局
        dto.setLayout(wmNews.getType());
        //频道
        WmChannel wmChannel = wmChannelMapper.selectById(wmNews.getChannelId());
        if(wmChannel != null){
    
    
            dto.setChannelName(wmChannel.getName());
        }

        //作者
        dto.setAuthorId(wmNews.getUserId().longValue());
        WmUser wmUser = wmUserMapper.selectById(wmNews.getUserId());
        if(wmUser != null){
    
    
            dto.setAuthorName(wmUser.getName());
        }

        //设置文章id
        if(wmNews.getArticleId() != null){
    
    
            dto.setId(wmNews.getArticleId());
        }
        dto.setCreatedTime(new Date());

        ResponseResult responseResult = articleClient.saveArticle(dto);
        return responseResult;

    }


    @Autowired
    private FileStorageService fileStorageService;

    @Autowired
    private GreenImageScan greenImageScan;

    /**
     * 审核图片
     * @param images
     * @param wmNews
     * @return
     */
    private boolean handleImageScan(List<String> images, WmNews wmNews) {
    
    

        boolean flag = true;

        if(images == null || images.size() == 0){
    
    
            return flag;
        }

        //下载图片 minIO
        //图片去重
        images = images.stream().distinct().collect(Collectors.toList());

        List<byte[]> imageList = new ArrayList<>();

        for (String image : images) {
    
    
            byte[] bytes = fileStorageService.downLoadFile(image);
            imageList.add(bytes);
        }


        //审核图片
        try {
    
    
            Map map = greenImageScan.imageScan(imageList);
            if(map != null){
    
    
                //审核失败
                if(map.get("suggestion").equals("block")){
    
    
                    flag = false;
                    updateWmNews(wmNews, (short) 2, "当前文章中存在违规内容");
                }

                //不确定信息  需要人工审核
                if(map.get("suggestion").equals("review")){
    
    
                    flag = false;
                    updateWmNews(wmNews, (short) 3, "当前文章中存在不确定内容");
                }
            }

        } catch (Exception e) {
    
    
            flag = false;
            e.printStackTrace();
        }
        return flag;
    }

    @Autowired
    private GreenTextScan greenTextScan;

    /**
     * 审核纯文本内容
     * @param content
     * @param wmNews
     * @return
     */
    private boolean handleTextScan(String content, WmNews wmNews) {
    
    

        boolean flag = true;

        if((wmNews.getTitle()+"-"+content).length() == 0){
    
    
            return flag;
        }

        try {
    
    
            Map map = greenTextScan.greeTextScan((wmNews.getTitle()+"-"+content));
            if(map != null){
    
    
                //审核失败
                if(map.get("suggestion").equals("block")){
    
    
                    flag = false;
                    updateWmNews(wmNews, (short) 2, "当前文章中存在违规内容");
                }

                //不确定信息  需要人工审核
                if(map.get("suggestion").equals("review")){
    
    
                    flag = false;
                    updateWmNews(wmNews, (short) 3, "当前文章中存在不确定内容");
                }
            }
        } catch (Exception e) {
    
    
            flag = false;
            e.printStackTrace();
        }

        return flag;

    }

    /**
     * 修改文章内容
     * @param wmNews
     * @param status
     * @param reason
     */
    private void updateWmNews(WmNews wmNews, short status, String reason) {
    
    
        wmNews.setStatus(status);
        wmNews.setReason(reason);
        wmNewsMapper.updateById(wmNews);
    }

    /**
     * 1。从自媒体文章的内容中提取文本和图片
     * 2.提取文章的封面图片
     * @param wmNews
     * @return
     */
    private Map<String, Object> handleTextAndImages(WmNews wmNews) {
    
    

        //存储纯文本内容
        StringBuilder stringBuilder = new StringBuilder();

        List<String> images = new ArrayList<>();

        //1。从自媒体文章的内容中提取文本和图片
        if(StringUtils.isNotBlank(wmNews.getContent())){
    
    
            List<Map> maps = JSONArray.parseArray(wmNews.getContent(), Map.class);
            for (Map map : maps) {
    
    
                if (map.get("type").equals("text")){
    
    
                    stringBuilder.append(map.get("value"));
                }

                if (map.get("type").equals("image")){
    
    
                    images.add((String) map.get("value"));
                }
            }
        }
        //2.提取文章的封面图片
        if(StringUtils.isNotBlank(wmNews.getImages())){
    
    
            String[] split = wmNews.getImages().split(",");
            images.addAll(Arrays.asList(split));
        }

        Map<String, Object> resultMap = new HashMap<>();
        resultMap.put("content",stringBuilder.toString());
        resultMap.put("images",images);
        return resultMap;

    }
}

4.3 Unit testing

package com.heima.wemedia.service;

import com.heima.wemedia.WemediaApplication;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.junit4.SpringRunner;

import static org.junit.Assert.*;


@SpringBootTest(classes = WemediaApplication.class)
@RunWith(SpringRunner.class)
public class WmNewsAutoScanServiceTest {
    
    

    @Autowired
    private WmNewsAutoScanService wmNewsAutoScanService;

    @Test
    public void autoScanWmNews() {
    
    

        wmNewsAutoScanService.autoScanWmNews(6238);
    }
}

Create a unit test for WmNewsAutoScanService.

The test parameters are as follows:

pass in id 6234

package com.heima.wemedia.service;

import com.heima.wemedia.WemediaApplication;
import org.junit.jupiter.api.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.autoconfigure.core.AutoConfigureCache;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.junit4.SpringRunner;

import static org.junit.jupiter.api.Assertions.*;

@SpringBootTest(classes = WemediaApplication.class)
@RunWith(SpringRunner.class)
class WmNewsAutoScanServiceTest {
    
    
    @Autowired
    private WmNewsAutoScanService wmNewsAutoScanService;

    @Test
    void autoScanWmNews() {
    
    
        wmNewsAutoScanService.autoScanWmNews(6234);
    }
}

After running, ap_article has one more piece of data

leadnews_wemedia.wm_news table audited successfully

4.4 feign remote interface calling method

The heima-leadnews-wemedia service has already relied on the heima-leadnews-feign-apis project, you only need to enable the remote call of feign in the self-media boot class

The annotation is: @EnableFeignClients(basePackages = "com.heima.apis")need to point to the apis package

4.5 Service downgrade processing

• Service degradation is a way for services to protect themselves, or to protect downstream services, to ensure that services will not become unavailable due to sudden spikes in requests and that services will not crash

• Although service degradation will cause the request to fail, it will not cause blocking.

Implementation steps:

①: Write downgrade logic in heima-leadnews-feign-api

package com.heima.apis.article.fallback;

import com.heima.apis.article.IArticleClient;
import com.heima.model.article.dtos.ArticleDto;
import com.heima.model.common.dtos.ResponseResult;
import com.heima.model.common.enums.AppHttpCodeEnum;
import org.springframework.stereotype.Component;

/**
 * feign失败配置
 * @author itheima
 */
@Component
public class IArticleClientFallback implements IArticleClient {
    
    
    @Override
    public ResponseResult saveArticle(ArticleDto dto)  {
    
    
        return ResponseResult.errorResult(AppHttpCodeEnum.SERVER_ERROR,"获取数据失败");
    }
}

Add classes in self-media microservices, scan packages for downgraded code classes

package com.heima.wemedia.config;

import org.springframework.context.annotation.ComponentScan;
import org.springframework.context.annotation.Configuration;

@Configuration
@ComponentScan("com.heima.apis.article.fallback")
public class InitConfig {
    
    
}

②: The remote interface points to the degraded code

package com.heima.apis.article;

import com.heima.apis.article.fallback.IArticleClientFallback;
import com.heima.model.article.dtos.ArticleDto;
import com.heima.model.common.dtos.ResponseResult;
import org.springframework.cloud.openfeign.FeignClient;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;

@FeignClient(value = "leadnews-article",fallback = IArticleClientFallback.class)
public interface IArticleClient {
    
    

    @PostMapping("/api/v1/article/save")
    public ResponseResult saveArticle(@RequestBody ArticleDto dto);
}

③: The client starts to downgrade heima-leadnews-wemedia

Add the following content in the nacos configuration center of wemedia, enable service downgrade, and also specify the timeout time of service response

feign:
  # 开启feign对hystrix熔断降级的支持
  hystrix:
    enabled: true
  # 修改调用超时时间
  client:
    config:
      default:
        connectTimeout: 2000
        readTimeout: 2000

④: Test

Add code to the saveArticle method in the ApArticleServiceImpl class.

The code is as follows:

try {
    
    
    Thread.sleep(3000);
} catch (InterruptedException e) {
    
    
    e.printStackTrace();
}

If the audit test is performed on the self-media side, there will be a phenomenon of service degradation.
Just start the article micro-service. We use the data with id = 6232 to test. Put

a breakpoint in WmNewsAutoScanServiceImpl.java and find that it is indeed downgraded

.

Forgot to restore ApArticleServiceImpl.java, restore downgrade

5. Publish article submission review integration

5.1 Synchronous call and asynchronous call

Synchronization: When a call is made, the call will not return until the result is obtained (real-time processing)

Asynchronous: After the call is issued, the call returns directly without returning a result (time-sharing processing)

Review articles in an asynchronous thread

5.2 Springboot integrated asynchronous thread call

①: Add the @Async annotation to the automatic review method (indicating that it is to be called asynchronously)

@Override
@Async  //标明当前方法是一个异步方法
public void autoScanWmNews(Integer id) {
    
    
	//代码略
}

②: Call the audit method after the article is published successfully

@Autowired
private WmNewsAutoScanService wmNewsAutoScanService;

/**
 * 发布修改文章或保存为草稿
 * @param dto
 * @return
 */
@Override
public ResponseResult submitNews(WmNewsDto dto) {
    
    

    //代码略

    //审核文章
    wmNewsAutoScanService.autoScanWmNews(wmNews.getId());

    return ResponseResult.okResult(AppHttpCodeEnum.SUCCESS);

}

③: Use the @EnableAsync annotation in the self-media boot class to enable asynchronous calls

@SpringBootApplication
@EnableDiscoveryClient
@MapperScan("com.heima.wemedia.mapper")
@EnableFeignClients(basePackages = "com.heima.apis")
@EnableAsync  //开启异步调用
public class WemediaApplication {
    
    

    public static void main(String[] args) {
    
    
        SpringApplication.run(WemediaApplication.class,args);
    }

    @Bean
    public MybatisPlusInterceptor mybatisPlusInterceptor() {
    
    
        MybatisPlusInterceptor interceptor = new MybatisPlusInterceptor();
        interceptor.addInnerInterceptor(new PaginationInnerInterceptor(DbType.MYSQL));
        return interceptor;
    }
}

6. Article review function - comprehensive test

6.1 Service startup list

1. nacos server

2, article Microservices

3. wemedia microservice

4. Start the wemedia gateway microservice

5. Start the front-end system wemedia

6.2 List of test cases

1. Publish a normal article from the front end of the self-media

After the audit is successful, whether the article-related data on the app side can be saved normally, and whether the status of self-media articles and the article id on the app side are echoed

2. The self-media front end publishes an article containing sensitive words

Normally, the audit fails, whether the status in the wm_news table has changed, and the success and failure reasons are saved normally

3. Publish an article containing sensitive pictures on the self-media front end

Normally, the audit fails, whether the status in the wm_news table has changed, and the success and failure reasons are saved normally

Finally, let's test whether an error in the asynchronous call will affect the publication of the article itself.
Modify WmNewsAutoScanServiceImpl.java

// 测试异步调用失败
        int a = 1 / 0;

Let's test it and publish another article

and find that the article is published successfully

Because it is asynchronous, the result is returned first, but in fact there is data in the news table

and no data in the article table

This is correct because 1/0 is wrong arithmetic.

7. New requirements - self-management sensitive words

7.1 Requirements Analysis

The article review function has been delivered, and articles can be published for review normally. Suddenly, the product manager came over and said there was a meeting.

The core contents of the meeting are as follows:

• Article review cannot filter out some sensitive words:

  Private detective, pinhole camera, credit card cash withdrawal, advertising agency, agency invoicing, engraving office, selling answers, small loans...

Functions to be completed:

You need to maintain a set of sensitive words yourself. When reviewing articles, you need to verify whether the articles contain these sensitive words

7.2 Sensitive words - filtering

Technology Selection

plan	illustrate
Database fuzzy query	too inefficient
String.indexOf("") find	If the database is large, it will be slower
Full Text Search	participle re-matching
DFA algorithm	Deterministic finite automata (a data structure)

7.3 Principle of DFA Implementation

The full name of DFA is: Deterministic Finite Automaton, which is to determine the finite automaton.

Storage: Store all sensitive words in multiple maps at one time, which is the structure shown in the figure below

Sensitive words: meth, marijuana, big bad

retrieval process

7.4 Integrate self-managed sensitive words into article review

①: Create a sensitive vocabulary, import wm_sensitive in the data into the leadnews_wemedia library

package com.heima.model.wemedia.pojos;

import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableField;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
import lombok.Data;

import java.io.Serializable;
import java.util.Date;

/**
 * <p>
 * 敏感词信息表
 * </p>
 *
 * @author itheima
 */
@Data
@TableName("wm_sensitive")
public class WmSensitive implements Serializable {
    
    

    private static final long serialVersionUID = 1L;

    /**
     * 主键
     */
    @TableId(value = "id", type = IdType.AUTO)
    private Integer id;

    /**
     * 敏感词
     */
    @TableField("sensitives")
    private String sensitives;

    /**
     * 创建时间
     */
    @TableField("created_time")
    private Date createdTime;

}

②: Copy the corresponding wm_sensitive mapper to the project

package com.heima.wemedia.mapper;

import com.baomidou.mybatisplus.core.mapper.BaseMapper;
import com.heima.model.wemedia.pojos.WmSensitive;
import org.apache.ibatis.annotations.Mapper;


@Mapper
public interface WmSensitiveMapper extends BaseMapper<WmSensitive> {
    
    
}

③: Add self-management sensitive word review to the article review code

First: Add the following code to the autoScanWmNews method in WmNewsAutoScanServiceImpl

//从内容中提取纯文本内容和图片
//.....省略

//自管理的敏感词过滤
boolean isSensitive = handleSensitiveScan((String) textAndImages.get("content"), wmNews);
if(!isSensitive) return;

//2.审核文本内容  阿里云接口
//.....省略

Add self-management sensitive word review code

@Autowired
private WmSensitiveMapper wmSensitiveMapper;

/**
     * 自管理的敏感词审核
     * @param content
     * @param wmNews
     * @return
     */
private boolean handleSensitiveScan(String content, WmNews wmNews) {
    
    

    boolean flag = true;

    //获取所有的敏感词
    List<WmSensitive> wmSensitives = wmSensitiveMapper.selectList(Wrappers.<WmSensitive>lambdaQuery().select(WmSensitive::getSensitives));
    List<String> sensitiveList = wmSensitives.stream().map(WmSensitive::getSensitives).collect(Collectors.toList());

    //初始化敏感词库
    SensitiveWordUtil.initMap(sensitiveList);

    //查看文章中是否包含敏感词
    Map<String, Integer> map = SensitiveWordUtil.matchWords(content);
    if(map.size() >0){
    
    
        updateWmNews(wmNews,(short) 2,"当前文章中存在违规内容"+map);
        flag = false;
    }

    return flag;
}

To test it, first restart all articles and self-media microservices, bring sensitive words,

and then we publish it.

The audit results are as follows:

8. New requirements - image recognition text review sensitive words

8.1 Requirements Analysis

The product manager called a meeting, the article review function has been delivered, and the article can be published for review normally. I am also very satisfied with the self-management sensitive words proposed last time. The core content of this meeting is as follows:

• The pictures contained in the article need to recognize the text, and filter out the sensitive words in the picture text
  

8.2 Image text recognition

What is OCR?

OCR (Optical Character Recognition, Optical Character Recognition) means that an electronic device (such as a scanner or a digital camera) checks characters printed on paper, determines its shape by detecting dark and light patterns, and then uses character recognition to translate the shape into a computer text process

plan	illustrate
Baidu OCR	TOLL
Tesseract-OCR	An open source OCR engine maintained by Google, which supports Java, Python and other language calls
Tess4J	Encapsulates Tesseract-OCR and supports Java calls

For specific training methods, please refer to the official documentation: https://tesser act-ocr.github.io/tessdoc/

8.3 Tess4j case

Create a new tess4j-demo and place it under the heima-leadnews-test module

①: Create a project to import the dependencies corresponding to tess4j

<dependency>
    <groupId>net.sourceforge.tess4j</groupId>
    <artifactId>tess4j</artifactId>
    <version>4.1.1</version>
</dependency>

②: Import the Chinese font library, copy the tessdata folder in the data to your own workspace

③: Write a test class for testing

package com.heima.tess4j;

import net.sourceforge.tess4j.ITesseract;
import net.sourceforge.tess4j.Tesseract;

import java.io.File;

public class Application {
    
    

    public static void main(String[] args) {
    
    
        try {
    
    
            //获取本地图片
            File file = new File("D:\\26.png");
            //创建Tesseract对象
            ITesseract tesseract = new Tesseract();
            //设置字体库路径
            tesseract.setDatapath("D:\\workspace\\tessdata");
            //中文识别
            tesseract.setLanguage("chi_sim");
            //执行ocr识别
            String result = tesseract.doOCR(file);
            //替换回车和tal键  使结果为一行
            result = result.replaceAll("\\r|\\n","-").replaceAll(" ","");
            System.out.println("识别的结果为："+result);
        } catch (Exception e) {
    
    
            e.printStackTrace();
        }
    }
}

The result of the operation is as follows:

8.4 Manage sensitive words and image text recognition integrated into article review

①: Create a tool class in heima-leadnews-common, and simply encapsulate tess4j

Need to import pom first

<dependency>
    <groupId>net.sourceforge.tess4j</groupId>
    <artifactId>tess4j</artifactId>
    <version>4.1.1</version>
</dependency>

Tools

package com.heima.common.tess4j;

import lombok.Getter;
import lombok.Setter;
import net.sourceforge.tess4j.ITesseract;
import net.sourceforge.tess4j.Tesseract;
import net.sourceforge.tess4j.TesseractException;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.stereotype.Component;

import java.awt.image.BufferedImage;

@Getter
@Setter
@Component
@ConfigurationProperties(prefix = "tess4j")
public class Tess4jClient {
    
    

    private String dataPath;
    private String language;

    public String doOCR(BufferedImage image) throws TesseractException {
    
    
        //创建Tesseract对象
        ITesseract tesseract = new Tesseract();
        //设置字体库路径
        tesseract.setDatapath(dataPath);
        //中文识别
        tesseract.setLanguage(language);
        //执行ocr识别
        String result = tesseract.doOCR(image);
        //替换回车和tal键  使结果为一行
        result = result.replaceAll("\\r|\\n", "-").replaceAll(" ", "");
        return result;
    }

}

Add this class in the spring.factories configuration, complete as follows:

org.springframework.boot.autoconfigure.EnableAutoConfiguration=\
  com.heima.common.exception.ExceptionCatch,\
  com.heima.common.swagger.SwaggerConfiguration,\
  com.heima.common.swagger.Swagger2Configuration,\
  com.heima.common.aliyun.GreenTextScan,\
  com.heima.common.aliyun.GreenImageScan,\
  com.heima.common.tess4j.Tess4jClient

②: Add two attributes to the configuration in heima-leadnews-wemedia

tess4j:
  data-path: D:\workspace\tessdata
  language: chi_sim

③: Add the following code to the handleImageScan method in WmNewsAutoScanServiceImpl

try {
    
    
    for (String image : images) {
    
    
        byte[] bytes = fileStorageService.downLoadFile(image);

        //图片识别文字审核---begin-----

        //从byte[]转换为butteredImage
        ByteArrayInputStream in = new ByteArrayInputStream(bytes);
        BufferedImage imageFile = ImageIO.read(in);
        //识别图片的文字
        String result = tess4jClient.doOCR(imageFile);

        //审核是否包含自管理的敏感词
        boolean isSensitive = handleSensitiveScan(result, wmNews);
        if(!isSensitive){
    
    
            return isSensitive;
        }

        //图片识别文字审核---end-----


        imageList.add(bytes);

    } 
}catch (Exception e){
    
    
    e.printStackTrace();
}

Finally, the complete code for article review is attached as follows:

package com.heima.wemedia.service.impl;

import com.alibaba.fastjson.JSONArray;
import com.baomidou.mybatisplus.core.toolkit.Wrappers;
import com.heima.apis.article.IArticleClient;
import com.heima.common.aliyun.GreenImageScan;
import com.heima.common.aliyun.GreenTextScan;
import com.heima.common.tess4j.Tess4jClient;
import com.heima.file.service.FileStorageService;
import com.heima.model.article.dtos.ArticleDto;
import com.heima.model.common.dtos.ResponseResult;
import com.heima.model.wemedia.pojos.WmChannel;
import com.heima.model.wemedia.pojos.WmNews;
import com.heima.model.wemedia.pojos.WmSensitive;
import com.heima.model.wemedia.pojos.WmUser;
import com.heima.utils.common.SensitiveWordUtil;
import com.heima.wemedia.mapper.WmChannelMapper;
import com.heima.wemedia.mapper.WmNewsMapper;
import com.heima.wemedia.mapper.WmSensitiveMapper;
import com.heima.wemedia.mapper.WmUserMapper;
import com.heima.wemedia.service.WmNewsAutoScanService;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.lang3.StringUtils;
import org.springframework.beans.BeanUtils;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.scheduling.annotation.Async;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;

import javax.imageio.ImageIO;
import java.awt.image.BufferedImage;
import java.io.ByteArrayInputStream;
import java.util.*;
import java.util.stream.Collectors;


@Service
@Slf4j
@Transactional
public class WmNewsAutoScanServiceImpl implements WmNewsAutoScanService {
    
    

    @Autowired
    private WmNewsMapper wmNewsMapper;

    /**
     * 自媒体文章审核
     *
     * @param id 自媒体文章id
     */
    @Override
    @Async  //标明当前方法是一个异步方法
    public void autoScanWmNews(Integer id) {
    
    

//        int a = 1/0;

        //1.查询自媒体文章
        WmNews wmNews = wmNewsMapper.selectById(id);
        if (wmNews == null) {
    
    
            throw new RuntimeException("WmNewsAutoScanServiceImpl-文章不存在");
        }

        if (wmNews.getStatus().equals(WmNews.Status.SUBMIT.getCode())) {
    
    
            //从内容中提取纯文本内容和图片
            Map<String, Object> textAndImages = handleTextAndImages(wmNews);

            //自管理的敏感词过滤
            boolean isSensitive = handleSensitiveScan((String) textAndImages.get("content"), wmNews);
            if(!isSensitive) return;

            //2.审核文本内容  阿里云接口
            boolean isTextScan = handleTextScan((String) textAndImages.get("content"), wmNews);
            if (!isTextScan) return;

            //3.审核图片  阿里云接口
            boolean isImageScan = handleImageScan((List<String>) textAndImages.get("images"), wmNews);
            if (!isImageScan) return;

            //4.审核成功，保存app端的相关的文章数据
            ResponseResult responseResult = saveAppArticle(wmNews);
            if (!responseResult.getCode().equals(200)) {
    
    
                throw new RuntimeException("WmNewsAutoScanServiceImpl-文章审核，保存app端相关文章数据失败");
            }
            //回填article_id
            wmNews.setArticleId((Long) responseResult.getData());
            updateWmNews(wmNews, (short) 9, "审核成功");

        }
    }

    @Autowired
    private WmSensitiveMapper wmSensitiveMapper;

    /**
     * 自管理的敏感词审核
     * @param content
     * @param wmNews
     * @return
     */
    private boolean handleSensitiveScan(String content, WmNews wmNews) {
    
    

        boolean flag = true;

        //获取所有的敏感词
        List<WmSensitive> wmSensitives = wmSensitiveMapper.selectList(Wrappers.<WmSensitive>lambdaQuery().select(WmSensitive::getSensitives));
        List<String> sensitiveList = wmSensitives.stream().map(WmSensitive::getSensitives).collect(Collectors.toList());

        //初始化敏感词库
        SensitiveWordUtil.initMap(sensitiveList);

        //查看文章中是否包含敏感词
        Map<String, Integer> map = SensitiveWordUtil.matchWords(content);
        if(map.size() >0){
    
    
            updateWmNews(wmNews,(short) 2,"当前文章中存在违规内容"+map);
            flag = false;
        }

        return flag;
    }

    @Autowired
    private IArticleClient articleClient;

    @Autowired
    private WmChannelMapper wmChannelMapper;

    @Autowired
    private WmUserMapper wmUserMapper;

    /**
     * 保存app端相关的文章数据
     *
     * @param wmNews
     */
    private ResponseResult saveAppArticle(WmNews wmNews) {
    
    

        ArticleDto dto = new ArticleDto();
        //属性的拷贝
        BeanUtils.copyProperties(wmNews, dto);
        //文章的布局
        dto.setLayout(wmNews.getType());
        //频道
        WmChannel wmChannel = wmChannelMapper.selectById(wmNews.getChannelId());
        if (wmChannel != null) {
    
    
            dto.setChannelName(wmChannel.getName());
        }

        //作者
        dto.setAuthorId(wmNews.getUserId().longValue());
        WmUser wmUser = wmUserMapper.selectById(wmNews.getUserId());
        if (wmUser != null) {
    
    
            dto.setAuthorName(wmUser.getName());
        }

        //设置文章id
        if (wmNews.getArticleId() != null) {
    
    
            dto.setId(wmNews.getArticleId());
        }
        dto.setCreatedTime(new Date());

        ResponseResult responseResult = articleClient.saveArticle(dto);
        return responseResult;

    }


    @Autowired
    private FileStorageService fileStorageService;

    @Autowired
    private GreenImageScan greenImageScan;

    @Autowired
    private Tess4jClient tess4jClient;

    /**
     * 审核图片
     *
     * @param images
     * @param wmNews
     * @return
     */
    private boolean handleImageScan(List<String> images, WmNews wmNews) {
    
    

        boolean flag = true;

        if (images == null || images.size() == 0) {
    
    
            return flag;
        }

        //下载图片 minIO
        //图片去重
        images = images.stream().distinct().collect(Collectors.toList());

        List<byte[]> imageList = new ArrayList<>();

        try {
    
    
            for (String image : images) {
    
    
                byte[] bytes = fileStorageService.downLoadFile(image);

                //图片识别文字审核---begin-----

                //从byte[]转换为butteredImage
                ByteArrayInputStream in = new ByteArrayInputStream(bytes);
                BufferedImage imageFile = ImageIO.read(in);
                //识别图片的文字
                String result = tess4jClient.doOCR(imageFile);

                //审核是否包含自管理的敏感词
                boolean isSensitive = handleSensitiveScan(result, wmNews);
                if(!isSensitive){
    
    
                    return isSensitive;
                }

                //图片识别文字审核---end-----


                imageList.add(bytes);

            }
        }catch (Exception e){
    
    
            e.printStackTrace();
        }


        //审核图片
        try {
    
    
            Map map = greenImageScan.imageScan(imageList);
            if (map != null) {
    
    
                //审核失败
                if (map.get("suggestion").equals("block")) {
    
    
                    flag = false;
                    updateWmNews(wmNews, (short) 2, "当前文章中存在违规内容");
                }

                //不确定信息  需要人工审核
                if (map.get("suggestion").equals("review")) {
    
    
                    flag = false;
                    updateWmNews(wmNews, (short) 3, "当前文章中存在不确定内容");
                }
            }

        } catch (Exception e) {
    
    
            flag = false;
            e.printStackTrace();
        }
        return flag;
    }

    @Autowired
    private GreenTextScan greenTextScan;

    /**
     * 审核纯文本内容
     *
     * @param content
     * @param wmNews
     * @return
     */
    private boolean handleTextScan(String content, WmNews wmNews) {
    
    

        boolean flag = true;

        if ((wmNews.getTitle() + "-" + content).length() == 0) {
    
    
            return flag;
        }
        try {
    
    
            Map map = greenTextScan.greeTextScan((wmNews.getTitle() + "-" + content));
            if (map != null) {
    
    
                //审核失败
                if (map.get("suggestion").equals("block")) {
    
    
                    flag = false;
                    updateWmNews(wmNews, (short) 2, "当前文章中存在违规内容");
                }

                //不确定信息  需要人工审核
                if (map.get("suggestion").equals("review")) {
    
    
                    flag = false;
                    updateWmNews(wmNews, (short) 3, "当前文章中存在不确定内容");
                }
            }
        } catch (Exception e) {
    
    
            flag = false;
            e.printStackTrace();
        }

        return flag;

    }

    /**
     * 修改文章内容
     *
     * @param wmNews
     * @param status
     * @param reason
     */
    private void updateWmNews(WmNews wmNews, short status, String reason) {
    
    
        wmNews.setStatus(status);
        wmNews.setReason(reason);
        wmNewsMapper.updateById(wmNews);
    }

    /**
     * 1。从自媒体文章的内容中提取文本和图片
     * 2.提取文章的封面图片
     *
     * @param wmNews
     * @return
     */
    private Map<String, Object> handleTextAndImages(WmNews wmNews) {
    
    

        //存储纯文本内容
        StringBuilder stringBuilder = new StringBuilder();

        List<String> images = new ArrayList<>();

        //1。从自媒体文章的内容中提取文本和图片
        if (StringUtils.isNotBlank(wmNews.getContent())) {
    
    
            List<Map> maps = JSONArray.parseArray(wmNews.getContent(), Map.class);
            for (Map map : maps) {
    
    
                if (map.get("type").equals("text")) {
    
    
                    stringBuilder.append(map.get("value"));
                }

                if (map.get("type").equals("image")) {
    
    
                    images.add((String) map.get("value"));
                }
            }
        }
        //2.提取文章的封面图片
        if (StringUtils.isNotBlank(wmNews.getImages())) {
    
    
            String[] split = wmNews.getImages().split(",");
            images.addAll(Arrays.asList(split));
        }

        Map<String, Object> resultMap = new HashMap<>();
        resultMap.put("content", stringBuilder.toString());
        resultMap.put("images", images);
        return resultMap;

    }
}

Upload a violation picture to test it

and find no problem

9. Article details - static file generation

9.1 Idea analysis

When creating an app-related article on the article side, generate a static page with article details and upload it to MinIO

9.2 Implementation steps

1. Create a new ArticleFreemarkerService to create static files and upload them to minIO

package com.heima.article.service;

import com.heima.model.article.pojos.ApArticle;

public interface ArticleFreemarkerService {
    
    

    /**
     * 生成静态文件上传到minIO中
     * @param apArticle
     * @param content
     */
    public void buildArticleToMinIO(ApArticle apArticle,String content);
}

accomplish

package com.heima.article.service.impl;

import com.alibaba.fastjson.JSON;
import com.alibaba.fastjson.JSONArray;
import com.baomidou.mybatisplus.core.toolkit.Wrappers;
import com.heima.article.mapper.ApArticleContentMapper;
import com.heima.article.service.ApArticleService;
import com.heima.article.service.ArticleFreemarkerService;
import com.heima.file.service.FileStorageService;
import com.heima.model.article.pojos.ApArticle;
import freemarker.template.Configuration;
import freemarker.template.Template;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.lang3.StringUtils;
import org.springframework.beans.BeanUtils;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.scheduling.annotation.Async;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;

import java.io.ByteArrayInputStream;
import java.io.InputStream;
import java.io.StringWriter;
import java.util.HashMap;
import java.util.Map;

@Service
@Slf4j
@Transactional
public class ArticleFreemarkerServiceImpl implements ArticleFreemarkerService {
    
    

    @Autowired
    private ApArticleContentMapper apArticleContentMapper;

    @Autowired
    private Configuration configuration;

    @Autowired
    private FileStorageService fileStorageService;

    @Autowired
    private ApArticleService apArticleService;

    /**
     * 生成静态文件上传到minIO中
     * @param apArticle
     * @param content
     */
    @Async
    @Override
    public void buildArticleToMinIO(ApArticle apArticle, String content) {
    
    
        //已知文章的id
        //4.1 获取文章内容
        if(StringUtils.isNotBlank(content)){
    
    
            //4.2 文章内容通过freemarker生成html文件
            Template template = null;
            StringWriter out = new StringWriter();
            try {
    
    
                template = configuration.getTemplate("article.ftl");
                //数据模型
                Map<String,Object> contentDataModel = new HashMap<>();
                contentDataModel.put("content", JSONArray.parseArray(content));
                //合成
                template.process(contentDataModel,out);
            } catch (Exception e) {
    
    
                e.printStackTrace();
            }

            //4.3 把html文件上传到minio中
            InputStream in = new ByteArrayInputStream(out.toString().getBytes());
            String path = fileStorageService.uploadHtmlFile("", apArticle.getId() + ".html", in);


            //4.4 修改ap_article表，保存static_url字段
            apArticleService.update(Wrappers.<ApArticle>lambdaUpdate().eq(ApArticle::getId,apArticle.getId())
                    .set(ApArticle::getStaticUrl,path));


        }
    }

}

2. Add the method of calling the generated file in the saveArticle implementation method of ApArticleService

/**
     * 保存app端相关文章
     * @param dto
     * @return
     */
@Override
public ResponseResult saveArticle(ArticleDto dto) {
    
    

    //        try {
    
    
    //            Thread.sleep(3000);
    //        } catch (InterruptedException e) {
    
    
    //            e.printStackTrace();
    //        }
    //1.检查参数
    if(dto == null){
    
    
        return ResponseResult.errorResult(AppHttpCodeEnum.PARAM_INVALID);
    }

    ApArticle apArticle = new ApArticle();
    BeanUtils.copyProperties(dto,apArticle);

    //2.判断是否存在id
    if(dto.getId() == null){
    
    
        //2.1 不存在id  保存  文章  文章配置  文章内容

        //保存文章
        save(apArticle);

        //保存配置
        ApArticleConfig apArticleConfig = new ApArticleConfig(apArticle.getId());
        apArticleConfigMapper.insert(apArticleConfig);

        //保存 文章内容
        ApArticleContent apArticleContent = new ApArticleContent();
        apArticleContent.setArticleId(apArticle.getId());
        apArticleContent.setContent(dto.getContent());
        apArticleContentMapper.insert(apArticleContent);

    }else {
    
    
        //2.2 存在id   修改  文章  文章内容

        //修改  文章
        updateById(apArticle);

        //修改文章内容
        ApArticleContent apArticleContent = apArticleContentMapper.selectOne(Wrappers.<ApArticleContent>lambdaQuery().eq(ApArticleContent::getArticleId, dto.getId()));
        apArticleContent.setContent(dto.getContent());
        apArticleContentMapper.updateById(apArticleContent);
    }

    //异步调用 生成静态文件上传到minio中
    articleFreemarkerService.buildArticleToMinIO(apArticle,dto.getContent());


    //3.结果返回  文章的id
    return ResponseResult.okResult(apArticle.getId());
}

3. The article microservice starts asynchronous calls.

We publish an article.

The database is inserted successfully. leadnews_article.ap_article

visits this URL and the access is successful.

10. Today's homework and thinking

This homework question, follow-up update (AT mode)

This question is solved on the fifth day of the course.