jenkins 使用 pipeline 方式部署 shop-boot java 工程

添加插件

pipeline

Workspace Cleanup

构建后清理工作空间

Config File Provider

为每个项目指定不同的maven配置文件，未使用

简单pipeline

此方式为简单记录，实际使用为pipeline多分支方式

新建项目

pipeline 脚本

pipeline {
    
    
    agent any

    options {
    
    
        // 保存最近历史构建记录的数量
        buildDiscarder(logRotator(numToKeepStr: '2'))
        // 拉去代码时默认目录
        // checkoutToSubdirectory('subdir')
        // 禁止同一个 pipeline 同时执行
        disableConcurrentBuilds()
    }

    stages {
    
    
        stage('pre') {
    
    
            steps {
    
    
                echo '打印环境变量'
                sh 'printenv'
            }
        }

        stage('pull code') {
    
    
            steps {
    
    
                checkout([$class: 'GitSCM', branches: [[name: '*/master']], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[url: '[email protected]:sanren2016/shop-boot.git']]])
            }
        }

        stage('build') {
    
    
            steps {
    
    
                sh 'mvn clean package'
            }
        }

        stage('deploy') {
    
    
            steps {
    
    
                sh 'scp ${WORKSPACE}/target/shop-boot.jar [email protected]:/home/stest/shop-boot'
                sh 'ssh [email protected] "cd /home/stest/shop-boot/sbin; ./control.sh restart"'
            }
        }
    }
}

pipeline

源码

https://gitee.com/sanren2016/shop-boot/tree/95874b81f21e42e3cb9c45a2b8bbfd1859316574/

新建项目

项目配置

注意点

1. stest与sprod的配置文件与启动脚本使用了绝对路径
2. 上传jar包的时候直接重命名，避免出现jenkins远程执行脚本状态码不正确问题

添加docker支持

安装docker

• 安装

yum install -y yum-utils
yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
yum makecache fast
yum install docker-ce docker-ce-cli containerd.io -y

• 非root用户运行docker

# 添加用户至docker组
gpasswd -a 用户名 docker
# 更新用户组
newgrp docker
# 重启docker
systemctl restart docker

• 配置镜像
  参见阿里云控制台

手动部署

• [devops] Dockerfile

# 基础镜像使用java
FROM java:8
# 作者
MAINTAINER sanren2016 <[email protected]>
# 端口
EXPOSE 8091
# 挂载目录
VOLUME /etc/shop-boot/conf
VOLUME /var/log/shop-boot
# 将jar包添加到容器中并更名为app.jar
ADD shop-boot-test.jar app.jar 
# 运行jar包
RUN bash -c 'touch /app.jar'
ENTRYPOINT ["java","-Djava.security.egd=file:/dev/./urandom","-Dspring.config.location=/etc/shop-boot/conf/application.yml","-jar","/app.jar"]

制作镜像

扫描二维码关注公众号，回复： 12424778 查看本文章

# 当前目录需要有 shop-boot-test.jar 文件
docker build -t shop-boot-test:0.1 .

• [devops] 打包镜像并scp到stest

docker save -o shop-boot-test.tar shop-boot-test:0.1

• [stest] 加载镜像

docker load -i ~/shop-boot-test.tar

• [stest] 配置文件

[stest@shopboot conf]$ pwd
/home/stest/docker/shop-boot/conf
[stest@shopboot conf]$ cat application.yml 
server:
  port: 8091
  servlet:
    context-path: /


logging:
  level:
    org.springframework.web.servlet.DispatcherServlet: DEBUG
    com.laolang.shop: DEBUG
  config: file:/etc/shop-boot/conf/logback-spring.xml
  path: /var/log/shop-boot
[stest@shopboot conf]$ cat logback-spring.xml 
<?xml version="1.0" encoding="UTF-8"?>
<!-- 日志级别从低到高分为TRACE < DEBUG < INFO < WARN < ERROR < FATAL，如果设置为WARN，则低于WARN的信息都不会输出 -->
<!-- scan:当此属性设置为true时，配置文件如果发生改变，将会被重新加载，默认值为true -->
<!-- scanPeriod:设置监测配置文件是否有修改的时间间隔，如果没有给出时间单位，默认单位是毫秒。当scan为true时，此属性生效。默认的时间间隔为1分钟。 -->
<!-- debug:当此属性设置为true时，将打印出logback内部日志信息，实时查看logback运行状态。默认值为false。 -->
<configuration scan="true" scanPeriod="10 seconds">

    <!--<include resource="org/springframework/boot/logging/logback/base.xml" />-->

    <contextName>logback</contextName>
    <!-- name的值是变量的名称，value的值时变量定义的值。通过定义的值会被插入到logger上下文中。定义变量后，可以使“${
    
    }”来使用变量。 -->
    <!--    <property name="log.path" value="/home/laolang/gitosc/km-boot/km-boot/logs/km-bbs/dev"/>-->

    <springProperty name="LOG_PATH" source="logging.path" defaultValue="/var/log/shop-boot" />
    <property name="LOG_FILE" value="shop-boot" />

    <!-- 彩色日志 -->
    <!-- 彩色日志依赖的渲染类 -->
    <conversionRule conversionWord="clr" converterClass="org.springframework.boot.logging.logback.ColorConverter"/>
    <conversionRule conversionWord="wex" converterClass="org.springframework.boot.logging.logback.WhitespaceThrowableProxyConverter"/>
    <conversionRule conversionWord="wEx"
                    converterClass="org.springframework.boot.logging.logback.ExtendedWhitespaceThrowableProxyConverter"/>
    <!-- 彩色日志格式 -->
    <property name="CONSOLE_LOG_PATTERN"
              value="${CONSOLE_LOG_PATTERN:-%clr(%d{yyyy-MM-dd HH:mm:ss.SSS}){faint} %clr(${LOG_LEVEL_PATTERN:-%5p}) %clr(${PID:- }){magenta} %clr(---){faint} %clr([%15.15t]){faint} %clr(%-40.40logger{39}){cyan} %clr(:){faint} %m%n${LOG_EXCEPTION_CONVERSION_WORD:-%wEx}}"/>


    <!--输出到控制台-->
    <appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
        <!--此日志appender是为开发使用，只配置最底级别，控制台输出的日志级别是大于或等于此级别的日志信息-->
        <filter class="ch.qos.logback.classic.filter.ThresholdFilter">
            <!--            <level>info</level>-->
            <level>debug</level>
        </filter>
        <encoder>
            <Pattern>${CONSOLE_LOG_PATTERN}</Pattern>
            <!-- 设置字符集 -->
            <charset>UTF-8</charset>
        </encoder>
    </appender>


    <!--日志输出到文件-->
    <appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
        <!--日志文件输出格式-->
        <encoder>
            <pattern>%d{
    
    yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{
    
    50} - %msg%n</pattern>
            <charset>UTF-8</charset> <!-- 此处设置字符集 -->
        </encoder>
        <!-- 日志记录器的滚动策略，按日期，按大小记录 -->
        <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
            <fileNamePattern>${LOG_PATH}/%d{
    
    yyyy-MM-dd}/info-%d{
    
    yyyy-MM-dd}.%i.log</fileNamePattern>
            <timeBasedFileNamingAndTriggeringPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
                <maxFileSize>100MB</maxFileSize>
            </timeBasedFileNamingAndTriggeringPolicy>
            <!--日志文件保留天数-->
            <maxHistory>150</maxHistory>
        </rollingPolicy>
    </appender>

    <!--开发环境:打印控制台-->
    <!--<springProfile name="dev">-->
    <!--<logger name="com.laolang" level="debug"/>-->
    <!--</springProfile>-->

    <root level="debug">
        <appender-ref ref="CONSOLE"/>
        <appender-ref ref="FILE"/>
    </root>
</configuration>
[stest@shopboot conf]$ 

• [stest] 运行镜像

docker run -d --name shop-boot-test --privileged=true -p 8091:8091 -v /home/stest/docker/shop-boot/conf:/etc/shop-boot/conf -v /home/stest/docker/shop-boot/logs:/var/log/shop-boot shop-boot-test:0.1

上传 dockerhub

• Dockerfile

# 基础镜像使用java
FROM java:8
# 作者
MAINTAINER khlbat <[email protected]>
# 修正时区问题
RUN bash -c 'echo "Asia/Shanghai" > /etc/timezone'
# 端口
EXPOSE 8091
# 挂载目录
VOLUME /etc/shop-boot/conf
VOLUME /var/log/shop-boot
# 将jar包添加到容器中并更名为app.jar
ADD shop-boot.jar app.jar 
# 运行jar包
RUN bash -c 'touch /app.jar'
ENTRYPOINT ["java","-Djava.security.egd=file:/dev/./urandom","-Dspring.config.location=/etc/shop-boot/conf/application.yml","-jar","/app.jar"]

• 构建镜像并上传

# 构建镜像
docker build -t khlbat/shop-boot:0.1 .
# 上传镜像
docker push khlbat/shop-boot:0.1

pipeline + python + docker 本地镜像

代码参见: https://gitee.com/sanren2016/shop-boot/tree/1d35cacac1de81185cf02ef8a191a0064e136079/

python 环境 搭建

安装依赖包

yum -y install zlib-devel bzip2-devel openssl-devel ncurses-devel sqlite-devel readline-devel tk-devel gdbm-devel db4-devel libpcap-devel xz-devel gcc libffi-devel gcc make automake autoconf libtool libffi-devel

下载源码包

从 https://www.python.org/ftp/python/ 下载python源码包

安装

./configure --prefix=/usr/local/python3 && make && make install

添加软链接

ln -s /usr/local/python3/bin/python3.7 /usr/bin/python3
ln -s /usr/local/python3/bin/pip3 /usr/bin/pip3

配置 pip 源

添加~/.pip/pip.conf 文件

[global]
index-url = https://pypi.tuna.tsinghua.edu.cn/simple
[install]
trusted-host = https://pypi.tuna.tsinghua.edu.cn

devops python 脚本打包本地镜像

需要lxml库
virtualenv 位置：/home/devops/pyenv/shop-boot
此脚本位置：/home/devops/scripts/sbin

#!/usr/bin/env python3
import os
import subprocess
import sys
from lxml import etree

# 工作空间路径，第一个参数
workspace_path = ''
# 分支名，第二个参数，默认：master
project_branch = ''
# 应用版本
project_version = '1.0'
# 临时目录
tmp_path = '/home/devops/.jenkinstmp'
# 本地镜像名称
image_name = ''

# 应用名称
app_name = 'shop-boot'


def init_env():
    """
    初始化环境
        1. 获取工作空间路径
        2. 解析项目版本, 解析pom.xml文件获得，忽略 -SNAPSHOT
    :return:
    """
    print('')
    print('初始化 docker 打包环境')
    global workspace_path
    global project_branch
    global project_version
    global image_name
    global tmp_path
    workspace_path = sys.argv[1]
    project_branch = sys.argv[2]
    print('应用名称:' + app_name)

    tree = etree.parse(workspace_path + '/pom.xml')
    root = tree.getroot()
    for el in root:
        if '{http://maven.apache.org/POM/4.0.0}version' == el.tag:
            project_version = el.text
            project_version = project_version.replace('-SNAPSHOT', '')
    print('应用版本:' + project_version)
    print('项目分支:' + project_branch)
    print('工作空间路径:' + workspace_path)
    image_name = app_name + '-' + project_branch + ':' + project_version
    print('本地镜像名称:' + image_name)
    tmp_path = '/home/devops/.jenkinstmp/' + workspace_path.split('/')[-1]
    if not os.path.isdir(tmp_path):
        os.mkdir(tmp_path)
    print('本地镜像打包临时目录:' + tmp_path)


def has_image():
    """
    判断本地是否已经存在项目对应的分支和版本的镜像
    :return: True | False
    """
    print('')
    print('搜索本地镜像')
    (status, output) = run_shell(
        'docker images -q --filter=reference=' + app_name + ':' + project_version)

    if 0 == status:
        if 0 == len(output):
            return False
        else:
            return True


def del_image():
    """
    删除本地镜像
    :return: 无
    """
    print('')
    print('删除本地镜像:' + image_name)
    sh = 'docker rmi ' + image_name
    run_shell(sh)


def create_image():
    """
    根据项目分支与版本创建镜像
    :return:
    """
    print('')
    print('构建本地镜像' + image_name)
    global tmp_path
    run_shell('rm -rf ' + tmp_path)
    run_shell('mkdir ' + tmp_path)
    run_shell('cp ' + workspace_path + '/Dockerfile ' + tmp_path + '/')
    run_shell('cp ' + workspace_path + '/target/' + app_name + '.jar ' + tmp_path + '/')
    run_shell(
        'cd ' + tmp_path + ' && docker build -t ' + image_name + ' .')


def tar_image():
    """
    打包本地镜像
    :return:
    """
    print('')
    print('打包本地镜像:' + image_name)
    image_tar_path = tmp_path + '/' + app_name + '-' + project_branch + ':' + project_version + '.tar'
    run_shell(
        'docker save -o ' + image_tar_path + ' ' + image_name)
    print('本地镜像打包文件:' + image_tar_path)
    run_shell('ls -lFh --time-style="+%Y-%m-%d %H:%I:%S" ' + image_tar_path)


def run_shell(sh):
    print('====================================================')
    print('shell:' + sh)
    (status, output) = subprocess.getstatusoutput(sh)
    print('shell status:', status)
    print('shell output:')
    print(output)
    print('====================================================')
    print('')
    return status, output


if __name__ == '__main__':
    init_env()
    if has_image():
        print('删除原镜像')
        del_image()
    create_image()
    tar_image()

测试

(shop-boot) [devops@devops sbin]$ ./shop-boot.py /home/devops/.jenkins/workspace/shop-boot-project_test test

初始化 docker 打包环境
应用名称:shop-boot
应用版本:0.1
项目分支:test
工作空间路径:/home/devops/.jenkins/workspace/shop-boot-project_test
本地镜像名称:shop-boot-test:0.1
本地镜像打包临时目录:/home/devops/.jenkinstmp/shop-boot-project_test

搜索本地镜像
====================================================
shell:docker images -q --filter=reference=shop-boot:0.1
shell status: 0
shell output:

====================================================


构建本地镜像shop-boot-test:0.1
====================================================
shell:rm -rf /home/devops/.jenkinstmp/shop-boot-project_test
shell status: 0
shell output:

====================================================

====================================================
shell:mkdir /home/devops/.jenkinstmp/shop-boot-project_test
shell status: 0
shell output:

====================================================

====================================================
shell:cp /home/devops/.jenkins/workspace/shop-boot-project_test/Dockerfile /home/devops/.jenkinstmp/shop-boot-project_test/
shell status: 0
shell output:

====================================================

====================================================
shell:cp /home/devops/.jenkins/workspace/shop-boot-project_test/target/shop-boot.jar /home/devops/.jenkinstmp/shop-boot-project_test/
shell status: 0
shell output:

====================================================

====================================================
shell:cd /home/devops/.jenkinstmp/shop-boot-project_test && docker build -t shop-boot-test:0.1 .
shell status: 0
shell output:
Sending build context to Docker daemon  21.67MB

Step 1/9 : FROM java:8
 ---> d23bdf5b1b1b
Step 2/9 : MAINTAINER khlbat <[email protected]>
 ---> Using cache
 ---> 2ea13a11d7b0
Step 3/9 : RUN bash -c 'echo "Asia/Shanghai" > /etc/timezone'
 ---> Using cache
 ---> 038c09f8acb2
Step 4/9 : EXPOSE 8091 8092
 ---> Using cache
 ---> 17a6d847a420
Step 5/9 : VOLUME /etc/shop-boot/conf
 ---> Using cache
 ---> fb21d9898bb9
Step 6/9 : VOLUME /var/log/shop-boot
 ---> Using cache
 ---> 946e56e38556
Step 7/9 : ADD shop-boot.jar app.jar
 ---> Using cache
 ---> f846764c0343
Step 8/9 : RUN bash -c 'touch /app.jar'
 ---> Using cache
 ---> b3546acfd1dc
Step 9/9 : ENTRYPOINT ["java","-Djava.security.egd=file:/dev/./urandom","-Dspring.config.location=/etc/shop-boot/conf/application.yml","-jar","/app.jar"]
 ---> Using cache
 ---> 323eb051f9b4
Successfully built 323eb051f9b4
Successfully tagged shop-boot-test:0.1
====================================================


打包本地镜像:shop-boot-test:0.1
====================================================
shell:docker save -o /home/devops/.jenkinstmp/shop-boot-project_test/shop-boot-test:0.1.tar shop-boot-test:0.1
shell status: 0
shell output:

====================================================

本地镜像打包文件:/home/devops/.jenkinstmp/shop-boot-project_test/shop-boot-test:0.1.tar
====================================================
shell:ls -lFh --time-style="+%Y-%m-%d %H:%I:%S" /home/devops/.jenkinstmp/shop-boot-project_test/shop-boot-test:0.1.tar
shell status: 0
shell output:
-rw------- 1 devops devops 671M 2020-10-21 22:10:54 /home/devops/.jenkinstmp/shop-boot-project_test/shop-boot-test:0.1.tar
====================================================

(shop-boot) [devops@devops sbin]$ 

stest python 脚本

#!/usr/bin/env python3
import os
import subprocess

# 工作目录
workdir = '/home/stest/docker/shop-boot'
# 镜像名称
image_name = ''


def del_container():
    print('')
    print('删除容器')
    (status, output) = run_shell('docker ps -a -q --filter name=shop-boot-test')
    if len(output) > 0:
        run_shell('docker rm -f ' + output)


def init_env():
    global image_name
    global workdir
    fs = os.listdir(workdir)

    for f in fs:
        if f.endswith('.tar'):
            image_name = f[:-4]
    print('镜像名称:' + image_name)


def has_image():
    """
    判断本地是否已经存在项目对应的分支和版本的镜像
    :return: True | False
    """
    print('')
    print('搜索本地镜像')
    global image_name
    (status, output) = run_shell(
        'docker images -q --filter=reference=' + image_name)

    if 0 == status:
        if 0 == len(output):
            return False
        else:
            return True


def del_image():
    print('')
    print('删除镜像')
    global image_name
    run_shell('docker rmi ' + image_name)


def load_image():
    print('')
    print('加载镜像')
    global image_name
    run_shell('docker load -i /home/stest/docker/shop-boot/' + image_name + '.tar')


def run_container():
    print('')
    print('运行容器')
    run_shell(
        'docker run -d --name shop-boot-test --privileged=true -p 8091:8091 -v '
        '/home/stest/docker/shop-boot/conf:/etc/shop-boot/conf -v '
        '/home/stest/docker/shop-boot/logs:/var/log/shop-boot shop-boot-test:0.1')


def run_shell(sh):
    print('====================================================')
    print('shell:' + sh)
    (status, output) = subprocess.getstatusoutput(sh)
    print('shell status:', status)
    print('shell output:')
    print(output)
    print('====================================================')
    print('')
    return status, output


if __name__ == '__main__':
    init_env()
    del_container()
    if has_image():
        del_image()
    load_image()
    run_container()