最近因工作需要,负责看护公司的生产环境,有16台生产服务节点,所以必须得引入自动化管理工具,不然我可不得累死啊。本文记录一下在使用Jenkins发布过程的整个过程。
1.启动Jenkins服务
2.登陆网页
3.配置
3.1点击选择“配置”
3.2添加项目描述
3.3 添加项目名称
3.4配置SVN路径
3.5配置SVN账户
3.6是否打基线
3.7配置项目列表
3.8配置发布到各节点的路径
3.9配置项目列表格式
4.构建环境
4.1配置SVN密码
4.2在Windows执行py脚本
build_spider.py编译脚本
# -*- coding: utf-8 -*-
from command import execute
import util
import os
import sys
COMMAMD_SVN_LIST = 'svn list '
COMMAMD_SVN_EXPORT = 'svn export --force '
COMMAMD_SVN_IMPORT = 'svn import -m -F '
COMMAMD_SVN_AUTHORIZATION = ' --username SVN账号 --password 密码***'
COMMAMD_PYTHON_BUILD = 'python setup.py sdist bdist_egg'
EXPORT_CODE_SUCCESS = '已导出版本'
UPLOAD_SUCCESS = '提交后的版本为'
COMPRESS_SUCCESS = '正在添加'
'''
编译spider
'''
class Build:
def __init__(self):
paramList = util.getParamList()
self.project_name = paramList[1]
self.project_list = paramList[2]
self.svn_code_repository = paramList[3]
self.svn_upload_repository = paramList[4]
self.svn_username = paramList[5]
self.svn_password = paramList[6]
self.svn_auth = ' --username ' + self.svn_username + ' --password ' + self.svn_password
self.workspace = os.getcwd()
self.conf_dir = util.getConfDir(self.project_name)
self.spiders = paramList[8]
print(self.spiders)
def getConfList(self):
print('conf dir: ' + self.conf_dir)
return os.listdir(self.conf_dir)
def exportCode2Local(self):
exportResult = False
print('export project')
project = self.project_list
if self.spiders == 'ALL':
# 全部的代码
result = execute(COMMAMD_SVN_EXPORT + util.myUrlJoin(self.svn_code_repository, project) + self.svn_auth)
print(result)
if not util.executeResult(EXPORT_CODE_SUCCESS, result):
return False
else:
exportResult = True
else:
spider_list = util.getCommaSepratedList(self.spiders)
for spider in spider_list:
pass
return exportResult
def getDetsConfFileThroughWar(self, conf):
project_folder = os.path.join(self.project_name,self.project_name)
# print(project_folder)
for fpathe,dirs,fs in os.walk(project_folder):
for f in fs:
if(conf == f):
return(os.path.join(fpathe,f))
return ''
def modifyConfig(self):
print(' in modifyConfig')
confList = self.getConfList()
print(confList)
for conf in confList:
dest = self.getDetsConfFileThroughWar(conf)
if(not len(dest) == 0):
# print('copy ' + os.path.join(self.conf_dir, conf) + ' ' + dest + ' ' +'/Y')
execute('copy ' + os.path.join(self.conf_dir, conf) + ' ' + dest + ' ' +'/Y')
print(dest)
def pypiBuild(self):
buildResult = False
for project in self.project_list:
command = COMMAMD_PYTHON_BUILD
print('build ' + project + ' start')
os.chdir(os.path.join(self.workspace, project))
print('plugin path is {}'.format(os.path))
result = execute(command)
for r in result:
print(r)
print('build ' + project + ' end')
os.chdir(os.path.join(self.workspace, project)+'\\dist')
relativeFileList = os.listdir(os.getcwd())
fileListStr = util.list2Str(relativeFileList)
print(fileListStr)
def startWork(self):
#step1: export code from svn
exportResult = self.exportCode2Local()
if exportResult == False:
print('export fail')
sys.exit(1)
else:
print('export success')
# modify config
self.modifyConfig()
#step2: upload code to svn baseline
#step3: python build
# self.pypiBuild()
#step4: update egg to svn
if __name__ == '__main__':
build = Build()
build.startWork()
4.3通过SSH推送代码到各节点指定路径,有多少节点就配置多少次即可。
4.4推送完代码后,在各节点执行start.sh脚本
这里的einfo-start.sh脚本主要是打包代码,具体代码如下:
sudo kill -9 `ps -ef |grep scrapyd|awk '{print $2}' `
sleep 3
sudo /etc/init.d/scrapyd start
sleep 3
cd /home/test/spiders/eie_nfo
sudo /root/.pyenv/versions/3.6.5/bin/scrapyd-deploy -p eie_info
4.5发布成功与否判断
保存好配置
5.构建发布
如下,控制台输出打印成功!
以上,就是我打包发布项目到各个生产服务节点的全部过程。