1. 创建一个testpack的项目
首先我们创建一个testpack的project,使用git初始化我们的代码仓库
stephen@stephen-OptiPlex-390:~/openstack/demo$ mkdir testpack
stephen@stephen-OptiPlex-390:~/openstack/demo$ cd testpack
stephen@stephen-OptiPlex-390:~/openstack/demo/testpack$ git init
Initialized empty Git repository in /home/stephen/openstack/demo/testpack/.git/
项目的目录结构:
stephen@stephen-OptiPlex-390:~/openstack/demo/testpack$ tree
.
├── LICENSE
├── README.md
├── requirements.txt
├── setup.cfg
├── setup.py
└── testpack
└── __init__.py
1 directory, 6 files
这是一个简单的项目结构:
* testpack:存放项目源代码
* 软件包管理所需的文件:setup.py,setup.cfg, requirements.txt
* 其他文件,LICENSE, README.md
2. 软件包管理的文件
2.1 setup.py
#!/usr/bin/env python
# Copyright (c) 2013 Hewlett-Packard Development Company, L.P.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# THIS FILE IS MANAGED BY THE GLOBAL REQUIREMENTS REPO - DO NOT EDIT
import setuptools
# In python < 2.7.4, a lazy loading of package `pbr` will break
# setuptools if some other modules registered functions in `atexit`.
# solution from: http://bugs.python.org/issue15881#msg170215
try:
import multiprocessing # noqa
except ImportError:
pass
setuptools.setup(
setup_requires=['pbr'],
pbr=True)
2.2 setup.cfg
[metadata] # 元数据段
name = testpack # 软件包名称
version = 0.0.1 # 软件包版本号,还可以指定preversoining, postversioning等值,具体的作用看pbr的文档。
summary = Web Application Demo # 简介
description-file = README.md # 指定README文件
author = author # 作者
author-email = author@example.com # 作者邮件
classifier = # 包的分类
Environment :: Web Environment
Intended Audience :: Developers
Intended Audience :: Education
License :: OSI Approved :: GNU General Public License v2 (GPLv2)
Operating System :: POSIX :: Linux
Programming Language :: Python
Programming Language :: Python :: 2
Programming Language :: Python :: 2.7
[global] # 全局段
setup-hooks = # 指定安装hook
pbr.hooks.setup_hook
[files] # 文件段
packages = # 包名称
testpack
[entry_points] # 指定入口点
console_scripts = # 指定要生成的可执行文件
# 下面是其他entry_points内容,主要用于指定不同功能的扩展,和打包无关。
2.3 requirements.txt
# The order of packages is significant, because pip processes them in the order
# of appearance. Changing the order has an impact on the overall integration
# process, which may cause wedges in the gate later.
pbr>=0.6
3. 打包格式
3.1 tar.gz格式:这个就是标准压缩格式,里面包含了项目元数据和代码,可以使用python setup.py sdist命令生成。
3.2 .egg格式:这个本质上也是一个压缩文件,只是扩展名换了,里面也包含了项目元数据以及源代码。这个格式由setuptools项目引入。可以通过命令python setup.py bdist_egg命令生成。
3.3 .whl格式:这个是Wheel包,也是一个压缩文件,只是扩展名换了,里面也包含了项目元数据和代码,还支持免安装直接运行。可以通过命令python setup.py bdist_wheel生成.
4. 打包python setup.py bdist_egg
stephen@stephen-OptiPlex-390:~/openstack/demo/testpack$ python setup.py bdist_egg
running bdist_egg
running egg_info
creating testpack.egg-info
writing pbr to testpack.egg-info/pbr.json
writing requirements to testpack.egg-info/requires.txt
writing testpack.egg-info/PKG-INFO
writing top-level names to testpack.egg-info/top_level.txt
writing dependency_links to testpack.egg-info/dependency_links.txt
writing entry points to testpack.egg-info/entry_points.txt
[pbr] Processing SOURCES.txt
writing manifest file 'testpack.egg-info/SOURCES.txt'
[pbr] In git context, generating filelist from git
warning: no files found matching 'AUTHORS'
warning: no files found matching 'ChangeLog'
warning: no previously-included files found matching '.gitignore'
warning: no previously-included files found matching '.gitreview'
warning: no previously-included files matching '*.pyc' found anywhere in distribution
writing manifest file 'testpack.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
creating build
creating build/lib.linux-x86_64-2.7
creating build/lib.linux-x86_64-2.7/testpack
copying testpack/__init__.py -> build/lib.linux-x86_64-2.7/testpack
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/egg
creating build/bdist.linux-x86_64/egg/testpack
copying build/lib.linux-x86_64-2.7/testpack/__init__.py -> build/bdist.linux-x86_64/egg/testpack
byte-compiling build/bdist.linux-x86_64/egg/testpack/__init__.py to __init__.pyc
creating build/bdist.linux-x86_64/egg/EGG-INFO
copying testpack.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying testpack.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying testpack.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying testpack.egg-info/entry_points.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying testpack.egg-info/not-zip-safe -> build/bdist.linux-x86_64/egg/EGG-INFO
copying testpack.egg-info/pbr.json -> build/bdist.linux-x86_64/egg/EGG-INFO
copying testpack.egg-info/requires.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying testpack.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
creating dist
creating 'dist/testpack-0.0.0-py2.7.egg' and adding 'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
在来看一下我们打包后的目录结构:
stephen@stephen-OptiPlex-390:~/openstack/demo/testpack$ tree
.
├── build
│ ├── bdist.linux-x86_64
│ └── lib.linux-x86_64-2.7
│ └── testpack
│ └── __init__.py
├── dist
│ └── testpack-0.0.0-py2.7.egg
├── LICENSE
├── README.md
├── requirements.txt
├── setup.cfg
├── setup.py
├── testpack
│ └── __init__.py
└── testpack.egg-info
├── dependency_links.txt
├── entry_points.txt
├── not-zip-safe
├── pbr.json
├── PKG-INFO
├── requires.txt
├── SOURCES.txt
└── top_level.txt
7 directories, 16 files
在dist/目录下生成了一个0.0.0版本的源码归档包,其实就是一个压缩包了,我们使用另外两个命令也会在dist/目录下生成相应文件格式的压缩包。
最后我们拷贝我们的
5. python setup.py –help-commands(查看更多是命令)
stephen@stephen-OptiPlex-390:~/openstack/demo/testpack$ python setup.py --help-commands
Standard commands:
build build everything needed to install
build_py "build" pure Python modules (copy to build directory)
build_ext build C/C++ extensions (compile/link to build directory)
build_clib build C/C++ libraries used by Python extensions
build_scripts "build" scripts (copy and fixup #! line)
clean clean up temporary files from 'build' command
install install everything from build directory
install_lib install all Python modules (extensions and pure Python)
install_headers install C/C++ header files
install_scripts install scripts (Python or otherwise)
install_data install data files
sdist create a source distribution (tarball, zip file, etc.)
register register the distribution with the Python package index
bdist create a built (binary) distribution
bdist_dumb create a "dumb" built distribution
bdist_rpm create an RPM distribution
bdist_wininst create an executable installer for MS Windows
upload upload binary package to PyPI
check perform some checks on the package
Extra commands:
saveopts save supplied options to setup.cfg or other config file
compile_catalog compile message catalogs to binary MO files
develop install package in 'development mode'
upload_docs Upload documentation to PyPI
easy_install Find/get/install Python packages
init_catalog create a new catalog based on a POT file
test run unit tests after in-place build
update_catalog update message catalogs from a POT file
testr Run unit tests using testr
setopt set an option in setup.cfg or another config file
nosetests Run unit tests using nosetests
install_egg_info Install an .egg-info directory for the package
rotate delete older distributions, keeping N newest files
bdist_wheel create a wheel distribution
egg_info create a distribution's .egg-info directory
alias define a shortcut to invoke one or more commands
extract_messages extract localizable strings from the project code
bdist_egg create an "egg" distribution
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
参考:http://segmentfault.com/a/1190000002940724
https://blog.csdn.net/u010571844/article/details/50498167
补充:https://blog.csdn.net/s1234567_89/article/details/53008444
openstack中的setup.py和setup.cfg的理解
- import setuptools
- # In python < 2.7.4, a lazy loading of package `pbr` will break
- # setuptools if some other modules registered functions in `atexit`.
- # solution from: http://bugs.python.org/issue15881#msg170215
- try:
- import multiprocessing # noqa
- except ImportError:
- pass
- setuptools.setup(
- setup_requires=['pbr>=1.8'],
- pbr=True)
这个是setup.py文件,从中看到只是使用了setuptools这个库
1.setuptools是什么和能做什么
它 是一组Python的 distutilsde工具的增强工具(适用于 Python 2.3.5 以上的版本,64 位平台则适用于 Python 2.4 以上的版本),可以让程序员更方便的创建和发布 Python 包,特别是那些对其它包具有依赖性的状况。
2.setuptools怎么使用
http://guoyunsky.iteye.com/blog/1659824,请参考该作者的实例,谢谢作者提供的例子
3.setuptools中的pbr是用来做什么的
setup.py会使用pbr从setup.cfg文件读取参数,执行命令
http://lingxiankong.github.io/blog/2013/12/23/python-setup/
我们从例子开始。假设你要分发一个叫foo的模块,文件名foo.py,那么setup.py内容如下:
- from distutils.core import setup
- setup(name='foo',
- version='1.0',
- py_modules=['foo'],
- )
然后,运行
python setup.py sdist
为模块创建一个源码包
python setup.py sdist
在当前目录下,会创建dist
目录,里面有个文件名为foo-1.0.tar.gz
,这个就是可以分发的包(如果使用命令python setup.py bdist_egg
,那么会在dist目录中生成foo-1.0-py2.7.egg包,setup.py中第一句引入需要改为from setuptools import setup
)。使用者拿到这个包后,解压,到foo-1.0目录下执行:python setup.py install
,那么,foo.py就会被拷贝到python类路径下,可以被导入使用(如果安装是egg文件,会把egg文件拷贝到dist-packages目录下)。
安装:
[root@xxx foo-0.0.2]# python setup.py install
running install
running build
running build_py
creating build
creating build/lib
copying app.py -> build/lib
running install_lib
copying build/lib/app.py -> /usr/lib/python2.7/site-packages
byte-compiling /usr/lib/python2.7/site-packages/app.py to app.pyc
running install_egg_info
Writing /usr/lib/python2.7/site-packages/foo-0.0.2-py2.7.egg-info
若要生成RPM包,执行python setup.py bdist_rpm
,但系统必须有rpm命令的支持。可以运行下面的命令查看所有格式的支持:
root@network:/kong/setup# python setup.py bdist --help-formats
List of available distribution formats:
--formats=rpm RPM distribution
--formats=gztar gzip'ed tar file
--formats=bztar bzip2'ed tar file
--formats=ztar compressed tar file
--formats=tar tar file
--formats=wininst Windows executable installer
--formats=zip ZIP file
--formats=msi Microsoft Installer
setup函数还有一些参数:
1、packages
告诉Distutils需要处理那些包(包含__init__.py
的文件夹)
2、package_dir
告诉Distutils哪些目录下的文件被映射到哪个源码包,感觉好像是一个相对路径的定义。一个例子:package_dir = {'': 'lib'}
,表示以lib为主目录。
3、ext_modules
是一个包含Extension实例的列表,Extension的定义也有一些参数。
4、ext_package
定义extension的相对路径
5、requires
定义依赖哪些模块
6、provides
定义可以为哪些模块提供依赖
7、scripts
指定python源码文件,可以从命令行执行。在安装时指定--install-script
8、package_data
通常包含与包实现相关的一些数据文件或类似于readme的文件。
package_data = {'': ['*.txt'], 'mypkg': ['data/*.dat'],}
表示包含所有目录下的txt文件和mypkg/data目录下的所有dat文件。
9、data_files
指定其他的一些文件(如配置文件)
9、data_files
指定其他的一些文件(如配置文件)
setup(...,
data_files=[('bitmaps', ['bm/b1.gif', 'bm/b2.gif']),
('config', ['cfg/data.cfg']),
('/etc/init.d', ['init-script'])]
)
规定了哪些文件被安装到哪些目录中。如果目录名是相对路径,则是相对于sys.prefix
或sys.exec_prefix
的路径。如果没有提供模板,会被添加到MANIFEST文件中。
执行sdist命令时,默认会打包哪些东西呢?
- 所有由
py_modules
或packages
指定的源码文件 - 所有由
ext_modules
或libraries
指定的C源码文件 - 由
scripts
指定的脚本文件 - 类似于test/test*.py的文件
- README.txt或README,setup.py,setup.cfg
- 所有
package_data
或data_files
指定的文件
还有一种方式是写一个manifest template,名为MANIFEST.in
,定义如何生成MANIFEST文件,内容就是需要包含在分发包中的文件。一个MANIFEST.in文件如下:
include *.txt
recursive-include examples *.txt *.py
prune examples/sample?/build
setup.cfg
setup.cfg提供一种方式,可以让包的开发者提供命令的默认选项,同时为用户提供修改的机会。对setup.cfg的解析,是在setup.py之后,在命令行执行前。
符合Distutils2的setup.cfg有些不同。包含一些sections:
1、global
定义Distutils2的全局选项,可能包含commands,compilers,setup_hook(定义脚本,在setup.cfg被读取后执行,可以修改setup.cfg的配置,pbr就用到了这个)
[global] setup-hooks = pbr.hooks.setup_hook2、
metadata
- [metadata]
- name = glance
- summary = OpenStack Image Service
- description-file =
- README.rst
- author = OpenStack
- author-email = openstack[email protected]
- home-page = http://docs.openstack.org/developer/glance/
- classifier =
- Environment :: OpenStack
- Intended Audience :: Information Technology
- Intended Audience :: System Administrators
- License :: OSI Approved :: Apache Software License
- Operating System :: POSIX :: Linux
- Programming Language :: Python
- Programming Language :: Python :: 2
- Programming Language :: Python :: 2.7
3.file
- [files]
- data_files =
- etc/glance/metadefs = etc/metadefs/*
- packages =
- glance
Setuptools
上面的setup.py和setup.cfg都是遵循python标准库中的Distutils,而setuptools工具针对Python官方的distutils做了很多针对性的功能增强,比如依赖检查,动态扩展等。很多高级功能我就不详述了,自己也没有用过,等用的时候再作补充。详情可参见这里。
setuptools相对distutils,增强的关键字:
extras_require
:当前包的高级/额外特性需要依赖的分发包。
entry_points
:这个很经典。见下面的讲解。
pbr会自动寻找 requirements.txt 进行安装
PBR
pbr是setuptools的辅助工具,最初是为OpenStack开发(https://launchpad.net/pbr),基于d2to1。> A library for managing setuptools packaging needs in a consistent manner.
pbr会读取和过滤setup.cfg中的数据,然后将解析后的数据提供给setup.py作为参数。包含如下功能:1、从git中获取Version、AUTHORS and ChangeLog信息2、Sphinx Autodoc。pbr会扫描project,找到所有模块,生成stub files3、Requirements。pbr会读取requirements.txt,生成setup函数需要的install_requires/tests_require/dependency_links
>这里需要注意,在requirements.txt文件的头部可以使用:--index https://pypi.python.org/simple/
,这一行把一个抽象的依赖声明如 requests==1.2.0 转变为一个具体的依赖声明 requests 1.2.0 from pypi.python.org/simple/
4、long_description。从README.rst, README.txt or README file中生成long_description
参数
使用pbr很简单:
from setuptools import setup
setup(
setup_requires=['pbr'],
pbr=True,
)
packages
:指定需要包含的包,行为类似于setuptools.find_packagesnamespace_packages
:指定namespace packagesdata_files
: 指定目的目录和源文件路径,一个示例:
[files] data_files = etc/glance/metadefs = etc/metadefs/* packages = glance
Babel
A collection of tools for internationalizing Python applications
Babel是 Python 的一个国际化工具包,提供了对distutils或setuptools的支持,包含一些命令。
1、compile_catalog
类似于msgfmt工具,takes a message catalog from a PO file and compiles it to a binary MO file.
./setup.py compile_catalog --directory foobar/locale--locale pt_BR
running compile_catalog
compiling catalog to foobar/locale/pt_BR/LC_MESSAGES/messages.mo
2、extract_messages
类似于xgettext,it can extract localizable messages from a variety of difference source files, and generate a PO (portable object) template file from the collected messages.
$ ./setup.py extract_messages --output-file foobar/locale/messages.pot
running extract_messages
extracting messages from foobar/__init__.py
extracting messages from foobar/core.py
...
writing PO template file to foobar/locale/messages.pot
3、update_catalog
类似于msgmerge,it updates an existing translations catalog based on a PO template file (POT).
setup.py和pip
表面上,python setup.py install
和pip install
都是用来安装python包的,实际上,pip
提供了更多的特性,更易于使用。体现在以下几个方面:
- pip会自动下载依赖,而如果使用setup.py,则需要手动搜索和下载;
- pip会自动管理包的信息,使卸载/更新更加方便和容易,使用
pip uninstall
即可。而使用setup.py,必须手动删除,有时容易出错。 - pip提供了对
virtualenv
更好的整合。