1. 使用命令历史
IPython 维护了一个小的磁盘数据库，包含执行的每条命令的文本。不同于notebook，其每个代码单元都会记录输入和输出。
1.1 搜索和复用命令历史
可以利用上下键，向上向下搜索已经执行的命令，可以键入部分开头的命令来搜索。
# 演示在命令行的操作
1 + 1
2
%run /Users/hhh/Documents/CS/利用python进行数据分析/test.py
hello word!
2 + 3
5
# 命令行中，上翻是2+3，再上翻是%run。。。，可以先键入%run，再下翻，就会只寻找最近的有重复字段的命令
%run /Users/hhh/Documents/CS/利用python进行数据分析/test.py
hello word!
# 在mac中，可以使用Ctrl-R来搜索命令（关键字）
1.2 输入和输出变量
对于输出变量：前一个输出_或__
2 ** 27
134217728
_
134217728
3 * 6
18
_
18
__
18
对于输入变量：_iX其中X比啊是输入的行号，相应的输入为_X
foo = 'bar'
foo
'bar'
_i12
"foo = 'bar'"
_i13
'foo'
_13
'bar'
_i14
'_i12'
_14
"foo = 'bar'"
exec(_i13)
print('hhh')
hhh
exec(_i20)
hhh
_i20
"print('hhh')"
一些魔术命令：
- %hist 可以用包含或者不包含行号的形式打印全部或者部分输入历史记录
- %reset 用于清除交互命名空间以及可选的输入和输出缓存
- %xdel 用于从IPython机中移除对特定对象的所有饮用。
%hist -n
1:
# 演示在命令行的操作
1 + 1
2: %run /Users/xxx/Documents/CS/利用python进行数据分析/test.py
3: 2 + 3
4:
# 命令行中，上翻是2+3，再上翻是%run。。。，可以先键入%run，再上翻，就会只寻找最近的有重复字段的命令
%run /Users/xxx/Documents/CS/利用python进行数据分析/test.py
5: # 在mac中，可以使用Ctrl-R来搜索命令（关键字）
6: 2 ** 27
7: _
8: __
9: 3 * 6
10: _
11: __
12: foo = 'bar'
13: foo
14: _i12
15: _i13
16: _13
17: _i14
18: _14
19: exec(_i13)
20: print('hhh')
21: exec(_i20)
22: _i20
23: %hist
24: %hist -l
25: %hist -l -n
26: %hist -n
%reset
Once deleted, variables cannot be recovered. Proceed (y/[n])? y
foo
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
<ipython-input-30-f1d2d2f924e9> in <module>
----> 1 foo
NameError: name 'foo' is not defined
%xdel
NameError: name '' is not defined
2. 与操作系统交互
2.1 shell命令及其别名
# 使用！开始一行，是告诉IPython在系统shell中执行其后的命令
# 也可以把命令行的输出储存在一个变量中
# 获取自己的ip
ip_info = !ifconfig en0 | grep "inet "
ip_info[0].strip()
'inet xxxxxxxx netmask 0xffffff00 broadcast xxxxxxxxxx'
# 也可以在shell命令中使用当前python环境的值
foo = 'test*'
!ls $foo
test.py
# %alias 魔术命令为shell命令定义别名
%alias ll ls -l
ll /usr
total 0
lrwxr-xr-x 1 root wheel 25 Mar 26 15:21 [35mX11[m[m -> ../private/var/select/X11
lrwxr-xr-x 1 root wheel 25 Mar 26 15:21 [35mX11R6[m[m -> ../private/var/select/X11
drwxr-xr-x 1012 root wheel 32384 Mar 26 15:21 [34mbin[m[m
drwxr-xr-x 30 root wheel 960 Mar 26 15:21 [34mlib[m[m
drwxr-xr-x 311 root wheel 9952 Mar 26 15:21 [34mlibexec[m[m
drwxr-xr-x 19 root wheel 608 Apr 11 18:06 [34mlocal[m[m
drwxr-xr-x 233 root wheel 7456 Mar 26 15:21 [34msbin[m[m
drwxr-xr-x 45 root wheel 1440 Mar 26 15:21 [34mshare[m[m
drwxr-xr-x 6 root wheel 192 Mar 26 15:21 [34mstandalone[m[m
%alias test_alias (cd examples; ls; cd ..)
test_alias
macrodata.csv spx.csv tips.csv
注意，关闭IPython窗口或者notebook，这些设置不会保留，因为没有更改系统配置。
2.2 目录书签系统
使用%bookmark创建目录的别名，方便跳转
%bookmark py4da /Users/xxx/Documents/CS/利用python进行数据分析
cd py4da
(bookmark:py4da) -> /Users/xxx/Documents/CS/利用python进行数据分析
/Users/xxx/Documents/CS/利用python进行数据分析
pwd
'/Users/xxx/Documents/CS/利用python进行数据分析'
IPython 会保存这个书签，即便关闭。另外，如果书签名称和当前工作目录中的目录名称冲突，可以使用-b标志进行覆盖，使用书签。使用%bookmark和-l可以列出所有的书签。
%bookmark -l
Current bookmarks:
py4da -> /Users/xxx/Documents/CS/利用python进行数据分析
%bookmark py3da /Users/xxx/Documents/CS/利用python进行数据分析
%bookmark -l
Current bookmarks:
py3da -> /Users/xxx/Documents/CS/利用python进行数据分析
py4da -> /Users/xxx/Documents/CS/利用python进行数据分析
删除全部书签：
%bookmark -r
%bookmark -l
Current bookmarks:
%bookmark py3da /Users/xxx/Documents/CS/利用python进行数据分析
%bookmark py4da /Users/xxx/Documents/CS/利用python进行数据分析
%bookmark -l
Current bookmarks:
py3da -> /Users/xxx/Documents/CS/利用python进行数据分析
py4da -> /Users/xxx/Documents/CS/利用python进行数据分析
删除指定书签：
%bookmark -d py3da
3. 软件开发工具
3.1 交互式调试器
在异常发生后立刻输入%debug命令，唤起“后现代”调试器，并进入抛出异常的堆栈区。
run examples/ipython_bug.py
---------------------------------------------------------------------------
AssertionError Traceback (most recent call last)
~/Documents/CS/利用python进行数据分析/examples/ipython_bug.py in <module>
1 a = 5
2 b = 6
----> 3 assert(a + b == 10)
AssertionError:
%debug
> [0;32m/Users/xxx/Documents/CS/利用python进行数据分析/examples/ipython_bug.py[0m(3)[0;36m<module>[0;34m()[0m
[0;32m 1 [0;31m[0ma[0m [0;34m=[0m [0;36m5[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m 2 [0;31m[0mb[0m [0;34m=[0m [0;36m6[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m----> 3 [0;31m[0;32massert[0m[0;34m([0m[0ma[0m [0;34m+[0m [0mb[0m [0;34m==[0m [0;36m10[0m[0;34m)[0m[0;34m[0m[0;34m[0m[0m
[0m
ipdb> u
> [0;32m/Users/xxx/opt/anaconda3/lib/python3.8/site-packages/IPython/utils/py3compat.py[0m(168)[0;36mexecfile[0;34m()[0m
[0;32m 166 [0;31m [0;32mwith[0m [0mopen[0m[0;34m([0m[0mfname[0m[0;34m,[0m [0;34m'rb'[0m[0;34m)[0m [0;32mas[0m [0mf[0m[0;34m:[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m 167 [0;31m [0mcompiler[0m [0;34m=[0m [0mcompiler[0m [0;32mor[0m [0mcompile[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m--> 168 [0;31m [0mexec[0m[0;34m([0m[0mcompiler[0m[0;34m([0m[0mf[0m[0;34m.[0m[0mread[0m[0;34m([0m[0;34m)[0m[0;34m,[0m [0mfname[0m[0;34m,[0m [0;34m'exec'[0m[0;34m)[0m[0;34m,[0m [0mglob[0m[0;34m,[0m [0mloc[0m[0;34m)[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m 169 [0;31m[0;34m[0m[0m
[0m[0;32m 170 [0;31m[0;31m# Refactor print statements in doctests.[0m[0;34m[0m[0;34m[0m[0;34m[0m[0m
[0m
ipdb> d
> [0;32m/Users/xxx/Documents/CS/利用python进行数据分析/examples/ipython_bug.py[0m(3)[0;36m<module>[0;34m()[0m
[0;32m 1 [0;31m[0ma[0m [0;34m=[0m [0;36m5[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m 2 [0;31m[0mb[0m [0;34m=[0m [0;36m6[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m----> 3 [0;31m[0;32massert[0m[0;34m([0m[0ma[0m [0;34m+[0m [0mb[0m [0;34m==[0m [0;36m10[0m[0;34m)[0m[0;34m[0m[0;34m[0m[0m
[0m
ipdb> d
*** Newest frame
ipdb> u
> [0;32m/Users/xxxx/opt/anaconda3/lib/python3.8/site-packages/IPython/utils/py3compat.py[0m(168)[0;36mexecfile[0;34m()[0m
[0;32m 166 [0;31m [0;32mwith[0m [0mopen[0m[0;34m([0m[0mfname[0m[0;34m,[0m [0;34m'rb'[0m[0;34m)[0m [0;32mas[0m [0mf[0m[0;34m:[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m 167 [0;31m [0mcompiler[0m [0;34m=[0m [0mcompiler[0m [0;32mor[0m [0mcompile[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m--> 168 [0;31m [0mexec[0m[0;34m([0m[0mcompiler[0m[0;34m([0m[0mf[0m[0;34m.[0m[0mread[0m[0;34m([0m[0;34m)[0m[0;34m,[0m [0mfname[0m[0;34m,[0m [0;34m'exec'[0m[0;34m)[0m[0;34m,[0m [0mglob[0m[0;34m,[0m [0mloc[0m[0;34m)[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m 169 [0;31m[0;34m[0m[0m
[0m[0;32m 170 [0;31m[0;31m# Refactor print statements in doctests.[0m[0;34m[0m[0;34m[0m[0;34m[0m[0m
[0m
ipdb> u
*** Oldest frame
ipdb> q
%run examples/ipython_bug.py
---------------------------------------------------------------------------
AssertionError Traceback (most recent call last)
~/Documents/CS/利用python进行数据分析/examples/ipython_bug.py in <module>
13 throws_an_exception()
14
---> 15 calling_things()
~/Documents/CS/利用python进行数据分析/examples/ipython_bug.py in calling_things()
11 def calling_things():
12 works_fine()
---> 13 throws_an_exception()
14
15 calling_things()
~/Documents/CS/利用python进行数据分析/examples/ipython_bug.py in throws_an_exception()
7 a = 5
8 b = 6
----> 9 assert(a + b == 10)
10
11 def calling_things():
AssertionError:
%debug
> [0;32m/Users/xxx/Documents/CS/利用python进行数据分析/examples/ipython_bug.py[0m(9)[0;36mthrows_an_exception[0;34m()[0m
[0;32m 7 [0;31m [0ma[0m [0;34m=[0m [0;36m5[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m 8 [0;31m [0mb[0m [0;34m=[0m [0;36m6[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m----> 9 [0;31m [0;32massert[0m[0;34m([0m[0ma[0m [0;34m+[0m [0mb[0m [0;34m==[0m [0;36m10[0m[0;34m)[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m 10 [0;31m[0;34m[0m[0m
[0m[0;32m 11 [0;31m[0;32mdef[0m [0mcalling_things[0m[0;34m([0m[0;34m)[0m[0;34m:[0m[0;34m[0m[0;34m[0m[0m
[0m
ipdb> u
> [0;32m/Users/xxx/Documents/CS/利用python进行数据分析/examples/ipython_bug.py[0m(13)[0;36mcalling_things[0;34m()[0m
[0;32m 11 [0;31m[0;32mdef[0m [0mcalling_things[0m[0;34m([0m[0;34m)[0m[0;34m:[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m 12 [0;31m [0mworks_fine[0m[0;34m([0m[0;34m)[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m---> 13 [0;31m [0mthrows_an_exception[0m[0;34m([0m[0;34m)[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m 14 [0;31m[0;34m[0m[0m
[0m[0;32m 15 [0;31m[0mcalling_things[0m[0;34m([0m[0;34m)[0m[0;34m[0m[0;34m[0m[0m
[0m
ipdb> q
%run -d:打开调试器执行
%run -d examples/ipython_bug.py
Breakpoint 1 at /Users/xxx/Documents/CS/利用python进行数据分析/examples/ipython_bug.py:1
NOTE: Enter 'c' at the ipdb> prompt to continue execution.
> [0;32m/Users/sss/Documents/CS/利用python进行数据分析/examples/ipython_bug.py[0m(1)[0;36m<module>[0;34m()[0m
[1;31m1[0;32m---> 1 [0;31m[0;32mdef[0m [0mworks_fine[0m[0;34m([0m[0;34m)[0m[0;34m:[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m 2 [0;31m [0ma[0m [0;34m=[0m [0;36m5[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m 3 [0;31m [0mb[0m [0;34m=[0m [0;36m6[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m 4 [0;31m [0;32massert[0m[0;34m([0m[0ma[0m [0;34m+[0m [0mb[0m [0;34m==[0m [0;36m11[0m[0;34m)[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m 5 [0;31m[0;34m[0m[0m
[0m
ipdb> s
> [0;32m/Users/xxx/Documents/CS/利用python进行数据分析/examples/ipython_bug.py[0m(6)[0;36m<module>[0;34m()[0m
[0;32m 4 [0;31m [0;32massert[0m[0;34m([0m[0ma[0m [0;34m+[0m [0mb[0m [0;34m==[0m [0;36m11[0m[0;34m)[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m 5 [0;31m[0;34m[0m[0m
[0m[0;32m----> 6 [0;31m[0;32mdef[0m [0mthrows_an_exception[0m[0;34m([0m[0;34m)[0m[0;34m:[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m 7 [0;31m [0ma[0m [0;34m=[0m [0;36m5[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m 8 [0;31m [0mb[0m [0;34m=[0m [0;36m6[0m[0;34m[0m[0;34m[0m[0m
[0m
ipdb> s
> [0;32m/Users/xxx/Documents/CS/利用python进行数据分析/examples/ipython_bug.py[0m(11)[0;36m<module>[0;34m()[0m
[0;32m 9 [0;31m [0;32massert[0m[0;34m([0m[0ma[0m [0;34m+[0m [0mb[0m [0;34m==[0m [0;36m10[0m[0xxx;34m)[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m 10 [0;31m[0;34m[0m[0m
[0m[0;32m---> 11 [0;31m[0;32mdef[0m [0mcalling_things[0m[0;34m([0m[0;34m)[0m[0;34m:[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m 12 [0;31m [0mworks_fine[0m[0;34m([0m[0;34m)[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m 13 [0;31m [0mthrows_an_exception[0m[0;34m([0m[0;34m)[0m[0;34m[0m[0;34m[0m[0m
[0m
ipdb> b 12
Breakpoint 2 at /Users/xxx/Documents/CS/利用python进行数据分析/examples/ipython_bug.py:12
ipdb> c
> [0;32m/Users/xxx/Documents/CS/利用python进行数据分析/examples/ipython_bug.py[0m(12)[0;36mcalling_things[0;34m()[0m
[0;32m 10 [0;31m[0;34m[0m[0m
[0m[0;32m 11 [0;31m[0;32mdef[0m [0mcalling_things[0m[0;34m([0m[0;34m)[0m[0;34m:[0m[0;34m[0m[0;34m[0m[0m
[0m[1;31m2[0;32m--> 12 [0;31m [0mworks_fine[0m[0;34m([0m[0;34m)[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m 13 [0;31m [0mthrows_an_exception[0m[0;34m([0m[0;34m)[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m 14 [0;31m[0;34m[0m[0m
[0m
ipdb> n
> [0;32m/Users/xxx/Documents/CS/利用python进行数据分析/examples/ipython_bug.py[0m(13)[0;36mcalling_things[0;34m()[0m
[0;32m 11 [0;31m[0;32mdef[0m [0mcalling_things[0m[0;34m([0m[0;34m)[0m[0;34m:[0m[0;34m[0m[0;34m[0m[0m
[0m[1;31m2[0;32m 12 [0;31m [0mworks_fine[0m[0;34m([0m[0;34m)[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m---> 13 [0;31m [0mthrows_an_exception[0m[0;34m([0m[0;34m)[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m 14 [0;31m[0;34m[0m[0m
[0m[0;32m 15 [0;31m[0mcalling_things[0m[0;34m([0m[0;34m)[0m[0;34m[0m[0;34m[0m[0m
[0m
ipdb> s
--Call--
> [0;32m/Users/xxx/Documents/CS/利用python进行数据分析/examples/ipython_bug.py[0m(6)[0;36mthrows_an_exception[0;34m()[0m
[0;32m 4 [0;31m [0;32massert[0m[0;34m([0m[0ma[0m [0;34m+[0m [0mb[0m [0;34m==[0m [0;36m11[0m[0;34m)[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m 5 [0;31m[0;34m[0m[0m
[0m[0;32m----> 6 [0;31m[0;32mdef[0m [0mthrows_an_exception[0m[0;34m([0m[0;34m)[0m[0;34m:[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m 7 [0;31m [0ma[0m [0;34m=[0m [0;36m5[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m 8 [0;31m [0mb[0m [0;34m=[0m [0;36m6[0m[0;34m[0m[0;34m[0m[0m
[0m
ipdb> n
> [0;32m/Users/xxx/Documents/CS/利用python进行数据分析/examples/ipython_bug.py[0m(7)[0;36mthrows_an_exception[0;34m()[0m
[0;32m 5 [0;31m[0;34m[0m[0m
[0m[0;32m 6 [0;31m[0;32mdef[0m [0mthrows_an_exception[0m[0;34m([0m[0;34m)[0m[0;34m:[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m----> 7 [0;31m [0ma[0m [0;34m=[0m [0;36m5[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m 8 [0;31m [0mb[0m [0;34m=[0m [0;36m6[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m 9 [0;31m [0;32massert[0m[0;34m([0m[0ma[0m [0;34m+[0m [0mb[0m [0;34m==[0m [0;36m10[0m[0;34m)[0m[0;34m[0m[0;34m[0m[0m
[0m
ipdb> n
> [0;32m/Users/xxx/Documents/CS/利用python进行数据分析/examples/ipython_bug.py[0m(8)[0;36mthrows_an_exception[0;34m()[0m
[0;32m 6 [0;31m[0;32mdef[0m [0mthrows_an_exception[0m[0;34m([0m[0;34m)[0m[0;34m:[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m 7 [0;31m [0ma[0m [0;34m=[0m [0;36m5[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m----> 8 [0;31m [0mb[0m [0;34m=[0m [0;36m6[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m 9 [0;31m [0;32massert[0m[0;34m([0m[0ma[0m [0;34m+[0m [0mb[0m [0;34m==[0m [0;36m10[0m[0;34m)[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m 10 [0;31m[0;34m[0m[0m
[0m
ipdb>
> [0;32m/Users/xxx/Documents/CS/利用python进行数据分析/examples/ipython_bug.py[0m(9)[0;36mthrows_an_exception[0;34m()[0m
[0;32m 7 [0;31m [0ma[0m [0;34m=[0m [0;36m5[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m 8 [0;31m [0mb[0m [0;34m=[0m [0;36m6[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m----> 9 [0;31m [0;32massert[0m[0;34m([0m[0ma[0m [0;34m+[0m [0mb[0m [0;34m==[0m [0;36m10[0m[0;34m)[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m 10 [0;31m[0;34m[0m[0m
[0m[0;32m 11 [0;31m[0;32mdef[0m [0mcalling_things[0m[0;34m([0m[0;34m)[0m[0;34m:[0m[0;34m[0m[0;34m[0m[0m
[0m
ipdb> !a
5
ipdb> !b
6
ipdb> q
在调试器模式下：
b + 行号：设置断点；c：运行直到断点位置；n：执行到下一行；s:进入脚本(函数)；!变量：检查变量内容；q: 退出；
3.1.1 调试器的其他用途
%pdb
Automatic pdb calling has been turned ON
%pdb
Automatic pdb calling has been turned OFF
# 将常用的技巧添加进IPython配置文件
from IPython.core.debugger import Pdb
# 第一个函数set_trace很简单，你可以将它放在代码中任何希望停下来查看一番的地方
def set_trace():
import sys
Pdb().set_trace(sys._getframe().f_back)
# 第二个函数(debug)，使你能够直接在任意函数上使用调试器。
def debug(f, *args, **kwargs):
pdb = Pdb()
return pdb.runcall(f, *args, **kwargs)
对于原来的py文件修改：
def works_fine():
a = 5
b = 6
assert(a + b == 11)
def throws_an_exception():
a = 5
b = 6
assert(a + b == 10)
def f(x, y, z=1):
tmp = x + y
return tmp / z
def set_trace():
from IPython.core.debugger import Pdb
import sys
Pdb().set_trace(sys._getframe().f_back)
def debug(f, *args, **kwargs):
from IPython.core.debugger import Pdb
pdb = Pdb()
return pdb.runcall(f, *args, **kwargs)
def calling_things():
set_trace() #自定义的调试函数
works_fine()
throws_an_exception()
calling_things()
run examples/ipython_bug.py
> [0;32m/Users/xxx/Documents/CS/利用python进行数据分析/examples/ipython_bug.py[0m(27)[0;36mcalling_things[0;34m()[0m
[0;32m 25 [0;31m[0;32mdef[0m [0mcalling_things[0m[0;34m([0m[0;34m)[0m[0;34m:[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m 26 [0;31m [0mset_trace[0m[0;34m([0m[0;34m)[0m [0;31m#自定义的调试函数[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m---> 27 [0;31m [0mworks_fine[0m[0;34m([0m[0;34m)[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m 28 [0;31m [0mthrows_an_exception[0m[0;34m([0m[0;34m)[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m 29 [0;31m[0;34m[0m[0m
[0m
ipdb> c
---------------------------------------------------------------------------
AssertionError Traceback (most recent call last)
~/Documents/CS/利用python进行数据分析/examples/ipython_bug.py in <module>
28 throws_an_exception()
29
---> 30 calling_things()
31
~/Documents/CS/利用python进行数据分析/examples/ipython_bug.py in calling_things()
25 def calling_things():
26 set_trace() #自定义的调试函数
---> 27 works_fine()
28 throws_an_exception()
29
~/Documents/CS/利用python进行数据分析/examples/ipython_bug.py in throws_an_exception()
7 a = 5
8 b = 6
----> 9 assert(a + b == 10)
10
11 def f(x, y, z=1):
AssertionError:
%debug
> [0;32m/Users/xxx/Documents/CS/利用python进行数据分析/examples/ipython_bug.py[0m(9)[0;36mthrows_an_exception[0;34m()[0m
[0;32m 7 [0;31m [0ma[0m [0;34m=[0m [0;36m5[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m 8 [0;31m [0mb[0m [0;34m=[0m [0;36m6[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m----> 9 [0;31m [0;32massert[0m[0;34m([0m[0ma[0m [0;34m+[0m [0mb[0m [0;34m==[0m [0;36m10[0m[0;34m)[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m 10 [0;31m[0;34m[0m[0m
[0m[0;32m 11 [0;31m[0;32mdef[0m [0mf[0m[0;34m([0m[0mx[0m[0;34m,[0m [0my[0m[0;34m,[0m [0mz[0m[0;34m=[0m[0;36m1[0m[0;34m)[0m[0;34m:[0m[0;34m[0m[0;34m[0m[0m
[0m
ipdb> !debug(f, 1, 2, z=3)
> [0;32m/Users/xxx/Documents/CS/利用python进行数据分析/examples/ipython_bug.py[0m(12)[0;36mf[0;34m()[0m
[0;32m 10 [0;31m[0;34m[0m[0m
[0m[0;32m 11 [0;31m[0;32mdef[0m [0mf[0m[0;34m([0m[0mx[0m[0;34m,[0m [0my[0m[0;34m,[0m [0mz[0m[0;34m=[0m[0;36m1[0m[0;34m)[0m[0;34m:[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m---> 12 [0;31m [0mtmp[0m [0;34m=[0m [0mx[0m [0;34m+[0m [0my[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m 13 [0;31m [0;32mreturn[0m [0mtmp[0m [0;34m/[0m [0mz[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m 14 [0;31m[0;34m[0m[0m
[0m
ipdb> c
1.0
ipdb> c
def f(x, y, z=1):
tmp = x + y
return tmp / z
# 打开pdb运行-d，并设置断点-b
%run -d -b27 examples/ipython_bug.py
Breakpoint 1 at /Users/xxx/Documents/CS/利用python进行数据分析/examples/ipython_bug.py:27
NOTE: Enter 'c' at the ipdb> prompt to continue execution.
> [0;32m/Users/xxx/Documents/CS/利用python进行数据分析/examples/ipython_bug.py[0m(1)[0;36m<module>[0;34m()[0m
[0;32m----> 1 [0;31m[0;32mdef[0m [0mworks_fine[0m[0;34m([0m[0;34m)[0m[0;34m:[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m 2 [0;31m [0ma[0m [0;34m=[0m [0;36m5[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m 3 [0;31m [0mb[0m [0;34m=[0m [0;36m6[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m 4 [0;31m [0;32massert[0m[0;34m([0m[0ma[0m [0;34m+[0m [0mb[0m [0;34m==[0m [0;36m11[0m[0;34m)[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m 5 [0;31m[0;34m[0m[0m
[0m
ipdb> c
> [0;32m/Users/xxx/Documents/CS/利用python进行数据分析/examples/ipython_bug.py[0m(27)[0;36mcalling_things[0;34m()[0m
[0;32m 25 [0;31m[0;32mdef[0m [0mcalling_things[0m[0;34m([0m[0;34m)[0m[0;34m:[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m 26 [0;31m [0mset_trace[0m[0;34m([0m[0;34m)[0m [0;31m#自定义的调试函数[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m---> 27 [0;31m [0mworks_fine[0m[0;34m([0m[0;34m)[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m 28 [0;31m [0mthrows_an_exception[0m[0;34m([0m[0;34m)[0m[0;34m[0m[0;34m[0m[0m
[0m[0;32m 29 [0;31m[0;34m[0m[0m
[0m
ipdb> c
[0;31m---------------------------------------------------------------------------[0m
[0;31mAssertionError[0m Traceback (most recent call last)
[0;32m~/opt/anaconda3/lib/python3.8/site-packages/IPython/core/interactiveshell.py[0m in [0;36msafe_execfile[0;34m(self, fname, exit_ignore, raise_exceptions, shell_futures, *where)[0m
[1;32m 2755[0m [0;32mtry[0m[0;34m:[0m[0;34m[0m[0;34m[0m[0m
[1;32m 2756[0m [0mglob[0m[0;34m,[0m [0mloc[0m [0;34m=[0m [0;34m([0m[0mwhere[0m [0;34m+[0m [0;34m([0m[0;32mNone[0m[0;34m,[0m [0;34m)[0m[0;34m)[0m[0;34m[[0m[0;34m:[0m[0;36m2[0m[0;34m][0m[0;34m[0m[0;34m[0m[0m
[0;32m-> 2757[0;31m py3compat.execfile(
[0m[1;32m 2758[0m [0mfname[0m[0;34m,[0m [0mglob[0m[0;34m,[0m [0mloc[0m[0;34m,[0m[0;34m[0m[0;34m[0m[0m
[1;32m 2759[0m self.compile if shell_futures else None)
[0;32m~/opt/anaconda3/lib/python3.8/site-packages/IPython/utils/py3compat.py[0m in [0;36mexecfile[0;34m(fname, glob, loc, compiler)[0m
[1;32m 166[0m [0;32mwith[0m [0mopen[0m[0;34m([0m[0mfname[0m[0;34m,[0m [0;34m'rb'[0m[0;34m)[0m [0;32mas[0m [0mf[0m[0;34m:[0m[0;34m[0m[0;34m[0m[0m
[1;32m 167[0m [0mcompiler[0m [0;34m=[0m [0mcompiler[0m [0;32mor[0m [0mcompile[0m[0;34m[0m[0;34m[0m[0m
[0;32m--> 168[0;31m [0mexec[0m[0;34m([0m[0mcompiler[0m[0;34m([0m[0mf[0m[0;34m.[0m[0mread[0m[0;34m([0m[0;34m)[0m[0;34m,[0m [0mfname[0m[0;34m,[0m [0;34m'exec'[0m[0;34m)[0m[0;34m,[0m [0mglob[0m[0;34m,[0m [0mloc[0m[0;34m)[0m[0;34m[0m[0;34m[0m[0m
[0m[1;32m 169[0m [0;34m[0m[0m
[1;32m 170[0m [0;31m# Refactor print statements in doctests.[0m[0;34m[0m[0;34m[0m[0;34m[0m[0m
[0;32m~/Documents/CS/利用python进行数据分析/examples/ipython_bug.py[0m in [0;36m<module>[0;34m[0m
[1;32m 28[0m [0mthrows_an_exception[0m[0;34m([0m[0;34m)[0m[0;34m[0m[0;34m[0m[0m
[1;32m 29[0m [0;34m[0m[0m
[0;32m---> 30[0;31m [0mcalling_things[0m[0;34m([0m[0;34m)[0m[0;34m[0m[0;34m[0m[0m
[0m[1;32m 31[0m [0;34m[0m[0m
[0;32m~/Documents/CS/利用python进行数据分析/examples/ipython_bug.py[0m in [0;36mcalling_things[0;34m()[0m
[1;32m 25[0m [0;32mdef[0m [0mcalling_things[0m[0;34m([0m[0;34m)[0m[0;34m:[0m[0;34m[0m[0;34m[0m[0m
[1;32m 26[0m [0mset_trace[0m[0;34m([0m[0;34m)[0m [0;31m#自定义的调试函数[0m[0;34m[0m[0;34m[0m[0m
[0;32m---> 27[0;31m [0mworks_fine[0m[0;34m([0m[0;34m)[0m[0;34m[0m[0;34m[0m[0m
[0m[1;32m 28[0m [0mthrows_an_exception[0m[0;34m([0m[0;34m)[0m[0;34m[0m[0;34m[0m[0m
[1;32m 29[0m [0;34m[0m[0m
[0;32m~/Documents/CS/利用python进行数据分析/examples/ipython_bug.py[0m in [0;36mthrows_an_exception[0;34m()[0m
[1;32m 7[0m [0ma[0m [0;34m=[0m [0;36m5[0m[0;34m[0m[0;34m[0m[0m
[1;32m 8[0m [0mb[0m [0;34m=[0m [0;36m6[0m[0;34m[0m[0;34m[0m[0m
[0;32m----> 9[0;31m [0;32massert[0m[0;34m([0m[0ma[0m [0;34m+[0m [0mb[0m [0;34m==[0m [0;36m10[0m[0;34m)[0m[0;34m[0m[0;34m[0m[0m
[0m[1;32m 10[0m [0;34m[0m[0m
[1;32m 11[0m [0;32mdef[0m [0mf[0m[0;34m([0m[0mx[0m[0;34m,[0m [0my[0m[0;34m,[0m [0mz[0m[0;34m=[0m[0;36m1[0m[0;34m)[0m[0;34m:[0m[0;34m[0m[0;34m[0m[0m
[0;31mAssertionError[0m:
3.2 对代码测时： %time 和 %timeit
import numpy as np
# 通常的时间测试方式
import time
iterations = 5
start = time.time()
for i in range(iterations):
np.random.randn(10000, 10000).mean(axis=1)
elapsed_per = (time.time() - start) / iterations
print(elapsed_per)
2.748708629608154
魔术命令：%time一次运行一条语句（即只测试运行一次的时间），报告总执行时间。
strings = ['foo', 'foobar', 'baz', 'qux', 'python', 'Guido Van Rossum'] * 100000
%time method1 = [x for x in strings if x.startswith('foo')]
CPU times: user 84.4 ms, sys: 2.98 ms, total: 87.4 ms
Wall time: 86.2 ms
%time method2 = [x for x in strings if x[:3] == 'foo']
CPU times: user 69.8 ms, sys: 2.64 ms, total: 72.4 ms
Wall time: 71.4 ms
其中，Wall time是wall-clock time的简写，表示壁钟时，是要关注的数字。
与%time相比%timeit可以平均多次结果，从而更加准确。
%timeit method1 = [x for x in strings if x.startswith('foo')]
69.4 ms ± 2.1 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
%timeit method2 = [x for x in strings if x[:3] == 'foo']
51.5 ms ± 984 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
x = 'foobar'
y = 'foo'
%timeit x.startswith(y)
121 ns ± 12.5 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)
%timeit x[:3] == y
117 ns ± 8.42 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)
3.3 基础分析：%prun和%run -p
python自身的分析工具cProfile模块可以在命令行运行整个程序，并输出每个函数的聚合时间。
import numpy as np
from numpy.linalg import eigvals
def run_experiment(niter=100):
K = 100
results = []
for _ in range(niter):
mat = np.random.randn(K, K)
max_eigenvalue = np.abs(eigvals(mat)).max()
results.append(max_eigenvalue)
return results
some_results = run_experiment()
print(f'Largest one we saw: {
np.max(some_results)}')
Largest one we saw: 11.393466801031785
# 上面代码的脚本
%run -m cProfile examples/cprof_example.py
# 命令行中可以改为：python -m cProfile examples/cprof_example.py
Largest one we saw: 11.81710052380441
4151 function calls (4051 primitive calls) in 0.192 seconds
Ordered by: standard name
ncalls tottime percall cumtime percall filename:lineno(function)
100 0.000 0.000 0.002 0.000 <__array_function__ internals>:2(all)
1 0.000 0.000 0.000 0.000 <__array_function__ internals>:2(amax)
100 0.000 0.000 0.162 0.002 <__array_function__ internals>:2(eigvals)
1 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap>:1017(_handle_fromlist)
100 0.000 0.000 0.000 0.000 _asarray.py:23(asarray)
100 0.000 0.000 0.001 0.000 _methods.py:37(_amax)
100 0.000 0.000 0.001 0.000 _methods.py:59(_all)
1 0.000 0.000 0.192 0.192 cprof_example.py:1(<module>)
1 0.001 0.001 0.192 0.192 cprof_example.py:4(run_experiment)
100 0.000 0.000 0.000 0.000 fromnumeric.py:2350(_all_dispatcher)
100 0.000 0.000 0.001 0.000 fromnumeric.py:2355(all)
1 0.000 0.000 0.000 0.000 fromnumeric.py:2612(_amax_dispatcher)
1 0.000 0.000 0.000 0.000 fromnumeric.py:2617(amax)
101 0.000 0.000 0.001 0.000 fromnumeric.py:70(_wrapreduction)
101 0.000 0.000 0.000 0.000 fromnumeric.py:71(<dictcomp>)
3 0.000 0.000 0.000 0.000 iostream.py:197(schedule)
2 0.000 0.000 0.000 0.000 iostream.py:310(_is_master_process)
2 0.000 0.000 0.000 0.000 iostream.py:323(_schedule_flush)
2 0.000 0.000 0.000 0.000 iostream.py:386(write)
3 0.000 0.000 0.000 0.000 iostream.py:93(_event_pipe)
100 0.000 0.000 0.000 0.000 linalg.py:102(get_linalg_error_extobj)
100 0.000 0.000 0.000 0.000 linalg.py:107(_makearray)
300 0.000 0.000 0.000 0.000 linalg.py:112(isComplexType)
100 0.000 0.000 0.000 0.000 linalg.py:125(_realType)
100 0.000 0.000 0.000 0.000 linalg.py:128(_complexType)
100 0.000 0.000 0.000 0.000 linalg.py:135(_commonType)
100 0.000 0.000 0.000 0.000 linalg.py:193(_assert_stacked_2d)
100 0.000 0.000 0.000 0.000 linalg.py:199(_assert_stacked_square)
100 0.001 0.000 0.001 0.000 linalg.py:205(_assert_finite)
100 0.000 0.000 0.000 0.000 linalg.py:472(_unary_dispatcher)
100 0.158 0.002 0.162 0.002 linalg.py:988(eigvals)
3 0.000 0.000 0.000 0.000 socket.py:432(send)
3 0.000 0.000 0.000 0.000 threading.py:1017(_wait_for_tstate_lock)
3 0.000 0.000 0.000 0.000 threading.py:1071(is_alive)
3 0.000 0.000 0.000 0.000 threading.py:513(is_set)
1 0.000 0.000 0.192 0.192 {built-in method builtins.exec}
101 0.000 0.000 0.000 0.000 {built-in method builtins.getattr}
1 0.000 0.000 0.000 0.000 {built-in method builtins.hasattr}
3 0.000 0.000 0.000 0.000 {built-in method builtins.isinstance}
400 0.000 0.000 0.000 0.000 {built-in method builtins.issubclass}
1 0.000 0.000 0.000 0.000 {built-in method builtins.print}
100 0.000 0.000 0.000 0.000 {built-in method numpy.array}
201/101 0.000 0.000 0.162 0.002 {built-in method numpy.core._multiarray_umath.implement_array_function}
2 0.000 0.000 0.000 0.000 {built-in method posix.getpid}
3 0.000 0.000 0.000 0.000 {method 'acquire' of '_thread.lock' objects}
100 0.000 0.000 0.001 0.000 {method 'all' of 'numpy.ndarray' objects}
3 0.000 0.000 0.000 0.000 {method 'append' of 'collections.deque' objects}
100 0.000 0.000 0.000 0.000 {method 'append' of 'list' objects}
100 0.000 0.000 0.000 0.000 {method 'astype' of 'numpy.ndarray' objects}
1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}
200 0.000 0.000 0.000 0.000 {method 'get' of 'dict' objects}
101 0.000 0.000 0.000 0.000 {method 'items' of 'dict' objects}
100 0.000 0.000 0.001 0.000 {method 'max' of 'numpy.ndarray' objects}
100 0.028 0.000 0.028 0.000 {method 'randn' of 'numpy.random.mtrand.RandomState' objects}
301 0.002 0.000 0.002 0.000 {method 'reduce' of 'numpy.ufunc' objects}
%run -m cProfile -s cumulative examples/cprof_example.py
Largest one we saw: 11.66632455651208
4151 function calls (4051 primitive calls) in 0.185 seconds
Ordered by: cumulative time
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.000 0.000 0.185 0.185 {built-in method builtins.exec}
1 0.000 0.000 0.185 0.185 cprof_example.py:1(<module>)
1 0.002 0.002 0.185 0.185 cprof_example.py:4(run_experiment)
100 0.000 0.000 0.156 0.002 <__array_function__ internals>:2(eigvals)
201/101 0.000 0.000 0.155 0.002 {built-in method numpy.core._multiarray_umath.implement_array_function}
100 0.152 0.002 0.155 0.002 linalg.py:988(eigvals)
100 0.027 0.000 0.027 0.000 {method 'randn' of 'numpy.random.mtrand.RandomState' objects}
301 0.001 0.000 0.001 0.000 {method 'reduce' of 'numpy.ufunc' objects}
100 0.001 0.000 0.001 0.000 linalg.py:205(_assert_finite)
100 0.000 0.000 0.001 0.000 <__array_function__ internals>:2(all)
100 0.000 0.000 0.001 0.000 fromnumeric.py:2355(all)
101 0.000 0.000 0.001 0.000 fromnumeric.py:70(_wrapreduction)
100 0.000 0.000 0.001 0.000 {method 'max' of 'numpy.ndarray' objects}
100 0.000 0.000 0.001 0.000 {method 'all' of 'numpy.ndarray' objects}
100 0.000 0.000 0.001 0.000 _methods.py:37(_amax)
100 0.000 0.000 0.001 0.000 _methods.py:59(_all)
100 0.000 0.000 0.000 0.000 linalg.py:135(_commonType)
100 0.000 0.000 0.000 0.000 linalg.py:107(_makearray)
100 0.000 0.000 0.000 0.000 {method 'astype' of 'numpy.ndarray' objects}
300 0.000 0.000 0.000 0.000 linalg.py:112(isComplexType)
100 0.000 0.000 0.000 0.000 _asarray.py:23(asarray)
100 0.000 0.000 0.000 0.000 linalg.py:102(get_linalg_error_extobj)
1 0.000 0.000 0.000 0.000 {built-in method builtins.print}
2 0.000 0.000 0.000 0.000 iostream.py:386(write)
100 0.000 0.000 0.000 0.000 linalg.py:199(_assert_stacked_square)
101 0.000 0.000 0.000 0.000 fromnumeric.py:71(<dictcomp>)
3 0.000 0.000 0.000 0.000 iostream.py:197(schedule)
100 0.000 0.000 0.000 0.000 linalg.py:128(_complexType)
400 0.000 0.000 0.000 0.000 {built-in method builtins.issubclass}
100 0.000 0.000 0.000 0.000 linalg.py:193(_assert_stacked_2d)
100 0.000 0.000 0.000 0.000 linalg.py:125(_realType)
100 0.000 0.000 0.000 0.000 {built-in method numpy.array}
3 0.000 0.000 0.000 0.000 socket.py:432(send)
200 0.000 0.000 0.000 0.000 {method 'get' of 'dict' objects}
101 0.000 0.000 0.000 0.000 {built-in method builtins.getattr}
100 0.000 0.000 0.000 0.000 fromnumeric.py:2350(_all_dispatcher)
101 0.000 0.000 0.000 0.000 {method 'items' of 'dict' objects}
100 0.000 0.000 0.000 0.000 linalg.py:472(_unary_dispatcher)
1 0.000 0.000 0.000 0.000 <__array_function__ internals>:2(amax)
100 0.000 0.000 0.000 0.000 {method 'append' of 'list' objects}
1 0.000 0.000 0.000 0.000 fromnumeric.py:2617(amax)
3 0.000 0.000 0.000 0.000 threading.py:1071(is_alive)
2 0.000 0.000 0.000 0.000 iostream.py:323(_schedule_flush)
3 0.000 0.000 0.000 0.000 threading.py:1017(_wait_for_tstate_lock)
1 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap>:1017(_handle_fromlist)
2 0.000 0.000 0.000 0.000 iostream.py:310(_is_master_process)
3 0.000 0.000 0.000 0.000 iostream.py:93(_event_pipe)
3 0.000 0.000 0.000 0.000 {method 'acquire' of '_thread.lock' objects}
3 0.000 0.000 0.000 0.000 {built-in method builtins.isinstance}
2 0.000 0.000 0.000 0.000 {built-in method posix.getpid}
3 0.000 0.000 0.000 0.000 {method 'append' of 'collections.deque' objects}
3 0.000 0.000 0.000 0.000 threading.py:513(is_set)
1 0.000 0.000 0.000 0.000 {built-in method builtins.hasattr}
1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}
1 0.000 0.000 0.000 0.000 fromnumeric.py:2612(_amax_dispatcher)
其中的cumtime表示每个函数话费的总时间。
在IPython中，可以使用%prun或者%run -p完成此功能。会有独立窗口出现。
%prun run_experiment()
前7行结果
%prun -l 7 run_experiment()
排序方式
%prun -l 7 -s cumulative run_experiment()
%run -p -s cumulative examples/cprof_example.py
Largest one we saw: 11.420482039182037
%%prun run_experiment()
UsageError: %%prun is a cell magic, but the cell body is empty. Did you mean the line magic %prun (single %)?
jupyter中，可以使用%%prun来分析单个的代码块
%%prun -s cumulative
import numpy as np
from numpy.linalg import eigvals
def run_experiment(niter=100):
K = 100
results = []
for _ in range(niter):
mat = np.random.randn(K, K)
max_eigenvalue = np.abs(eigvals(mat)).max()
results.append(max_eigenvalue)
return results
some_results = run_experiment()
print(f'Largest one we saw: {
np.max(some_results)}')
Largest one we saw: 12.581609374118653
一个好用的运行时间的可视化工具SnakeViz
pip install snakeviz
Collecting snakeviz
Downloading snakeviz-2.1.1-py2.py3-none-any.whl (282 kB)
[K |████████████████████████████████| 282 kB 1.1 MB/s eta 0:00:01
[?25hRequirement already satisfied: tornado>=2.0 in /Users/xxx/opt/anaconda3/lib/python3.8/site-packages (from snakeviz) (6.1)
Installing collected packages: snakeviz
Successfully installed snakeviz-2.1.1
Note: you may need to restart the kernel to use updated packages.
%load_ext snakeviz
%%snakeviz
import numpy as np
from numpy.linalg import eigvals
def run_experiment(niter=100):
K = 100
results = []
for _ in range(niter):
mat = np.random.randn(K, K)
max_eigenvalue = np.abs(eigvals(mat)).max()
results.append(max_eigenvalue)
return results
some_results = run_experiment()
print(f'Largest one we saw: {
np.max(some_results)}')
Largest one we saw: 12.068346019407974
*** Profile stats marshalled to file '/var/folders/bj/9d8bzmns5q936gy102k3t0m40000gn/T/tmppa3ok_kq'.
Embedding SnakeViz in this document...
%snakeviz run_experiment()
*** Profile stats marshalled to file '/var/folders/bj/9d8bzmns5q936gy102k3t0m40000gn/T/tmpukezgj5q'.
Embedding SnakeViz in this document...
3.4 逐行分析函数
%lprun可以对一个或者多个函数进行逐行分析
一般语法为：%lprun -f func1 -f func2 statement_to_profile
pip install line_profiler
Collecting line_profiler
Downloading line_profiler-3.5.1-cp38-cp38-macosx_10_9_x86_64.whl (53 kB)
[K |████████████████████████████████| 53 kB 842 kB/s eta 0:00:01
[?25hInstalling collected packages: line-profiler
Successfully installed line-profiler-3.5.1
Note: you may need to restart the kernel to use updated packages.
%load_ext line_profiler
The line_profiler extension is already loaded. To reload it, use:
%reload_ext line_profiler
import numpy as np
def add_and_sum(x, y):
added = x + y
summed = added.sum(axis=1)
return summed
def call_function():
x = np.random.randn(1000, 1000)
y = np.random.randn(1000, 1000)
return add_and_sum(x, y)
x = np.random.randn(3000, 3000)
y = np.random.randn(3000, 3000)
%prun add_and_sum(x, y)
%lprun -f add_and_sum add_and_sum(x, y)
%lprun -f add_and_sum -f call_function call_function()
4. 使用IPython进行高效代码开发的技巧
4.1 重载模块依赖项
注意依赖库的重载问题
4.2 代码设计技巧
if __name__ == '__main__':
main()
- 扁平化
- balance between 多的小文件 and 少的大文件
5. 高阶IPython特性
5.1 定义IPython友好的类
class Message:
def __init__(self, msg):
self.msg = msg
x = Message('I have a secrete')
x
<__main__.Message at 0x7fad92258820>
class Message:
def __init__(self, msg):
self.msg = msg
def __repr__(self):
return 'Message: %s' % self.msg
x = Message('I have a secrete')
x
Message: I have a secrete