首先在电脑上创建一个文件夹,下面有5个文件:
1.helloworld_mydocker.py
code1
import json
import pandas as pd
df = pd.read_csv('/tcdata/num_list.csv', header=None)
# df = pd.read_csv(
# '/home/xutengfei/test_for_tianchi_submit/num_list.csv', header=None) #测试用
ans_2 = 0
for i, num in enumerate(df[0]):
ans_2 += num
series = df.iloc[:, 0]
# ans2 = series.sum()
ans3 = series.sort_values(ascending=False) # descending
ans3 = ans3[:10] # get top-10
list3 = ans3.tolist()
ans = {
"Q1": 'Hello world', "Q2": ans_2, "Q3": list3}
with open('result.json', 'w', encoding='utf-8') as f:
json.dump(ans, f)
2.requirements.txt
pandas
numpy
3.Dockerfile
# Base Images
FROM python:3.8
## 把当前文件夹里的文件构建到镜像的根目录下(.后面有空格,不能直接跟/)
ADD . /
## 指定默认工作目录为根目录(需要把run.sh和生成的结果文件都放在该文件夹下,提交后才能运行)
WORKDIR /
## Install Requirements(requirements.txt包含python包的版本)
## 这里使用清华镜像加速安装
## RUN pip install -i https://pypi.tuna.tsinghua.edu.cn/simple -r requirements.txt
RUN pip install -i https://pypi.mirrors.ustc.edu.cn/simple/ -r requirements.txt
## 镜像启动后统一执行 sh run.sh
CMD ["sh", "run.sh"]
4.run.sh
python helloworld_mydocker.py
5测试用文件 num_lists.csv
1,
2,
3,
4,
5,
6,
7,
8,
9,
10,
参赛步骤:本机打开终端–登陆账号–构建本机镜像–按照阿里云容器镜像推送命令进行推送(就是那三行,第二行tag好像不很需要)–提交
遇到的问题以及解决
1.TypeError: Object of type int64 is not JSON serializable
这个是由于pandas的sum操作引起json无法识别导致的报错。可以采用迭代求和方式解决问题。
转成字符串之后也可以运行,但是稍有区别:
code1运行出来的json:
{“Q1”: “Hello world”, “Q2”: 55, “Q3”: [10, 9, 8, 7, 6, 5, 4, 3, 2, 1]}
code2:
import json
import pandas as pd
# df = pd.read_csv('/tcdata/num_list.csv', header=None)
df = pd.read_csv(
'/home/xutengfei/test_for_tianchi_submit/num_list.csv', header=None)
# result_2 = 0
# for i, num in enumerate(df[0]):
# result_2 += num
series = df.iloc[:, 0]
result_2 = series.sum()
ans3 = series.sort_values(ascending=False) # descending
ans3 = ans3[:10] # get top-10
list3 = ans3.tolist()
ans = {
"Q1": 'Hello world', "Q2": str(result_2), "Q3": list3}
with open('result.json', 'w', encoding='utf-8') as f:
json.dump(ans, f)
code2运行出来的json:
{“Q1”: “Hello world”, “Q2”: “55”, “Q3”: [10, 9, 8, 7, 6, 5, 4, 3, 2, 1]}
这样会导致结果里有双引号,不能得分。
2.怎么修改本地镜像?
从本地镜像创建容器–使用cp命令将修改后文件(在本机)替换掉镜像里面的文件–使用commit命令将容器提升为一个新版本的镜像
以下是整个过程的终端记录
base) xutengfei@xutengfei-G3-3579:~/test_for_tianchi_submit$ sudo docker login --username=利维_阿克曼 registry.cn-shanghai.aliyuncs.com
[sudo] xutengfei 的密码:
Password:
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
(base) xutengfei@xutengfei-G3-3579:~/test_for_tianchi_submit$ docker build -t registry.cn-shanghai.aliyuncs.com/xutengfei123/test_for_tianchi_submit:1.0
"docker build" requires exactly 1 argument.
See 'docker build --help'.
Usage: docker build [OPTIONS] PATH | URL | -
Build an image from a Dockerfile
(base) xutengfei@xutengfei-G3-3579:~/test_for_tianchi_submit$ docker build -t registry.cn-shanghai.aliyuncs.com/xutengfei123/test_for_tianchi_submit:1.0 .
Sending build context to Docker daemon 8.192kB
Step 1/5 : FROM python:3.8
3.8: Pulling from library/python
b9a857cbf04d: Pull complete
d557ee20540b: Pull complete
3b9ca4f00c2e: Pull complete
667fd949ed93: Pull complete
4ad46e8a18e5: Pull complete
381aea9d4031: Pull complete
8a9e78e1993b: Pull complete
9eff4cbaa677: Pull complete
c3c6df49b780: Pull complete
Digest: sha256:9a6b07ef1ecb923abfc444cf90d8bda47d5ae54bda1337e346bf596201b4eb29
Status: Downloaded newer image for python:3.8
---> 4d53664a7025
Step 2/5 : ADD . /
---> 85ea8aa0b600
Step 3/5 : WORKDIR /
---> Running in 2b03d8c9296c
Removing intermediate container 2b03d8c9296c
---> 340cb24ffdc4
Step 4/5 : RUN pip install -i https://pypi.mirrors.ustc.edu.cn/simple/ -r requirements.txt
---> Running in 71041c9feb98
Looking in indexes: https://pypi.mirrors.ustc.edu.cn/simple/
Collecting pandas
Downloading https://mirrors.bfsu.edu.cn/pypi/web/packages/ac/00/b52d3ae41ce14c4adef5d2a6952c46ed733ff9d1b33fc1aa423db0a6c1cd/pandas-1.2.1-cp38-cp38-manylinux1_x86_64.whl (9.7 MB)
Collecting numpy
Downloading https://mirrors.bfsu.edu.cn/pypi/web/packages/66/d7/3b133b17e185f14137bc8afe7a41daf1f31556900f10238312a5ae9c7345/numpy-1.19.5-cp38-cp38-manylinux2010_x86_64.whl (14.9 MB)
Collecting python-dateutil>=2.7.3
Downloading https://mirrors.bfsu.edu.cn/pypi/web/packages/d4/70/d60450c3dd48ef87586924207ae8907090de0b306af2bce5d134d78615cb/python_dateutil-2.8.1-py2.py3-none-any.whl (227 kB)
Collecting pytz>=2017.3
Downloading https://mirrors.bfsu.edu.cn/pypi/web/packages/89/06/2c2d3034b4d6bf22f2a4ae546d16925898658a33b4400cfb7e2c1e2871a3/pytz-2020.5-py2.py3-none-any.whl (510 kB)
Collecting six>=1.5
Downloading https://mirrors.bfsu.edu.cn/pypi/web/packages/ee/ff/48bde5c0f013094d729fe4b0316ba2a24774b3ff1c52d924a8a4cb04078a/six-1.15.0-py2.py3-none-any.whl (10 kB)
Installing collected packages: six, pytz, python-dateutil, numpy, pandas
Successfully installed numpy-1.19.5 pandas-1.2.1 python-dateutil-2.8.1 pytz-2020.5 six-1.15.0
Removing intermediate container 71041c9feb98
---> 9b6d42898a7b
Step 5/5 : CMD ["sh", "run.sh"]
---> Running in 38bb18f66f39
Removing intermediate container 38bb18f66f39
---> 8fcfbebff40a
Successfully built 8fcfbebff40a
Successfully tagged registry.cn-shanghai.aliyuncs.com/xutengfei123/test_for_tianchi_submit:1.0
(base) xutengfei@xutengfei-G3-3579:~/test_for_tianchi_submit$ sudo docker tag 8fcfbebff40a registry.cn-shanghai.aliyuncs.com/xutengfei123/test_for_tianchi_submit:1.0
(base) xutengfei@xutengfei-G3-3579:~/test_for_tianchi_submit$ sudo docker push registry.cn-shanghai.aliyuncs.com/xutengfei123/test_for_tianchi_submit:1.0
The push refers to repository [registry.cn-shanghai.aliyuncs.com/xutengfei123/test_for_tianchi_submit]
b15681e9284b: Pushed
57e1e4251f87: Pushed
0236dd92a81f: Pushed
7df99d8c2ab3: Pushed
ac9092d7b19b: Pushed
9f5b4cdea532: Layer already exists
cd702377e4e5: Layer already exists
aa7af8a465c6: Layer already exists
ef9a7b8862f4: Layer already exists
a1f2f42922b1: Layer already exists
4762552ad7d8: Layer already exists
1.0: digest: sha256:c66a211118d4635df8f218a7bd25ddf1a2d12844169c82c977da2d072b4bbc55 size: 2637
(base) xutengfei@xutengfei-G3-3579:~/test_for_tianchi_submit$ docker create --mycontainer1 8fcfbebff40a
unknown flag: --mycontainer1
See 'docker create --help'.
(base) xutengfei@xutengfei-G3-3579:~/test_for_tianchi_submit$ ^C
(base) xutengfei@xutengfei-G3-3579:~/test_for_tianchi_submit$ docker create --name mycontainer1 8fcfbebff40a
cbe8ea3c47795bde7adbb9ee669d308950f42049b4e4699b020fe6f92be31f8e
(base) xutengfei@xutengfei-G3-3579:~/test_for_tianchi_submit$ docker cp helloworld_mydocker.py mycontainer1:helloworld_mydocker.py
(base) xutengfei@xutengfei-G3-3579:~/test_for_tianchi_submit$ docker commit -a "xutengfei" -m "corrected image" mycontainer1 registry.cn-shanghai.aliyuncs.com/xutengfei123/test_for_tianchi_submit:1.0
sha256:cfea8d55bbf5f42e3ff0ae04039c149cfcaad1a1d04a89a498dff949f69be122
(base) xutengfei@xutengfei-G3-3579:~/test_for_tianchi_submit$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
registry.cn-shanghai.aliyuncs.com/xutengfei123/test_for_tianchi_submit 1.0 cfea8d55bbf5 9 seconds ago 1.02GB
registry.cn-shanghai.aliyuncs.com/xutengfei123/test_for_tianchi_submit <none> 8fcfbebff40a 44 minutes ago 1.02GB
python 3.8 4d53664a7025 4 days ago 883MB
(base) xutengfei@xutengfei-G3-3579:~/test_for_tianchi_submit$ sudo docker push registry.cn-shanghai.aliyuncs.com/xutengfei123/test_for_tianchi_submit:1.0
[sudo] xutengfei 的密码:
The push refers to repository [registry.cn-shanghai.aliyuncs.com/xutengfei123/test_for_tianchi_submit]
3ea622f0b5c7: Pushed
b15681e9284b: Layer already exists
57e1e4251f87: Layer already exists
0236dd92a81f: Layer already exists
7df99d8c2ab3: Layer already exists
ac9092d7b19b: Layer already exists
9f5b4cdea532: Layer already exists
cd702377e4e5: Layer already exists
aa7af8a465c6: Layer already exists
ef9a7b8862f4: Layer already exists
a1f2f42922b1: Layer already exists
4762552ad7d8: Layer already exists
1.0: digest: sha256:9a965f6345e35f63144fd6cd8b91d3eb6dba9c968536823b50a2127165ccec06 size: 2844
(base) xutengfei@xutengfei-G3-3579:~/test_for_tianchi_submit$
上面的操作有个缺憾,就是没有给镜像标注新版本,原来版本镜像的tag被覆盖为None了。
转载清注明出处,谢谢!