参加天池docker练习赛记录

其他 2021-03-06 12:30:09 阅读次数: 0

首先在电脑上创建一个文件夹，下面有5个文件：

1.helloworld_mydocker.py

code1

import json
import pandas as pd
df = pd.read_csv('/tcdata/num_list.csv', header=None)
# df = pd.read_csv(
#     '/home/xutengfei/test_for_tianchi_submit/num_list.csv', header=None) #测试用
ans_2 = 0
for i, num in enumerate(df[0]):
    ans_2 += num
series = df.iloc[:, 0]
# ans2 = series.sum()
ans3 = series.sort_values(ascending=False)  # descending
ans3 = ans3[:10]  # get top-10
list3 = ans3.tolist()
ans = {
    
    "Q1": 'Hello world', "Q2": ans_2, "Q3": list3}
with open('result.json', 'w', encoding='utf-8') as f:
    json.dump(ans, f)

2.requirements.txt

pandas
numpy

3.Dockerfile

# Base Images
FROM python:3.8

## 把当前文件夹里的文件构建到镜像的根目录下（.后面有空格，不能直接跟/）
ADD . /

## 指定默认工作目录为根目录（需要把run.sh和生成的结果文件都放在该文件夹下，提交后才能运行）
WORKDIR /

## Install Requirements（requirements.txt包含python包的版本）
## 这里使用清华镜像加速安装
## RUN pip install -i https://pypi.tuna.tsinghua.edu.cn/simple -r requirements.txt
RUN pip install -i https://pypi.mirrors.ustc.edu.cn/simple/ -r requirements.txt
## 镜像启动后统一执行 sh run.sh
CMD ["sh", "run.sh"]

4.run.sh

python helloworld_mydocker.py

5测试用文件 num_lists.csv

1,
2,
3,
4,
5,
6,
7,
8,
9,
10,

参赛步骤：本机打开终端–登陆账号–构建本机镜像–按照阿里云容器镜像推送命令进行推送(就是那三行，第二行tag好像不很需要)–提交

遇到的问题以及解决

1.TypeError: Object of type int64 is not JSON serializable

这个是由于pandas的sum操作引起json无法识别导致的报错。可以采用迭代求和方式解决问题。
转成字符串之后也可以运行,但是稍有区别：
code1运行出来的json:
{“Q1”: “Hello world”, “Q2”: 55, “Q3”: [10, 9, 8, 7, 6, 5, 4, 3, 2, 1]}
code2:

import json
import pandas as pd
# df = pd.read_csv('/tcdata/num_list.csv', header=None)
df = pd.read_csv(
    '/home/xutengfei/test_for_tianchi_submit/num_list.csv', header=None)
# result_2 = 0
# for i, num in enumerate(df[0]):
#     result_2 += num
series = df.iloc[:, 0]
result_2 = series.sum()
ans3 = series.sort_values(ascending=False)  # descending
ans3 = ans3[:10]  # get top-10
list3 = ans3.tolist()
ans = {
    
    "Q1": 'Hello world', "Q2": str(result_2), "Q3": list3}
with open('result.json', 'w', encoding='utf-8') as f:
    json.dump(ans, f)

code2运行出来的json:
{“Q1”: “Hello world”, “Q2”: “55”, “Q3”: [10, 9, 8, 7, 6, 5, 4, 3, 2, 1]}
这样会导致结果里有双引号，不能得分。

2.怎么修改本地镜像？

从本地镜像创建容器–使用cp命令将修改后文件（在本机）替换掉镜像里面的文件–使用commit命令将容器提升为一个新版本的镜像

以下是整个过程的终端记录

base) xutengfei@xutengfei-G3-3579:~/test_for_tianchi_submit$ sudo docker login --username=利维_阿克曼 registry.cn-shanghai.aliyuncs.com
[sudo] xutengfei 的密码： 
Password: 
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded
(base) xutengfei@xutengfei-G3-3579:~/test_for_tianchi_submit$ docker build -t registry.cn-shanghai.aliyuncs.com/xutengfei123/test_for_tianchi_submit:1.0
"docker build" requires exactly 1 argument.
See 'docker build --help'.

Usage:  docker build [OPTIONS] PATH | URL | -

Build an image from a Dockerfile
(base) xutengfei@xutengfei-G3-3579:~/test_for_tianchi_submit$ docker build -t registry.cn-shanghai.aliyuncs.com/xutengfei123/test_for_tianchi_submit:1.0 .
Sending build context to Docker daemon  8.192kB
Step 1/5 : FROM python:3.8
3.8: Pulling from library/python
b9a857cbf04d: Pull complete 
d557ee20540b: Pull complete 
3b9ca4f00c2e: Pull complete 
667fd949ed93: Pull complete 
4ad46e8a18e5: Pull complete 
381aea9d4031: Pull complete 
8a9e78e1993b: Pull complete 
9eff4cbaa677: Pull complete 
c3c6df49b780: Pull complete 
Digest: sha256:9a6b07ef1ecb923abfc444cf90d8bda47d5ae54bda1337e346bf596201b4eb29
Status: Downloaded newer image for python:3.8
 ---> 4d53664a7025
Step 2/5 : ADD . /
 ---> 85ea8aa0b600
Step 3/5 : WORKDIR /
 ---> Running in 2b03d8c9296c
Removing intermediate container 2b03d8c9296c
 ---> 340cb24ffdc4
Step 4/5 : RUN pip install -i https://pypi.mirrors.ustc.edu.cn/simple/ -r requirements.txt
 ---> Running in 71041c9feb98
Looking in indexes: https://pypi.mirrors.ustc.edu.cn/simple/
Collecting pandas
  Downloading https://mirrors.bfsu.edu.cn/pypi/web/packages/ac/00/b52d3ae41ce14c4adef5d2a6952c46ed733ff9d1b33fc1aa423db0a6c1cd/pandas-1.2.1-cp38-cp38-manylinux1_x86_64.whl (9.7 MB)
Collecting numpy
  Downloading https://mirrors.bfsu.edu.cn/pypi/web/packages/66/d7/3b133b17e185f14137bc8afe7a41daf1f31556900f10238312a5ae9c7345/numpy-1.19.5-cp38-cp38-manylinux2010_x86_64.whl (14.9 MB)
Collecting python-dateutil>=2.7.3
  Downloading https://mirrors.bfsu.edu.cn/pypi/web/packages/d4/70/d60450c3dd48ef87586924207ae8907090de0b306af2bce5d134d78615cb/python_dateutil-2.8.1-py2.py3-none-any.whl (227 kB)
Collecting pytz>=2017.3
  Downloading https://mirrors.bfsu.edu.cn/pypi/web/packages/89/06/2c2d3034b4d6bf22f2a4ae546d16925898658a33b4400cfb7e2c1e2871a3/pytz-2020.5-py2.py3-none-any.whl (510 kB)
Collecting six>=1.5
  Downloading https://mirrors.bfsu.edu.cn/pypi/web/packages/ee/ff/48bde5c0f013094d729fe4b0316ba2a24774b3ff1c52d924a8a4cb04078a/six-1.15.0-py2.py3-none-any.whl (10 kB)
Installing collected packages: six, pytz, python-dateutil, numpy, pandas
Successfully installed numpy-1.19.5 pandas-1.2.1 python-dateutil-2.8.1 pytz-2020.5 six-1.15.0
Removing intermediate container 71041c9feb98
 ---> 9b6d42898a7b
Step 5/5 : CMD ["sh", "run.sh"]
 ---> Running in 38bb18f66f39
Removing intermediate container 38bb18f66f39
 ---> 8fcfbebff40a
Successfully built 8fcfbebff40a
Successfully tagged registry.cn-shanghai.aliyuncs.com/xutengfei123/test_for_tianchi_submit:1.0
(base) xutengfei@xutengfei-G3-3579:~/test_for_tianchi_submit$ sudo docker tag 8fcfbebff40a registry.cn-shanghai.aliyuncs.com/xutengfei123/test_for_tianchi_submit:1.0
(base) xutengfei@xutengfei-G3-3579:~/test_for_tianchi_submit$ sudo docker push registry.cn-shanghai.aliyuncs.com/xutengfei123/test_for_tianchi_submit:1.0
The push refers to repository [registry.cn-shanghai.aliyuncs.com/xutengfei123/test_for_tianchi_submit]
b15681e9284b: Pushed 
57e1e4251f87: Pushed 
0236dd92a81f: Pushed 
7df99d8c2ab3: Pushed 
ac9092d7b19b: Pushed 
9f5b4cdea532: Layer already exists 
cd702377e4e5: Layer already exists 
aa7af8a465c6: Layer already exists 
ef9a7b8862f4: Layer already exists 
a1f2f42922b1: Layer already exists 
4762552ad7d8: Layer already exists 
1.0: digest: sha256:c66a211118d4635df8f218a7bd25ddf1a2d12844169c82c977da2d072b4bbc55 size: 2637
(base) xutengfei@xutengfei-G3-3579:~/test_for_tianchi_submit$ docker create --mycontainer1 8fcfbebff40a
unknown flag: --mycontainer1
See 'docker create --help'.
(base) xutengfei@xutengfei-G3-3579:~/test_for_tianchi_submit$ ^C
(base) xutengfei@xutengfei-G3-3579:~/test_for_tianchi_submit$ docker create --name mycontainer1 8fcfbebff40a
cbe8ea3c47795bde7adbb9ee669d308950f42049b4e4699b020fe6f92be31f8e
(base) xutengfei@xutengfei-G3-3579:~/test_for_tianchi_submit$ docker cp helloworld_mydocker.py mycontainer1:helloworld_mydocker.py
(base) xutengfei@xutengfei-G3-3579:~/test_for_tianchi_submit$ docker commit -a "xutengfei" -m "corrected image" mycontainer1 registry.cn-shanghai.aliyuncs.com/xutengfei123/test_for_tianchi_submit:1.0
sha256:cfea8d55bbf5f42e3ff0ae04039c149cfcaad1a1d04a89a498dff949f69be122
(base) xutengfei@xutengfei-G3-3579:~/test_for_tianchi_submit$ docker images
REPOSITORY                                                               TAG       IMAGE ID       CREATED          SIZE
registry.cn-shanghai.aliyuncs.com/xutengfei123/test_for_tianchi_submit   1.0       cfea8d55bbf5   9 seconds ago    1.02GB
registry.cn-shanghai.aliyuncs.com/xutengfei123/test_for_tianchi_submit   <none>    8fcfbebff40a   44 minutes ago   1.02GB
python                                                                   3.8       4d53664a7025   4 days ago       883MB
(base) xutengfei@xutengfei-G3-3579:~/test_for_tianchi_submit$ sudo docker push registry.cn-shanghai.aliyuncs.com/xutengfei123/test_for_tianchi_submit:1.0
[sudo] xutengfei 的密码： 
The push refers to repository [registry.cn-shanghai.aliyuncs.com/xutengfei123/test_for_tianchi_submit]
3ea622f0b5c7: Pushed 
b15681e9284b: Layer already exists 
57e1e4251f87: Layer already exists 
0236dd92a81f: Layer already exists 
7df99d8c2ab3: Layer already exists 
ac9092d7b19b: Layer already exists 
9f5b4cdea532: Layer already exists 
cd702377e4e5: Layer already exists 
aa7af8a465c6: Layer already exists 
ef9a7b8862f4: Layer already exists 
a1f2f42922b1: Layer already exists 
4762552ad7d8: Layer already exists 
1.0: digest: sha256:9a965f6345e35f63144fd6cd8b91d3eb6dba9c968536823b50a2127165ccec06 size: 2844
(base) xutengfei@xutengfei-G3-3579:~/test_for_tianchi_submit$

上面的操作有个缺憾，就是没有给镜像标注新版本，原来版本镜像的tag被覆盖为None了。

转载清注明出处，谢谢！

猜你喜欢

转载自blog.csdn.net/qq_44065334/article/details/113419597
今日推荐
周排行
教你如何约女孩子的方式去理解（TCP三次握手与四次挥手）
android按压背景
【量化小讲堂-Python&amp;Pandas系列10】如何判断一个策略的好坏？(附代码)
编程题：利用链表实现栈
盘点47条 Allegro 使用技巧，你都知道吗？
在VMware Workstation中安装CentOS
二叉树的实现
cmake安装jsoncpp
ReactNative开发城市列表页
最全前端学习资源
每日归档
更多
2025-03-20(0)
2025-03-19(0)
2025-03-18(0)
2025-03-17(0)
2025-03-16(0)
2025-03-15(0)
2025-03-14(0)
2025-03-13(0)
2025-03-12(0)
2025-03-11(0)

代码天地 © 2018 codetd.com

境外服务器   最新电视剧2022