1. Project document well in advance of filing
Every time you start a new job, before I was always the sake of convenience, Code, Data, documents are concentrated in one folder, it looks messed up, so that once back process is very painful, or changed computers, files are all running to die, we need to modify the path on their own, very painful.
After some exploration of their own, we can roughly divide the project into several sub-folders, code in your main folder:
2. Never manually modify the source data and make a backup ✍️
We need a good backup of the source data to help us next time backtracking, may be the next operation or modification of the intermediate steps, and, on the other codes and other documents also need to be backed up to avoid accidental loss.
Xu Liang Linux here from an article, recommended four tools:
- Git version control system
- Rsync backup file
- Dropbox cloud storage
- Time Machine Time Machine
Introduction and use more tools I will not start here, you can go to inform themselves about the pictures.
3. do properly configured path
Many students at the time of writing are like direct path absolute path, although not a problem under normal circumstances, but if the code is shared with other people learning or running when the problem comes, are in many cases can not be directly run through,
It is recommended that:
- Using a relative path: the script is located in the main directory, other resources (such as data, third-party packages, etc.) in which the same or lower directories, such as ./data/processed/test1.csv
- Global Path configuration variables:
# Setting the Home
HOME_PATH = r'E: ML90615- PROJECT1 '
# reads the data
Data = Open (HOME_PATH +' / Data / Processed / Test1.csv ')
Data = pd.read_csv (Data)
data.head ()
4. Code where necessary make Notes and Remarks
I believe that most people identify with, do not believe? Back a month ago to write the code to see it, to see what can understand how much (if not done Remarks words)
5. Accelerate your Python loop code ⚡️
It is recommended Brother Yun (US Python and Algorithms) article: 24 accelerometers your python
Covered up, see more and more times, to form good habits chant, this way you will write code faster and faster -
6. Visualize your loop code progress
Here introduce a Python library, tqdm, install it: pip install tqdm
This is a cycle can show the progress of the library, it can have a more strategizing.
You can look at the following example:
7. Efficient use of abnormal capture tool
Abnormal positioning bug, I often also a former print () function to go in the end, although that is also no problem, but the efficiency is still relatively slow, but later found a man named PySnooper decorator, if discovered the New World.
We generally debug, we may feel are in place to have a problem, go to the printout, look at what the actual output, and thinking the problem, which requires us to change the code, very carefully change compared directly add decoration device, it is very troublesome.
You can look at Example:
import pysnooper
@pysnooper.snoop('./file.log')
def number_to_bits(number):
if number:
bits = []
while number:
number, remainder = divmod(number, 2)
bits.insert(0, remainder)
return bits
else:
return [0]
number_to_bits(6)
We put every step of the output function are saved as file.log, we can see directly what went wrong in the end.
8. Consider the code to be more robust
何为代码的健壮性,顾名思义，就是可以抵挡得住各种异常场景的测试，异常处理工作由“捕获”和“抛出”两部分组成。“捕获”指的是使用 try ... except 包裹特定语句，妥当的完成错误流程处理。而恰当的使用 raise 主动“抛出”异常，更是优雅代码里必不可少的组成部分，下面总结几点供大家参考：
1）知道要传入的参数是什么,类型,个数 （异常处理,逻辑判断）
def add(a, b):
if isinstance(a, int) and isinstance(b, int):
return a+b
else:
return '参数类型错误'
print(add(1, 2))
print(add(1, 'a'))
2）只做最精准的异常捕获
我们有的时候想着让脚本work才是王道，所以不管三七二十一就搞一个大大的try...except把整块代码包裹起来，但这样很容易把原本该被抛出的 AttibuteError 吞噬了。从而给我们的 debug 过程增加了不必要的麻烦。
所以，我们永远只捕获那些可能会抛出异常的语句块，而且尽量只捕获精确的异常类型，而不是模糊的 Exception。
from requests.exceptions import RequestException
def save_website_title(url, filename):
try:
resp = requests.get(url)
except RequestException as e:
print(f'save failed: unable to get page content: {e}')
return False
# 这段正则操作本身就是不应该抛出异常的，所以我们没必要使用 try 语句块
# 假如 group 被误打成了 grop 也没关系，程序马上就会通过 AttributeError 来
# 告诉我们。
obj = re.search(r'<title>(.*)</title>', resp.text)
if not obj:
print('save failed: title tag not found in page content')
return False
title = obj.group(1)
try: with open(filename, 'w') as fp:
fp.write(title)
except IOError as e:
print(f'save failed: unable to write to file {filename}: {e}')
return False
else:
return True
3）异常处理不应该喧宾夺主
像上一条说到的异常捕获要精准，但如果每一个都很精准的话，其实我们的代码里就会有很多try...except语句块，以至于扰乱核心代码，代码整体阅读性。
这里，我们可以利用上下文管理器来改善我们的异常处理流程，简化重复的异常处理逻辑。
class raise_api_error:
"""captures specified exception and raise ApiErrorCode instead
:raises: AttributeError if code_name is not valid
"""
def __init__(self, captures, code_name):
self.captures = captures
self.code = getattr(error_codes, code_name)
def __enter__(self):
# 该方法将在进入上下文时调用
return self
def __exit__(self, exc_type, exc_val, exc_tb):
# 该方法将在退出上下文时调用
# exc_type, exc_val, exc_tb 分别表示该上下文内抛出的
# 异常类型、异常值、错误栈
if exc_type is None:
return False
if exc_type == self.captures:
raise self.code from exc_val
return False
In the above code, we define a context manager named raise_api_error, which is entering the context and do nothing. But when you exit context, it will determine whether the current context type self.captures thrown exception, if so, replace it with APIErrorCode exception class.
After use context manager, concise code is as follows:
def upload_avatar(request):
"""用户上传新头像"""
with raise_api_error(KeyError, 'AVATAR_FILE_NOT_PROVIDED'):
avatar_file = request.FILES['avatar']
with raise_api_error(ResizeAvatarError, 'AVATAR_FILE_INVALID'),
raise_api_error(FileTooLargeError, 'AVATAR_FILE_TOO_LARGE'):
resized_avatar_file = resize_avatar(avatar_file)
with raise_api_error(Exception, 'INTERNAL_SERVER_ERROR'):
request.user.avatar = resized_avatar_file
request.user.save()
return HttpResponse({})