Numpy from entry to proficiency - randomly generated arrays | specific generated arrays | rule generated arrays

This column is called " Numpy from entry to proficiency ". As the name suggests, it is to record the learning process of learning numpy, and it is also convenient for you to review later! Lay the foundation for further study in deep learning! I hope it can help everyone, and I wish you a happy life if you love to sleep! This article introduces " Numpy from entry to proficiency - randomly generated arrays | specific generated arrays | rule generated arrays "

0, written in front

Although I have done many projects before, I feel that they are all running demos. I don’t know the grammar, model modification, and algorithm reconstruction at all, so I prepare to learn Pytorch systematically and truly understand the creation, training and modification of models. I plan to learn from the book I recommended before, step by step, and finish it before June. During the winter vacation, I decided to officially start deep learning and finish learning numpy first. Because after being exposed to numpy and a small amount of deep learning knowledge before , I found that the matrix in numpy is very similar to the tensor tensor in deep learning , and there are often transformations between the two. So I think that learning numpy well is essential for deep learning.

1. Basic knowledge of Numpy

We all know that in computer vision, the essence of an image is a matrix of pixels. It is said that the image is processed, but it is actually processed by the matrix. Numpy is the general language of data science, and it is the cornerstone of scientific computing, matrix operations, and deep learning. Objects in numpy are called matrix objects ( ndarray objects ), which directly store values, which are similar to one-dimensional arrays in C language.

Numpy mainly has the following main features:

1. Fast, space-saving, array-based arithmetic operations and advanced broadcasting capabilities
2. Use standard math functions to perform fast operations on entire arrays of data without writing loops.
3. Provides tools to read/write array data on disk and manipulate memory image files (storage read functions)
4. Provides capabilities for linear algebra, random number generation, and Fourier transforms
5. Provides tools for integrating C, C++, Fortran code

We use the Opencv library to explore images in depth. First, we write the most basic code for Opencv to read and display images:

#!/usr/bin/env python
# -*- coding: UTF-8 -*-
"""
@Project ：numpy学习 
@File    ：task_1.py
@IDE     ：PyCharm 
@Author  ：咋
@Date    ：2023/4/16 18:22 
"""
import cv2
image = cv2.imread("image.jpg")
cv2.imshow("image",image)
cv2.waitKey(0)
cv2.destroyAllWindows()

Use the type function to view the type of image:

print(type(image))
# <class 'numpy.ndarray'>

As can be seen from the above example, the essence of image is actually a matrix, np.ndarray.

2. Array properties

In Numpy, dimensions are called axes. np.ndarray has three important properties:

• nd.array.ndim: the number of dimensions (axes) of the array
• nd.array.shape: The dimension of the array, the value is a tuple of integers, and the value of the tuple represents the length of the corresponding axis. The x in (x,y) represents how many rows this array has, and y represents how many columns this array has.
• nd.array.dtpye: data type, describing the type of elements in the array.

Similarly, we explain with specific examples:

#!/usr/bin/env python
# -*- coding: UTF-8 -*-

#!/usr/bin/env python
# -*- coding: UTF-8 -*-
"""
@Project ：numpy学习 
@File    ：task_1.py
@IDE     ：PyCharm 
@Author  ：咋
@Date    ：2023/4/16 18:22 
"""
import numpy as np
import cv2
image = cv2.imread("image.jpg")
image_gray = cv2.cvtColor(image,cv2.COLOR_BGR2GRAY) # 转成灰度图
print("image的维度为：",image.ndim)
print("image_gray的维度为：",image_gray.ndim)
print("image的大小为：",image.shape)
print("image_gray的大小为：",image_gray.shape)
print("image的数据类型为：",image.dtype)
print("image_gray的数据类型为：",image_gray.dtype)
# image的维度为： 3
# image_gray的维度为： 2
# image的大小为： (1080, 1920, 3)
# image_gray的大小为： (1080, 1920)
# image的数据类型为： uint8
# image_gray的数据类型为： uint8

ndim is the dimension, the number of color image channels is 3, and the dimension is 3; the number of grayscale image channels is 1, and the dimension is 2. The size of the image before and after conversion remains the same, both are 1080*1920, and the data type also remains the same, both are uint8.

Convert a list to a matrix using the array function:

#!/usr/bin/env python
# -*- coding: UTF-8 -*-
"""
@Project ：numpy学习 
@File    ：task_2.py
@IDE     ：PyCharm 
@Author  ：咋
@Date    ：2023/4/16 19:48
"""
import numpy as np
import cv2
# 将列表转化为ndarray
list_1 = [0,1,2,3,4,5]
image_1 = np.array(list_1)
print(image_1.ndim)
print(image_1.shape)
# 将嵌套列表转化为ndarray
list_2 = [[0,1,2,3,4],[5,6,7,8,9]]
image_2 = np.array(list_2)
print(image_2.ndim)
print(image_2.shape)
# 将嵌套列表转化为ndarray，并用cv2显示
list_3 = [[[1,2,3],[1,2,3],[1,2,3]],
          [[1,2,3],[1,2,3],[1,2,3]],
          [[1,2,3],[1,2,3],[1,2,3]]]
image_3 = np.array(list_3,dtype="uint8")
print(image_3.ndim)
print(image_3.shape)
print(image_3.dtype)
cv2.imshow("image",image_3)
cv2.waitKey(0)
cv2.destroyAllWindows()

Use the array function to convert the list list to ndarray format. Note that if you want to use opencv for image display, pay attention to the array conversion. Set dtpye to "uint8", so that it can be displayed normally with opencv.

3. The random module generates an array

In deep learning, we often need to initialize parameters, which involves np.random to generate arrays. Next, use a table and a piece of code to introduce np.random to generate an array:

function	describe
np.random.random	Generate random numbers from 0 to 1
np.random.uniform	Generate uniformly distributed random numbers
np.random.randn	Generate random numbers from a standard normal distribution
np.random.randint	generate random integers
np.random.normal	generate normally distributed integers
np.random.shuffle	Shuffle the order randomly
np.random.seed	Set random number seed
random_sample	Generate random floating point numbers

#!/usr/bin/env python
# -*- coding: UTF-8 -*-
"""
@Project ：numpy学习 
@File    ：task_3.py
@IDE     ：PyCharm 
@Author  ：咋
@Date    ：2023/4/17 9:06 
"""

import numpy as np

print('生成形状(4, 4)，值在0-1之间的随机数:')
print(np.random.random((4, 4)), end='\n\n')

# 产生一个取值范围在[1, 50)之间的数组，数组的shape是(3, 3)
# 参数起始值（low）默认为0， 终止值（high）默认为1。
print('生成形状(3, 3)，值在low-high之间的随机整数::')
print(np.random.randint(low=1, high=50, size=(3, 3)), end='\n\n')
print('产生的数组元素是均匀分布的随机数:')
print(np.random.uniform(low=1, high=3, size=(3, 3)), end='\n\n')

print('生成满足正态分布的形状为(3, 3)的矩阵:')
print(np.random.randn(3, 3))

The result is:

生成形状(4, 4)，值在0-1之间的随机数:
[[1.09478767e-01 8.28542687e-01 2.29063164e-01 6.74524184e-01]
 [3.95004570e-01 2.21405468e-01 7.33547514e-02 1.52782087e-01]
 [9.89312618e-01 2.67019311e-01 6.23058383e-01 2.03517166e-01]
 [9.67375188e-01 4.01401820e-04 8.69373856e-01 4.89925436e-01]]

生成形状(3, 3)，值在low-high之间的随机整数::
[[48 29 41]
 [17 29 17]
 [12 30 13]]

产生的数组元素是均匀分布的随机数:
[[2.53303787 1.30406591 1.73541217]
 [1.45465375 2.78644509 1.58166964]
 [2.7118102  2.50493356 1.31396379]]

生成满足正态分布的形状为(3, 3)的矩阵:
[[ 1.18399228 -0.17099359 -2.02865943]
 [-0.86478027 -0.68082124  0.84941314]
 [-0.03038969  2.32215063 -0.57330676]]

The meaning of the random number seed is further explained below:
the matrix generated by the above random method cannot be reproduced. If you want to generate the same matrix twice, you use the random number seed, np.random.seed()

import numpy as np
np.random.seed(10) 
print("按指定随机种子，第1次生成随机数:")
print(np.random.randint(1, 5, (2, 2)))
 
# 想要生成同样的数组，必须再次设置相同的种子
np.random.seed(10)
print("按相同随机种子，第2次生成的数据:")
print(np.random.randint(1, 5, (2, 2)))

output:

按指定随机种子，第1次生成随机数:
[[2 2]
 [1 4]]
按相同随机种子，第2次生成的数据:
[[2 2]
 [1 4]]

Therefore, the random number you want to generate is the same, you only need to add a random number seed in front, the code in the seed represents what the random number is, here is 10, you can also replace it with other numbers, representing different Random number seed!

4. A function to generate a multidimensional array of a specific shape

Sometimes, when initializing parameters, it is required to generate some special matrices, such as a matrix full of 1 or all 0, which also leaves us with corresponding functions in numpy.

function	describe
np.zeros((3,.4))	Generate a 3x4 array with all 0 elements
np.ones((3,4))	Generate a 3x4 array with all elements being 1
np.empty((2,3))	Generates a 2x3 empty array, the values ​​in the empty array are uninitialized garbage values
np.zeros_like(ndarr)	Generate an array with all 0 elements in the same dimension as ndarr
np.ones_like(ndarr)	Generate an array with all 1 elements in the same dimension as ndarr
np.empty_like(ndarr)	Generate an empty array with the same dimension as ndarr
np.eye(5)	Generate a 5x5 matrix with all 1s on the diagonal and 0s on the rest
np.full((3,5),666)	Generate a 3x5 array with all 666 elements
np.diag([1, 2, 3])	Matrix with Diagonals 1,2,3

Let's look at the specific code below:

#!/usr/bin/env python
# -*- coding: UTF-8 -*-
"""
@Project ：numpy学习 
@File    ：task_5.py
@IDE     ：PyCharm 
@Author  ：咋
@Date    ：2023/4/17 9:19 
"""
import numpy as np

# 生成全是 0 的 3x3 矩阵
nd5 = np.zeros([3, 3])
# 生成与nd5形状一样的全0矩阵
# np.zeros_like(nd5)
# 生成全是 1 的 3x3 矩阵
nd6 = np.ones([3, 3])
# 生成 3 阶的单位矩阵
nd7 = np.eye(3)
# 生成 3 阶对角矩阵
nd8 = np.diag([1, 2, 3])
print("*" * 6 + "nd5" + "*" * 6)
print(nd5)
print("*" * 6 + "nd6" + "*" * 6)
print(nd6)
print("*" * 6 + "nd7" + "*" * 6)
print(nd7)
print("*" * 6 + "nd8" + "*" * 6)
print(nd8)

The output is:

******nd5******
[[0. 0. 0.]
 [0. 0. 0.]
 [0. 0. 0.]]
******nd6******
[[1. 1. 1.]
 [1. 1. 1.]
 [1. 1. 1.]]
******nd7******
[[1. 0. 0.]
 [0. 1. 0.]
 [0. 0. 1.]]
******nd8******
[[1 0 0]
 [0 2 0]
 [0 0 3]]

5. Use arange and linspace functions to generate arrays

In some specific cases, we also want to generate a series of regular matrices. Students who have learned the basics of Python must be familiar with the range function. In numpy, we can use arange and linspace to generate specific regular arrays.

np.arange([start,]stop[,step,],dtype=None)

• start: starting number
• stop: stop number, not included
• step: step size, can be a decimal

np.linspace(start,stop,num,endpoint=True,retstep=False,dtype=None)

• start: starting number
• stop: stop number
• num: how many parts to divide
• endpoint: whether to include the endpoint
• retstep: whether to take the step size
• dtype: data type

#!/usr/bin/env python
# -*- coding: UTF-8 -*-
"""
@Project ：numpy学习 
@File    ：task_6.py
@IDE     ：PyCharm 
@Author  ：咋
@Date    ：2023/4/17 9:57 
"""
import numpy as np

print(np.arange(10))
# [0 1 2 3 4 5 6 7 8 9]
print(np.arange(0, 10))
# [0 1 2 3 4 5 6 7 8 9]
print(np.arange(1, 4, 0.5))
# [1.  1.5 2.  2.5 3.  3.5]
print(np.arange(9, -1, -1))
# [9 8 7 6 5 4 3 2 1 0]

#!/usr/bin/env python
# -*- coding: UTF-8 -*-
"""
@Project ：numpy学习 
@File    ：task_7.py
@IDE     ：PyCharm 
@Author  ：咋
@Date    ：2023/4/17 9:58 
"""
import numpy as np

print(np.linspace(0, 1, 10))
# [0.         0.11111111 0.22222222 0.33333333 0.44444444 0.55555556
# 0.66666667 0.77777778 0.88888889 1.        ]
print(np.linspace(0,1,10,retstep=True))
# ([0.        , 0.11111111, 0.22222222, 0.33333333, 0.44444444,
       0.55555556, 0.66666667, 0.77777778, 0.88888889, 1.        ]), 0.1111111111111111)

Note that np.linspace(0, 1, 10) generates a matrix with a step size of 0.11111111, because 0 also occupies one bit, and the step size is calculated as (1-0)/9=0.11111, if you want the step size to be 0.1, just change start=0 to 0.1.
In addition to the arange and linspace functions learned above, numpy also provides the logspace function. You can also write your own code and try the effect yourself!