In the process of building a deep learning model, Tensor and Numpy are indispensable, and we often face the mutual conversion of the two data structures.

What exactly is the difference between these two data structures?
When to choose which data structure to use?
How to convert between the two data structures?

This paper gives answers to questions through multiple dimensions such as definitions, application scenarios, and conversion methods.

1 Tensor in pytorch

1.1 What is Tensor

Tensor is the most basic and critical data structure in deep learning:

Tensor is to PyTorch what array is to Numpy or DataFrame is to Pandas, and both build the lowest-level data structure in the entire framework;
Tensor is different from ordinary data structures, and has an extremely critical feature-automatic differentiation.

Definition: The original English meaning of Tensor is tensor, and it is introduced as follows in Pytorch official website:

A Tensor is a high-dimensional matrix containing a single data type. Generally speaking, the high-dimensional characteristics of a Tensor are usually described by a three-dimensional or higher matrix, as shown in the figure below: a single element is called a scalar (scalar), and a sequence is called A vector, a plane composed of multiple sequences is called a matrix, and a cube composed of multiple planes is called a tensor. Of course, just like a matrix has a one-dimensional matrix, a two-dimensional matrix or even a multi-dimensional matrix, a tensor does not need to be strictly limited to more than three dimensions to be called a tensor. In the field of deep learning, scalars, vectors, and matrices are collectively called tensors.

Tensor is not a unique definition of PyTorch, but a data structure widely used by many deep learning frameworks, such as TensorFlow.

1.2 Why do you need Tensor

Tensor in PyTorch is a data structure widely used in deep learning. It is essentially a high-dimensional matrix. It is not an exaggeration to understand it as the promotion and upgrade of array in NumPy. However, due to some special features it supports, Tensor is more convenient when used to support deep learning models and training.

1.3 How to create Tensor

Generally speaking, there are three ways to create a Tensor:

Convert and create Tensor from other existing data structures
Randomly initialize a Tensor
Load a Tensor from a saved file

1.3.1 Convert and create Tensor from other existing data structures

This may be the most commonly used form in practical applications. It is often used for loading data sets, such as reading data from a list or a NumPy array, and then generating a new Tensor. In order to achieve this goal, there are two commonly used methods:

torch.tensor
torch.Tensor

The difference between the two is that the former uses the tensor function (t is lowercase), while the latter uses the Tensor class (T is uppercase). The two are in creating the default data type of Tensor, supporting parameter passing and handling of individual details.

The first is that the default data type of the created Tensor is different:

import torch

t1 = torch.tensor([1,2,3])
t2 = torch.Tensor([1,2,3])
print(t1, t2)
print(t1.dtype, t2.dtype)

The running display is as follows:

tensor([1, 2, 3]) tensor([1., 2., 3.])
torch.int64 torch.float32

Secondly, using the Tensor class to initialize and input an integer will return an all-zero one-dimensional tensor of this length, while the tensor function returns a zero-dimensional tensor with only this element:

import torch

t1 = torch.tensor(3)
t2 = torch.Tensor(3)
print(t1, t2)

The running result shows:

tensor(3) tensor([0., 0., 0.])

There is one detail that needs to be mentioned first: when using the Tensor class to receive a sequence to create a Tensor, the returned data type is float type, because Tensor is the equivalent form of FloatTensor, that is, there are ByteTensor, IntTensor, LongTensor and Different default data types like DoubleTensor.

Create two other commonly used functions of Tensor based on existing data:

from_numpy
as_tensor

The biggest difference between the two methods and the above methods is that the Tensor they return shares memory with the original data, while the aforementioned tensor function and Tensor class create a new object after copying.

import torch
import numpy as np

n1 = np.array([1,2,3])
n2 = np.array([1,2,3])
n3 = np.array([1,2,3])
t1 = torch.tensor(n1)
t2 = torch.from_numpy(n2)
t3 = torch.as_tensor(n3)
print('t1:', t1)
print('t2:', t2)
print('t3:', t3)

n1[0] = 100
n2[0] = 100
n3[0] = 100

print('t1:', t1)
print('t2:', t2)
print('t2:', t3)

The running result shows:

t1: tensor([1, 2, 3])
t2: tensor([1, 2, 3])
t3: tensor([1, 2, 3])
t1: tensor([1, 2, 3])
t2: tensor([100,   2,   3])
t2: tensor([100,   2,   3])

1.3.2 Randomly initialize a Tensor

Random initialization is a commonly used form. When building a neural network model, the weight of the module is automatically randomly initialized after adding a module. These methods fall into two forms:

Create a specific type of tensor, such as torch.ones, torch.randn, etc.
Create a specific type of tensor consistent with its shape based on an existing Tensor, such as torch.ones_like, torch.randn_like, etc.

import torch

func_name = [x for x in dir(torch) if x.endswith('like')]
print(func_name)

The running result shows:

['empty_like', 'full_like', 'ones_like', 'rand_like', 'randint_like', 'randn_like', 'zeros_like']

Randomly construct a fully connected unit Linear in PyTorch, which will create corresponding weight coefficients and offsets by default (because network parameters generally need to participate in subsequent model training, the default requires_grad=True):

import torch

linear = torch.nn.Linear(2,3)
print('weight:', linear.weight)
print('bias:', linear.bias)

The running result shows:

weight: Parameter containing:
tensor([[-0.5380, -0.6481],
        [ 0.0144,  0.0627],
        [-0.6560,  0.0382]], requires_grad=True)
bias: Parameter containing:
tensor([-0.2800,  0.0235, -0.6940], requires_grad=True)

1.3.3 Load a Tensor from a saved file

PyTorch does not deliberately distinguish the form of the object to be saved and loaded. It can be a trained network or data. This is pickle in Python, which is realized by the two functions torch.save and torch.load. There is no suffix format requirement for the saved file. Commonly used suffix formats are .pkl, .pth, and .pt. This method is often used for saving models and large-scale data sets.

import torch

t1 = torch.tensor([1,2,3,4,5])
torch.save(t1, 'tensor.pkl')

ret_t = torch.load('tensor.pkl')
print('ret_t', ret_t)

The result of the operation is as follows:

ret_t tensor([1, 2, 3, 4, 5])

1.4 Characteristics of Tensors

The reason why PyTorch defines Tensor to support deep learning, instead of directly using a list in Python or an array in NumPy, is because Tensor is endowed with some unique features. The main features of Tensor are the following three aspects:

Rich common operation functions
Flexible dtype and CPU/GPU free switching storage
Automatic Gradient Solving

1.4.1 Rich common function operations

Tensor is essentially a high-dimensional matrix composed of numerical elements. The process of deep learning is the process of various matrix operations. As its basic data structure, Tensor supports rich function operations.

Tensor itself operates functions, such as tensor.max(), tensor.abs(), etc.,
Tensor performs related operations on other tensors, such as tensor.add() is used to add another tensor, tensor.mm() is used to perform matrix multiplication with another tensor, and so on. The addition and multiplication here have requirements for the two tensor sizes of the operation.

In addition to supporting rich enough function operations, tensor's API functions have another important convenience feature: most functions support two versions: underlined version and non-underscored version, such as tensor.abs() and tensor. abs_(), both return the Tensor after the operation, but at the same time, the underlined version belongs to the inplace operation, that is, its own data will also change after the call.

1.4.2 Flexible dtype and CPU/GPU free switching

The concept of dtype in Tensor is similar to the usage in NumPy and Pandas, which is used to specify the data structure of Tensor to be created. There are 10 different data structures defined in PyTorch, including integer types of different lengths and floating point types of different lengths. All elements of the entire Tesor must have the same data type and must be of numeric type (the array in NumPy also requires elements are homogeneous, but support string types):

In addition to supporting different numerical data types, another major feature of Tensor is that it supports different computing units: CPU or GPU. Supporting GPU acceleration is also a key to the large-scale application of deep learning. In order to switch between CPU computing (data stored in memory) or GPU computing (data stored in video memory), Tensor supports flexible configuration of storage devices, including the following two methods:

When creating a tensor, specify it directly through the device parameter
Switch through the tensor.to() function, to() can be used to switch storage devices and data types

In addition, in addition to the two major features of dtype and device, Tensor actually has a third feature, namely layout. It mainly includes strided and sparse_coo, and this feature generally does not require additional consideration.

1.4.3 Automatic Gradient Solving

Tensor automatic gradient solving is the cornerstone of deep learning. The core of the deep learning model lies in the connection of neurons, and the key to the connection between neurons is the network weight, which is the parameters of each module. It is precisely because of the different network parameters that the same network structure can achieve different model application values. So, how to learn the optimal network parameters? This is the optimization weapon in deep learning: gradient descent method, and a major premise of gradient descent method is to support automatic gradient solution.

In order to support automatic gradient solving, the general process of Tensor is as follows:

Tensor supports grad solution, ie requires_grad=True
According to the process of Tensor participating in the calculation, the Tensor and the corresponding operation functions are recorded as a tree structure (or called a directed acyclic graph: DAG), and the direction of calculation is from the leaf node to the root node
Calculate the corresponding difference (loss) according to the root node Tensor and the target value, and then use the chain derivation rule to calculate the gradient step by step in reverse (that is, the backpropagation of the gradient)

2 Numpy

2.1 What is Numpy

Numpy, short for Numerical Python, is a library consisting of multidimensional array objects (ndarray) and a collection of functions (function) for manipulating these arrays. Using the Numpy library, you can perform mathematical operations and related logical operations on arrays. Numpy is not only an extension package of Python, but also the basic package of Python scientific computing. Numpy provides a large number of mathematical functions, mainly used to calculate and process one-dimensional or multi-dimensional arrays, and supports common array and matrix operations. The bottom layer of NumPy is mainly written in C language, so it can perform numerical calculations at high speed. Numpy also provides a variety of data structures, which can be very suitable for the operation of arrays and matrices.
As an open source project, Numpy is jointly developed and maintained by many collaborators. With the vigorous development of data science (Data Science, referred to as DS, including big data analysis and processing, big data storage, data capture and other branches), like Numpy, Data analysis libraries such as SciPy (Python scientific computing library), Pandas (NumPy-based data processing library) and other data analysis libraries have a lot of growth, and they all have a simpler syntax format. Numpy is usually used together with Matplotlib and so on.

2.2 What are the advantages of NumPy

2.2.1 ndarray supports parallelized operations (vectorized operations)

Numpy has a built-in parallel computing function. When the system has multiple cores, when doing some kind of calculation, numpy will automatically do parallel computing.

2.2.2 Efficiency is much higher than pure Python code

The bottom layer of Numpy is written in C language, and the GIL (Global Interpreter Lock) is unlocked internally. Its operation speed on arrays is not limited by the Python interpreter, so its efficiency is much higher than that of pure Python code.

2.2.3 Storage data address continuous

When ndarray stores data, the addresses of data and data are continuous, which makes it faster to operate array elements in batches.

The types of all elements in ndarray are the same, while the element types in Python lists are arbitrary, so the memory of ndarray can be continuous when storing elements, while python's native list can only find the next element by addressing. Although Numpy's ndarray is not as good as Python's native list in terms of general performance, in scientific computing, Numpy's ndarray can save a lot of loop statements, and the code usage is much simpler than Python's native list.

2.3 How to create Numpy objects

Numpy defines an n-dimensional array object, referred to as ndarray object, which is an array collection composed of a series of elements of the same type. Each element in the array occupies a memory block of the same size, and each element in the array can be obtained by indexing or slicing. ndarray objects have a dtype attribute that describes the data type of the elements. The ndarray object adopts the index mechanism of the array, maps each element in the array to a memory block, and arranges the memory block according to a certain layout. There are two commonly used layout methods, that is, by row or by column.

The ndarray object can be created through the built-in function array() of Numpy, and its syntax format is as follows:

numpy.array(object, dtype = None, copy = True, order = None,ndmin = 0)

2.3.1 Create an ndarray array using Numpy

import numpy as np

a = np.array([1,2,3])
b = np.array([[1,2,3], [4,5,6]])

print(f'a:{a}')
print(f'type(a):{type(a)}')

print('b:{b}')

The running result shows:

a:[1 2 3]
type(a):<class 'numpy.ndarray'>
b:{b}

2.3.2 Check the dimension of the array and change the dimension

import numpy as np

a = np.array([[1,2], [3,4], [5,6]])

print(a.ndim)
print("old array:", a)

a = a.reshape(2,3)
print("new array:", a.ndim, a)

The running result shows:

old array: [[1 2]
 [3 4]
 [5 6]]
new array: 2 [[1 2 3]
 [4 5 6]]

2.3.3 Arrays can also be manipulated through the attributes of ndarray

Properties of ndarray

import numpy as np

a = np.array([[1,2], [3,4], [5,6]])
print('a.shape:', a.shape)
print('a:', a)
a.shape = (2, 3)
print('a[2,3]:', a)
print('a[3,2]:', a.reshape(3, 2))

print('a.ndim:', a.ndim)
print('a.itemsize:', a.itemsize)
print('a.flags:', a.flags)

a.shape: (3, 2)
a: [[1 2]
 [3 4]
 [5 6]]
a[2,3]: [[1 2 3]
 [4 5 6]]
a[3,2]: [[1 2]
 [3 4]
 [5 6]]
a.ndim: 2
a.itemsize: 4
a.flags:   C_CONTIGUOUS : True
  F_CONTIGUOUS : False
  OWNDATA : True
  WRITEABLE : True
  ALIGNED : True
  WRITEBACKIFCOPY : False

type of ndarray

The data type identification code in Numpy, each data type in Numpy has a uniquely identified character code, as follows:

i- integer
b- Boolean
u- unsigned integer
f- floating point
c- Composite floating point numbers
m- from timed
M - datetime
O- object
S- string
U- unicode string
V- pinned memory blocks of other types ( void )

Usually, Numpy is used together with the SciPy package, and SciPy can be regarded as an extension of the Numpy library, which adds many engineering calculation functions on the basis of Numpy.

2.3.4 Creating uninitialized arrays

numpy.empty(shape, dtype = float, order = 'C')

numpy.empty() creates an uninitialized array, and you can specify the shape (shape) and data type (dtype) of the created array.

参数：

    shape：指定数组的形状；
    dtype：数组元素的数据类型，默认值是值 float；
    order：指数组元素在计算机内存中的储存顺序，默认顺序是“C”（“C”代表以行顺序存储，“F”则表示以列顺序存储）。

import numpy as np
n1 = np.empty([2, 2])
print(n1)

The running result shows:

[[0.0e+000 5.4e-323]
 [9.9e-324 9.9e-324]]

2.3.4 Create an array whose elements are all 0

numpy.zeros(shape,dtype=float,order="C")

This function is used to create an array whose elements are all 0, and can also specify the shape of the array.

参数：

    shape：指定数组的形状；
    dtype：数组元素的数据类型，默认值是值 float；
    order：指数组元素在计算机内存中的储存顺序，默认顺序是“C”（“C”代表以行顺序存储，“F”则表示以列顺序存储）。

import numpy as np
n1 = np.zeros([2, 2])
print(n1)

The running result shows:

[[0. 0.]
 [0. 0.]]

2.3.4 Create a new array with a specified shape size and data type filled with 1

numpy.ones(shape, dtype = None, order = 'C')

Returns a new array of the specified shape, size and data type, and each element in the new array is filled with 1.

import numpy as np
n1 = np.ones([2, 2])
print(n1)

The running result shows

[[1. 1.]
 [1. 1.]]

2.3.5 Converting a Python sequence to an ndarray object

numpy.asarray（sequence，dtype = None ，order = None ）

asarray() converts a Python sequence into an ndarray object.

参数：

    sequence：接受一个 Python 序列，可以是列表或者元组；
    dtype：可选参数，数组的数据类型，默认值是值 float；
    order：指数组元素在计算机内存中的储存顺序，默认顺序是“C”（“C”代表以行顺序存储，“F”则表示以列顺序存储）。

import numpy as np


array = [1, 2, 3, 4, 5]
n1 = np.asarray(array)
print(n1)
print(type(n1))

The running result shows

[1 2 3 4 5]
<class 'numpy.ndarray'>

2.3.6 Create an array using a specified buffer

numpy.frombuffer(buffer, dtype = float, count = -1, offset = 0)

Indicates to create an array using the specified buffer.

参数：

    buffer：将任意对象转换为流的形式读入缓冲区；
    dtype：返回数组的数据类型，默认是 float32；
    count：要读取的数据数量，默认为 -1 表示读取所有数据；
    offset：读取数据的起始位置，默认为 0。

import numpy as np

data =b'hello world!'
res = np.frombuffer(data,dtype='S3',offset=0)
print(res)

The running result shows:

[b'hel' b'lo ' b'wor' b'ld!']

2.3.7 Convert iterative object to ndarray array

numpy.fromiter(iterable, dtype, count = -1)

Convert the iteration object to an ndarray array, and its return value is a one-dimensional array.

参数：

    iterable：可迭代对象；
    dtype：数组元素的数据类型，默认值是值 float；
    count：读取的数据数量，默认为 -1，读取所有数据。

import numpy as np

iterable = (x * x*x for x in range(4))
n1 = np.fromiter(iterable, int)

print (n1)

The running result shows

[ 0  1  8 27]

2.3.8 Create an array with a given range of values

numpy.arange(start, stop, step, dtype)

参数：

    start： 起始值，默认是 0。
    stop： 终止值，注意生成的数组元素值不包含终止值。
    step： 步长，默认为 1。
    dtype： 可选参数，指定 ndarray 数组的数据类型。

import numpy as np

A = np.arange(5)
print(A)
print(type(A))

A = np.arange(1, 5)
print(A)

A = np.arange(1, 5, 2)
print(A)

A = np.arange(1, 5.2, 0.6)
print(A)

The running result shows

[0 1 2 3 4]
<class 'numpy.ndarray'>
[1 2 3 4]
[1 3]
[1.  1.6 2.2 2.8 3.4 4.  4.6 5.2]

2.3.9 Create an array within a specified range of values

numpy.linspace(start, stop, num=50, endpoint=True, retstep=False, dtype=None)

参数：

    start：代表数值区间的起始值；
    stop：代表数值区间的终止值；
    num：表示数值区间内要生成多少个均匀的样本。默认值为 50；
    endpoint：默认为 True，表示数列包含 stop 终止值，反之不包含；
    retstep：默认为 True，表示生成的数组中会显示公差项，反之不显示；
    dtype：代表数组元素值的数据类型。

import numpy as np

n1 = np.linspace(start = 0, stop = 1, num = 11)
n2 = np.linspace(start = 0, stop = 100, num = 11)
n3 = np.linspace(start = 1, stop = 5, num = 4, endpoint = False)
n4 = np.linspace(start = 0, stop = 100, num = 5, dtype = int)
print(n1)
print(n2)
print(n3)
print(n4)

The running result shows:

[0.  0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1. ]
[  0.  10.  20.  30.  40.  50.  60.  70.  80.  90. 100.]
[1. 2. 3. 4.]
[  0  25  50  75 100]

2.3.10 Create a proportional array

numpy.logspace(start, stop, num=50, endpoint=True, base=10.0, dtype=None)

参数：

    start： 序列的起始值：base**start。
    stop： 序列的终止值：base**stop。
    num： 数值范围区间内样本数量，默认为 50。
    endpoint： 默认为 True 包含终止值，反之不包含。
    base： 对数函数的 log 底数，默认为10。
    dtype： 可选参数，指定 ndarray 数组的数据类型。

In logspace, the start point and end point are powers of 10

import numpy as np

n1 = np.logspace(0,0,10)
n2 = np.logspace(0,9,10)
n3 = np.logspace(0,9,10,base=2)

print(n1)
print(n2)
print(n3)

The running result shows:

[1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]
[1.e+00 1.e+01 1.e+02 1.e+03 1.e+04 1.e+05 1.e+06 1.e+07 1.e+08 1.e+09]
[  1.   2.   4.   8.  16.  32.  64. 128. 256. 512.]

2.4 Numpy operations

2.4.1 Tiling array elements

numpy.ndarray.flat()

Returns an array iterator, each element of which can be traversed with a for loop.

import numpy as np

a = np.arange(8).reshape(2, 4)
print ('a:', a)

for e in a.flat:
    print (e, end=" ")

The running result shows:

a: [[0 1 2 3]
 [4 5 6 7]]
0 1 2 3 4 5 6 7

2.4.2 Returning a copy of the array as a one-dimensional array

numpy.ndarray.flatten()

Returns a copy of the array as a one-dimensional array. Modifications to the copy will not affect the original array.

import numpy as np

a = np.arange(8).reshape(2, 4)
print(a)
print(a.flatten())

The running result shows:

[[0 1 2 3]
 [4 5 6 7]]
[0 1 2 3 4 5 6 7]

2.4.2 Convert to a continuous flat array

numpy.ravel()

Returns a contiguous flattened array (i.e. an expanded one-dimensional array). Unlike flatten, it returns an array view (modifying the view will affect the original array).

import numpy as np

a1, a2 = np.mgrid[1:4:1, 2:3:1]
print("a1:\n", a1, "\n", "a2:\n", a2)

n1 = a1.ravel() 
n2 = a2.ravel()

print("n1:", n1)
print("n2:", n2)
sum = n1 + n2
print(sum)

The running result shows:

a1:
 [[1]
 [2]
 [3]] 
 a2:
 [[2]
 [2]
 [2]]
n1: [1 2 3]
n2: [2 2 2]
[3 4 5]

2.4.3 Bit operations in Numpy

import numpy as np

n1 = np.bitwise_and([11,7], [4,25])
n2 = np.bitwise_and(np.array([2,5,255]), np.array([3,14,16]))
n3 = np.bitwise_and([True, True], [False, True])


n4 = np.bitwise_or([11,7], [4,25])
n5 = np.bitwise_or(np.array([2,5,255]), np.array([3,14,16]))
n6 = np.bitwise_or([True, True], [False, True])

print(n1)
print(n2)
print(n3)
print(n4)
print(n5)
print(n6)

The running result shows:

[0 1]
[ 2  4 16]
[False  True]
[15 31]
[  3  15 255]
[ True  True]

2.4.4 Arithmetic operations

The "addition, subtraction, multiplication and division" arithmetic operations of NumPy arrays correspond to the add(), subtract(), multiple() and divide() functions respectively.

The numpy.reciprocal() function takes the reciprocal of each element in the array and returns them as an array.
numpy.power() takes the elements in the a array as the base, exponentiates the elements in the b array corresponding to a, and finally returns the calculation results of the two in the form of an array.
numpy.mod() returns the remainder after dividing the elements at the corresponding positions of the two arrays

import numpy as np

a = np.random.randint(9, size=(3, 3))
print("a =", a)
b = np.random.randint(9, size=(3, 3))
print("b =", b)
print("a + b =", np.add(a, b))
print("a - b =", np.subtract(a, b))
print("a x b =", np.multiply(a, b))
print("a / b =", np.divide(a, b))

The running result shows:

a = [[8 0 2]
 [7 8 4]
 [3 5 1]]
b = [[2 3 5]
 [1 8 3]
 [4 1 2]]
a + b = [[10  3  7]
 [ 8 16  7]
 [ 7  6  3]]
a - b = [[ 6 -3 -3]
 [ 6  0  1]
 [-1  4 -1]]
a x b = [[16  0 10]
 [ 7 64 12]
 [12  5  2]]
a / b = [[4.         0.         0.4       ]
 [7.         1.         1.33333333]
 [0.75       5.         0.5       ]]

2.4.5 Numpy Matrix Operations

NumPy provides a matrix library module numpy.matlib, the functions in this module return a matrix object instead of an ndarray object. The matrix is composed of m rows and n columns (m*n) elements, and the elements in the matrix can be numbers, symbols or mathematical formulas.

numpy.matlib.empty() returns an empty matrix, which is very fast to create.
numpy.matlib.zeros() creates a matrix filled with zeros.
numpy.matlib.ones() creates a matrix filled with ones.
numpy.matlib.eye() returns a matrix with 1s on the diagonal and 0s on the other elements.
numpy.matlib.identity() This function returns an identity matrix of a given size, where the diagonal elements of the matrix are 1 and the other elements are 0.
numpy.matlib.rand() creates a matrix of given dimensions filled with random numbers.

import numpy as np
import numpy.matlib

print(np.matlib.empty((3, 4)))
print(np.matlib.zeros((3, 4)))
print(np.matlib.ones((3, 4)))

The running result shows:

[[0.00000000e+000             nan 1.69759663e-313 0.00000000e+000]
 [0.00000000e+000 0.00000000e+000 0.00000000e+000 0.00000000e+000]
 [0.00000000e+000 0.00000000e+000 0.00000000e+000 0.00000000e+000]]
[[0. 0. 0. 0.]
 [0. 0. 0. 0.]
 [0. 0. 0. 0.]]
[[1. 1. 1. 1.]
 [1. 1. 1. 1.]
 [1. 1. 1. 1.]]

2.4.5 Element-wise multiplication of two matrices

multiple()

Function for element-wise multiplication of two matrices.

import numpy as np

a1=np.array([[1,2], [3,4]], ndmin=2)
a2=np.array([[1,2], [3,4]], ndmin=2)

print(np.multiply(a1, a2))

[[ 1  4]
 [ 9 16]]

2.4.6 Matrix product of two arrays

matmul()
is used to calculate the matrix product of two arrays.

import numpy as np

a1=np.array([[1,2], [3,4]], ndmin=2)
a2=np.array([[1,2], [3,4]], ndmin=2)

print(np.matmul(a1, a2))

The running result shows:

[[ 7 10]
 [15 22]]

2.4.7 Dot Product of Two Matrices

The dot()
function is used to calculate the dot product of two matrices.

import numpy as np

a1=np.array([[1,2], [3,4]], ndmin=2)
a2=np.array([[1,2], [3,4]], ndmin=2)

print(np.dot(a1, a2))

[[ 7 10]
 [15 22]]

2.4.8 Inner product of two matrices

numpy.inner()
is used to calculate the inner product between arrays.

import numpy as np
n1=np.inner(np.array([1,2,3,4]),np.array([1,2,3,4]))
a1=np.array(([[1,2],[3,4]]))
a2=np.array(([[11,12],[13,14]]))
n2=np.inner(a1,a2)

print(n1)
print(n2)

The running result shows:

30
[[35 41]
 [81 95]]

2.4.9 Solving Linear Matrix Equations

numpy.linalg.solve()
This function is used to solve linear matrix equations and represents the solution of linear equations in the form of a matrix.

Solve the system of equations x0 + 2 * x1 = 1and 3 * x0 + 5 * x1 = 2:

import numpy as np

a = np.array([[1, 2], [3, 5]])
b = np.array([1, 2])
x = np.linalg.solve(a, b)
print(x)

The running result shows:

[-1.  1.]

2.4.10 Computing the Inverse of a Matrix

numpy.linalg.inv()
This function is used to calculate the inverse matrix of the matrix, and the inverse matrix is multiplied by the original matrix to obtain the identity matrix.

import numpy as np

a = np.array([[[1., 2.], [3., 4.]], [[1, 3], [3, 5]]])
n =np.linalg.inv(a)
print(a)
print(n)

The running result shows:

[[[1. 2.]
  [3. 4.]]

 [[1. 3.]
  [3. 5.]]]
[[[-2.    1.  ]
  [ 1.5  -0.5 ]]

 [[-1.25  0.75]
  [ 0.75 -0.25]]]
[[[-2.    2.  ]
  [ 4.5  -2.  ]]

 [[-1.25  2.25]
  [ 2.25 -1.25]]]

3 Mutual conversion between Tensor and Numpy

3.1 Tensor to Numpy

import numpy as np
import torch

t = torch.arange(1, 10).reshape(3, 3)
x = t.numpy()
print(f'a={t}')
print(f'b={x}')

The running result shows:

a=tensor([[1, 2, 3],
          [4, 5, 6],
          [7, 8, 9]])
b=[[1 2 3]
   [4 5 6]
   [7 8 9]]

3.2 Numpy to Tensor

3.2.1 Using torch.tensor

import numpy as np
import torch
a = np.random.normal(0, 1, (2, 3))
b = torch.tensor(a)
print(f'a={a}, b={b}')

The running result shows:

a=[[ 1.77128009  2.02509013  0.38144148]
   [ 1.14920329 -0.30733646 -1.20198951]]
b=tensor([[ 1.7713,  2.0251,  0.3814],
          [ 1.1492, -0.3073, -1.2020]], dtype=torch.float64)

3.2.2 Using torch.from_numpy

import numpy as np
import torch

a = np.random.normal(0, 1, (4, 5))
b = torch.from_numpy(a)
print(f'a={a}')
print(f'b={b}')

The running result shows:

a=[[ 1.27357033  0.43722359 -0.74243293 -0.19531152  0.95053336]
   [-0.52235811  0.95262418 -0.11157708  0.65592213  0.04188334]
   [ 0.14932165 -0.40966126  0.09067062  0.3212764  -2.41411188]
   [ 0.63391603  0.29742247 -0.43064322  1.08767221 -0.95699876]]
b=tensor([[ 1.2736,  0.4372, -0.7424, -0.1953,  0.9505],
          [-0.5224,  0.9526, -0.1116,  0.6559,  0.0419],
          [ 0.1493, -0.4097,  0.0907,  0.3213, -2.4141],
          [ 0.6339,  0.2974, -0.4306,  1.0877, -0.9570]], dtype=torch.float64)

4 Tensor to other

4.1 Tensor conversion value

import torch

a = torch.tensor([1])
b = a.item()

print(f'a={a}')
print(f'b={b}')

The running result shows:

a=tensor([1])
b=1

4.2 Tensor to List

import torch

t1 = torch.arange(10)
t2 = t1.tolist() 

print(f't1={t1}')
print(f't2={t2}')

The running result shows:

t1=tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
t2=[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

4.3 Tensor to List Tensor

import torch

t1 = torch.arange(10)
t3 = list(t1)

print(f't1={t1}')
print(f't3={t3}')

The running result shows

t1=tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
t3=[tensor(0), tensor(1), tensor(2), tensor(3), tensor(4), tensor(5), tensor(6), tensor(7), tensor(8), tensor(9)]

Deep Learning: Exploring Tensor and Numpy

introduction

1 Tensor in pytorch

1.1 What is Tensor

1.2 Why do you need Tensor

1.3 How to create Tensor

1.3.1 Convert and create Tensor from other existing data structures

1.3.2 Randomly initialize a Tensor

1.3.3 Load a Tensor from a saved file

1.4 Characteristics of Tensors

1.4.1 Rich common function operations

1.4.2 Flexible dtype and CPU/GPU free switching

1.4.3 Automatic Gradient Solving

2 Numpy

2.1 What is Numpy

2.2 What are the advantages of NumPy

2.2.1 ndarray supports parallelized operations (vectorized operations)

2.2.2 Efficiency is much higher than pure Python code

2.2.3 Storage data address continuous

2.3 How to create Numpy objects

2.3.1 Create an ndarray array using Numpy

2.3.2 Check the dimension of the array and change the dimension

2.3.3 Arrays can also be manipulated through the attributes of ndarray

2.3.4 Create an array whose elements are all 0

2.3.4 Create a new array with a specified shape size and data type filled with 1

2.3.5 Converting a Python sequence to an ndarray object

2.3.6 Create an array using a specified buffer

2.3.7 Convert iterative object to ndarray array

2.3.8 Create an array with a given range of values

2.3.9 Create an array within a specified range of values

2.3.10 Create a proportional array

2.4 Numpy operations

2.4.1 Tiling array elements

2.4.2 Returning a copy of the array as a one-dimensional array

2.4.2 Convert to a continuous flat array

2.4.3 Bit operations in Numpy

2.4.4 Arithmetic operations

2.4.5 Numpy Matrix Operations

2.4.5 Element-wise multiplication of two matrices

2.4.6 Matrix product of two arrays

2.4.7 Dot Product of Two Matrices

2.4.8 Inner product of two matrices

2.4.9 Solving Linear Matrix Equations

2.4.10 Computing the Inverse of a Matrix

3 Mutual conversion between Tensor and Numpy

3.1 Tensor to Numpy

3.2 Numpy to Tensor

3.2.1 Using torch.tensor

3.2.2 Using torch.from_numpy

4 Tensor to other

4.1 Tensor conversion value

4.2 Tensor to List

4.3 Tensor to List Tensor

Guess you like