The text is released under the CC-BY-NC-ND license, and code is released under the MIT license. If you find this content useful, please consider supporting the work by buying the book!
在 Python 中的数据操作与 NumPy 中基本是一致的:甚至像 Pandas 这样的工具 (第三章) 都是基于 NumPy 构建的。
本章节讲介绍一些操作 NumPy 数组的方法,包含数据的读取、获取子数组、拆分数组、操纵数组维度、连接多个数组等。 This section will present several examples of using NumPy array manipulation to access data and subarrays, and to split, reshape, and join the arrays.
这些操作可能显得非常枯燥,但它们在后续其他示例中会被大量的使用,所以要充分的掌握。
While the types of operations shown here may seem a bit dry and pedantic, they comprise the building blocks of many other examples used throughout the book. Get to know them well!
下面是我们需要了解的数据操作的几个方面:
We'll cover a few categories of basic array manipulations here:
Data manipulation in Python is nearly synonymous with NumPy array manipulation: even newer tools like Pandas (Chapter 3) are built around the NumPy array. This section will present several examples of using NumPy array manipulation to access data and subarrays, and to split, reshape, and join the arrays. While the types of operations shown here may seem a bit dry and pedantic, they comprise the building blocks of many other examples used throughout the book. Get to know them well!
We'll cover a few categories of basic array manipulations here:
首相我们来介绍一些数组的属性。我们先创建三个随机数组:一个一维数组,一个二维数组,一个三维数组。我们会使用 NumPy 的随机数生成模块并通过 np.random.seed
来保证每次执行所生成的数据是一致的:
First let's discuss some useful array attributes. We'll start by defining three random arrays, a one-dimensional, tw-dimensional, and three-dimensional array. We'll use NumPy's random number generator, which we will seed with a set value in order to ensure that the same random arrays are generated each time this code is run:
import numpy as np
np.random.seed(0) # seed for reproducibility
x1 = np.random.randint(10, size=6) # One-dimensional array
x2 = np.random.randint(10, size=(3, 4)) # Two-dimensional array
x3 = np.random.randint(10, size=(3, 4, 5)) # Three-dimensional array
每个数组都有属性 ndim
(维度),shape
(每个维度的长度), size
(整个数组的长度):
Each array has attributes ndim
(the number of dimensions), shape
(the size of each dimension), and size
(the total size of the array):
print("x3 ndim: ", x3.ndim)
print("x3 shape:", x3.shape)
print("x3 size: ", x3.size)
x3 ndim: 3 x3 shape: (3, 4, 5) x3 size: 60
另一个有用的属性是 dtype
,数组中数据的类型(在理解 Python 的数据类型中我们有提及):
Another useful attribute is the dtype
, the data type of the array (which we discussed previously in Understanding Data Types in Python):
print("dtype:", x3.dtype)
dtype: int64
itemsize
是数组中每个元素的(字节)长度,nbytes
是整个数组的(字节)长度。
Other attributes include itemsize
, which lists the size (in bytes) of each array element, and nbytes
, which lists the total size (in bytes) of the array:
print("itemsize:", x3.itemsize, "bytes")
print("nbytes:", x3.nbytes, "bytes")
itemsize: 8 bytes nbytes: 480 bytes
通常,nbytes
等于 itemsize
x size
。
In general, we expect that nbytes
is equal to itemsize
times size
.
如果你熟悉 Python list 的索引,那么你会发现 NumPy 的所以非常类似。在一维数组中,$i^{th}$ (从 0 开始)的值可以通过方括号加索引值的方式访问:
If you are familiar with Python's standard list indexing, indexing in NumPy will feel quite familiar. In a one-dimensional array, the $i^{th}$ value (counting from zero) can be accessed by specifying the desired index in square brackets, just as with Python lists:
x1
array([5, 0, 3, 3, 7, 9])
x1[0]
5
x1[4]
7
采用负数索引值则可以从数组的末尾开始索引:
To index from the end of the array, you can use negative indices:
x1[-1]
9
x1[-2]
7
多维数组中,元素可是通过逗号分隔的索引 n 元组(tuple)进行访问:
In a multi-dimensional array, items can be accessed using a comma-separated tuple of indices:
x2
array([[3, 5, 2, 4], [7, 6, 8, 8], [1, 6, 7, 7]])
x2[0, 0]
3
x2[2, 0]
1
x2[2, -1]
7
数组的值也可以采用以上的索引方式获取后进行修改:
Values can also be modified using any of the above index notation:
x2[0, 0] = 12
x2
array([[12, 5, 2, 4], [ 7, 6, 8, 8], [ 1, 6, 7, 7]])
与 Python list 不同,NumPy 数组元素有固定类型,因此如果你要讲一个浮点数赋值给一个整型数组的元素,其赋值会被转为整型。
Keep in mind that, unlike Python lists, NumPy arrays have a fixed type. This means, for example, that if you attempt to insert a floating-point value to an integer array, the value will be silently truncated. Don't be caught unaware by this behavior!
x1[0] = 3.14159 # this will be truncated!
x1
array([3, 0, 3, 3, 7, 9])
与采用方括号访问单个元素相似,我们可以采用方括号与 切片 标记 :
来访问子数组。NumPy 切片同样遵循标准 Python list 的语法规则;利用如下语法访问 x
的切片:
Just as we can use square brackets to access individual array elements, we can also use them to access subarrays with the slice notation, marked by the colon (:
) character.
The NumPy slicing syntax follows that of the standard Python list; to access a slice of an array x
, use this:
x[start:stop:step]
``
如果任意一个变量没有指定,则默认为 ``start=0``,``start=0``, ``stop=``*``维度的长度``*, ``step=1``。后面有访问一维和多维数组切片的例子。
If any of these are unspecified, they default to the values ``start=0``, ``stop=``*``size of dimension``*, ``step=1``.
We'll take a look at accessing sub-arrays in one dimension and in multiple dimensions.
x = np.arange(10)
x
array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
x[:5] # first five elements
array([0, 1, 2, 3, 4])
x[5:] # elements after index 5
array([5, 6, 7, 8, 9])
x[4:7] # middle sub-array
array([4, 5, 6])
x[::2] # every other element
array([0, 2, 4, 6, 8])
x[1::2] # every other element, starting at index 1
array([1, 3, 5, 7, 9])
一个容易令人迷惑的情况是当 step
为负数的情况。此时 start
和 stop
的默认值互换。这也是一种将数组倒序的方式:
A potentially confusing case is when the step
value is negative.
In this case, the defaults for start
and stop
are swapped.
This becomes a convenient way to reverse an array:
x[::-1] # all elements, reversed
array([9, 8, 7, 6, 5, 4, 3, 2, 1, 0])
x[5::-2] # reversed every other from index 5
array([5, 3, 1])
x2
array([[12, 5, 2, 4], [ 7, 6, 8, 8], [ 1, 6, 7, 7]])
x2[:2, :3] # two rows, three columns
array([[12, 5, 2], [ 7, 6, 8]])
x2[:3, ::2] # all rows, every other column
array([[12, 2], [ 7, 8], [ 1, 7]])
子数组的维度也可以变为逆序的:
Finally, subarray dimensions can even be reversed together:
x2[::-1, ::-1]
array([[ 7, 7, 6, 1], [ 8, 8, 6, 7], [ 4, 2, 5, 12]])
print(x2[:, 0]) # first column of x2
[12 7 1]
print(x2[0, :]) # first row of x2
[12 5 2 4]
行访问时,全切片也可以被忽略:
In the case of row access, the empty slice can be omitted for a more compact syntax:
print(x2[0]) # equivalent to x2[0, :]
[12 5 2 4]
切片返回的是数组的一个视图(或者说是引用)而不是一个拷贝。这与 Python list 的切片是不同的:在 Python list 中 切片是一个拷贝。考虑前面提到的二维数组的情况:
One important–and extremely useful–thing to know about array slices is that they return views rather than copies of the array data. This is one area in which NumPy array slicing differs from Python list slicing: in lists, slices will be copies. Consider our two-dimensional array from before:
print(x2)
[[12 5 2 4] [ 7 6 8 8] [ 1 6 7 7]]
我们提取一个 $2 \times 2$ 的子数组:
Let's extract a $2 \times 2$ subarray from this:
x2_sub = x2[:2, :2]
print(x2_sub)
[[12 5] [ 7 6]]
如果我们修改了这个子数组,我们会看到原始的数组也同样发生了变化:
Now if we modify this subarray, we'll see that the original array is changed! Observe:
x2_sub[0, 0] = 99
print(x2_sub)
[[99 5] [ 7 6]]
print(x2)
[[99 5 2 4] [ 7 6 8 8] [ 1 6 7 7]]
这个默认的行为非常的有用:我们可以在大型的数据集中采用切片、访问和更改大型数据的任意部分而不用担心因为拷贝带来的冗余的数据缓存。
This default behavior is actually quite useful: it means that when we work with large datasets, we can access and process pieces of these datasets without the need to copy the underlying data buffer.
x2_sub_copy = x2[:2, :2].copy()
print(x2_sub_copy)
[[99 5] [ 7 6]]
现在如果我们修改了子数组,原始数组讲不会被修改。
If we now modify this subarray, the original array is not touched:
x2_sub_copy[0, 0] = 42
print(x2_sub_copy)
[[42 5] [ 7 6]]
print(x2)
[[99 5 2 4] [ 7 6 8 8] [ 1 6 7 7]]
另一个常用的操作是对数组进行重塑,即修改数组的维度信息。最灵活的方式就是采用 reshape
方法。例如如果你希望将数字 1 到 9 放入一个 $3 \times 3$ 的数组中,你可以这么做:
Another useful type of operation is reshaping of arrays.
The most flexible way of doing this is with the reshape
method.
For example, if you want to put the numbers 1 through 9 in a $3 \times 3$ grid, you can do the following:
grid = np.arange(1, 10).reshape((3, 3))
print(grid)
[[1 2 3] [4 5 6] [7 8 9]]
为了让这个操作有效,原始数组的大小必须和重塑后的数组的大小一致。reshape
会尽量采用非拷贝的方式来处理初始数组,但是在初始数据没有提供连续的内存空间的时候就无法实施。
Note that for this to work, the size of the initial array must match the size of the reshaped array.
Where possible, the reshape
method will use a no-copy view of the initial array, but with non-contiguous memory buffers this is not always the case.
重塑方法另一个常用的情况是将一个一维的数组转换为一个两位的行或者列矩阵。可以通过 reshape
或者 newaxis
方法实现。
Another common reshaping pattern is the conversion of a one-dimensional array into a two-dimensional row or column matrix.
This can be done with the reshape
method, or more easily done by making use of the newaxis
keyword within a slice operation:
x = np.array([1, 2, 3])
# row vector via reshape
x.reshape((1, 3))
array([[1, 2, 3]])
# row vector via newaxis
x[np.newaxis, :]
array([[1, 2, 3]])
# column vector via reshape
x.reshape((3, 1))
array([[1], [2], [3]])
# column vector via newaxis
x[:, np.newaxis]
array([[1], [2], [3]])
这种形式的转换在之后的章节中会频繁出现。
We will see this type of transformation often throughout the remainder of the book.
连接两个 NumPy 数组可以通过 np.concatenate
, np.vstack
,np.hstack
来实现。np.concatenate
需要一个数组的 list 或 tuple 作为第一个参数:
Concatenation, or joining of two arrays in NumPy, is primarily accomplished using the routines np.concatenate
, np.vstack
, and np.hstack
.
np.concatenate
takes a tuple or list of arrays as its first argument, as we can see here:
x = np.array([1, 2, 3])
y = np.array([3, 2, 1])
np.concatenate([x, y])
array([1, 2, 3, 3, 2, 1])
你也可以连接多个数组:
You can also concatenate more than two arrays at once:
z = [99, 99, 99]
print(np.concatenate([x, y, z]))
[ 1 2 3 3 2 1 99 99 99]
同样也适用于二维数组:
It can also be used for two-dimensional arrays:
grid = np.array([[1, 2, 3],
[4, 5, 6]])
# 默认按照第一个维度连接
# concatenate along the first axis
np.concatenate([grid, grid])
array([[1, 2, 3], [4, 5, 6], [1, 2, 3], [4, 5, 6]])
# 显式的按照第二维连接
# concatenate along the second axis (zero-indexed)
np.concatenate([grid, grid], axis=1)
array([[1, 2, 3, 1, 2, 3], [4, 5, 6, 4, 5, 6]])
在数组维度不一致的情况下采用 np.vstack
与 np.hstack
更清晰一些:
For working with arrays of mixed dimensions, it can be clearer to use the np.vstack
(vertical stack) and np.hstack
(horizontal stack) functions:
x = np.array([1, 2, 3])
grid = np.array([[9, 8, 7],
[6, 5, 4]])
# vertically stack the arrays
np.vstack([x, grid])
array([[1, 2, 3], [9, 8, 7], [6, 5, 4]])
# horizontally stack the arrays
y = np.array([[99],
[99]])
np.hstack([grid, y])
array([[ 9, 8, 7, 99], [ 6, 5, 4, 99]])
类似,np.dstack
会按照数组的第三维连接。
Similary, np.dstack
will stack arrays along the third axis.
x = [1, 2, 3, 99, 99, 3, 2, 1]
x1, x2, x3 = np.split(x, [3, 5])
print(x1, x2, x3)
[1 2 3] [99 99] [3 2 1]
N 个分割点会生成 N + 1 个子数组。
np.hsplit
与 np.vsplit
类似:
Notice that N split-points, leads to N + 1 subarrays.
The related functions np.hsplit
and np.vsplit
are similar:
grid = np.arange(16).reshape((4, 4))
grid
array([[ 0, 1, 2, 3], [ 4, 5, 6, 7], [ 8, 9, 10, 11], [12, 13, 14, 15]])
upper, lower = np.vsplit(grid, [2])
print(upper)
print(lower)
[[0 1 2 3] [4 5 6 7]] [[ 8 9 10 11] [12 13 14 15]]
left, right = np.hsplit(grid, [2])
print(left)
print(right)
[[ 0 1] [ 4 5] [ 8 9] [12 13]] [[ 2 3] [ 6 7] [10 11] [14 15]]
类似地, np.dsplit
会按照数组的第三维分割。