Understand PyTorch tensors - PyTorch Tutorial

- [Narrator] A PyTorch tensor is the core data structure in PyTorch, a multidimensional array similar to NumPy's ndarray, and it supports automatic differentiation for deep learning. Think of tensor as a generalized matrix, which can be of any dimension. For example, zero-dimensional tensors are scalar numbers, like 0, 10, 100, etcetera. One-dimensional tensors are arrays of numbers like an array of 9, 8, 7, or an array of 50, 30, 10, etcetera. Tensors can also be 2D. They are matrix of numbers like a 2 by 3 matrix with a two-element array of an array of three. The 3D+ tensors are used for images, sequences, and models. Here are the primary data types of tensors. There are four categories including floating point, integer, boolean, and complex. For floating points, float32 is mostly used, whereas float64 are rare for deep learning due to the slowness. float16 is half the size of float32, and it speeds up training on GPU while using less memory. bfloat16 is good for TPUs and newer GPUs with larger exponent range. For integer data types, int64 is the most common, used for classification tasks. 32-bit integers are used less commonly and only for smaller ranges. The other three are smaller ints used for quantization purposes, for storage, and speed efficiency. Boolean, as we all know, stores true or false. Complex numbers are less used in deep learning. Now let's look at some examples. First, we'll create some tensors by importing torch and using torch.tensor method. We created a 0D, 1D, and 2D tensor, and let's print them up. You can set the underlying data type of a tensor with an optional argument at creation time. Here's an example. To cast between data types, use the .to(dtype) method. As you can see, the data type for the new tensor b is now floating points. While training the model, you'll deal with higher dimensions. The neural network only accepts the dimension which is defined at the input layer while architecting the model. To get the size, you can use tensor.size method or tensor.shape property. Let's see the results here. And to get the dimension of the tensor use tensor.dimension method or tensor.ndim property. The dimension basically tells whether the tensor is zero dimension, or, 1D or 2D, or even higher than that. The dimension of the tensor is also called the rank of the tensor. Sometimes working with a neural network, you will need to change the dimension of the tensor. This can be done by using the tensor.view(nrows, ncols) method of the tensor. PyTorch tensors also provide flexibilities such as in-place alterations, tensor copying, and changing the shape of the tensor, such as altering the dimensions of the tensor. We'll cover more tensor operations in the next lesson.