Mastering TensorFlow: Building Neural Networks with Keras
School
The University of Hong Kong**We aren't endorsed by this school
Course
HKU 3312
Subject
Mechanical Engineering
Date
Dec 10, 2024
Pages
11
Uploaded by marvinlie98
12/10/20241Reading Assignment (Assignment 3)●Chapter 14○Going Deeper – The Mechanics of TensorFlow●Chapter 15○Classifying Images with Deep Convolutional Neural Networks79Chapter 11Logistic Function Recap●We can obtain a class label using the argmax function69Chapter 11TF Keras - Building a MLP●Now, we are ready to use the Keras API to build a model efficiently○In particular, using the tf.keras.Sequential class, we can stack a few Keras layers and build a NN○You can see the list of all the Keras layers that are already available here●For this problem, we are going to use the Dense layer○Aka a fully connected (FC) layer or linear layer●Recall that in a NN, each layer receives its inputs from the preceding layer○Therefore, its dimensionality (rank and shape) is fixed○Typically, we need to concern ourselves with the dimensionality of output only when we design an NN architecture■Note: the first layer is the exception, but Keras allows us to decide the input dimensionality of the first layer after defining the model through late variable creation59Chapter 11796959
12/10/2024249Chapter 11Creating Dataset from Stored Images●Nest, let us extract labels from the list of filenames and assign label 1 to dogs and label 0 to cats39Chapter 11Transformation on Dataset●We can apply transformations to each individual element of a dataset○For this, we will use the previous ds_joint dataset and apply feature-scaling to scale the values to the range [-1, 1)29Chapter 11493929
12/10/20243Mathematical Operations - Multiply●Let's instantiate two random tensors, one with uniform distribution in the range [–1, 1) and the other with a standard normal distribution●Then we compute the element-wise product of them as follows19Chapter 11Tensor●Mathematically, tensors can be understood as a generalization of scalars, vectors,and matrices●More concretely, a scalar can be defined as a rank-0 tensor, a vector can be defined as a rank-1 tensor, a matrix can be defined as a rank-2 tensor, and matrices stacked in a third dimension can be defined as rank-3 tensors●Values are stored in NumPy arrays, and the tensors provide references to these arrays9Chapter 11Logistic Function Recap●In the last chapter, we used the one-hot encoding to represent multiclass labels and used an output layer consisting of multiple logistic activation units○However, an output layer consisting of multiple logistic activation units does not produce probability values68Chapter 1119968
12/10/20244TF Keras - Building a MLP●Next, we apply a transformation via the .map() method to convert the dictionary to a tuple58Chapter 11TF Keras - Linear Regression●We will define a new class derived from the tf.keras.Model class○Subclassing tf.keras.Model allows us to use the Keras tools for exploring a model, training, and evaluation○In the constructor of our class, we will define the parameters of our model■w - the weights■b - the bias parameters○Finally, we will define the call() method to determine how this model uses the input data to generate its output48Chapter 1138Chapter 11584838
12/10/20245Joint Dataset●We may need to build a dataset that combines tensors, e.g., those for features and labels28Chapter 11Manipulating the Shape●Certain operations require that the input tensors have a certain number of dimensions (that is, rank) associated with a certain number of elements (shape)○TF provides useful functions achieve this, such as tf.transpose(), tf.reshape(), and tf.squeeze() 18Chapter 11Tensor●Mathematically, tensors can be understood as a generalization of scalars, vectors,and matrices●More concretely, a scalar can be defined as a rank-0 tensor, a vector can be defined as a rank-1 tensor, a matrix can be defined as a rank-2 tensor, and matrices stacked in a third dimension can be defined as rank-3 tensors●Values are stored in NumPy arrays, and the tensors provide references to these arrays9Chapter 1128189
12/10/20246Graphics Processing Units - GPU●Unfortunately, writing code to target GPUs is not as simple as executing Python code in our interpreter○There are special packages, such as CUDAand OpenCL, that allow us to target the GPU○These libraries are for General-Purpose computing on Graphics Processing Units (GPGPU)■They are not specific to machine learning●Writing machine learning code in CUDA or OpenCL is therefore not very convenient○Better to use a library specific to machine learning7Chapter 11Performance Challenges●In chapter 10, we implemented a very simple multilayer perceptron (MLP) with only one hidden layer consisting of 100 units○We had to optimize approximately 80,000 weight parameters to learn a model for a very simple image classification task●The images in MNIST are rather small (28 × 28)●There will be an explosion in the number of parameters if we wanted to add additional hidden layers or work with images that have higher pixel densities●Such a task would quickly become unfeasible for a single processing unit○How can we tackle such problems more effectively?5Chapter 11Outline●Performance Challenges●GPU●TensorFlow○Tensor○Computation Graph○Creating Tensors○Data Type○Shape○Multiply○Mean, Sum, Std, Product, Norm○Splitting Tensors○Concatenation and Stacking●TF Dataset3Chapter 11●TF Dataset API○Shuffle, Batch, Repeat○From stored Images○Tensorflow-Datasets○TF Keras API■Linear Regression■MLP■Saving / Reloading○Activation Function■Logistic Function Recap■Softmax Function■Hyperbolic Tangent■Rectified Linear Unit753
12/10/20247Installing TF (GPU version)●In case you want to use GPUs on your own system (recommended), you need○A compatible NVIDIA graphics card○CUDA Toolkit○NVIDIA cuDNN library●If your machine satisfies these requirements, you can install TensorFlow with GPU support, as follows13Chapter 11Creating Tensors●You can create a tensors as follows 15Chapter 11Manipulating the Data Type●Learning ways to manipulate tensors is necessary to make them compatible for input to a model or an operation●The tf.cast() function can be used to change the data type of a tensor toa desired type17Chapter 11131517
12/10/20248Mathematical Operations - Multiply●Let's instantiate two random tensors, one with uniform distribution in the range [–1, 1) and the other with a standard normal distribution●Then we compute the element-wise product of them as follows19Chapter 11Mathematical Operations - Matrix Product●The matrix product between t1 and t2 can be computed as follows21Chapter 11Splitting●Assume that we have a tensor and we want to split it into two or more tensors●TF provides a convenient tf.split() function, which divides an input tensor into a list of equally-sized tensors23Chapter 11192123
12/10/20249Concatenation and Stacking●Sometimes, we are working with multiple tensors and need to concatenate or stack them to create a single tensor24Chapter 11TF API●The TF API has many operations that you can use for building a model,processing your data, and more●However, covering every function is outside the scope of this chapter, where we will focus on the most essential ones●For the full list of operations and functions, you can refer to the documentation page of TF here25Chapter 11Dataset from Existing Tensors●If the data already exists in the form of a tensor object, Python list, or NumPy array, we can easily create a dataset using thefrom_tensor_ slices()function○This function returns an object of class Dataset, which we can use to iterate through the individual elements in the input dataset27Chapter 11242527
12/10/202410Transformation on Dataset●We can apply transformations to each individual element of a dataset○For this, we will use the previous ds_joint dataset and apply feature-scaling to scale the values to the range [-1, 1)29Chapter 1138Chapter 1136Chapter 11293836
12/10/202411Shuffle + Batch + Repeat●Finally, to get a better understanding of how these three operations (batch, shuffle, and repeat) behave, let's experiment with them in different orders34Chapter 11Batch●Two more examples of using the .batch()function32Chapter 11Mathematical Operations - Mean, Sum, Std●Compute the mean, sum, and standard deviation along a certain axis (or axes) as follows20Chapter 11343220