[exercise] 2.2: hint for plotting

fe0d7dfd · Dennis Noll · 7d5e506b · fe0d7dfd · fe0d7dfd · fe0d7dfd
Commit fe0d7dfd authored Aug 8, 2022 by Dennis Noll
--- a/lecture2/checkerboard.pdf
+++ b/lecture2/checkerboard.pdf
--- a/lecture2/exercise_2_2_checkerboard.ipynb
+++ b/lecture2/exercise_2_2_checkerboard.ipynb
@@ -222,7 +222,9 @@
   "metadata": {},
   "source": [
    "## Model Evaluation\n",
-    "Now we want to visualize the predicitons of our model and the training data in the two dimensions ($x_1$ and $x_2$)."
+    "Now we want to visualize the predicitons of our model and the training data in the two dimensions ($x_1$ and $x_2$).\n",
+    "\n",
+    "Matplotlib has many great functions to visualize data. A function which can be used to visualize data in two dimensions is `imshow` ([Documentation](https://matplotlib.org/stable/api/_as_gen/matplotlib.pyplot.imshow.html)). "
   ]
  },
  {

 %% Cell type:markdown id:1401ffeb tags:

 # Checkerboard Task - Regression
 In this exercise we will solve the checkerboard task using a regression approach.

 ## Introduction
 The the data consists of 2D vectors (x1, x2) uniformly sampled in the range (-1, 1).
 The data samples are classified by the XOR (exclusive or) operation.

 The task is to train a simple neural network to correctly classify the data.
 For now we formulate this problem as regression task with the network predicting a single value y_model and the optimizer minimizing the mean squared error between model and data.

 %% Cell type:markdown id:c1ac1e6d tags:

 ## Imports and Seeding
 First we will do the necessary imports:
 * `numpy` for general data handling and array manipulation
 * `tensorflow` to build and train the regression model
 * `matplotlib.pyplot` for plotting

 %% Cell type:code id:f9e44804 tags:

 ``` python
 import numpy as np
 import tensorflow as tf
 from matplotlib import pyplot as plt
 ```

 %% Cell type:markdown id:904d7138 tags:

 Then we set a random seed for the `np.random` module. This makes our code reproducible as the random operations will yield the same results in every run through the notebook.

 %% Cell type:code id:681e7c51 tags:

 ``` python
 np.random.seed(42)
 ```

 %% Cell type:markdown id:3a0c76d5 tags:

 ## Data creation
 The the data consists of 2D vectors $(x_1, x_2)$ uniformly sampled in the range $(-1, 1)$.

 The data samples are classified as signal $y = 1$ if either $x_1>0$ or $x_2>0$ (but not both).

 * x = $(x_1, x_2)$ with random numbers $x_1, x_2$ ~ Uniform(-1, 1)
 * y = XOR$(x_1, x_2)$

 %% Cell type:code id:e7c1faa2 tags:

 ``` python
 N = 1000
 xdata = np.random.uniform(low=-1, high=1, size=(N, 2)) # shape = (N, 2)
 ydata = (xdata[:, :1] * xdata[:, 1:]) < 0  # shape = (N, 1)
 ```

 %% Cell type:markdown id:d69b17d4 tags:

 ## Model Creation
 We build a simple neural netowork.

 **TODO**: Inspect the implemented model. What is the total number of parameters? Do the calculation by pen&paper.

 %% Cell type:code id:9e3f4fa1 tags:

 ``` python
 model = tf.keras.models.Sequential(
    [
        tf.keras.layers.Dense(4, activation="relu", input_dim=2),
        tf.keras.layers.Dense(4, activation="relu"),
        tf.keras.layers.Dense(1, activation="sigmoid"),
    ]
 )
 ```

 %% Output

    2022-08-05 12:03:53.151970: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  SSE4.1 SSE4.2 AVX AVX2 FMA
    To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.

 %% Cell type:markdown id:707ba1bd tags:

 **TODO**: Now verify your pen&paper result using the `model.summary` function.

 %% Cell type:code id:69ea5b48 tags:

 ``` python
 """
 TODO: Extract the number of parameters of your model.
 """
 ```

 %% Output

    '\nTODO: Extract the number of parameters of your model.\n'

 %% Cell type:markdown id:6263bc2d tags:

 Define objective, optimizer and prepare TF computation graph:
 * objective: mean squared error
 * optimizer: (stochastic) gradient descent
 * metrics: accuracy

 %% Cell type:code id:ff81b536 tags:

 ``` python
 learning_rate = 5e-2
 sgd = tf.keras.optimizers.SGD(learning_rate)
 model.compile(
    loss="mse",
    optimizer=sgd,
    metrics=["accuracy"],
 )
 ```

 %% Cell type:markdown id:cd34d6e6 tags:

 ## Model Training
 Now train the model to an accuracy of at least 90%. How many epochs are needed to this?

 **Hint:** The `verbose` argument sets the verbosity of shell output `(0: none, 1: high, 2: low)`.

 %% Cell type:code id:0470cb40 tags:

 ``` python
 """
 TODO: How many epochs of training are needed to achieve an accuracy of 90%?
 """
 n_epochs = 1
 fit = model.fit(
    xdata,
    ydata,
    epochs=n_epochs,
    verbose=0,
 )
 ```

 %% Output

    2022-08-05 12:07:24.923012: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)

 %% Cell type:markdown id:5ee7b6c2 tags:

 ## Model Evaluation
 Now we want to visualize the predicitons of our model and the training data in the two dimensions ($x_1$ and $x_2$).

+Matplotlib has many great functions to visualize data. A function which can be used to visualize data in two dimensions is `imshow` ([Documentation](https://matplotlib.org/stable/api/_as_gen/matplotlib.pyplot.imshow.html)).
+
 %% Cell type:code id:0d6e02db tags:

 ``` python
 """
 TODO: Evaluate the output of your model output depending on the two dimensions ($x_1$ and $x_2$).
 Visualize the output of your model along with the training data.
 """
 ```

 %% Cell type:markdown id:45eafe59 tags:

 ## Further Tasks
 Change the network according to the following configurations and retrain the model.
 - 8 neurons in the hidden layer
 - 2 neurons in the hidden layer
 - Add an additional hidden layer of 4 neurons with a ReLU activation

 Describe your observations.

 %% Cell type:markdown id:10210c5c tags:

 ## Summary
 This concludes our tutorial on solving the checkerboard task using a regression

 In this tutorial you have learned:
 * How to build a non-linear tf.keras model
 * How to calculate the number of parameters of a neural network
 * How to train a tf.keras model on a given data distribution
 * How to visualize two dimensional data distributions

 %% Cell type:code id:d1a8b9e1 tags:

 ``` python
 ```

--- a/lecture2/exercise_2_2_checkerboard_solution.ipynb
+++ b/lecture2/exercise_2_2_checkerboard_solution.ipynb