{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###ANSWER\n",
    "(for making sure this gets removed)\n",
    "\n",
    "### INSTRUCTOR CONTENT -- REMOVE THIS IN STUDENT VERSION\n",
    "#### The goals/notes for this class are:\n",
    "\n",
    "* This assignment just involves review the take-away concepts from the PCA and making sure student understand all of the components of a neural network (e.g. input layer, hidden layer, outpuer layer, neurons, activation functions, etc)\n",
    "* Once you review the PCA content, you have project groups give a short update on their projects to the class. Any time that is left over becomes project work time."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Day 21 In-Class: Intro to Artificial Neural Networks\n",
    "\n",
    "<img src=\"http://www.bogotobogo.com/python/scikit-learn/images/NeuralNetwork1/NN-with-components-w11-etc.png\" width=600px>\n",
    "\n",
    "### Agenda:\n",
    "\n",
    "1. Review pre-class assignment\n",
    "3. Work on semester projects!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "----\n",
    "# 1. Review pre-class assignment\n",
    "\n",
    "Below is the solution to the pre-class assignment, we're going to talk through this to make sure we all understand what's happening."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X_norm =\n",
      "[[0.3 0.5]\n",
      " [0.5 0.1]\n",
      " [1.  0.2]]\n",
      "y_norm =\n",
      "[[0.75]\n",
      " [0.83]\n",
      " [0.93]]\n",
      "W1---\n",
      "[[-0.02822558  1.16350777 -1.45913933]\n",
      " [ 1.62802952  1.25411256  0.10798232]]\n",
      "Z2 =\n",
      "[[ 0.80554709  0.97610861 -0.38375064]\n",
      " [ 0.14869016  0.70716514 -0.71877143]\n",
      " [ 0.29738033  1.41433029 -1.43754286]]\n",
      "a2 =\n",
      "[[0.6911598  0.7263354  0.40522261]\n",
      " [0.53710421 0.66977446 0.32766358]\n",
      " [0.57380199 0.80444805 0.19192614]]\n",
      "W2 ---\n",
      "[[-0.88251562]\n",
      " [ 1.13869737]\n",
      " [ 1.24748509]]\n",
      "Z3 =\n",
      "[[0.72262604]\n",
      " [0.69742299]\n",
      " [0.64905865]]\n",
      "--------------\n",
      "How did we do?\n",
      "y_norm:\n",
      "[[0.75]\n",
      " [0.83]\n",
      " [0.93]]\n",
      "yHat:\n",
      "[[0.67318503]\n",
      " [0.66761617]\n",
      " [0.6567983 ]]\n"
     ]
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "\n",
    "# input data (hours of sleep, hours of study)\n",
    "X = np.array([[3,5], [5,1], [10,2]])\n",
    "\n",
    "# normalized X\n",
    "X_norm = X / np.max(X)\n",
    "print(\"X_norm =\")\n",
    "print(X_norm)\n",
    "\n",
    "# output data (test score)\n",
    "y = np.array([[75], [83], [93]])\n",
    "\n",
    "# normalized y\n",
    "y_norm = y/100\n",
    "print(\"y_norm =\")\n",
    "print(y_norm)\n",
    "\n",
    "# Neural Network properties\n",
    "inputLayerSize = 2\n",
    "outputLayerSize = 1\n",
    "hiddenLayerSize = 3\n",
    "\n",
    "# Define synapse weights\n",
    "W1 = np.random.randn(inputLayerSize, hiddenLayerSize)\n",
    "W2 = np.random.randn(hiddenLayerSize, outputLayerSize)\n",
    "print(\"W1---\")\n",
    "print(W1)\n",
    "\n",
    "\n",
    "# Apply the first weights to inputs to get hidden layer activity\n",
    "Z2 = np.dot(X_norm, W1)\n",
    "print(\"Z2 =\")\n",
    "print(Z2)\n",
    "\n",
    "# Define our activation function\n",
    "def sigmoid(z):\n",
    "    #Apply sigmoid activation function to scalar, vector, or matrix\n",
    "    return 1/(1+np.exp(-z))\n",
    "\n",
    "# Apply the activation function to our activity levels\n",
    "a2 = sigmoid(Z2)\n",
    "print(\"a2 =\")\n",
    "print(a2)\n",
    "\n",
    "print(\"W2 ---\")\n",
    "print(W2)\n",
    "# Apply the second weights to the hidden layer\n",
    "Z3 = np.dot(a2, W2)\n",
    "print(\"Z3 =\")\n",
    "print(Z3)\n",
    "\n",
    "# Apply the activation function to produce prediction for yHat\n",
    "yHat = sigmoid(Z3)\n",
    "\n",
    "# Compare yhat to known y_norm\n",
    "print(\"--------------\")\n",
    "print(\"How did we do?\")\n",
    "print(\"y_norm:\")\n",
    "print(y_norm)\n",
    "print(\"yHat:\")\n",
    "print(yHat)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "----\n",
    "# 2. Work on semester projects\n",
    "\n",
    "This is a chance to make some progress on your project with your group while the instructors are around to help. Make use of it!\n",
    "\n",
    "Look back at the <a href=\"https://www.egr.msu.edu/classes/cmse202/days/Day-20-Git2/Day-20_In-Class_GroupInventory-STUDENT.ipynb\"> Day-20 group inventory </a>. Are you on track? If you have questions, make sure to ask."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Put your answers here!*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "-----\n",
    "### Congratulations, you're done!\n",
    "\n",
    "Submit this notebook to D2L so that your instructor knows where you're at with your project!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "&#169; Copyright Michigan State University Board of Trustees"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}