{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "##

✅ Put your name here

" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Day 21 Pre-Class assignment: Introduction to Artificial Neural Networks\n", "\n", "This entire Artificial Neural Networks module is from Neural Networks Demystified by @stephencwelch. We have streamlined the content to better fit the format of the class. However, if you have questions or are just curious I highly recommend downloading everything from the following git repository. It is a great reference to have:\n", "\n", " git clone https://github.com/stephencwelch/Neural-Networks-Demystified\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Goals for today's pre-class assignment \n", "\n", "

\n", "\n", "1. Think about the architecture of Artificial Neural Networks\n", "1. Model data flow by performing forward propagation\n", "1. Explore A Neural Network (through a visualization)\n", "\n", "## Assignment instructions\n", "\n", "**This assignment is due by 11:59 p.m. the day before class** and should be uploaded into the appropriate \"Pre-class assignments\" submission folder in the Desire2Learn website.\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# 1. The architecture of Artificial Neural Networks" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Watch the following video:" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "data": { "image/jpeg": "/9j/4AAQSkZJRgABAQAAAQABAAD/2wCEAAUDBAgICAgICAgICAgGCAgICAgICAgICAgICAgICAgICAgIChALCAgOCQcIDBUNDhERExMTCAsWGBYSGBASExIBBQUFCAcIDQgICBIICA0SEhISEhISEhISEhISEhISEhISEhISEhISEhISEhISEhISEhISEhISEhISEhISEhISEv/AABEIAWgB4AMBIgACEQEDEQH/xAAcAAEAAgMBAQEAAAAAAAAAAAAAAggEBQYDBwH/xABWEAABAwIDBAUHBgoECQ0AAAAAAgMEAQUGEhMYVJTUERQiIzIhMTNCQ1NjByRSYnODFTRBRFFxcoKTo2SRofBhhJKxssHDxOEWdIGipLO00dLT8fLz/8QAGQEBAQEBAQEAAAAAAAAAAAAAAAIDBAEF/8QAKBEBAAIBAwMDAwUAAAAAAAAAAAIDEgEiMgQTQhEzUiEjMRRBU2Fj/9oADAMBAAIRAxEAPwCmQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAALMbF+J9/sHE3HkBsX4n3+wcTceQArOCzGxfiff7BxNx5AbF+J9/sHE3HkAKzgsxsX4n3+wcTceQGxfiff7BxNx5ACs4LMbF+J9/sHE3HkBsX4n3+wcTceQArOCzGxfiff7BxNx5AbF+J9/sHE3HkAKzgsxsX4n3+wcTceQGxfiff7BxNx5ACs4LMbF+J9/sHE3HkBsX4n3+wcTceQArOCzGxfiff7BxNx5AbF+J9/sHE3HkAKzgsxsX4n3+wcTceQGxfiff7BxNx5ACs4LMbF+J9/sHE3HkBsX4n3+wcTceQArOCzGxfiff7BxNx5AbF+J9/sHE3HkAKzgsxsX4n3+wcTceQGxfiff7BxNx5ACs4LMbF+J9/sHE3HkBsX4n3+wcTceQArOCzGxfiff7BxNx5AbF+J9/sHE3HkAKzgsxsX4n3+wcTceQGxfiff7BxNx5ACs4LMbF+J9/sHE3HkBsX4n3+wcTceQArOCzGxfiff7BxNx5AbF+J9/sHE3HkAKzgsxsX4n3+wcTceQGxfiff7BxNx5ACs4LMbF+J9/sHE3HkBsX4n3+wcTceQArOCzGxfiff7BxNx5AbF+J9/sHE3HkAKzgsxsX4n3+wcTceQGxfiff7BxNx5ACs4LMbF+J9/sHE3HkBsX4n3+wcTceQArOCzGxfiff7BxNx5AbF+J9/sHE3HkAKzgsxsX4n3+wcTceQGxfiff7BxNx5ACs4LMbF+J9/sHE3HkBsX4n3+wcTceQArOCzGxfiff7BxNx5AbF+J9/sHE3HkAKzgsxsX4n3+wcTceQGxfiff7BxNx5ACs4LMbF+J9/sHE3HkBsX4n3+wcTceQArOCzGxfiff7BxNx5AbF+J9/sHE3HkAKzgsxsX4n3+wcTceQGxfiff7BxNx5AC/YAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACAAAAAAAAAAAmAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAgCZAAAAAAAAACYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACAAAAAAAAAAAmAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAgAJgEAJggELSrzATB4IfRWvRRaVK/aPQCYIZ0mBabs1L19Lp+ZyXYzuZPtWPOBsgYiJrNXFtUcSp1pOdTWbpcQedLtEqp9HWGc0OnTITnp3P2v/EDPBz8jEzCZjURPSvWjOy1SErTpMtM+9I0xfblRkym5TbzLzmiytmupqO+5p8cDogaPDmJodwaW7Gdz6PkeQtORxv7VoxcN4mQ9bE3SSlMNnvVr1V9hDWt6atQOmBz9nxNElvrjsLUpaWGpKF5ehp9p72zCvbHQAAAAAAAAAAAAAAAAgAAAAAAAAAAAEwAAAAAAAAABD1gPWNHjeXViE68mq6aKml10vH6YDeA5G3XZqsuW+3qUquC09ldS836DX9hUna58plTNJLqXEXL0Kqewle5+w7IQ6sZ0nDqv7qaWzvaq7Xz+uTzNfi/e/wCMGRIgxqyrlJlU1OrJaWj4bWiB2AOXtLrjlv0mpKetPMurZrn1HEe4M3CD6KM9XzPasf0qZCu8C27AAEwAAAAEATINgcF8oV7pbLlapT0lxuC91qLIZR0rbcd0NZjuv3VGpt0iZKTeGrdOXGqiZWZR1TWddYs+D1hjQTI9D3x3d8sLct6E6pakqtknrSKJ9fuasiVCjuqmoYd0Zj7TSXXmvSNeTuAOWl212XhdSZEl6U69A6zrL7tzV0df83MRKoMJ22/gxSm13bSzW5pTziHIrzX4xpZvm+l2e+Orwfh+ltj9WVMkTEqrlb6041XJ8Fo/MNWC2WhDUeK02yp3soUrtuOfegfMLTZLTbYMtMhLn4SZnz40N5pb3Xn3dbuNDiGDPvl/kxJduTcszqnLa061EZe6s2ufrd+8++fRH3rfApIlVyJ73PJVTtuNuvHpOttumutLeajyHo3bZqtGo42BxtvtkpV2u0qO6qPMT1BVY6l6kZ9rq/oX/wD3jo/k1jyEsSnpbHVnp86S+trN06dFf/U9YuLGF1VVLD3bZdeQrJ6ekcWzFaXm4rlYr7fXXtNCV/o96Bo7rOVAvy5KoUt9mfbmmdWIz1jvWXvNX91RzVvjT03qXMZtzzqZVydiy6urZbbXA6uxofzz6JIxZFZddbeXRtLelkV49fWMi7XvQdjdKU9WkampIUvsIA5vBmDmYcu+Kbj6LVyU0zHT2fI1o9/pf0fXkVONwBhqeq1TYDqZbDsBbXUayGWW4rUmO7mYkQer9/6p9ZrNW1Ipnr0sTNOjKvdudPojbAcjYsPONzXrjm0aXVpus6E4nOtt6jPmafSaFcGi7PPtbseRIrAkusoRHSzq1a1usQXvnH7v9p9MAHy3C9xlS5toblMdXnQ405ctGSnkjegj1d3fV859VPNtlNKqVRKcy/Er6Z6AAAAAAAAAAAAAAAgTAEAAAAAAAAAABMAAAAAAAAAAQ9YxbvARJZcYX4XkZOyZXrHm/mo2qqKZlZOwgDFpbGcza1pzuMtaOdXujGgYfjMrS4lKlKR5Uai8+n9iYWGVLezOuSVKkZMi4/o22PuDZ4clLcis1dVmc6Mi1fFCHoizxqa3Qw3859N8QxmrEyhxt1ClUo3G6sprN3bjfxTRrWp2TKouPKdUy7kT1d3Ta0v4/pz9uKpapGSnWFNaLS+qsqZacb+3eA3Mm3Q2G1K7uElftWfmzh+4fgxWqKdiq1Nbxu59Rxw1d7S2ilufk9DaGXsi0vq1Mmsz/wACcGU2qW3+D6Z23vxuqE9Ef7X7csdQGwEELDzcfTTzqSk5W6XGTKfVEhq06I/GH/d/q/wnqjBTCqd8t55f0lu1ObW6cuDr06aEfds9GyexLBQrKqQ3m/bNq06ldMyKpUmvrJOas+GaMKW0ujbsdaexVae8R8H9R4TYa7bVL0equq5/nDCuleT4zIznHdNU6apSxqsdiQbDC0rolSfCvths6XEHP4cj0omVIqjv5L0jMv1+hmuVmn/RQ6AwWLfVuQ44hdNJ7tKa+L74DkLFbJ8yMmr0jMllWtDezd6478cyLXa1sz3NV115uNH1oya9vT1+y/oHWwIaI6NNtOVGZxXRT8hkBDgcMRHVplQ1xl9XktO/O3UabjjvxyFotElN21lsvN0W01RbzVe66dHL0H0E1dMRQet0g9aZ64pOfq+fvAdtiw8P0oqis6m1MvSasqa90+eMbCbSY3VlvyHk0VnQta+8b+wOkAWwY9pjo0+hhvMyjRQrJ5mjGuGHIryG2qoUllnzNIVkbN0ANBdLFV/Ra1tKIxVqtWUo8q6M+bvTeEwAAAAgTIACZAmAAAAAAAAAAAAgAAAAAAAAAAAAEwCAEwQJgAAAAAEPWMe4MuLRVLTmi5Xwry6nQZHrHhPk6bbi8qlZEZ8ifSAaxFhzPsyHn1PORvB0o0zHRhCHnccVqOUeVnqlTvdoMhjEzC6JqlqWrP8AojPHo5fk/kjzFf4s8EPKThyi39dEmSz2GkrQ0robXo+End8OR5D6ZK9RLiE5KZHcmcl+G1dGbqsv+CPw3XcZv8ELYmK7JSRC6uimZTCmls0Ur2rHmN82jop2aZfqHCfKHcdaOyjq81tWs0tK1I02vv3TWWq4zGMuiuS71ryRmXu81Pjf0cIfUTVYpnVjxHnaeKnZQZtvW7VlFXkpS9l7aUmix/8Aiqfo9YazkXbYujp45WwZuFLZ1WOlHtF9t1Xxfym4qKA8hXimyWc8pB5TGkrbUivhWgyDBvctLEd11XhbQLCHNqMBO1VDSlXsVuM/1HQo837xosCR8kJrpp2ne2btHmV96TTwg16n3JJtuUVTppVKv2SZzVpt8dxGpF1YavCtppXRpufHaMlfW2G31aiZNatV6ulSaNuLkeXoa/s/tNnO3hrrjc6Mqoijbjzqk1WhKEnzj5EHsR9FwdvylM0eX82RIr6N34H9GO5kXN2KhSpqE6SPPIa9H9616oHP41uM3uY3ZZ64h1a2kr7xDTPxzaQcGW6M8qdGhsJnraydYVTy+iMBitX1Tpa6pUp9DUaMn3bT/wD8nbo8lElog4HAsyU824tT6lS0Ou60V30f3B07V+YplpIV1VxaslEu93qOHGOSqsXCRHj01JK3etRko/nsvm+QtpbdJlxW3XIvKhhSe7iu8wHkGRacdWqVImRWJbbr1sTnk9HszoGJKHaZkLS4n6qz49h7DTce3yLpEYRGkTZkp6RpNfjEHW/F/wDmx2GHKx5DbeWiIzq052nYvdtv/wB/ch67gHJ4gmSIEfXXNbUhCvC613jvwTncI/Kkm5u3JLUB9TFpdabzI7xxz7ghb6cDGgT2n052lpUk9gJkCZAATIEwAAAAAAAAAAAgAAAAAAAAAAAAAmQBMCAJkABMgTAAACHrAAAAAJkAANViu0VmxlMJc0s6215suc9LTamoyezmU5Xxur9IbEZPrADFucNL7LjS/C8jIZWQAhsc/h+5KSvqcnyPtU7CvfNfpN/00qYN1tTElGV5Gb6KvaINY5h51PkbnyG0/RV3hjvi6Ptz3ey37riW01rWvRRJxlwkqurqY7NPmjKs7znvP8BsK4Racr0yX35H7a+7/qOgixW2UZG6JTT6KSJ1yt5LjOqrdH71r2aQmlEpp4UEGPN++TPCvTpqyZc3e5Mx0uRrpTlGX6vIqmjfglpT0dj3Dx+or1qQryZmYfh+u6Ytgl6jknOjKp1LTi0f9nf/APDmJZE1hxFSPKqKvVlLb9djxPd19IIbHG0XVgSqJ8VEVWmn2JzmEpqZr2RTmpDjJ1o2f2n/AOJ4YY+UqJfIUty3ocS5G7C0vJ6Mjb/tjnbLclpiPQ4GVTlsW6uNK/2JbyxubhIbhKiyEKWlt+ZrSWkVz6nWHu47g1uH6YnRdJjlymMMWd1daMrVot6fl7rQ+5M/B0BqW/Br23HI3zl1bvs9D5uwz/359Hu1tYlNKYkIS42v1FB64y8rhxlRZMVbHYXkXkX6Q1/yiSc8yOxFX2ZKM8zL7Np/23D6561YYaiSbS8005IR+J0y+na9g6cjgFcWK3IfddebkRmc60td5r/cBD7fDjNNspaQnukIyJp8M+cypLFtYyLopKdZ1DOl3jjc9j0P8Y4uz40xJEnq6xFS1Z8ruil2rNHdL2Dx1UBljWclTn2JUi4M9yhC/Ru+5C2ZbWHLm/pXCqusIW089E9nFa9h/GO/tlsjxtTq7DTOsrO7pI09R05yw22vRVt1fRcWu1WR7w6iAt3L3yUpc+p64K2LOtVFq1WlaL/00+v9qfltuFVK0H06chFP0d25T3rRslrTSmaqsppbrOhv0yKzPKR7pGpkIW3pD1jQW1MnIlbLq3EVV6KajI4b/wBYCfQQoeMuSltOapqayJb1ewhLKK+sv0hhO7Hb+7GV0Y7Py3nSflf1Gl/Bsrz9bV0/sUIwrg8hWnIRly9jV/I4T3vlWjTqflW3pMgvzEzpdIAAABACdSBjXCWlpPSpXj8Jr4FVsr6XFdmQr/Icr/qMZXb8WMrsZYtyAGzZsAAAAAAAAAAAAABMgAJggAJggTAgCYAAAAAAAIEwIAmAIAmAIEGPN++TIMeYDjsUKXE1lo8Su3G+u6/6eH/tjeYbltPMUaRTyxkNMupUj2miYeMsj1EsK9ROt97+b/ziGG4biWdSnR11Cnet/HdCEZMWPZ2lKiW5lERVXVzExWUtuU+NofnBwOPUQXG25Vtysqkr8bXduNux/gH2CLIS4n/TSr1D4B8qkbqUxuMjOllHbh5f57JbObscBSJUVmtxcYVIjz69pTSe8bisege0Dp7N8otpndY6lKRIXDRrLSj3ZpsN3qMtTNrhZlMyWc//ADVr27Jv7Lge02/rDsSC1GVJQ6h51nyOVbC63G46o+5pyWFK/CCOwtSPRsRfbsnM2JbVaZU+mek55i1brIeOofuCo0WS5EWmbGjI6sylXdyNV/8Aw/nBp7TGSw8ytCdZy2I+fpV6R+K/3Gj9yGT6dfcOsyFNvtpT1mMnoaWvttr+C8aaVEjXGji6R221W32WmzqdaNvGvGSPkpmU70NIj6nduOa/oDS4xZkMtsotfQq4s1pmy+0a9rrhsybxabZVEZeesVyRXojL1nvK77n+/wCgwPk/mTJjtxjuxJsBqL3CHluvd+775jrB44NjNP3Cuq51hTKOtdv83lSPTs6B9KBW1jFhYp49V5X9IXqGwYZSinQhKUp+qgmQlyUMozuqypIWmF+Q1UGZJdWmtGUtx/pOK7xf/QZV3r0NL/Y/1Gc54wyROWMMnhAb1a1cX2qeok2R5wkdCEJ+p/qPcmmGMEUwxigQfZStCk19c9iBrq11a+0uqqlSK+JleQ2ZrLb5VPV+uYt3xOxGe0FtyVOZM/cs6hlTxZU8W6MW7T24zLj682mwjOvKaWmLa18EC4q/UyeFxxK5VpVK2uetK05VdCGjZs3NpvcWVTuX21K6PDnpqINkfH8NRkvVbWtLjkjRaX839Jq/7von1OwtvpjM0k1zPJT21Fogm/RpbiULTmUjtoPydKQiqEq9qvsHhPXpu0drRVaZMnZIUh1dzLc8XqfUPn2T8YsZz8YtqGzxgPZ20qr4j2bO3k6awAAAAAAAAAAAAAAAAAAA4tKKdKqpSkOHPfKC9VuApyiEuaL0VWRX2zAG5hy2ns2kuitFbjavtP79JkHJRVyNGe2pCGnnF61NLvO6fMHPWinFw6SHo0ZcZako1u897ofuhDt5clppOd1aW0/SUoQ5KHU5mlpcT9JJwUqC9oyKyG1OUeeauSE5dTT9+z/AJSs6fwihLchlMhba2VR2XtPT/wAX995QO/Jmrw1PVJjtuqacZV4VId85tAtAmQAEwCAEwQAAhF8JM1l2eq3DkLorKpCHcigNPBc61cHFU7TbK6Lqr7H0H+3NtO7h9DqKdmT3KvtfYHlhC29WipSqmVxfbdGI6KkpchtV6HFo7bvuQhzGBV39py4Sb+1E02/xPqfeOaRxmJ20Xm6qrNU7Fiwo3YTke/nn2G2SU0htuqV2UM9pSj59bkO0dcccdUpu5yfnKMvoIv8AfuC2c2p+T2HkjNTEZUuIk5GVp+B+ZP8A23vj7BLm6bCn1py5E58qjio1mjt3GcjOuI281GmJU07pN/RfOcxXje71mvWlqyLciyq9Wh3Bb3418319YhdbMuTKVuQ5KdJPVns6835076d8x7k0u5RF3GIhTSlrdW7KV3fzX3P9IM62Q41E+VS3noUPRySPSNypHwDeXqnUUoZj5aJlo0Vo9mx/TC0vmCP+Va7pFkR0IlwZLLWeanR7tr2+hu8k+vQ7lFjR1Jj5tbwUad/GFumN8nTNIaHbZRXkt72Zr67Ujv8A/eDMlQ2p0rpWiimYXgV5s8kLaOXCXbJEWbmopp9eS4+Xwuv+3O+R5aHI3x11CXIuVNyS6nT6ur0iPt3zjvk0+UB2Y69bWIslzq3kZlyE91Rv3L/2OYgfVblOQzTtdpS/AhHpFmLBgrUrXldpz1EezZJW220bVquK1n1ed1X+Zk2QWmY0tnO2pP0zJIdJ5rDIYttezIy+sjsKMoxJMTy5265F/wCn+s86PP09RKjnzx2yY547ZM+prrrM6O6R2lrPxyslfk7Lf9p6w4CUdpXaX9NXnPJzlbtj9ETnOeyKcBnTQlJm1IA6Y6YtYRw2AWjNQA9aPCBb2o6KIZQltKPyJPcDtfRAhIeQjzqyng/NRRPTnSe7iM3nSlRj1t7Xu0mE9JeLGeXgWandprX1u0ZbRBH7JM2hpjHFcIYwAAerAAAAAAAAAAAAAAAAHPVDmVVOitMyRX1SC83Qro8QDsJ8qsqSaOjo7OU4iHao7jTsWcpxUuYrOrVX7X4B7QYfRCiOss5lRns7rSPX0NdgM83STrkwxl1nUt5/BmC7kxRTKNVOaT6H4hykVl+jmeRTM8zJakoa9JpxZ/cPsntKQpxMmiYiuraMrOmQjzu+w0A9dJargmQ2paUqTkW4jtfBqbE1eFoyWYcZuicuRlo2gWg56pMh+VJMDzok8pcttpKluKSlKPWUajFF3UxRLTNNSTI9Ej9JrE4Q6x2577jznjyJVpto/UY2Wy4w/Lqrpjyts9NGRTGCFqyx470j6yEd3/Wfn/KeS32pUB5lH00d7l/qNnYrG1DzpZqvTX5mlV7tv9RtctDOuFvnY9nOiMvSFf0ecOShxCVoVmSvwqMeXCS+hKK17ObOtPvDTWGlIsx+NSvQ2/8AOWU/Q/I8baetdI7imqLU56mT0hvGeTG2OMkbjOyq0Gu1IX4Ee7+M6e9sh0ZTX1nF9t1fvHTTWZMhtPQ1EWlSvG7Ke7xwy5tvkSGlsyOrVYdTkdSlDvStstztN+G4M3pgQX0P6MiiJdGl59Bv8Y/9JgYOnNLQ/VFOtOyHnUpQn0SGtYycIfJda7UzKahUeTWfTI86teo7p+5zG+/A9WUpRCdTFbQn0SGWdNwDjnrQtqdCVMqhxLTnVqtU9GiM/wCg/nn0jRTSiehKex4PqHEY5W91ZWs13jKOw616NZtl4hQu1dep4Vs/zSxpZrLMl7VebplZ1Zq1J9Jq/i7DJmx48qKlTrq25Dkz2T3pPsTDsTy3nMjSEqydt5SvRtu+wZ+5OwgW9LVc9a6jqvE6oD4f8qt7lWlcV+UtEVL68kZEdWpJ/uyfXLVncZaRFrVmOlHRqqR3jn2JlX3DkG4KZXLitSFQ1Z2aup9G6bcg7bEgwGmU9CKeOnbV7Rwy22Uop0JSlP7KCYCwAACBMgBMAAAAAAIATBAAAAAAAAAAAAAAAAAAAAAAAAAAAAAr6pi3OAiS3pOZtP6itMyq+qYtzk6LSnUoceyeo16RYGJEw/EbWldGqajPaSpSs+Q2LTKUZstEpz9s0bd+lrr2LZJ+9Wy2eyJtxr5obKf2pPSBt8ienNlTm+kect5LTbi1eFKM6zVpXda08EJv9brzh4uNXeqVd9CbV+w84EMywYhiz26ORnUqp0eHp6HKfdG1PgU6HMhXBXWNVWR7OhUTu9frHuD7ThdcpUVus1KUyPoghNtPWJ1IesAvRy2Hka86XIV2tFWi19T9J1JzODq0Q7NZ/K3I1P4/lOmMqeLo6zl/QCZ4yHkoSpdVZUoNdXPp9XMvqz3htKfZR+3/AFnSxfCc3ghmrq5M5X54vuvsjpI/hMaeOTo6nlj8HoDwRKbq4pqiu8QnOpJ7mznTIAAea0UXRSVUzJX6ij4J8pjF7iSWrNbYbr1tuEhqTrIaecoxTW79nW9ifcn7wwhzSqvvE+qe7cxFUqXSuZKC8ZM84PC021qKylhlOVKP+uZpgwrk27XobzV+tk7sUuKKqUimZSkdjwDCSe/BnA0Kb04hCXXkJS2v6Ku8N8JQxIXwlxCZAZyGvqmQIdZbp66f8s80TmaryUdSpX0cw7aM4PcGDb3l1W+hdenKvsfZmcCE8kwAGgAABAmQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABkGQDErbWKv0k1QnWQjJm+EZYyDIkB6xMgAOdvkFxl5M5hOZaEZHmveNGZCxDFep00cSmvroX2HE/rNsa6fY4r/pWW1K+lkMcZ+LoyhL3XlNxFDaT0reb/d7f+Y0S6SLrXo6FR4X8x//AMqG9iYahN16Ux2qftIobWlMpGE5c192qv2vyhBjIZbS03TK2hORJ+x/CTIMeY6dHJq0dxcdak9LKEuOPteFS9P0JCQq7K8CIrafrVqs278Oi3mnulSVM6nZ+mZZtk5ex/o07cef+V2P/CqZdtZfRRWs+l39zTM0EZtIU4tM68lp91VUKVmabV2EdP6aHkwh1x1Tulpt6TiMntHDfAvNnr0rl8KOVcrRVXnlZfZKRptoJpZkVlOZml6aHc6Fpd02/wCr8p0oHe3Ir6XZi1dqgZauZ2k5kL7C/ptGTMhqcqlSXXG0/RSZYIzb9iGGDX/gtFfEpxX7SybdqYp6pmkxnI/TVfxsLqDHukns3FapXpolKVHuDzKS+zBrY0ZaZLjnT3bqG/J9ehngEkK8EwQAWAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABBjzEyCPMBMAAAAAAAAAATIAATAAAAACBMgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAII8xMAAO0O0ABDtE+0AA7Q7QAEO0T7QAEO0T7QAAdoAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAEAJgoPtnYn3CwcNcefG2difcLBw1x58C/AKD7Z2J9wsHDXHnxtnYn3CwcNcefAvwCg+2hifcLBw1x58bZ2J9wsHDXHnwL8AoPtoYn3CwcNcefG2hifcLBw1x58C/AKD7aGJ9wsHDXHnxtoYn3CwcNcefAvwCg+2hifcLBw1x58baGJ9wsHDXHnwL8AoPtoYn3CwcNcefG2hifcLBw1x58C/AKD7aGJ9wsHDXHnxtoYn3CwcNcefAvwCg+2hifcLBw1x58baGJ9wsHDXHnwL8AoPtoYn3CwcNcefG2hifcLBw1x58C/AKD7aGJ9wsHDXHnxtoYn3CwcNcefAvwCg+2hifcLBw1x58baGJ9wsHDXHnwL8AoPtoYn3CwcNcefG2hifcLBw1x58C/AKD7aGJ9wsHDXHnxtoYn3CwcNcefAvwCg+2hifcLBw1x58baGJ9wsHDXHnwL8AoPtoYn3CwcNcefG2hifcLBw1x58C/AKD7aGJ9wsHDXHnxtoYn3CwcNcefAvwCg+2hifcLBw1x58baGJ9wsHDXHnwL8AoPtoYn3CwcNcefG2hifcLBw1x58C/AKD7aGJ9wsHDXHnxtoYn3CwcNcefAvwCg+2hifcLBw1x58baGJ9wsHDXHnwL8AoPtoYn3CwcNcefG2hifcLBw1x58C/AKD7aGJ9wsHDXHnxtoYn3CwcNcefAvwCg+2hifcLBw1x58baGJ9wsHDXHnwL8AoPtoYn3CwcNcefG2hifcLBw1x58C/AKD7Z2J9wsHDXHnxtnYn3CwcNcefAvwCg+2difcLBw1x58bZ2J9wsHDXHnwL8AoPtnYn3CwcNcefG2difcLBw1x58C/AKD7Z2J9wsHDXHnxtnYn3CwcNcefAvqChW2difcLBw1x58bZ2J9wsHDXHnwKzgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/9k=\n", "text/html": [ "\n", " \n", " " ], "text/plain": [ "" ] }, "execution_count": 1, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from IPython.display import YouTubeVideo\n", "YouTubeVideo('bxe2T-V8XRs',width=640,height=360)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We will use the data from the video above:\n", "$$X = \\left[\\begin{matrix} 3 & 5 \\\\ 5 & 1 \\\\ 10 & 2 \\end{matrix}\\right] \\hspace{1cm} , \\hspace{1cm}y = \\left[ \\begin{matrix} 75 \\\\ 82 \\\\ 93 \\end{matrix}\\right] $$\n", "\n", "\n", "### ✅ Step 1: Initialize your inputs\n", "Create two numpy arrays to store the values of the variables $X$ and $y$, as well as their normalized counterparts \n", "$X_{norm}$ and $y_{norm}$." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# your code here (Note include needed libraries):\n", "\n", "\n", "\n", "# input data (hours of sleep, hours of study)\n", "X = \n", "\n", "# normalized X\n", "X_norm = \n", "\n", "\n", "# output data (test score)\n", "y = \n", "\n", "# normalized y\n", "y_norm = " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# 2. Data flow: forward propagation" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Data in a neural network flows via a process called **forward propagation**. Watch the following video:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": true }, "outputs": [], "source": [ "from IPython.display import YouTubeVideo\n", "YouTubeVideo('UJwK6jAStmg',width=640,height=360, align='Center')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Question 1:** How many input layers, hidden layers and output layers are there in the neural network shown in the video?" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Put your answer here\n", "inputLayerSize = \n", "outputLayerSize = \n", "hiddenLayerSize = " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### ✅ Step 2: Initialize random weights\n", "Randomly initialize two numpy arrays W1 and W2, of the right dimensions, to store the weights (zero-one) in the synapses between input layer --> hidden layer, and hidden layer --> output layer " ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# your code here:\n", "\n", "W1 = \n", "\n", "\n", "W2 =" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### ✅ Step 3: Multiply the normalized input matrix by $W^{(1)}$\n", "$$Z^{(2)} = X W^{(1)} $$ \n", "Here is the code using the NumPy `dot` function. If you get an error you may have initilized the size of your variables incorrectly. Make sure the second dimension of ```X_norm``` matches the first dimension of ```W1```:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "Z2 = np.dot(X_norm, W1)\n", "Z2" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### ✅ **Do This:** Implement and test the sigmoid function \n", "\n", "$$a(z) = \\frac{1}{1 + e^{-z}} $$ \n", "\n", "The implemented sigmoid function should take as input a numpy array and return a numpy array of the same dimension, with the function $f$ applied to each entry." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# your code here:\n", "def sigmoid(z):\n", " # apply sigmoid activation function\n", " \n", " return " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Test your sigmoid function using the following testing code:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "testInput = np.arange(-6,6,0.01)\n", "plt.plot(testInput, sigmoid(testInput), linewidth= 2)\n", "plt.grid(1)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### ✅ Step 4: Apply the sigmoid function to $Z^{(2)}$\n", "$$a^{(2)} = f({Z^{(2)}})$$ \n", "Here is the code to apply the sigmoid function to $Z^{(2)}$ and display the results" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "a2 = sigmoid(Z2)\n", "a2" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### ✅ Step 5: multiply $A^{(2)}$ by $W^{(2)}$ to get $Z^{(3)}$\n", "$$Z^{(3)} = A^{(2)} W^{(2)} $$ " ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "Z3 = np.dot(a2, W2)\n", "Z3" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### ✅ Step 6: Apply the sigmoid function again to $Z^{(3)}$ to produce $\\hat{y}$\n", "$$\\hat{y} = f({Z^{(3)}})$$ " ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# your code here:\n", "yHat = " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Final Comparison\n", "Now compare the estimation output ($\\hat{y}$) to the actual output ```y_norm```. " ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "y_norm" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "yHat" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Of course the results from forward propagation are terrible; no surprises here! The weights have not been properly chosen. That's what training a network does: the goal is to find a combination of weights so that the result of forward propagation fits the intended output data as best as possible. \n", "\n", "We will be covering this topic in class." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# 3. Exploring A Neural Network\n", "\n", "Please go to the following website : http://playground.tensorflow.org/\n", "\n", "There, you'll have the opportunity to play with an actual neural network (e.g., choosing its architecture and the type of activation function) for classification purpose. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "----\n", "# Assignment wrap-up\n", "\n", "Please fill out the form that appears when you run the code below. **You must completely fill this out in order to receive credit for the assignment!**" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from IPython.display import HTML\n", "HTML(\n", "\"\"\"\n", "\n", "\"\"\"\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---------\n", "### Congratulations, you're done with your pre-class assignment!\n", "\n", "Now, you just need to submit this assignment by uploading it to the course Desire2Learn web page for today's submission folder (Don't forget to add your name in the first cell)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "© Copyright Michigan State University Board of Trustees" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.6" } }, "nbformat": 4, "nbformat_minor": 4 }