-
Notifications
You must be signed in to change notification settings - Fork 11
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
92 changed files
with
107 additions
and
120 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,4 +1,4 @@ | ||
# Sphinx build info version 1 | ||
# This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done. | ||
config: aaf901aee1afcd16a615dedd3e634e41 | ||
config: 65e8070b5576a2323dd614509064877e | ||
tags: 645f666f9bcd5a90fca523b33c5a78b7 |
Binary file renamed
BIN
+74.6 KB
...e94298c7ac454f018589c5d93860443d26859.png → ...99f2e4625133cd4d689be8cd932863dc01760.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file removed
BIN
-611 KB
_images/041b29cda9b72a2907fe21cd4f2207ce4de51e12d52a426b44128921f1b32246.png
Binary file not shown.
Binary file renamed
BIN
+765 KB
...295fcd89f4c76f6d685f3be500ca593c03505.png → ...1368f19e74afa90e5708d12e72134919ed05b.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+714 KB
_images/0df4d8edccc07cb273e547e7e22a41a610e41409dc683ff936a4f38c8a5154a0.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file renamed
BIN
+145 KB
...a58eb8d94bfd11bb397ef9eaf3c87e1479d5a.png → ...00355e7674135d3b3d33e07417b9db32b54eb.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file renamed
BIN
+643 KB
...ca4a4e90f2fe1f5dc4a4b316fbaaf72deb5a4.png → ...3dd51366f0d383ef19cea6fa2e02df661466a.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+599 KB
_images/16c0988f8dcbdf684083d04c0aad6044a915dca8233ccd0d4ce77809dfe74dcc.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file renamed
BIN
+124 KB
...4e7020fdf672242e900266a08d86ae0db3d9d.png → ...d68826b912b3a258ca7053ac33b34bb5dd302.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file renamed
BIN
+72.4 KB
...b7df070138fa29eee2689d72878b436f76e01.png → ...ab90e61ccdd4e79469d10af84ce15d6812eb5.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file renamed
BIN
+609 KB
...56214b62a22ea5d5be2ae9fff8d96d5f8e4ec.png → ...a0032c60e8d67350560413dd778170899771c.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file renamed
BIN
+60.7 KB
...2eba1370a0405ef498658c314229c26d9bcca.png → ...5730e5e549a9def917e5986fb0bc7705bce17.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+107 KB
_images/34b824595ea1185a2aa56c707e4f748b738aa1e63ecd78e7ebe2713f9d601919.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+292 KB
_images/3a1d404a0ae5c5da1d7813b7bee0579d1da1dfde237b1ced76eba752d3e13329.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+232 KB
_images/3eff8a4b322a194cc508504c70be7e83e92f73bd5d83a6d5fd6818104cf1c6c0.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file renamed
BIN
+549 KB
...37cfdd580ecdcd6bcbbab4c32be6205d91631.png → ...3ee85b0ecf2fc0f28c40f4dbc1b5b1d4ec199.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file removed
BIN
-59.9 KB
_images/52133f6c14b96f46a6131e6634f2191978274952e44cdd0685472719283dda8e.png
Binary file not shown.
Binary file renamed
BIN
+95.3 KB
...59a3e2efd20441b76ee54e3837ad7017ca1ab.png → ...c03d6921e745adbb4aab1f9b20438c2d662c9.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file renamed
BIN
+141 KB
...5e4425900f3b54f90d651651f455b61b9c7d0.png → ...6da0f62ba1ce332f54200f83abe35bbdf7a80.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file removed
BIN
-221 KB
_images/7817aacfa469a5f75abd2a57c7c166a78d3d8094e56e4f4f9aeda12165ccdae6.png
Binary file not shown.
Binary file renamed
BIN
+55.7 KB
...62393a50dfa5e4209e497c49514c8e73f2911.png → ...c4cbcccad81550ae5e1eeff4d1a12a98ffd9f.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+1.39 MB
_images/80e0c945417f0a8e8b7fb4760a4944c03a37f926d53e07c9e4db4d7d21689411.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file removed
BIN
-1.32 MB
_images/8f3ffcffd63694125267c4f3f6494ae934b5dc14a4a8136874793feeb40a3b8f.png
Binary file not shown.
Binary file renamed
BIN
+416 KB
...abc39067e127e3331d90ff0ab904a6348c334.png → ...a323b962ed1e112707b71a69ce081cb8451ec.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file removed
BIN
-45.7 KB
_images/95a4ab486788b3b6d95242c6ca59ec4003415bccd1316a34a9e803364a863e40.png
Binary file not shown.
Binary file renamed
BIN
+160 KB
...4bfd2dbb941eb1795c1666ed48a7b2b245120.png → ...b366cbce0e9d398ef0b480812cc63aaf9af5a.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file renamed
BIN
+82.9 KB
...9c5f975b660c6935e84d2412d2cf2876047a6.png → ...18cb3b256a9a5953827cd1499dc218ded1582.png
Oops, something went wrong.
Binary file renamed
BIN
+215 KB
...aa7a09e27e0956a88f223fa68eaf964526ce4.png → ...a8d5dc729915759c6e58771b095397edd44f7.png
Oops, something went wrong.
Binary file added
BIN
+1.39 MB
_images/a842fe3bb8275ad72c2a19c55ee300f08a96539a6b8effa84e88d29041462ca3.png
Oops, something went wrong.
Binary file renamed
BIN
+80.1 KB
...826c860b51a2438ffbd10054023085b3a891c.png → ...413ad0ca6daed4747c16b396cdcb168946640.png
Oops, something went wrong.
Binary file renamed
BIN
+928 KB
...7cb27d04eb661050e7107f000371c23b1842b.png → ...be3b26a9dab65bea9bbeb43391faa466175f6.png
Oops, something went wrong.
Binary file removed
BIN
-291 KB
_images/b9a7df4fec596a3628e39371013dfafe9f57eef598f7db7438f979e74c2b8552.png
Diff not rendered.
Binary file renamed
BIN
+52.9 KB
...ff33f7fb55071b0c01edc00a02374c362bf4e.png → ...06e1d86e9d2e98a8c1bcfdd55f7c32c1dce01.png
Oops, something went wrong.
Binary file removed
BIN
-1.33 MB
_images/c7c38d411191ee89743265c59d35bff5520ee39f844484b8e46bb5197a11518e.png
Diff not rendered.
Binary file renamed
BIN
+314 KB
...2fabe5fda67baf488137490c4d290cbcb1b5b.png → ...b3c657f97907ead343ecf7f148be404aaac3b.png
Oops, something went wrong.
Binary file removed
BIN
-589 KB
_images/ca752a386abc4c0c59a03dbbb2cda8573b71837f577aedad254ee5a607878eb1.png
Diff not rendered.
Binary file added
BIN
+621 KB
_images/ce15b2079d4d30a7331e4f6c5aef7c9e4e729d1a10c379105092cd91eb601a35.png
Oops, something went wrong.
Binary file renamed
BIN
+97.6 KB
...7c1d444d54bfd0f87975524b17d6cf4eb8783.png → ...5a99d362145d85f1265d7c2dbaa7c61560579.png
Oops, something went wrong.
Binary file removed
BIN
-107 KB
_images/dab685006e4b28b14500b2e138fe41f38bc6e6c273e6ea47b1bc998949a75e47.png
Diff not rendered.
Binary file renamed
BIN
+53.7 KB
...54f1dc86120410dbd92c94b0e9ede66b4f6b2.png → ...4a2e293ee28089134c9e7efc2aada7e659f76.png
Oops, something went wrong.
Binary file added
BIN
+45.8 KB
_images/e275fa1d53ccadc9fb754ad543d7ad730b254ff64f54b0a4b56371509eea5b6e.png
Oops, something went wrong.
Binary file removed
BIN
-654 KB
_images/e2c5069067026e4c39edc7151ce328fc5a07553cf1b5ac60d97c16e16e385964.png
Diff not rendered.
Binary file renamed
BIN
+238 KB
...63ee526d73082c14ca6629d5d87b8fd043543.png → ...bf95fc6432a0908ea038266a05708c6f30088.png
Oops, something went wrong.
Binary file renamed
BIN
+305 KB
...dd0a3e788bfb13302c454350db1f8aee84c93.jpg → ...99d704be2c8895c2fc1ddd7f6cbddc24394d3.jpg
Oops, something went wrong.
Binary file renamed
BIN
+415 KB
...15300ab749b2856ac7e2b68c286fcd99bb00e.png → ...d8b3d96157676ce49b9452d11a5480f78def5.png
Oops, something went wrong.
Binary file added
BIN
+106 KB
_images/edaaa8a8995da4dce60881ff88a6cb193bdc232c63b316c5222c4e37e07dc449.png
Oops, something went wrong.
Binary file renamed
BIN
+277 KB
...32351653563f830df28ac01801a26ee676bea.png → ...2b2c5cc6f05acfba0fe39e2f52ff74282aa13.png
Oops, something went wrong.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -38,8 +38,6 @@ | |
<script src="https://buttons.github.io/buttons.js"></script> | ||
<script>window.MathJax = {"options": {"processHtmlClass": "tex2jax_process|mathjax_process|math|output_area"}}</script> | ||
<script defer="defer" src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js"></script> | ||
<script src="https://unpkg.com/[email protected]/dist/mermaid.min.js"></script> | ||
<script>mermaid.initialize({startOnLoad:true});</script> | ||
<link rel="shortcut icon" href="_static/favicon.ico"/> | ||
<link rel="index" title="Index" href="genindex.html" /> | ||
<link rel="search" title="Search" href="search.html" /> | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -38,8 +38,6 @@ | |
<script src="https://buttons.github.io/buttons.js"></script> | ||
<script>window.MathJax = {"options": {"processHtmlClass": "tex2jax_process|mathjax_process|math|output_area"}}</script> | ||
<script defer="defer" src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js"></script> | ||
<script src="https://unpkg.com/[email protected]/dist/mermaid.min.js"></script> | ||
<script>mermaid.initialize({startOnLoad:true});</script> | ||
<link rel="shortcut icon" href="../_static/favicon.ico"/> | ||
<link rel="index" title="Index" href="../genindex.html" /> | ||
<link rel="search" title="Search" href="../search.html" /> | ||
|
@@ -585,7 +583,7 @@ <h2>Optimizing a Function with Gradient Descent<a class="headerlink" href="#opti | |
</div> | ||
</div> | ||
<div class="cell_output docutils container"> | ||
<img alt="../_images/aa4014aa819c4fd1240e4a412fa9c5f975b660c6935e84d2412d2cf2876047a6.png" src="../_images/aa4014aa819c4fd1240e4a412fa9c5f975b660c6935e84d2412d2cf2876047a6.png" /> | ||
<img alt="../_images/9e6515d7281d0636faf745b79b518cb3b256a9a5953827cd1499dc218ded1582.png" src="../_images/9e6515d7281d0636faf745b79b518cb3b256a9a5953827cd1499dc218ded1582.png" /> | ||
</div> | ||
</div> | ||
<p>We see we need to go to the right to go down toward the minimum. For a multivariate function, the gradient will be a vector pointing in the direction of the steepest downward slope. When we take steps, we find the x coordinate of our new location by:</p> | ||
|
@@ -649,7 +647,7 @@ <h2>Optimizing a Function with Gradient Descent<a class="headerlink" href="#opti | |
</div> | ||
</div> | ||
<div class="cell_output docutils container"> | ||
<img alt="../_images/ade4bc80dd522959106b566d6fde94298c7ac454f018589c5d93860443d26859.png" src="../_images/ade4bc80dd522959106b566d6fde94298c7ac454f018589c5d93860443d26859.png" /> | ||
<img alt="../_images/034e030958c936689acf508e6dd99f2e4625133cd4d689be8cd932863dc01760.png" src="../_images/034e030958c936689acf508e6dd99f2e4625133cd4d689be8cd932863dc01760.png" /> | ||
</div> | ||
</div> | ||
<p>The gradient at our new point (shown in orange) is still not close to zero, meaning we haven’t reached the minimum. We’ll continue this process of checking if the gradient is nearly zero, and take a step in the direction of steepest descent until we reach the bottom of the valley. We’ll say we’ve reached the bottom of the valley when the absolute value of the gradient is <span class="math notranslate nohighlight">\(<0.1\)</span>:</p> | ||
|
@@ -696,7 +694,7 @@ <h2>Optimizing a Function with Gradient Descent<a class="headerlink" href="#opti | |
</div> | ||
</div> | ||
<div class="cell_output docutils container"> | ||
<img alt="../_images/b555dc26be48899b117e71e690cb7df070138fa29eee2689d72878b436f76e01.png" src="../_images/b555dc26be48899b117e71e690cb7df070138fa29eee2689d72878b436f76e01.png" /> | ||
<img alt="../_images/1d8abadb937d7eafab8c2f0405fab90e61ccdd4e79469d10af84ce15d6812eb5.png" src="../_images/1d8abadb937d7eafab8c2f0405fab90e61ccdd4e79469d10af84ce15d6812eb5.png" /> | ||
</div> | ||
</div> | ||
<p>This works, but it takes a long time since we have several small steps.</p> | ||
|
@@ -749,7 +747,7 @@ <h2>Optimizing a Function with Gradient Descent<a class="headerlink" href="#opti | |
</div> | ||
</div> | ||
<div class="cell_output docutils container"> | ||
<img alt="../_images/03507cfedf12a44d611a8d154f2826c860b51a2438ffbd10054023085b3a891c.png" src="../_images/03507cfedf12a44d611a8d154f2826c860b51a2438ffbd10054023085b3a891c.png" /> | ||
<img alt="../_images/ac4b397a7095065b3dc48ea2d72413ad0ca6daed4747c16b396cdcb168946640.png" src="../_images/ac4b397a7095065b3dc48ea2d72413ad0ca6daed4747c16b396cdcb168946640.png" /> | ||
</div> | ||
</div> | ||
<p><em>Note the change in scale!</em> With only one step, we already see that we stepped <em>right over</em> the minimum to somewhere far up the other side of the valley (orange point)! This is not good. If we kept iterating with the same learning rate, we’d find that the optimization process diverges and the step sizes start blowing up. This is why it is important to pick the proper step size by setting the learning rate appropriately. Steps that are too small take a long time while steps that are too large render the optimization process invalid. In this case, a reasonable choice appears to be <code class="docutils literal notranslate"><span class="pre">step</span> <span class="pre">size</span> <span class="pre">=</span> <span class="pre">0.6</span></code>, which would have reached pretty close to the minimum after only 3 steps.</p> | ||
|
Oops, something went wrong.