diff --git a/_downloads/3195443a0ced3cabc0ad643537bdb5cd/introyt1_tutorial.ipynb b/_downloads/3195443a0ced3cabc0ad643537bdb5cd/introyt1_tutorial.ipynb index 7c7c307175..c23d5542c0 100644 --- a/_downloads/3195443a0ced3cabc0ad643537bdb5cd/introyt1_tutorial.ipynb +++ b/_downloads/3195443a0ced3cabc0ad643537bdb5cd/introyt1_tutorial.ipynb @@ -34,7 +34,7 @@ { "cell_type": "code", "execution_count": null, - "id": "7990a69c", + "id": "de9d7e17", "metadata": {}, "outputs": [], "source": [ @@ -50,7 +50,7 @@ }, { "cell_type": "markdown", - "id": "cd424fde", + "id": "38547d24", "metadata": {}, "source": [ "\n", diff --git a/_downloads/4355e2cef7d17548f1e25f97a62828c4/template_tutorial.ipynb b/_downloads/4355e2cef7d17548f1e25f97a62828c4/template_tutorial.ipynb index d5dd2551ee..4aaffb7973 100644 --- a/_downloads/4355e2cef7d17548f1e25f97a62828c4/template_tutorial.ipynb +++ b/_downloads/4355e2cef7d17548f1e25f97a62828c4/template_tutorial.ipynb @@ -31,7 +31,7 @@ { "cell_type": "code", "execution_count": null, - "id": "41f06fa1", + "id": "6eaa09f5", "metadata": {}, "outputs": [], "source": [ @@ -47,7 +47,7 @@ }, { "cell_type": "markdown", - "id": "a8f13a31", + "id": "70db1f88", "metadata": {}, "source": [ "\n", diff --git a/_downloads/63a0f0fc7b3ffb15d3a5ac8db3d521ee/tensors_deeper_tutorial.ipynb b/_downloads/63a0f0fc7b3ffb15d3a5ac8db3d521ee/tensors_deeper_tutorial.ipynb index 821328bcb4..4f6ed15bad 100644 --- a/_downloads/63a0f0fc7b3ffb15d3a5ac8db3d521ee/tensors_deeper_tutorial.ipynb +++ b/_downloads/63a0f0fc7b3ffb15d3a5ac8db3d521ee/tensors_deeper_tutorial.ipynb @@ -34,7 +34,7 @@ { "cell_type": "code", "execution_count": null, - "id": "e9a0317a", + "id": "8cf9d4a9", "metadata": {}, "outputs": [], "source": [ @@ -50,7 +50,7 @@ }, { "cell_type": "markdown", - "id": "cb107f47", + "id": "32d07b59", "metadata": {}, "source": [ "\n", diff --git a/_downloads/770632dd3941d2a51b831c52ded57aa2/trainingyt.ipynb b/_downloads/770632dd3941d2a51b831c52ded57aa2/trainingyt.ipynb index 58fe4ee2ce..a57b4c1acd 100644 --- a/_downloads/770632dd3941d2a51b831c52ded57aa2/trainingyt.ipynb +++ b/_downloads/770632dd3941d2a51b831c52ded57aa2/trainingyt.ipynb @@ -35,7 +35,7 @@ { "cell_type": "code", "execution_count": null, - "id": "9e2a7722", + "id": "b0907d17", "metadata": {}, "outputs": [], "source": [ @@ -51,7 +51,7 @@ }, { "cell_type": "markdown", - "id": "a175d8e7", + "id": "e734a4d7", "metadata": {}, "source": [ "\n", diff --git a/_downloads/c28f42852d456daf9af72da6c6909556/captumyt.ipynb b/_downloads/c28f42852d456daf9af72da6c6909556/captumyt.ipynb index 16d9254cf1..941213c0b0 100644 --- a/_downloads/c28f42852d456daf9af72da6c6909556/captumyt.ipynb +++ b/_downloads/c28f42852d456daf9af72da6c6909556/captumyt.ipynb @@ -37,7 +37,7 @@ { "cell_type": "code", "execution_count": null, - "id": "cea1749f", + "id": "594c72cf", "metadata": {}, "outputs": [], "source": [ @@ -53,7 +53,7 @@ }, { "cell_type": "markdown", - "id": "9b221904", + "id": "475c2a63", "metadata": {}, "source": [ "\n", diff --git a/_downloads/e2e556f6b4693c2cef716dd7f40caaf6/tensorboardyt_tutorial.ipynb b/_downloads/e2e556f6b4693c2cef716dd7f40caaf6/tensorboardyt_tutorial.ipynb index 98c5fa212b..96207f51b9 100644 --- a/_downloads/e2e556f6b4693c2cef716dd7f40caaf6/tensorboardyt_tutorial.ipynb +++ b/_downloads/e2e556f6b4693c2cef716dd7f40caaf6/tensorboardyt_tutorial.ipynb @@ -35,7 +35,7 @@ { "cell_type": "code", "execution_count": null, - "id": "34ea9b0f", + "id": "8c63d153", "metadata": {}, "outputs": [], "source": [ @@ -51,7 +51,7 @@ }, { "cell_type": "markdown", - "id": "a5bc0df0", + "id": "6dd50681", "metadata": {}, "source": [ "\n", diff --git a/_downloads/ed9d4f94afb79f7dada6742a06c486a5/autogradyt_tutorial.ipynb b/_downloads/ed9d4f94afb79f7dada6742a06c486a5/autogradyt_tutorial.ipynb index 9df7e529f6..6fb3baf08b 100644 --- a/_downloads/ed9d4f94afb79f7dada6742a06c486a5/autogradyt_tutorial.ipynb +++ b/_downloads/ed9d4f94afb79f7dada6742a06c486a5/autogradyt_tutorial.ipynb @@ -34,7 +34,7 @@ { "cell_type": "code", "execution_count": null, - "id": "99f4bac3", + "id": "cd6a315d", "metadata": {}, "outputs": [], "source": [ @@ -50,7 +50,7 @@ }, { "cell_type": "markdown", - "id": "17dc5c5a", + "id": "9a958047", "metadata": {}, "source": [ "\n", diff --git a/_downloads/fe726e041160526cf828806536922cf6/modelsyt_tutorial.ipynb b/_downloads/fe726e041160526cf828806536922cf6/modelsyt_tutorial.ipynb index b188778e77..e508e626f9 100644 --- a/_downloads/fe726e041160526cf828806536922cf6/modelsyt_tutorial.ipynb +++ b/_downloads/fe726e041160526cf828806536922cf6/modelsyt_tutorial.ipynb @@ -34,7 +34,7 @@ { "cell_type": "code", "execution_count": null, - "id": "bb07cc8e", + "id": "26b54310", "metadata": {}, "outputs": [], "source": [ @@ -50,7 +50,7 @@ }, { "cell_type": "markdown", - "id": "d4ef5611", + "id": "34de35c5", "metadata": {}, "source": [ "\n", diff --git a/_images/sphx_glr_coding_ddpg_001.png b/_images/sphx_glr_coding_ddpg_001.png index cf9dff5ac7..fb8cc02ff8 100644 Binary files a/_images/sphx_glr_coding_ddpg_001.png and b/_images/sphx_glr_coding_ddpg_001.png differ diff --git a/_images/sphx_glr_dqn_with_rnn_tutorial_001.png b/_images/sphx_glr_dqn_with_rnn_tutorial_001.png index 5292f46462..f1abe12cf0 100644 Binary files a/_images/sphx_glr_dqn_with_rnn_tutorial_001.png and b/_images/sphx_glr_dqn_with_rnn_tutorial_001.png differ diff --git a/_images/sphx_glr_neural_style_tutorial_004.png b/_images/sphx_glr_neural_style_tutorial_004.png index 3ca654ec86..2488344654 100644 Binary files a/_images/sphx_glr_neural_style_tutorial_004.png and b/_images/sphx_glr_neural_style_tutorial_004.png differ diff --git a/_images/sphx_glr_reinforcement_ppo_001.png b/_images/sphx_glr_reinforcement_ppo_001.png index ea93ac4941..d298981958 100644 Binary files a/_images/sphx_glr_reinforcement_ppo_001.png and b/_images/sphx_glr_reinforcement_ppo_001.png differ diff --git a/_images/sphx_glr_reinforcement_q_learning_001.png b/_images/sphx_glr_reinforcement_q_learning_001.png index 8a2ef37b62..5895ed0a44 100644 Binary files a/_images/sphx_glr_reinforcement_q_learning_001.png and b/_images/sphx_glr_reinforcement_q_learning_001.png differ diff --git a/_images/sphx_glr_spatial_transformer_tutorial_001.png b/_images/sphx_glr_spatial_transformer_tutorial_001.png index dbc64859b7..06ba3b7f02 100644 Binary files a/_images/sphx_glr_spatial_transformer_tutorial_001.png and b/_images/sphx_glr_spatial_transformer_tutorial_001.png differ diff --git a/_images/sphx_glr_torchvision_tutorial_002.png b/_images/sphx_glr_torchvision_tutorial_002.png index e531567d7f..6ec4d824b5 100644 Binary files a/_images/sphx_glr_torchvision_tutorial_002.png and b/_images/sphx_glr_torchvision_tutorial_002.png differ diff --git a/_sources/advanced/coding_ddpg.rst.txt b/_sources/advanced/coding_ddpg.rst.txt index d7db549d18..61be16b148 100644 --- a/_sources/advanced/coding_ddpg.rst.txt +++ b/_sources/advanced/coding_ddpg.rst.txt @@ -1632,26 +1632,26 @@ modules we need. 0%| | 0/10000 [00:00 + @@ -513,199 +513,167 @@ up our dataset. Downloading tokenizer_config.json: 0%| | 0.00/49.0 [00:00 @@ -1530,49 +1498,49 @@ zero-shot pruning, or pruning without fine-tuning / retraining. 0%| | 0/43 [00:00 + @@ -806,7 +806,7 @@ https://colab.research.google.com/drive/1HiICg6jRkBnr5hvK2-VnMi88Vi9pUzEJ .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 0 minutes 0.219 seconds) + **Total running time of the script:** ( 0 minutes 0.224 seconds) .. _sphx_glr_download_beginner_Intro_to_TorchScript_tutorial.py: diff --git a/_sources/beginner/basics/autogradqs_tutorial.rst.txt b/_sources/beginner/basics/autogradqs_tutorial.rst.txt index b7d4dab5a0..28bb181e62 100644 --- a/_sources/beginner/basics/autogradqs_tutorial.rst.txt +++ b/_sources/beginner/basics/autogradqs_tutorial.rst.txt @@ -113,8 +113,8 @@ documentation `__. .. code-block:: none - Gradient function for z = - Gradient function for loss = + Gradient function for z = + Gradient function for loss = diff --git a/_sources/beginner/basics/buildmodel_tutorial.rst.txt b/_sources/beginner/basics/buildmodel_tutorial.rst.txt index 2f0fef9a00..56e8c50d53 100644 --- a/_sources/beginner/basics/buildmodel_tutorial.rst.txt +++ b/_sources/beginner/basics/buildmodel_tutorial.rst.txt @@ -482,7 +482,7 @@ Further Reading .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 0 minutes 0.322 seconds) + **Total running time of the script:** ( 0 minutes 0.309 seconds) .. _sphx_glr_download_beginner_basics_buildmodel_tutorial.py: diff --git a/_sources/beginner/basics/data_tutorial.rst.txt b/_sources/beginner/basics/data_tutorial.rst.txt index ec9fc72954..8d308dc647 100644 --- a/_sources/beginner/basics/data_tutorial.rst.txt +++ b/_sources/beginner/basics/data_tutorial.rst.txt @@ -103,43 +103,62 @@ We load the `FashionMNIST Dataset `_. Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/train-images-idx3-ubyte.gz to data/FashionMNIST/raw/train-images-idx3-ubyte.gz 0%| | 0/26421880 [00:00`_. .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 1 minutes 3.247 seconds) + **Total running time of the script:** ( 1 minutes 6.083 seconds) .. _sphx_glr_download_beginner_basics_quickstart_tutorial.py: diff --git a/_sources/beginner/basics/saveloadrun_tutorial.rst.txt b/_sources/beginner/basics/saveloadrun_tutorial.rst.txt index b17bb4646b..66f7a01ccc 100644 --- a/_sources/beginner/basics/saveloadrun_tutorial.rst.txt +++ b/_sources/beginner/basics/saveloadrun_tutorial.rst.txt @@ -76,38 +76,37 @@ method: Downloading: "https://download.pytorch.org/models/vgg16-397923af.pth" to /var/lib/ci-user/.cache/torch/hub/checkpoints/vgg16-397923af.pth 0%| | 0.00/528M [00:00 - - + + + @@ -436,7 +436,7 @@ implements all these methods. Using it is very simple: .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 0 minutes 0.062 seconds) + **Total running time of the script:** ( 0 minutes 0.068 seconds) .. _sphx_glr_download_beginner_blitz_neural_networks_tutorial.py: diff --git a/_sources/beginner/blitz/tensor_tutorial.rst.txt b/_sources/beginner/blitz/tensor_tutorial.rst.txt index 49ba66e34b..91709a6c9d 100644 --- a/_sources/beginner/blitz/tensor_tutorial.rst.txt +++ b/_sources/beginner/blitz/tensor_tutorial.rst.txt @@ -555,7 +555,7 @@ Changes in the NumPy array reflects in the tensor. .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 0 minutes 0.036 seconds) + **Total running time of the script:** ( 0 minutes 0.035 seconds) .. _sphx_glr_download_beginner_blitz_tensor_tutorial.py: diff --git a/_sources/beginner/chatbot_tutorial.rst.txt b/_sources/beginner/chatbot_tutorial.rst.txt index b928539f34..1803f2a7ca 100644 --- a/_sources/beginner/chatbot_tutorial.rst.txt +++ b/_sources/beginner/chatbot_tutorial.rst.txt @@ -5767,7 +5767,7 @@ in PyTorch! .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 5 minutes 51.748 seconds) + **Total running time of the script:** ( 5 minutes 53.106 seconds) .. _sphx_glr_download_beginner_chatbot_tutorial.py: diff --git a/_sources/beginner/data_loading_tutorial.rst.txt b/_sources/beginner/data_loading_tutorial.rst.txt index 7968d2cf65..d4a8cba249 100644 --- a/_sources/beginner/data_loading_tutorial.rst.txt +++ b/_sources/beginner/data_loading_tutorial.rst.txt @@ -63,7 +63,7 @@ installed: .. code-block:: none - + @@ -661,7 +661,7 @@ For an example with training code, please see .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 0 minutes 2.734 seconds) + **Total running time of the script:** ( 0 minutes 2.829 seconds) .. _sphx_glr_download_beginner_data_loading_tutorial.py: diff --git a/_sources/beginner/dcgan_faces_tutorial.rst.txt b/_sources/beginner/dcgan_faces_tutorial.rst.txt index 8e760aa809..0c9b8549a8 100644 --- a/_sources/beginner/dcgan_faces_tutorial.rst.txt +++ b/_sources/beginner/dcgan_faces_tutorial.rst.txt @@ -1284,42 +1284,42 @@ animation.
- +
- + oninput="anim50f3b675e3ef4e27b2eaabe76970a12c.set_frame(parseInt(this.value));">
- - - - - - - - -
-
- - - Once + - - Loop + - +
@@ -1329,9 +1329,9 @@ animation. /* Instantiate the Animation class. */ /* The IDs given should match those used in the template above. */ (function() { - var img_id = "_anim_img5c4a5c884f6a4b178d66187683458b26"; - var slider_id = "_anim_slider5c4a5c884f6a4b178d66187683458b26"; - var loop_select_id = "_anim_loop_select5c4a5c884f6a4b178d66187683458b26"; + var img_id = "_anim_img50f3b675e3ef4e27b2eaabe76970a12c"; + var slider_id = "_anim_slider50f3b675e3ef4e27b2eaabe76970a12c"; + var loop_select_id = "_anim_loop_select50f3b675e3ef4e27b2eaabe76970a12c"; var frames = new Array(17); frames[0] = "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAyAAAAMgCAYAAADbcAZoAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90\ @@ -233662,7 +233662,7 @@ animation. /* set a timeout to make sure all the above elements are created before the object is initialized. */ setTimeout(function() { - anim5c4a5c884f6a4b178d66187683458b26 = new Animation(frames, img_id, slider_id, 1000.0, + anim50f3b675e3ef4e27b2eaabe76970a12c = new Animation(frames, img_id, slider_id, 1000.0, loop_select_id); }, 0); })() @@ -233735,7 +233735,7 @@ could go from here. You could: .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 6 minutes 57.402 seconds) + **Total running time of the script:** ( 7 minutes 1.022 seconds) .. _sphx_glr_download_beginner_dcgan_faces_tutorial.py: diff --git a/_sources/beginner/deploy_seq2seq_hybrid_frontend_tutorial.rst.txt b/_sources/beginner/deploy_seq2seq_hybrid_frontend_tutorial.rst.txt index 9d1500e70d..0dfb6c9db5 100644 --- a/_sources/beginner/deploy_seq2seq_hybrid_frontend_tutorial.rst.txt +++ b/_sources/beginner/deploy_seq2seq_hybrid_frontend_tutorial.rst.txt @@ -1135,7 +1135,7 @@ of torch.save(model, PATH). .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 0 minutes 0.733 seconds) + **Total running time of the script:** ( 0 minutes 0.725 seconds) .. _sphx_glr_download_beginner_deploy_seq2seq_hybrid_frontend_tutorial.py: diff --git a/_sources/beginner/examples_nn/polynomial_nn.rst.txt b/_sources/beginner/examples_nn/polynomial_nn.rst.txt index 001b137c4b..b7404e8404 100644 --- a/_sources/beginner/examples_nn/polynomial_nn.rst.txt +++ b/_sources/beginner/examples_nn/polynomial_nn.rst.txt @@ -144,7 +144,7 @@ input and may have some trainable weights. .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 0 minutes 0.849 seconds) + **Total running time of the script:** ( 0 minutes 0.883 seconds) .. _sphx_glr_download_beginner_examples_nn_polynomial_nn.py: diff --git a/_sources/beginner/examples_tensor/polynomial_numpy.rst.txt b/_sources/beginner/examples_tensor/polynomial_numpy.rst.txt index 1e19f8f9b8..74951431f8 100644 --- a/_sources/beginner/examples_tensor/polynomial_numpy.rst.txt +++ b/_sources/beginner/examples_tensor/polynomial_numpy.rst.txt @@ -113,7 +113,7 @@ generic numeric computations. .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 0 minutes 0.353 seconds) + **Total running time of the script:** ( 0 minutes 0.350 seconds) .. _sphx_glr_download_beginner_examples_tensor_polynomial_numpy.py: diff --git a/_sources/beginner/fgsm_tutorial.rst.txt b/_sources/beginner/fgsm_tutorial.rst.txt index c3eee78706..82c749822c 100644 --- a/_sources/beginner/fgsm_tutorial.rst.txt +++ b/_sources/beginner/fgsm_tutorial.rst.txt @@ -180,7 +180,7 @@ follows: .. code-block:: none - + @@ -267,7 +267,7 @@ pretrained weights. Downloading https://ossci-datasets.s3.amazonaws.com/mnist/train-images-idx3-ubyte.gz to ../data/MNIST/raw/train-images-idx3-ubyte.gz 0%| | 0/9912422 [00:00\n', 'nokia\n', 'ec\n', 'virgin\n', '2011\n'] Downloading builder script: 0%| | 0.00/5.02k [00:00] + [] @@ -363,17 +363,17 @@ to the function with no history of its own. .. code-block:: none d: - - ((, 0), (None, 0)) - ((, 0), (None, 0)) - ((, 0),) + + ((, 0), (None, 0)) + ((, 0), (None, 0)) + ((, 0),) () c: - + b: - + a: None @@ -417,7 +417,7 @@ call the ``backward()`` method on the output, and check the input’s -1.4142e+00, -1.0000e+00, -5.1764e-01, 2.3850e-08, 5.1764e-01, 1.0000e+00, 1.4142e+00, 1.7321e+00, 1.9319e+00, 2.0000e+00]) - [] + [] @@ -928,17 +928,17 @@ example usage: ------------------------------------ ------------ ------------ ------------ ------------ ------------ ------------ ------------ ------------ ------------ ------------ Name Self CPU % Self CPU CPU total % CPU total CPU time avg Self CUDA Self CUDA % CUDA total CUDA time avg # of Calls ------------------------------------ ------------ ------------ ------------ ------------ ------------ ------------ ------------ ------------ ------------ ------------ - cudaEventRecord 45.07% 9.215ms 45.07% 9.215ms 2.304us 0.000us 0.00% 0.000us 0.000us 4000 - aten::div 27.27% 5.575ms 27.27% 5.575ms 5.575us 16.286ms 50.14% 16.286ms 16.286us 1000 - aten::mul 26.58% 5.433ms 26.58% 5.433ms 5.433us 16.193ms 49.86% 16.193ms 16.193us 1000 - cudaGetDeviceProperties_v2 0.96% 196.000us 0.96% 196.000us 196.000us 0.000us 0.00% 0.000us 0.000us 1 + cudaEventRecord 46.67% 9.570ms 46.67% 9.570ms 2.393us 0.000us 0.00% 0.000us 0.000us 4000 + aten::div 26.28% 5.389ms 26.28% 5.389ms 5.389us 16.622ms 50.62% 16.622ms 16.622us 1000 + aten::mul 26.07% 5.346ms 26.07% 5.346ms 5.346us 16.216ms 49.38% 16.216ms 16.216us 1000 + cudaGetDeviceProperties_v2 0.85% 174.000us 0.85% 174.000us 174.000us 0.000us 0.00% 0.000us 0.000us 1 cudaDeviceSynchronize 0.06% 12.000us 0.06% 12.000us 12.000us 0.000us 0.00% 0.000us 0.000us 1 - cudaStreamIsCapturing 0.04% 8.000us 0.04% 8.000us 2.000us 0.000us 0.00% 0.000us 0.000us 4 + cudaStreamIsCapturing 0.04% 9.000us 0.04% 9.000us 2.250us 0.000us 0.00% 0.000us 0.000us 4 cudaDeviceGetStreamPriorityRange 0.01% 3.000us 0.01% 3.000us 3.000us 0.000us 0.00% 0.000us 0.000us 1 cudaGetDeviceCount 0.01% 2.000us 0.01% 2.000us 1.000us 0.000us 0.00% 0.000us 0.000us 2 ------------------------------------ ------------ ------------ ------------ ------------ ------------ ------------ ------------ ------------ ------------ ------------ - Self CPU time total: 20.444ms - Self CUDA time total: 32.479ms + Self CPU time total: 20.505ms + Self CUDA time total: 32.838ms @@ -1214,7 +1214,7 @@ API ` .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 0 minutes 0.803 seconds) + **Total running time of the script:** ( 0 minutes 0.778 seconds) .. _sphx_glr_download_beginner_introyt_autogradyt_tutorial.py: diff --git a/_sources/beginner/introyt/introyt1_tutorial.rst.txt b/_sources/beginner/introyt/introyt1_tutorial.rst.txt index 5760ef0c92..32a873c8f7 100644 --- a/_sources/beginner/introyt/introyt1_tutorial.rst.txt +++ b/_sources/beginner/introyt/introyt1_tutorial.rst.txt @@ -598,22 +598,22 @@ automobile, ship, truck): Downloading https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz to ./data/cifar-10-python.tar.gz 0%| | 0/170498071 [00:00 - tensor([[-9.6404e-31, 4.5629e-41, 2.4354e-05, 0.0000e+00], - [ 2.0735e-37, 0.0000e+00, 0.0000e+00, 0.0000e+00], - [ 5.5904e+07, 0.0000e+00, -6.3661e-37, 4.5629e-41]]) + tensor([[ 0.0000e+00, 0.0000e+00, 1.4013e-45, 1.4013e-45], + [ 1.7331e-37, 0.0000e+00, 0.0000e+00, 0.0000e+00], + [ 6.6902e-29, 0.0000e+00, -2.4643e-31, 4.5907e-41]]) @@ -271,17 +271,17 @@ have the ``torch.*_like()`` methods: .. code-block:: none torch.Size([2, 2, 3]) - tensor([[[ 9.5868e-10, 0.0000e+00, 0.0000e+00], - [ 0.0000e+00, 8.9683e-44, 0.0000e+00]], + tensor([[[-3.6793e-25, 4.5907e-41, 7.1389e-14], + [ 0.0000e+00, 1.7331e-37, 0.0000e+00]], - [[ 1.5695e-43, 0.0000e+00, 2.0005e+03], - [ 0.0000e+00, -6.3661e-37, 4.5629e-41]]]) + [[ 0.0000e+00, 0.0000e+00, 7.1043e-28], + [ 0.0000e+00, -2.4643e-31, 4.5907e-41]]]) torch.Size([2, 2, 3]) - tensor([[[ 1.5769e-31, 0.0000e+00, 7.4182e-13], - [ 0.0000e+00, 4.4842e-44, 0.0000e+00]], + tensor([[[ 2.0829e-07, 0.0000e+00, 4.5751e-29], + [ 0.0000e+00, 8.9683e-44, 0.0000e+00]], - [[ 1.5695e-43, 0.0000e+00, 1.3849e-31], - [ 0.0000e+00, -6.3661e-37, 4.5629e-41]]]) + [[ 1.5695e-43, 0.0000e+00, 2.4362e-27], + [ 0.0000e+00, -2.4643e-31, 4.5907e-41]]]) torch.Size([2, 2, 3]) tensor([[[0., 0., 0.], [0., 0., 0.]], @@ -1777,7 +1777,7 @@ are reflected in the other: .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 0 minutes 0.275 seconds) + **Total running time of the script:** ( 0 minutes 0.299 seconds) .. _sphx_glr_download_beginner_introyt_tensors_deeper_tutorial.py: diff --git a/_sources/beginner/introyt/trainingyt.rst.txt b/_sources/beginner/introyt/trainingyt.rst.txt index a87e8ea1c0..d8e3d9e65b 100644 --- a/_sources/beginner/introyt/trainingyt.rst.txt +++ b/_sources/beginner/introyt/trainingyt.rst.txt @@ -127,88 +127,78 @@ and download both training and validation data splits. Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/train-images-idx3-ubyte.gz to ./data/FashionMNIST/raw/train-images-idx3-ubyte.gz 0%| | 0/26421880 [00:00 + @@ -458,7 +458,7 @@ Pick up some real data and do a comparison! .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 0 minutes 10.515 seconds) + **Total running time of the script:** ( 0 minutes 11.222 seconds) .. _sphx_glr_download_beginner_nlp_advanced_tutorial.py: diff --git a/_sources/beginner/nlp/deep_learning_tutorial.rst.txt b/_sources/beginner/nlp/deep_learning_tutorial.rst.txt index ddc798e54d..fa31475bbb 100644 --- a/_sources/beginner/nlp/deep_learning_tutorial.rst.txt +++ b/_sources/beginner/nlp/deep_learning_tutorial.rst.txt @@ -73,7 +73,7 @@ output below is the mapping of the :math:`i`'th row of the input under .. code-block:: none - + diff --git a/_sources/beginner/nlp/pytorch_tutorial.rst.txt b/_sources/beginner/nlp/pytorch_tutorial.rst.txt index be55f56c53..0cc5e947e1 100644 --- a/_sources/beginner/nlp/pytorch_tutorial.rst.txt +++ b/_sources/beginner/nlp/pytorch_tutorial.rst.txt @@ -49,7 +49,7 @@ let's look what we can do with tensors. .. code-block:: none - + @@ -386,7 +386,7 @@ created. Let's see it in action. .. code-block:: none tensor([5., 7., 9.], grad_fn=) - + @@ -421,7 +421,7 @@ But how does that help us compute a gradient? .. code-block:: none tensor(21., grad_fn=) - + @@ -536,7 +536,7 @@ successful programmer in deep learning. False False None - + True None diff --git a/_sources/beginner/nlp/sequence_models_tutorial.rst.txt b/_sources/beginner/nlp/sequence_models_tutorial.rst.txt index 3e78e31c70..0dfa902762 100644 --- a/_sources/beginner/nlp/sequence_models_tutorial.rst.txt +++ b/_sources/beginner/nlp/sequence_models_tutorial.rst.txt @@ -90,7 +90,7 @@ Let's see a quick example. .. code-block:: none - + @@ -376,7 +376,7 @@ this LSTM. Hints: .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 0 minutes 0.845 seconds) + **Total running time of the script:** ( 0 minutes 0.809 seconds) .. _sphx_glr_download_beginner_nlp_sequence_models_tutorial.py: diff --git a/_sources/beginner/nlp/word_embeddings_tutorial.rst.txt b/_sources/beginner/nlp/word_embeddings_tutorial.rst.txt index 2ccb80ff4e..3a9105a42d 100644 --- a/_sources/beginner/nlp/word_embeddings_tutorial.rst.txt +++ b/_sources/beginner/nlp/word_embeddings_tutorial.rst.txt @@ -195,7 +195,7 @@ indices are integers, not floats). .. code-block:: none - + @@ -343,9 +343,9 @@ examples and update the parameters with backpropagation. .. code-block:: none [(['forty', 'When'], 'winters'), (['winters', 'forty'], 'shall'), (['shall', 'winters'], 'besiege')] - [521.698427438736, 519.1963937282562, 516.7113490104675, 514.2416417598724, 511.7873330116272, 509.3473746776581, 506.92100834846497, 504.5057735443115, 502.10225200653076, 499.70783376693726] - tensor([-1.1033, -0.6981, 0.2359, 1.9145, 1.8357, 1.3259, -0.0693, 0.3463, - -0.6511, 1.5578], grad_fn=) + [519.4899263381958, 516.9370996952057, 514.4005351066589, 511.87999200820923, 509.37452578544617, 506.88408493995667, 504.40851354599, 501.94547271728516, 499.4936594963074, 497.0529816150665] + tensor([ 0.6867, 0.4197, -1.0219, 0.9891, 0.7819, -2.2055, -1.4977, -0.9028, + 0.8012, -0.6630], grad_fn=) @@ -439,14 +439,14 @@ tips: [(['are', 'We', 'to', 'study'], 'about'), (['about', 'are', 'study', 'the'], 'to'), (['to', 'about', 'the', 'idea'], 'study'), (['study', 'to', 'idea', 'of'], 'the'), (['the', 'study', 'of', 'a'], 'idea')] - tensor([48, 14, 34, 7]) + tensor([35, 30, 27, 19]) .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 0 minutes 0.738 seconds) + **Total running time of the script:** ( 0 minutes 0.784 seconds) .. _sphx_glr_download_beginner_nlp_word_embeddings_tutorial.py: diff --git a/_sources/beginner/nn_tutorial.rst.txt b/_sources/beginner/nn_tutorial.rst.txt index 1cc0a7bdc3..614b9f6684 100644 --- a/_sources/beginner/nn_tutorial.rst.txt +++ b/_sources/beginner/nn_tutorial.rst.txt @@ -1648,8 +1648,8 @@ You should find it runs faster now: .. code-block:: none - 0 0.18058827521800994 - 1 0.1704494299888611 + 0 0.1840453363776207 + 1 0.1712360767841339 @@ -1692,7 +1692,7 @@ what we've seen: .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 0 minutes 25.080 seconds) + **Total running time of the script:** ( 0 minutes 24.457 seconds) .. _sphx_glr_download_beginner_nn_tutorial.py: diff --git a/_sources/beginner/onnx/export_simple_model_to_onnx_tutorial.rst.txt b/_sources/beginner/onnx/export_simple_model_to_onnx_tutorial.rst.txt index 489a9aba6c..c460535e7b 100644 --- a/_sources/beginner/onnx/export_simple_model_to_onnx_tutorial.rst.txt +++ b/_sources/beginner/onnx/export_simple_model_to_onnx_tutorial.rst.txt @@ -344,7 +344,7 @@ sit tight and have fun going through all of them to learn all there is about the .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 0 minutes 1.014 seconds) + **Total running time of the script:** ( 0 minutes 1.084 seconds) .. _sphx_glr_download_beginner_onnx_export_simple_model_to_onnx_tutorial.py: diff --git a/_sources/beginner/onnx/onnx_registry_tutorial.rst.txt b/_sources/beginner/onnx/onnx_registry_tutorial.rst.txt index 24bab75dae..cb67ab087f 100644 --- a/_sources/beginner/onnx/onnx_registry_tutorial.rst.txt +++ b/_sources/beginner/onnx/onnx_registry_tutorial.rst.txt @@ -140,26 +140,6 @@ We will override the registration of ``aten::add.Tensor`` with our custom implem ) - - - -.. rst-class:: sphx-glr-script-out - - .. code-block:: none - - /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/onnx/_internal/exporter.py:136: UserWarning: - - torch.onnx.dynamo_export only implements opset version 18 for now. If you need to use a different opset version, please register them with register_custom_op. - - aten::add.Tensor is supported by ONNX registry: True - /opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py:309: UserWarning: - - torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead. - - - - - .. GENERATED FROM PYTHON SOURCE LINES 122-126 Now let's inspect the model and verify the model has a ``custom_aten_add`` instead of ``aten::add.Tensor``. @@ -184,12 +164,6 @@ operator, and one for constant attribute. - - - - - - .. GENERATED FROM PYTHON SOURCE LINES 139-155 This is how ``custom_aten_add_model`` looks in the ONNX graph using Netron: @@ -237,12 +211,6 @@ As an additional step, we can use ONNX Runtime to run the model, and compare the - - - - - - .. GENERATED FROM PYTHON SOURCE LINES 179-193 Custom operators with existing ONNX Runtime support @@ -271,22 +239,6 @@ Before we begin, let's check whether ``aten::gelu.default`` is really supported - - - -.. rst-class:: sphx-glr-script-out - - .. code-block:: none - - /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/onnx/_internal/exporter.py:136: UserWarning: - - torch.onnx.dynamo_export only implements opset version 18 for now. If you need to use a different opset version, please register them with register_custom_op. - - aten::gelu.default is supported by ONNX registry: True - - - - .. GENERATED FROM PYTHON SOURCE LINES 200-202 In our example, ``aten::gelu.default`` operator is supported by the ONNX registry, @@ -328,22 +280,6 @@ so :meth:`onnx_registry.is_registered_op` returns ``True``. - - - -.. rst-class:: sphx-glr-script-out - - .. code-block:: none - - 'Gelu' is not a known op in 'com.microsoft' - /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/onnx/_internal/exporter.py:136: UserWarning: - - torch.onnx.dynamo_export only implements opset version 18 for now. If you need to use a different opset version, please register them with register_custom_op. - - - - - .. GENERATED FROM PYTHON SOURCE LINES 234-239 Let's inspect the model and verify the model uses :func:`custom_aten_gelu` instead of @@ -368,12 +304,6 @@ node for ``Gelu`` with namespace ``com.microsoft``. - - - - - - .. GENERATED FROM PYTHON SOURCE LINES 251-265 The following diagram shows ``custom_aten_gelu_model`` ONNX graph using Netron: @@ -416,12 +346,6 @@ and compare the results with PyTorch. torch.testing.assert_close(torch_output, torch.tensor(onnxruntime_output)) - - - - - - .. GENERATED FROM PYTHON SOURCE LINES 286-308 Custom operators without ONNX Runtime support @@ -478,12 +402,6 @@ This can be done by using ``torch._custom_op``. - - - - - - .. GENERATED FROM PYTHON SOURCE LINES 335-344 Custom Ops Registration in ONNX Registry @@ -530,23 +448,6 @@ and ``CustomOpTwo``. These two ops are registered and built in - - - -.. rst-class:: sphx-glr-script-out - - .. code-block:: none - - 'CustomOpOne' is not a known op in 'test.customop' - 'CustomOpTwo' is not a known op in 'test.customop' - /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/onnx/_internal/exporter.py:136: UserWarning: - - torch.onnx.dynamo_export only implements opset version 18 for now. If you need to use a different opset version, please register them with register_custom_op. - - - - - .. GENERATED FROM PYTHON SOURCE LINES 375-379 The ``onnx_program`` exposes the exported model as protobuf through ``onnx_program.model_proto``. @@ -568,12 +469,6 @@ there are two function nodes, one for each operator. - - - - - - .. GENERATED FROM PYTHON SOURCE LINES 389-469 This is how ``custom_addandround_model`` ONNX graph looks using Netron: @@ -660,7 +555,7 @@ sit tight and have fun going through all of them to learn all there is about the .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 0 minutes 0.920 seconds) + **Total running time of the script:** ( 0 minutes 0.000 seconds) .. _sphx_glr_download_beginner_onnx_onnx_registry_tutorial.py: diff --git a/_sources/beginner/template_tutorial.rst.txt b/_sources/beginner/template_tutorial.rst.txt index c4d16117d8..1962901a9f 100644 --- a/_sources/beginner/template_tutorial.rst.txt +++ b/_sources/beginner/template_tutorial.rst.txt @@ -134,7 +134,7 @@ Further Reading .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 0 minutes 0.021 seconds) + **Total running time of the script:** ( 0 minutes 0.023 seconds) .. _sphx_glr_download_beginner_template_tutorial.py: diff --git a/_sources/beginner/transfer_learning_tutorial.rst.txt b/_sources/beginner/transfer_learning_tutorial.rst.txt index a894982002..0794f9630e 100644 --- a/_sources/beginner/transfer_learning_tutorial.rst.txt +++ b/_sources/beginner/transfer_learning_tutorial.rst.txt @@ -80,7 +80,7 @@ These two major transfer learning scenarios look as follows: .. code-block:: none - + @@ -377,8 +377,8 @@ Load a pretrained model and reset final fully connected layer. Downloading: "https://download.pytorch.org/models/resnet18-f37072fd.pth" to /var/lib/ci-user/.cache/torch/hub/checkpoints/resnet18-f37072fd.pth 0%| | 0.00/44.7M [00:00Sobol False 16810.0 - 0.897621 + 0.911009 19 66 0.003182 @@ -1644,7 +1717,7 @@ an easy way to sanity check the optimization. Sobol False 21926.0 - 0.879206 + 0.883842 23 118 0.000145 @@ -1660,7 +1733,7 @@ an easy way to sanity check the optimization. Sobol True 37560.0 - 0.954121 + 0.949112 40 124 0.002745 @@ -1676,7 +1749,7 @@ an easy way to sanity check the optimization. Sobol False 14756.0 - 0.892503 + 0.899586 18 23 0.000166 @@ -1692,7 +1765,7 @@ an easy way to sanity check the optimization. Sobol True 71630.0 - 0.952770 + 0.948751 80 99 0.000642 @@ -1708,7 +1781,7 @@ an easy way to sanity check the optimization. Sobol False 13948.0 - 0.925682 + 0.923867 16 54 0.000444 @@ -1724,7 +1797,7 @@ an easy way to sanity check the optimization. Sobol False 24686.0 - 0.855924 + 0.862653 29 50 0.000177 @@ -1740,7 +1813,7 @@ an easy way to sanity check the optimization. Sobol False 18290.0 - 0.877489 + 0.875520 20 87 0.000119 @@ -1756,7 +1829,7 @@ an easy way to sanity check the optimization. Sobol False 20996.0 - 0.723215 + 0.841835 26 17 0.005245 @@ -1771,13 +1844,13 @@ an easy way to sanity check the optimization. COMPLETED BoTorch True - 41030.0 - 0.960493 - 44 - 121 - 0.001528 + 52088.0 + 0.954132 + 56 + 124 + 0.002046 3 - 0.089977 + 0.168286 64 @@ -1829,14 +1902,14 @@ validation accuracy. The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. - [WARNING 06-25 21:49:23] ax.service.utils.report_utils: Column reason missing for all trials. Not appending column. + [WARNING 06-25 22:32:56] ax.service.utils.report_utils: Column reason missing for all trials. Not appending column. .. raw:: html
-
+


@@ -1881,7 +1954,7 @@ much easier to model than the validation accuracy (``val_acc``) metric.
-
+


@@ -1914,7 +1987,7 @@ as the hidden sizes increase.
-
+


@@ -1943,7 +2016,7 @@ is much larger).
-
+


@@ -1960,7 +2033,7 @@ for their help with integrating TorchX with Ax. .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 16 minutes 27.058 seconds) + **Total running time of the script:** ( 16 minutes 21.583 seconds) .. _sphx_glr_download_intermediate_ax_multiobjective_nas_tutorial.py: diff --git a/_sources/intermediate/char_rnn_classification_tutorial.rst.txt b/_sources/intermediate/char_rnn_classification_tutorial.rst.txt index 002210eefd..60ec06a47f 100644 --- a/_sources/intermediate/char_rnn_classification_tutorial.rst.txt +++ b/_sources/intermediate/char_rnn_classification_tutorial.rst.txt @@ -594,26 +594,26 @@ average of the loss. .. code-block:: none - 5000 5% (0m 34s) 2.2208 Horigome / Japanese ✓ + 5000 5% (0m 35s) 2.2208 Horigome / Japanese ✓ 10000 10% (1m 10s) 1.6752 Miazga / Japanese ✗ (Polish) - 15000 15% (1m 46s) 0.1778 Yukhvidov / Russian ✓ - 20000 20% (2m 21s) 1.5856 Mclaughlin / Irish ✗ (Scottish) - 25000 25% (2m 57s) 0.6552 Banh / Vietnamese ✓ - 30000 30% (3m 32s) 1.5547 Machado / Japanese ✗ (Portuguese) - 35000 35% (4m 8s) 0.0168 Fotopoulos / Greek ✓ - 40000 40% (4m 43s) 1.1464 Quirke / Irish ✓ - 45000 45% (5m 18s) 1.7532 Reier / French ✗ (German) - 50000 50% (5m 54s) 0.8413 Hou / Chinese ✓ - 55000 55% (6m 29s) 0.8587 Duan / Vietnamese ✗ (Chinese) - 60000 60% (7m 5s) 0.2047 Giang / Vietnamese ✓ - 65000 65% (7m 40s) 2.5534 Cober / French ✗ (Czech) - 70000 70% (8m 16s) 1.5163 Mateus / Arabic ✗ (Portuguese) - 75000 75% (8m 51s) 0.2217 Hamilton / Scottish ✓ - 80000 80% (9m 30s) 0.4456 Maessen / Dutch ✓ - 85000 85% (10m 6s) 0.0239 Gan / Chinese ✓ - 90000 90% (10m 41s) 0.0521 Bellomi / Italian ✓ - 95000 95% (11m 16s) 0.0867 Vozgov / Russian ✓ - 100000 100% (11m 52s) 0.2730 Tong / Vietnamese ✓ + 15000 15% (1m 47s) 0.1778 Yukhvidov / Russian ✓ + 20000 20% (2m 22s) 1.5856 Mclaughlin / Irish ✗ (Scottish) + 25000 25% (2m 58s) 0.6552 Banh / Vietnamese ✓ + 30000 30% (3m 34s) 1.5547 Machado / Japanese ✗ (Portuguese) + 35000 35% (4m 10s) 0.0168 Fotopoulos / Greek ✓ + 40000 40% (4m 49s) 1.1464 Quirke / Irish ✓ + 45000 45% (5m 24s) 1.7532 Reier / French ✗ (German) + 50000 50% (6m 0s) 0.8413 Hou / Chinese ✓ + 55000 55% (6m 36s) 0.8587 Duan / Vietnamese ✗ (Chinese) + 60000 60% (7m 12s) 0.2047 Giang / Vietnamese ✓ + 65000 65% (7m 48s) 2.5534 Cober / French ✗ (Czech) + 70000 70% (8m 23s) 1.5163 Mateus / Arabic ✗ (Portuguese) + 75000 75% (8m 59s) 0.2217 Hamilton / Scottish ✓ + 80000 80% (9m 34s) 0.4456 Maessen / Dutch ✓ + 85000 85% (10m 10s) 0.0239 Gan / Chinese ✓ + 90000 90% (10m 46s) 0.0521 Bellomi / Italian ✓ + 95000 95% (11m 21s) 0.0867 Vozgov / Russian ✓ + 100000 100% (11m 57s) 0.2730 Tong / Vietnamese ✓ @@ -653,7 +653,7 @@ learning: .. code-block:: none - [] + [] @@ -844,7 +844,7 @@ Exercises .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 12 minutes 4.757 seconds) + **Total running time of the script:** ( 12 minutes 10.610 seconds) .. _sphx_glr_download_intermediate_char_rnn_classification_tutorial.py: diff --git a/_sources/intermediate/char_rnn_generation_tutorial.rst.txt b/_sources/intermediate/char_rnn_generation_tutorial.rst.txt index f0cbae6867..fcc42866de 100644 --- a/_sources/intermediate/char_rnn_generation_tutorial.rst.txt +++ b/_sources/intermediate/char_rnn_generation_tutorial.rst.txt @@ -464,26 +464,26 @@ in ``all_losses`` for plotting later. .. code-block:: none - 0m 39s (5000 5%) 3.1506 - 1m 19s (10000 10%) 2.5070 - 1m 59s (15000 15%) 3.3047 - 2m 39s (20000 20%) 2.4247 - 3m 20s (25000 25%) 2.6406 - 4m 0s (30000 30%) 2.0266 - 4m 40s (35000 35%) 2.6520 - 5m 20s (40000 40%) 2.4261 - 6m 0s (45000 45%) 2.2302 - 6m 41s (50000 50%) 1.6496 - 7m 22s (55000 55%) 2.7101 - 8m 3s (60000 60%) 2.5396 - 8m 43s (65000 65%) 2.5978 - 9m 23s (70000 70%) 1.6029 - 10m 4s (75000 75%) 0.9634 - 10m 44s (80000 80%) 3.0950 - 11m 24s (85000 85%) 2.0512 - 12m 4s (90000 90%) 2.5302 - 12m 45s (95000 95%) 3.2365 - 13m 25s (100000 100%) 1.7113 + 0m 38s (5000 5%) 3.1506 + 1m 17s (10000 10%) 2.5070 + 1m 57s (15000 15%) 3.3047 + 2m 37s (20000 20%) 2.4247 + 3m 17s (25000 25%) 2.6406 + 3m 57s (30000 30%) 2.0266 + 4m 37s (35000 35%) 2.6520 + 5m 17s (40000 40%) 2.4261 + 5m 57s (45000 45%) 2.2302 + 6m 37s (50000 50%) 1.6496 + 7m 17s (55000 55%) 2.7101 + 7m 58s (60000 60%) 2.5396 + 8m 38s (65000 65%) 2.5978 + 9m 18s (70000 70%) 1.6029 + 9m 58s (75000 75%) 0.9634 + 10m 38s (80000 80%) 3.0950 + 11m 18s (85000 85%) 2.0512 + 11m 59s (90000 90%) 2.5302 + 12m 40s (95000 95%) 3.2365 + 13m 20s (100000 100%) 1.7113 @@ -522,7 +522,7 @@ learning: .. code-block:: none - [] + [] @@ -641,7 +641,7 @@ Exercises .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 13 minutes 25.694 seconds) + **Total running time of the script:** ( 13 minutes 21.102 seconds) .. _sphx_glr_download_intermediate_char_rnn_generation_tutorial.py: diff --git a/_sources/intermediate/custom_function_conv_bn_tutorial.rst.txt b/_sources/intermediate/custom_function_conv_bn_tutorial.rst.txt index fdf8b2c28e..1fe1d40700 100644 --- a/_sources/intermediate/custom_function_conv_bn_tutorial.rst.txt +++ b/_sources/intermediate/custom_function_conv_bn_tutorial.rst.txt @@ -568,22 +568,22 @@ allocate one fewer buffer per fused ``conv-bn`` pair. Test set: Average loss: 0.4197, Accuracy: 8681/10000 (87%) Train Epoch: 0 [0/60000 (0%)] Loss: 2.349030 - Train Epoch: 0 [4096/60000 (7%)] Loss: 7.435157 - Train Epoch: 0 [8192/60000 (13%)] Loss: 5.443542 - Train Epoch: 0 [12288/60000 (20%)] Loss: 2.457858 - Train Epoch: 0 [16384/60000 (27%)] Loss: 1.739216 - Train Epoch: 0 [20480/60000 (33%)] Loss: 1.448291 - Train Epoch: 0 [24576/60000 (40%)] Loss: 1.312150 - Train Epoch: 0 [28672/60000 (47%)] Loss: 1.145356 - Train Epoch: 0 [32768/60000 (53%)] Loss: 1.496005 - Train Epoch: 0 [36864/60000 (60%)] Loss: 1.251128 - Train Epoch: 0 [40960/60000 (67%)] Loss: 1.076815 - Train Epoch: 0 [45056/60000 (73%)] Loss: 0.892333 - Train Epoch: 0 [49152/60000 (80%)] Loss: 0.829717 - Train Epoch: 0 [53248/60000 (87%)] Loss: 0.740473 - Train Epoch: 0 [57344/60000 (93%)] Loss: 0.789344 - - Test set: Average loss: 0.4183, Accuracy: 8869/10000 (89%) + Train Epoch: 0 [4096/60000 (7%)] Loss: 7.435156 + Train Epoch: 0 [8192/60000 (13%)] Loss: 5.443526 + Train Epoch: 0 [12288/60000 (20%)] Loss: 2.457884 + Train Epoch: 0 [16384/60000 (27%)] Loss: 1.739214 + Train Epoch: 0 [20480/60000 (33%)] Loss: 1.448314 + Train Epoch: 0 [24576/60000 (40%)] Loss: 1.312073 + Train Epoch: 0 [28672/60000 (47%)] Loss: 1.144974 + Train Epoch: 0 [32768/60000 (53%)] Loss: 1.495681 + Train Epoch: 0 [36864/60000 (60%)] Loss: 1.249685 + Train Epoch: 0 [40960/60000 (67%)] Loss: 1.080824 + Train Epoch: 0 [45056/60000 (73%)] Loss: 0.894487 + Train Epoch: 0 [49152/60000 (80%)] Loss: 0.845663 + Train Epoch: 0 [53248/60000 (87%)] Loss: 0.700515 + Train Epoch: 0 [57344/60000 (93%)] Loss: 0.743951 + + Test set: Average loss: 0.4384, Accuracy: 8719/10000 (87%) cuDNN version: 8902 @@ -598,7 +598,7 @@ allocate one fewer buffer per fused ``conv-bn`` pair. .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 0 minutes 36.520 seconds) + **Total running time of the script:** ( 0 minutes 37.817 seconds) .. _sphx_glr_download_intermediate_custom_function_conv_bn_tutorial.py: diff --git a/_sources/intermediate/dqn_with_rnn_tutorial.rst.txt b/_sources/intermediate/dqn_with_rnn_tutorial.rst.txt index 526a996746..9bd04e7471 100644 --- a/_sources/intermediate/dqn_with_rnn_tutorial.rst.txt +++ b/_sources/intermediate/dqn_with_rnn_tutorial.rst.txt @@ -803,15 +803,15 @@ every 50 data collection, and plot the results after training. device=cpu, is_shared=False) - 0%| | 50/1000000 [00:00<2:26:56, 113.41it/s] - 0%| | 50/1000000 [00:15<2:26:56, 113.41it/s] - steps: 16, loss_val: 0.0002, action_spread: tensor([ 8, 42]): 0%| | 50/1000000 [00:25<2:26:56, 113.41it/s] - steps: 16, loss_val: 0.0002, action_spread: tensor([ 8, 42]): 0%| | 100/1000000 [00:26<85:13:26, 3.26it/s] - steps: 16, loss_val: 0.0003, action_spread: tensor([40, 10]): 0%| | 100/1000000 [00:50<85:13:26, 3.26it/s] - steps: 16, loss_val: 0.0003, action_spread: tensor([40, 10]): 0%| | 150/1000000 [00:51<110:30:44, 2.51it/s] - steps: 16, loss_val: 0.0003, action_spread: tensor([ 8, 42]): 0%| | 150/1000000 [01:16<110:30:44, 2.51it/s] - steps: 16, loss_val: 0.0003, action_spread: tensor([ 8, 42]): 0%| | 200/1000000 [01:16<122:26:43, 2.27it/s] - steps: 16, loss_val: 0.0003, action_spread: tensor([43, 7]): 0%| | 200/1000000 [01:41<122:26:43, 2.27it/s] + 0%| | 50/1000000 [00:00<2:24:15, 115.53it/s] + 0%| | 50/1000000 [00:10<2:24:15, 115.53it/s] + steps: 14, loss_val: 0.0005, action_spread: tensor([ 9, 41]): 0%| | 50/1000000 [00:26<2:24:15, 115.53it/s] + steps: 14, loss_val: 0.0005, action_spread: tensor([ 9, 41]): 0%| | 100/1000000 [00:27<87:57:07, 3.16it/s] + steps: 14, loss_val: 0.0006, action_spread: tensor([42, 8]): 0%| | 100/1000000 [00:52<87:57:07, 3.16it/s] + steps: 14, loss_val: 0.0006, action_spread: tensor([42, 8]): 0%| | 150/1000000 [00:53<114:57:29, 2.42it/s] + steps: 14, loss_val: 0.0007, action_spread: tensor([ 7, 43]): 0%| | 150/1000000 [01:19<114:57:29, 2.42it/s] + steps: 14, loss_val: 0.0007, action_spread: tensor([ 7, 43]): 0%| | 200/1000000 [01:20<127:54:56, 2.17it/s] + steps: 14, loss_val: 0.0007, action_spread: tensor([41, 9]): 0%| | 200/1000000 [01:45<127:54:56, 2.17it/s] @@ -867,7 +867,7 @@ Further Reading .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 1 minutes 46.960 seconds) + **Total running time of the script:** ( 1 minutes 51.213 seconds) .. _sphx_glr_download_intermediate_dqn_with_rnn_tutorial.py: diff --git a/_sources/intermediate/ensembling.rst.txt b/_sources/intermediate/ensembling.rst.txt index bfef212f9c..f9f5cf5453 100644 --- a/_sources/intermediate/ensembling.rst.txt +++ b/_sources/intermediate/ensembling.rst.txt @@ -306,13 +306,13 @@ Curious about performance numbers? Here's how the numbers look. .. code-block:: none - Predictions without vmap + Predictions without vmap [model(minibatch) for model, minibatch in zip(models, minibatches)] - 2.08 ms + 2.26 ms 1 measurement, 100 runs , 1 thread - Predictions with vmap + Predictions with vmap vmap(fmodel)(params, buffers, minibatches) - 795.37 us + 858.11 us 1 measurement, 100 runs , 1 thread @@ -332,7 +332,7 @@ on GitHub. .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 0 minutes 0.835 seconds) + **Total running time of the script:** ( 0 minutes 0.879 seconds) .. _sphx_glr_download_intermediate_ensembling.py: diff --git a/_sources/intermediate/forward_ad_usage.rst.txt b/_sources/intermediate/forward_ad_usage.rst.txt index d534b7cfcc..9797e2e59c 100644 --- a/_sources/intermediate/forward_ad_usage.rst.txt +++ b/_sources/intermediate/forward_ad_usage.rst.txt @@ -370,7 +370,7 @@ to the module. .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 0 minutes 0.152 seconds) + **Total running time of the script:** ( 0 minutes 0.155 seconds) .. _sphx_glr_download_intermediate_forward_ad_usage.py: diff --git a/_sources/intermediate/fx_profiling_tutorial.rst.txt b/_sources/intermediate/fx_profiling_tutorial.rst.txt index 4d48d3daae..eb6c50b5dc 100644 --- a/_sources/intermediate/fx_profiling_tutorial.rst.txt +++ b/_sources/intermediate/fx_profiling_tutorial.rst.txt @@ -477,77 +477,77 @@ characteristics of our ResNet18 model; Op type Op Average runtime (s) Pct total runtime ------------- --------------------- --------------------- ------------------- - call_module maxpool 0.00575376 8.70941 - call_module layer4_1_conv2 0.00491071 7.4333 - call_module conv1 0.00479746 7.26187 - call_module layer4_0_conv2 0.00369048 5.58625 - call_module layer1_0_conv1 0.00338578 5.12503 - call_module layer1_1_conv2 0.00320458 4.85075 - call_module layer4_1_conv1 0.00319433 4.83523 - call_module layer1_0_conv2 0.00295234 4.46893 - call_module layer2_1_conv1 0.0028708 4.3455 - call_module layer3_1_conv1 0.00283694 4.29426 - call_module layer3_1_conv2 0.00275111 4.16434 - call_module layer3_0_conv2 0.00265932 4.02539 - call_module layer1_1_conv1 0.00259066 3.92146 - call_module layer2_1_conv2 0.0025208 3.81571 - call_module layer2_0_conv2 0.00239658 3.62769 - call_module layer4_0_conv1 0.00222993 3.37543 - call_module layer3_0_conv1 0.00175953 2.66338 - call_module bn1 0.00150132 2.27254 - call_module layer2_0_conv1 0.00146842 2.22274 - call_module layer2_0_downsample_0 0.000971079 1.46991 - call_module layer3_0_downsample_0 0.000450134 0.681365 - call_module layer4_0_downsample_0 0.000447512 0.677395 - call_function add 0.000422955 0.640223 - call_function add_1 0.00041914 0.634449 - call_module relu 0.000286579 0.433793 - call_function add_3 0.000248194 0.375689 - call_module layer1_0_bn1 0.000195265 0.295571 - call_module fc 0.000192881 0.291962 - call_module layer1_1_bn2 0.000168562 0.255151 - call_module layer1_0_bn2 0.000168085 0.254429 - call_module avgpool 0.000153065 0.231693 - call_module layer1_1_bn1 0.000148773 0.225197 - call_module layer4_1_bn2 0.000141859 0.214731 - call_module layer2_0_downsample_1 0.000134468 0.203543 - call_module layer2_1_bn1 0.000132799 0.201017 - call_module layer2_0_bn2 0.000128746 0.194882 - call_module layer1_1_relu_1 0.000128508 0.194521 - call_module layer3_1_bn1 0.000123501 0.186942 - call_module layer2_0_bn1 0.000122309 0.185138 - call_module layer3_1_bn2 0.000121117 0.183333 - call_module layer2_1_bn2 0.000119686 0.181168 - call_module layer4_1_bn1 0.000119448 0.180807 - call_module layer3_0_bn1 0.000116348 0.176115 - call_module layer4_0_bn2 0.000114679 0.173589 - call_module layer1_0_relu 0.000113726 0.172146 - call_module layer3_0_bn2 0.00011301 0.171063 - call_module layer4_0_bn1 0.000112057 0.169619 - call_module layer3_0_downsample_1 0.00011158 0.168898 - call_module layer4_0_downsample_1 0.000108957 0.164928 - call_module layer1_1_relu 9.84669e-05 0.149049 - call_module layer1_0_relu_1 9.75132e-05 0.147605 - call_module layer4_1_relu 8.63075e-05 0.130643 - call_function add_2 8.05855e-05 0.121982 - call_module layer2_1_relu 8.03471e-05 0.121621 - call_module layer4_0_relu 8.01086e-05 0.12126 - call_function add_7 8.01086e-05 0.12126 - call_module layer2_0_relu 7.96318e-05 0.120538 - call_module layer2_1_relu_1 7.82013e-05 0.118373 - call_module layer2_0_relu_1 7.67708e-05 0.116207 - call_module layer3_1_relu 7.39098e-05 0.111877 - call_module layer3_0_relu 7.31945e-05 0.110794 - call_function add_4 7.29561e-05 0.110433 - call_module layer4_1_relu_1 7.27177e-05 0.110072 - call_function add_6 7.15256e-05 0.108268 - call_module layer4_0_relu_1 7.10487e-05 0.107546 - call_module layer3_1_relu_1 6.98566e-05 0.105741 - call_function add_5 6.96182e-05 0.105381 - call_module layer3_0_relu_1 6.93798e-05 0.10502 - call_function flatten 4.29153e-05 0.0649606 - placeholder x 2.5034e-05 0.0378937 - output output 1.85966e-05 0.0281496 + call_module maxpool 0.00839758 8.8298 + call_module conv1 0.00785089 8.25497 + call_module layer1_1_conv1 0.00579381 6.09202 + call_module layer1_0_conv2 0.00575066 6.04664 + call_module layer1_0_conv1 0.00556517 5.85161 + call_module layer2_1_conv2 0.00535631 5.632 + call_module layer2_1_conv1 0.00509667 5.359 + call_module layer1_1_conv2 0.00482035 5.06845 + call_module layer2_0_conv2 0.00472021 4.96316 + call_module layer4_0_conv2 0.00393105 4.13338 + call_module layer4_1_conv1 0.00334334 3.51543 + call_module layer4_1_conv2 0.00333142 3.50289 + call_module layer2_0_conv1 0.00315976 3.32239 + call_module layer3_1_conv2 0.00291038 3.06017 + call_module layer3_1_conv1 0.00289321 3.04212 + call_module layer3_0_conv1 0.00289011 3.03886 + call_module layer3_0_conv2 0.00272608 2.86639 + call_module bn1 0.0023737 2.49587 + call_module layer4_0_conv1 0.00233245 2.4525 + call_module layer2_0_downsample_0 0.00140858 1.48108 + call_function add_1 0.000641108 0.674105 + call_module relu 0.000547171 0.575334 + call_module layer3_0_downsample_0 0.000541925 0.569818 + call_module layer4_0_downsample_0 0.000512362 0.538733 + call_function add 0.000449419 0.472551 + call_module layer1_1_bn1 0.000344038 0.361746 + call_module layer1_0_bn2 0.00033474 0.351969 + call_module layer1_0_bn1 0.00032258 0.339184 + call_module layer1_1_bn2 0.000280857 0.295313 + call_module layer2_1_bn2 0.000243187 0.255704 + call_module layer2_1_bn1 0.000236273 0.248434 + call_module layer2_0_bn1 0.000234127 0.246178 + call_module layer2_0_bn2 0.000226974 0.238657 + call_module layer2_0_downsample_1 0.000204802 0.215343 + call_module fc 0.000177145 0.186263 + call_module layer1_0_relu 0.000175714 0.184759 + call_module layer4_0_bn2 0.000174522 0.183505 + call_module layer1_1_relu 0.000173569 0.182502 + call_module layer1_0_relu_1 0.000165939 0.17448 + call_function add_3 0.000164747 0.173227 + call_module layer3_0_bn1 0.00016427 0.172725 + call_module layer1_1_relu_1 0.000163555 0.171973 + call_module layer4_1_bn1 0.000163078 0.171472 + call_module layer4_1_bn2 0.000160694 0.168965 + call_module layer4_0_downsample_1 0.00015974 0.167962 + call_module avgpool 0.000156164 0.164202 + call_function add_2 0.000155926 0.163951 + call_module layer3_1_bn1 0.000144958 0.15242 + call_module layer2_1_relu 0.000142813 0.150163 + call_module layer3_1_bn2 0.000141621 0.14891 + call_module layer2_1_relu_1 0.000133753 0.140637 + call_module layer4_0_bn1 0.000132799 0.139634 + call_module layer3_0_bn2 0.000131845 0.138632 + call_module layer2_0_relu_1 0.000128746 0.135373 + call_module layer2_0_relu 0.000125408 0.131863 + call_module layer3_0_downsample_1 0.000123739 0.130108 + call_module layer4_1_relu 0.000109196 0.114816 + call_function add_6 0.000101089 0.106293 + call_module layer4_0_relu_1 9.13143e-05 0.0960143 + call_module layer4_0_relu 8.82149e-05 0.0927553 + call_function add_7 8.79765e-05 0.0925046 + call_function add_5 8.44002e-05 0.0887443 + call_module layer4_1_relu_1 8.32081e-05 0.0874908 + call_module layer3_0_relu 8.13007e-05 0.0854853 + call_module layer3_1_relu 7.9155e-05 0.0832291 + call_module layer3_0_relu_1 7.65324e-05 0.0804715 + call_module layer3_1_relu_1 7.43866e-05 0.0782153 + call_function add_4 7.27177e-05 0.0764605 + call_function flatten 4.48227e-05 0.0471297 + placeholder x 2.93255e-05 0.0308349 + output output 1.90735e-05 0.0200552 @@ -580,7 +580,7 @@ you might have. .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 0 minutes 0.418 seconds) + **Total running time of the script:** ( 0 minutes 0.493 seconds) .. _sphx_glr_download_intermediate_fx_profiling_tutorial.py: diff --git a/_sources/intermediate/inductor_debug_cpu.rst.txt b/_sources/intermediate/inductor_debug_cpu.rst.txt index cd880855c4..f748cd114e 100644 --- a/_sources/intermediate/inductor_debug_cpu.rst.txt +++ b/_sources/intermediate/inductor_debug_cpu.rst.txt @@ -76,12 +76,6 @@ Here is a simple example to run the ``torch.compile`` using Inductor and compare result = compiled_foo1(x1, x2) - - - - - - .. GENERATED FROM PYTHON SOURCE LINES 57-58 The correct implementation of ``neg`` in the ``cpp`` codegen is as follows: @@ -95,12 +89,6 @@ The correct implementation of ``neg`` in the ``cpp`` codegen is as follows: return f"decltype({x})(-{x})" - - - - - - .. GENERATED FROM PYTHON SOURCE LINES 63-102 In order to demonstrate the debugging, we will modify the function to a wrong one later. @@ -155,12 +143,6 @@ Here are the main parts of code extracted from the files and we correlate the C+ return (clone,) - - - - - - .. GENERATED FROM PYTHON SOURCE LINES 110-112 C++ kernel in ``output_code``: @@ -202,12 +184,6 @@ C++ kernel in ``output_code``: - - - - - - .. GENERATED FROM PYTHON SOURCE LINES 144-176 Determine component of error @@ -252,12 +228,6 @@ For example, the ``neg`` function is modified like this: return f"-{x}" - - - - - - .. GENERATED FROM PYTHON SOURCE LINES 181-229 The logging gives the following compile error with a rather clear reason. @@ -388,12 +358,6 @@ For instance, we modify the example like this: assert same(expected_result, actual_result) == True - - - - - - .. GENERATED FROM PYTHON SOURCE LINES 302-303 And also modify the ``neg`` function: @@ -407,12 +371,6 @@ And also modify the ``neg`` function: return f"decltype({x})(2 * {x})" - - - - - - .. GENERATED FROM PYTHON SOURCE LINES 308-339 An accuracy problem would be raised as follows: @@ -457,12 +415,6 @@ After running, we get the final minified graph with the target node ``neg``: return (neg,) - - - - - - .. GENERATED FROM PYTHON SOURCE LINES 345-346 For more usage details about Minifier, please refer to `Troubleshooting `_. @@ -529,33 +481,6 @@ We set following environment variable as a best practice to benchmark on Intel(R - - - -.. rst-class:: sphx-glr-script-out - - .. code-block:: none - - /opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py:309: UserWarning: - - torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead. - - /opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py:309: UserWarning: - - torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead. - - - Downloading config.json: 0%| | 0.00/765 [00:00`_ @@ -627,12 +546,6 @@ We are able to get the profiling table and trace files. p.step() - - - - - - .. GENERATED FROM PYTHON SOURCE LINES 454-484 We get the following performance profiling table for the eager-mode model (omitting some columns): @@ -754,12 +667,6 @@ searching ``cpp_fused__mkl_linear_add_mul_relu_151`` in ``output_code.py`` }''') - - - - - - .. GENERATED FROM PYTHON SOURCE LINES 568-571 From the generated code above, we can see this kernel has done a typical `Loop Fusion `_ on ``[add, add, mul, add]``. @@ -809,12 +716,6 @@ we can infer the sizes and stride of the inputs and further benchmark this ``[ad # print(f"speed up ratio: {eager_t / inductor_t}") - - - - - - .. GENERATED FROM PYTHON SOURCE LINES 610-621 Output: @@ -848,7 +749,7 @@ We show how to find the time-consuming hotspot with PyTorch Profiler and figure .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 9 minutes 30.097 seconds) + **Total running time of the script:** ( 0 minutes 0.000 seconds) .. _sphx_glr_download_intermediate_inductor_debug_cpu.py: diff --git a/_sources/intermediate/jacobians_hessians.rst.txt b/_sources/intermediate/jacobians_hessians.rst.txt index 00d4528be7..7d0b641a4b 100644 --- a/_sources/intermediate/jacobians_hessians.rst.txt +++ b/_sources/intermediate/jacobians_hessians.rst.txt @@ -261,13 +261,13 @@ And then run the performance comparison: .. code-block:: none - + compute_jac(xp) - 2.41 ms + 2.40 ms 1 measurement, 500 runs , 1 thread - + jacrev(predict, argnums=2)(weight, bias, x) - 678.92 us + 680.98 us 1 measurement, 500 runs , 1 thread @@ -292,7 +292,7 @@ Let's do a relative performance comparison of the above with our ``get_perf`` fu .. code-block:: none - Performance delta: 71.8454 percent improvement with vmap + Performance delta: 71.6275 percent improvement with vmap @@ -394,13 +394,13 @@ First, let's benchmark with more inputs than outputs: .. code-block:: none torch.Size([2048, 32]) - jacfwd time: + jacfwd time: jacfwd(predict, argnums=2)(weight, bias, x) - 1.23 ms + 1.31 ms 1 measurement, 500 runs , 1 thread - jacrev time: + jacrev time: jacrev(predict, argnums=2)(weight, bias, x) - 13.41 ms + 13.13 ms 1 measurement, 500 runs , 1 thread @@ -425,7 +425,7 @@ and then do a relative benchmark: .. code-block:: none - Performance delta: 985.8145 percent improvement with jacrev + Performance delta: 904.8346 percent improvement with jacrev @@ -462,13 +462,13 @@ and now the reverse - more outputs (M) than inputs (N): .. code-block:: none - jacfwd time: + jacfwd time: jacfwd(predict, argnums=2)(weight, bias, x) - 6.73 ms + 8.65 ms 1 measurement, 500 runs , 1 thread - jacrev time: + jacrev time: jacrev(predict, argnums=2)(weight, bias, x) - 783.92 us + 822.61 us 1 measurement, 500 runs , 1 thread @@ -493,7 +493,7 @@ and a relative performance comparison: .. code-block:: none - Performance delta: 758.4283 percent improvement with jacfwd + Performance delta: 951.6169 percent improvement with jacfwd @@ -749,7 +749,7 @@ instead compose reverse-mode AD with reverse-mode AD: .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 0 minutes 13.847 seconds) + **Total running time of the script:** ( 0 minutes 14.813 seconds) .. _sphx_glr_download_intermediate_jacobians_hessians.py: diff --git a/_sources/intermediate/mario_rl_tutorial.rst.txt b/_sources/intermediate/mario_rl_tutorial.rst.txt index 017ca9286f..9246945efa 100644 --- a/_sources/intermediate/mario_rl_tutorial.rst.txt +++ b/_sources/intermediate/mario_rl_tutorial.rst.txt @@ -993,9 +993,9 @@ his world, we suggest running the loop for at least 40,000 episodes! Using CUDA: True - Episode 0 - Step 163 - Epsilon 0.9999592508251706 - Mean Reward 635.0 - Mean Length 163.0 - Mean Loss 0.0 - Mean Q Value 0.0 - Time Delta 1.995 - Time 2024-06-25T20:49:45 - Episode 20 - Step 5007 - Epsilon 0.9987490329557962 - Mean Reward 667.429 - Mean Length 238.429 - Mean Loss 0.0 - Mean Q Value 0.0 - Time Delta 59.381 - Time 2024-06-25T20:50:45 - Episode 39 - Step 8854 - Epsilon 0.9977889477081997 - Mean Reward 656.6 - Mean Length 221.35 - Mean Loss 0.0 - Mean Q Value 0.0 - Time Delta 47.646 - Time 2024-06-25T20:51:32 + Episode 0 - Step 163 - Epsilon 0.9999592508251706 - Mean Reward 635.0 - Mean Length 163.0 - Mean Loss 0.0 - Mean Q Value 0.0 - Time Delta 2.018 - Time 2024-06-25T21:23:13 + Episode 20 - Step 5007 - Epsilon 0.9987490329557962 - Mean Reward 667.429 - Mean Length 238.429 - Mean Loss 0.0 - Mean Q Value 0.0 - Time Delta 59.836 - Time 2024-06-25T21:24:12 + Episode 39 - Step 8854 - Epsilon 0.9977889477081997 - Mean Reward 656.6 - Mean Length 221.35 - Mean Loss 0.0 - Mean Q Value 0.0 - Time Delta 48.305 - Time 2024-06-25T21:25:01 @@ -1012,7 +1012,7 @@ to train an AI to play any of the games at the `OpenAI gym `_. .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 0 minutes 0.064 seconds) + **Total running time of the script:** ( 0 minutes 0.062 seconds) .. _sphx_glr_download_intermediate_memory_format_tutorial.py: diff --git a/_sources/intermediate/model_parallel_tutorial.rst.txt b/_sources/intermediate/model_parallel_tutorial.rst.txt index 21996ae957..0c2e3777bf 100644 --- a/_sources/intermediate/model_parallel_tutorial.rst.txt +++ b/_sources/intermediate/model_parallel_tutorial.rst.txt @@ -485,7 +485,7 @@ inputs. .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 2 minutes 42.800 seconds) + **Total running time of the script:** ( 2 minutes 42.102 seconds) .. _sphx_glr_download_intermediate_model_parallel_tutorial.py: diff --git a/_sources/intermediate/neural_tangent_kernels.rst.txt b/_sources/intermediate/neural_tangent_kernels.rst.txt index 5601466d7f..253e162bea 100644 --- a/_sources/intermediate/neural_tangent_kernels.rst.txt +++ b/_sources/intermediate/neural_tangent_kernels.rst.txt @@ -360,7 +360,7 @@ for details. .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 0 minutes 0.538 seconds) + **Total running time of the script:** ( 0 minutes 0.528 seconds) .. _sphx_glr_download_intermediate_neural_tangent_kernels.py: diff --git a/_sources/intermediate/optimizer_step_in_backward_tutorial.rst.txt b/_sources/intermediate/optimizer_step_in_backward_tutorial.rst.txt index 9aba214f91..1a0cf03afe 100644 --- a/_sources/intermediate/optimizer_step_in_backward_tutorial.rst.txt +++ b/_sources/intermediate/optimizer_step_in_backward_tutorial.rst.txt @@ -68,127 +68,34 @@ but, again, feel free to substitute with your own optimizer. Downloading: "https://download.pytorch.org/models/vit_l_16-852ce7e3.pth" to /var/lib/ci-user/.cache/torch/hub/checkpoints/vit_l_16-852ce7e3.pth 0%| | 0.00/1.13G [00:00 + Per-sample-grads without vmap compute_sample_grads(data, targets) - 57.70 ms + 57.50 ms 1 measurement, 100 runs , 1 thread - Per-sample-grads with vmap + Per-sample-grads with vmap ft_compute_sample_grad(params, buffers, data, targets) - 2.88 ms + 2.84 ms 1 measurement, 100 runs , 1 thread - Performance delta: 1904.7805 percent improvement with vmap + Performance delta: 1921.5607 percent improvement with vmap @@ -419,7 +419,7 @@ at on GitHub. .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 0 minutes 7.223 seconds) + **Total running time of the script:** ( 0 minutes 7.194 seconds) .. _sphx_glr_download_intermediate_per_sample_grads.py: diff --git a/_sources/intermediate/pruning_tutorial.rst.txt b/_sources/intermediate/pruning_tutorial.rst.txt index 64dc2ac572..ed511ba664 100644 --- a/_sources/intermediate/pruning_tutorial.rst.txt +++ b/_sources/intermediate/pruning_tutorial.rst.txt @@ -459,7 +459,7 @@ present. .. code-block:: none - OrderedDict([(3, )]) + OrderedDict([(3, )]) @@ -652,7 +652,7 @@ module attributes, and the module will now have two ``forward_pre_hooks``. .. code-block:: none - OrderedDict([(3, ), (4, )]) + OrderedDict([(3, ), (4, )]) @@ -762,7 +762,7 @@ pruning applied to the ``weight`` parameter. .. code-block:: none - [, ] + [, ] @@ -1359,7 +1359,7 @@ Let's try it out! .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 0 minutes 0.218 seconds) + **Total running time of the script:** ( 0 minutes 0.211 seconds) .. _sphx_glr_download_intermediate_pruning_tutorial.py: diff --git a/_sources/intermediate/reinforcement_ppo.rst.txt b/_sources/intermediate/reinforcement_ppo.rst.txt index cc55e6c1c4..db25f70932 100644 --- a/_sources/intermediate/reinforcement_ppo.rst.txt +++ b/_sources/intermediate/reinforcement_ppo.rst.txt @@ -1046,106 +1046,106 @@ The steps include: 0%| | 0/50000 [00:00 + @@ -124,7 +124,7 @@ network. Downloading https://ossci-datasets.s3.amazonaws.com/mnist/train-images-idx3-ubyte.gz to ./MNIST/raw/train-images-idx3-ubyte.gz 0%| | 0/9912422 [00:00 ([l__self___features_pool0], 1) {} + call_function concated_features ([l__self___features_pool0], 1) {} call_module l__self___features_denseblock1_denselayer1_norm1 L__self___features_denseblock1_denselayer1_norm1 (concated_features,) {} call_module l__self___features_denseblock1_denselayer1_relu1 L__self___features_denseblock1_denselayer1_relu1 (l__self___features_denseblock1_denselayer1_norm1,) {} call_module bottleneck_output L__self___features_denseblock1_denselayer1_conv1 (l__self___features_denseblock1_denselayer1_relu1,) {} call_module l__self___features_denseblock1_denselayer1_norm2 L__self___features_denseblock1_denselayer1_norm2 (bottleneck_output,) {} call_module l__self___features_denseblock1_denselayer1_relu2 L__self___features_denseblock1_denselayer1_relu2 (l__self___features_denseblock1_denselayer1_norm2,) {} call_module new_features L__self___features_denseblock1_denselayer1_conv2 (l__self___features_denseblock1_denselayer1_relu2,) {} - call_function concated_features_1 ([l__self___features_pool0, new_features], 1) {} + call_function concated_features_1 ([l__self___features_pool0, new_features], 1) {} call_module l__self___features_denseblock1_denselayer2_norm1 L__self___features_denseblock1_denselayer2_norm1 (concated_features_1,) {} call_module l__self___features_denseblock1_denselayer2_relu1 L__self___features_denseblock1_denselayer2_relu1 (l__self___features_denseblock1_denselayer2_norm1,) {} call_module bottleneck_output_2 L__self___features_denseblock1_denselayer2_conv1 (l__self___features_denseblock1_denselayer2_relu1,) {} call_module l__self___features_denseblock1_denselayer2_norm2 L__self___features_denseblock1_denselayer2_norm2 (bottleneck_output_2,) {} call_module l__self___features_denseblock1_denselayer2_relu2 L__self___features_denseblock1_denselayer2_relu2 (l__self___features_denseblock1_denselayer2_norm2,) {} call_module new_features_2 L__self___features_denseblock1_denselayer2_conv2 (l__self___features_denseblock1_denselayer2_relu2,) {} - call_function concated_features_2 ([l__self___features_pool0, new_features, new_features_2], 1) {} + call_function concated_features_2 ([l__self___features_pool0, new_features, new_features_2], 1) {} call_module l__self___features_denseblock1_denselayer3_norm1 L__self___features_denseblock1_denselayer3_norm1 (concated_features_2,) {} call_module l__self___features_denseblock1_denselayer3_relu1 L__self___features_denseblock1_denselayer3_relu1 (l__self___features_denseblock1_denselayer3_norm1,) {} call_module bottleneck_output_4 L__self___features_denseblock1_denselayer3_conv1 (l__self___features_denseblock1_denselayer3_relu1,) {} call_module l__self___features_denseblock1_denselayer3_norm2 L__self___features_denseblock1_denselayer3_norm2 (bottleneck_output_4,) {} call_module l__self___features_denseblock1_denselayer3_relu2 L__self___features_denseblock1_denselayer3_relu2 (l__self___features_denseblock1_denselayer3_norm2,) {} call_module new_features_4 L__self___features_denseblock1_denselayer3_conv2 (l__self___features_denseblock1_denselayer3_relu2,) {} - call_function concated_features_3 ([l__self___features_pool0, new_features, new_features_2, new_features_4], 1) {} + call_function concated_features_3 ([l__self___features_pool0, new_features, new_features_2, new_features_4], 1) {} call_module l__self___features_denseblock1_denselayer4_norm1 L__self___features_denseblock1_denselayer4_norm1 (concated_features_3,) {} call_module l__self___features_denseblock1_denselayer4_relu1 L__self___features_denseblock1_denselayer4_relu1 (l__self___features_denseblock1_denselayer4_norm1,) {} call_module bottleneck_output_6 L__self___features_denseblock1_denselayer4_conv1 (l__self___features_denseblock1_denselayer4_relu1,) {} call_module l__self___features_denseblock1_denselayer4_norm2 L__self___features_denseblock1_denselayer4_norm2 (bottleneck_output_6,) {} call_module l__self___features_denseblock1_denselayer4_relu2 L__self___features_denseblock1_denselayer4_relu2 (l__self___features_denseblock1_denselayer4_norm2,) {} call_module new_features_6 L__self___features_denseblock1_denselayer4_conv2 (l__self___features_denseblock1_denselayer4_relu2,) {} - call_function concated_features_4 ([l__self___features_pool0, new_features, new_features_2, new_features_4, new_features_6], 1) {} + call_function concated_features_4 ([l__self___features_pool0, new_features, new_features_2, new_features_4, new_features_6], 1) {} call_module l__self___features_denseblock1_denselayer5_norm1 L__self___features_denseblock1_denselayer5_norm1 (concated_features_4,) {} call_module l__self___features_denseblock1_denselayer5_relu1 L__self___features_denseblock1_denselayer5_relu1 (l__self___features_denseblock1_denselayer5_norm1,) {} call_module bottleneck_output_8 L__self___features_denseblock1_denselayer5_conv1 (l__self___features_denseblock1_denselayer5_relu1,) {} call_module l__self___features_denseblock1_denselayer5_norm2 L__self___features_denseblock1_denselayer5_norm2 (bottleneck_output_8,) {} call_module l__self___features_denseblock1_denselayer5_relu2 L__self___features_denseblock1_denselayer5_relu2 (l__self___features_denseblock1_denselayer5_norm2,) {} call_module new_features_8 L__self___features_denseblock1_denselayer5_conv2 (l__self___features_denseblock1_denselayer5_relu2,) {} - call_function concated_features_5 ([l__self___features_pool0, new_features, new_features_2, new_features_4, new_features_6, new_features_8], 1) {} + call_function concated_features_5 ([l__self___features_pool0, new_features, new_features_2, new_features_4, new_features_6, new_features_8], 1) {} call_module l__self___features_denseblock1_denselayer6_norm1 L__self___features_denseblock1_denselayer6_norm1 (concated_features_5,) {} call_module l__self___features_denseblock1_denselayer6_relu1 L__self___features_denseblock1_denselayer6_relu1 (l__self___features_denseblock1_denselayer6_norm1,) {} call_module bottleneck_output_10 L__self___features_denseblock1_denselayer6_conv1 (l__self___features_denseblock1_denselayer6_relu1,) {} call_module l__self___features_denseblock1_denselayer6_norm2 L__self___features_denseblock1_denselayer6_norm2 (bottleneck_output_10,) {} call_module l__self___features_denseblock1_denselayer6_relu2 L__self___features_denseblock1_denselayer6_relu2 (l__self___features_denseblock1_denselayer6_norm2,) {} call_module new_features_10 L__self___features_denseblock1_denselayer6_conv2 (l__self___features_denseblock1_denselayer6_relu2,) {} - call_function cat_6 ([l__self___features_pool0, new_features, new_features_2, new_features_4, new_features_6, new_features_8, new_features_10], 1) {} + call_function cat_6 ([l__self___features_pool0, new_features, new_features_2, new_features_4, new_features_6, new_features_8, new_features_10], 1) {} call_module l__self___features_transition1_norm L__self___features_transition1_norm (cat_6,) {} call_module l__self___features_transition1_relu L__self___features_transition1_relu (l__self___features_transition1_norm,) {} call_module l__self___features_transition1_conv L__self___features_transition1_conv (l__self___features_transition1_relu,) {} call_module l__self___features_transition1_pool L__self___features_transition1_pool (l__self___features_transition1_conv,) {} - call_function concated_features_6 ([l__self___features_transition1_pool], 1) {} + call_function concated_features_6 ([l__self___features_transition1_pool], 1) {} call_module l__self___features_denseblock2_denselayer1_norm1 L__self___features_denseblock2_denselayer1_norm1 (concated_features_6,) {} call_module l__self___features_denseblock2_denselayer1_relu1 L__self___features_denseblock2_denselayer1_relu1 (l__self___features_denseblock2_denselayer1_norm1,) {} call_module bottleneck_output_12 L__self___features_denseblock2_denselayer1_conv1 (l__self___features_denseblock2_denselayer1_relu1,) {} call_module l__self___features_denseblock2_denselayer1_norm2 L__self___features_denseblock2_denselayer1_norm2 (bottleneck_output_12,) {} call_module l__self___features_denseblock2_denselayer1_relu2 L__self___features_denseblock2_denselayer1_relu2 (l__self___features_denseblock2_denselayer1_norm2,) {} call_module new_features_12 L__self___features_denseblock2_denselayer1_conv2 (l__self___features_denseblock2_denselayer1_relu2,) {} - call_function concated_features_7 ([l__self___features_transition1_pool, new_features_12], 1) {} + call_function concated_features_7 ([l__self___features_transition1_pool, new_features_12], 1) {} call_module l__self___features_denseblock2_denselayer2_norm1 L__self___features_denseblock2_denselayer2_norm1 (concated_features_7,) {} call_module l__self___features_denseblock2_denselayer2_relu1 L__self___features_denseblock2_denselayer2_relu1 (l__self___features_denseblock2_denselayer2_norm1,) {} call_module bottleneck_output_14 L__self___features_denseblock2_denselayer2_conv1 (l__self___features_denseblock2_denselayer2_relu1,) {} call_module l__self___features_denseblock2_denselayer2_norm2 L__self___features_denseblock2_denselayer2_norm2 (bottleneck_output_14,) {} call_module l__self___features_denseblock2_denselayer2_relu2 L__self___features_denseblock2_denselayer2_relu2 (l__self___features_denseblock2_denselayer2_norm2,) {} call_module new_features_14 L__self___features_denseblock2_denselayer2_conv2 (l__self___features_denseblock2_denselayer2_relu2,) {} - call_function concated_features_8 ([l__self___features_transition1_pool, new_features_12, new_features_14], 1) {} + call_function concated_features_8 ([l__self___features_transition1_pool, new_features_12, new_features_14], 1) {} call_module l__self___features_denseblock2_denselayer3_norm1 L__self___features_denseblock2_denselayer3_norm1 (concated_features_8,) {} call_module l__self___features_denseblock2_denselayer3_relu1 L__self___features_denseblock2_denselayer3_relu1 (l__self___features_denseblock2_denselayer3_norm1,) {} call_module bottleneck_output_16 L__self___features_denseblock2_denselayer3_conv1 (l__self___features_denseblock2_denselayer3_relu1,) {} call_module l__self___features_denseblock2_denselayer3_norm2 L__self___features_denseblock2_denselayer3_norm2 (bottleneck_output_16,) {} call_module l__self___features_denseblock2_denselayer3_relu2 L__self___features_denseblock2_denselayer3_relu2 (l__self___features_denseblock2_denselayer3_norm2,) {} call_module new_features_16 L__self___features_denseblock2_denselayer3_conv2 (l__self___features_denseblock2_denselayer3_relu2,) {} - call_function concated_features_9 ([l__self___features_transition1_pool, new_features_12, new_features_14, new_features_16], 1) {} + call_function concated_features_9 ([l__self___features_transition1_pool, new_features_12, new_features_14, new_features_16], 1) {} call_module l__self___features_denseblock2_denselayer4_norm1 L__self___features_denseblock2_denselayer4_norm1 (concated_features_9,) {} call_module l__self___features_denseblock2_denselayer4_relu1 L__self___features_denseblock2_denselayer4_relu1 (l__self___features_denseblock2_denselayer4_norm1,) {} call_module bottleneck_output_18 L__self___features_denseblock2_denselayer4_conv1 (l__self___features_denseblock2_denselayer4_relu1,) {} call_module l__self___features_denseblock2_denselayer4_norm2 L__self___features_denseblock2_denselayer4_norm2 (bottleneck_output_18,) {} call_module l__self___features_denseblock2_denselayer4_relu2 L__self___features_denseblock2_denselayer4_relu2 (l__self___features_denseblock2_denselayer4_norm2,) {} call_module new_features_18 L__self___features_denseblock2_denselayer4_conv2 (l__self___features_denseblock2_denselayer4_relu2,) {} - call_function concated_features_10 ([l__self___features_transition1_pool, new_features_12, new_features_14, new_features_16, new_features_18], 1) {} + call_function concated_features_10 ([l__self___features_transition1_pool, new_features_12, new_features_14, new_features_16, new_features_18], 1) {} call_module l__self___features_denseblock2_denselayer5_norm1 L__self___features_denseblock2_denselayer5_norm1 (concated_features_10,) {} call_module l__self___features_denseblock2_denselayer5_relu1 L__self___features_denseblock2_denselayer5_relu1 (l__self___features_denseblock2_denselayer5_norm1,) {} call_module bottleneck_output_20 L__self___features_denseblock2_denselayer5_conv1 (l__self___features_denseblock2_denselayer5_relu1,) {} call_module l__self___features_denseblock2_denselayer5_norm2 L__self___features_denseblock2_denselayer5_norm2 (bottleneck_output_20,) {} call_module l__self___features_denseblock2_denselayer5_relu2 L__self___features_denseblock2_denselayer5_relu2 (l__self___features_denseblock2_denselayer5_norm2,) {} call_module new_features_20 L__self___features_denseblock2_denselayer5_conv2 (l__self___features_denseblock2_denselayer5_relu2,) {} - call_function concated_features_11 ([l__self___features_transition1_pool, new_features_12, new_features_14, new_features_16, new_features_18, new_features_20], 1) {} + call_function concated_features_11 ([l__self___features_transition1_pool, new_features_12, new_features_14, new_features_16, new_features_18, new_features_20], 1) {} call_module l__self___features_denseblock2_denselayer6_norm1 L__self___features_denseblock2_denselayer6_norm1 (concated_features_11,) {} call_module l__self___features_denseblock2_denselayer6_relu1 L__self___features_denseblock2_denselayer6_relu1 (l__self___features_denseblock2_denselayer6_norm1,) {} call_module bottleneck_output_22 L__self___features_denseblock2_denselayer6_conv1 (l__self___features_denseblock2_denselayer6_relu1,) {} call_module l__self___features_denseblock2_denselayer6_norm2 L__self___features_denseblock2_denselayer6_norm2 (bottleneck_output_22,) {} call_module l__self___features_denseblock2_denselayer6_relu2 L__self___features_denseblock2_denselayer6_relu2 (l__self___features_denseblock2_denselayer6_norm2,) {} call_module new_features_22 L__self___features_denseblock2_denselayer6_conv2 (l__self___features_denseblock2_denselayer6_relu2,) {} - call_function concated_features_12 ([l__self___features_transition1_pool, new_features_12, new_features_14, new_features_16, new_features_18, new_features_20, new_features_22], 1) {} + call_function concated_features_12 ([l__self___features_transition1_pool, new_features_12, new_features_14, new_features_16, new_features_18, new_features_20, new_features_22], 1) {} call_module l__self___features_denseblock2_denselayer7_norm1 L__self___features_denseblock2_denselayer7_norm1 (concated_features_12,) {} call_module l__self___features_denseblock2_denselayer7_relu1 L__self___features_denseblock2_denselayer7_relu1 (l__self___features_denseblock2_denselayer7_norm1,) {} call_module bottleneck_output_24 L__self___features_denseblock2_denselayer7_conv1 (l__self___features_denseblock2_denselayer7_relu1,) {} call_module l__self___features_denseblock2_denselayer7_norm2 L__self___features_denseblock2_denselayer7_norm2 (bottleneck_output_24,) {} call_module l__self___features_denseblock2_denselayer7_relu2 L__self___features_denseblock2_denselayer7_relu2 (l__self___features_denseblock2_denselayer7_norm2,) {} call_module new_features_24 L__self___features_denseblock2_denselayer7_conv2 (l__self___features_denseblock2_denselayer7_relu2,) {} - call_function concated_features_13 ([l__self___features_transition1_pool, new_features_12, new_features_14, new_features_16, new_features_18, new_features_20, new_features_22, new_features_24], 1) {} + call_function concated_features_13 ([l__self___features_transition1_pool, new_features_12, new_features_14, new_features_16, new_features_18, new_features_20, new_features_22, new_features_24], 1) {} call_module l__self___features_denseblock2_denselayer8_norm1 L__self___features_denseblock2_denselayer8_norm1 (concated_features_13,) {} call_module l__self___features_denseblock2_denselayer8_relu1 L__self___features_denseblock2_denselayer8_relu1 (l__self___features_denseblock2_denselayer8_norm1,) {} call_module bottleneck_output_26 L__self___features_denseblock2_denselayer8_conv1 (l__self___features_denseblock2_denselayer8_relu1,) {} call_module l__self___features_denseblock2_denselayer8_norm2 L__self___features_denseblock2_denselayer8_norm2 (bottleneck_output_26,) {} call_module l__self___features_denseblock2_denselayer8_relu2 L__self___features_denseblock2_denselayer8_relu2 (l__self___features_denseblock2_denselayer8_norm2,) {} call_module new_features_26 L__self___features_denseblock2_denselayer8_conv2 (l__self___features_denseblock2_denselayer8_relu2,) {} - call_function concated_features_14 ([l__self___features_transition1_pool, new_features_12, new_features_14, new_features_16, new_features_18, new_features_20, new_features_22, new_features_24, new_features_26], 1) {} + call_function concated_features_14 ([l__self___features_transition1_pool, new_features_12, new_features_14, new_features_16, new_features_18, new_features_20, new_features_22, new_features_24, new_features_26], 1) {} call_module l__self___features_denseblock2_denselayer9_norm1 L__self___features_denseblock2_denselayer9_norm1 (concated_features_14,) {} call_module l__self___features_denseblock2_denselayer9_relu1 L__self___features_denseblock2_denselayer9_relu1 (l__self___features_denseblock2_denselayer9_norm1,) {} call_module bottleneck_output_28 L__self___features_denseblock2_denselayer9_conv1 (l__self___features_denseblock2_denselayer9_relu1,) {} call_module l__self___features_denseblock2_denselayer9_norm2 L__self___features_denseblock2_denselayer9_norm2 (bottleneck_output_28,) {} call_module l__self___features_denseblock2_denselayer9_relu2 L__self___features_denseblock2_denselayer9_relu2 (l__self___features_denseblock2_denselayer9_norm2,) {} call_module new_features_28 L__self___features_denseblock2_denselayer9_conv2 (l__self___features_denseblock2_denselayer9_relu2,) {} - call_function concated_features_15 ([l__self___features_transition1_pool, new_features_12, new_features_14, new_features_16, new_features_18, new_features_20, new_features_22, new_features_24, new_features_26, new_features_28], 1) {} + call_function concated_features_15 ([l__self___features_transition1_pool, new_features_12, new_features_14, new_features_16, new_features_18, new_features_20, new_features_22, new_features_24, new_features_26, new_features_28], 1) {} call_module l__self___features_denseblock2_denselayer10_norm1 L__self___features_denseblock2_denselayer10_norm1 (concated_features_15,) {} call_module l__self___features_denseblock2_denselayer10_relu1 L__self___features_denseblock2_denselayer10_relu1 (l__self___features_denseblock2_denselayer10_norm1,) {} call_module bottleneck_output_30 L__self___features_denseblock2_denselayer10_conv1 (l__self___features_denseblock2_denselayer10_relu1,) {} call_module l__self___features_denseblock2_denselayer10_norm2 L__self___features_denseblock2_denselayer10_norm2 (bottleneck_output_30,) {} call_module l__self___features_denseblock2_denselayer10_relu2 L__self___features_denseblock2_denselayer10_relu2 (l__self___features_denseblock2_denselayer10_norm2,) {} call_module new_features_30 L__self___features_denseblock2_denselayer10_conv2 (l__self___features_denseblock2_denselayer10_relu2,) {} - call_function concated_features_16 ([l__self___features_transition1_pool, new_features_12, new_features_14, new_features_16, new_features_18, new_features_20, new_features_22, new_features_24, new_features_26, new_features_28, new_features_30], 1) {} + call_function concated_features_16 ([l__self___features_transition1_pool, new_features_12, new_features_14, new_features_16, new_features_18, new_features_20, new_features_22, new_features_24, new_features_26, new_features_28, new_features_30], 1) {} call_module l__self___features_denseblock2_denselayer11_norm1 L__self___features_denseblock2_denselayer11_norm1 (concated_features_16,) {} call_module l__self___features_denseblock2_denselayer11_relu1 L__self___features_denseblock2_denselayer11_relu1 (l__self___features_denseblock2_denselayer11_norm1,) {} call_module bottleneck_output_32 L__self___features_denseblock2_denselayer11_conv1 (l__self___features_denseblock2_denselayer11_relu1,) {} call_module l__self___features_denseblock2_denselayer11_norm2 L__self___features_denseblock2_denselayer11_norm2 (bottleneck_output_32,) {} call_module l__self___features_denseblock2_denselayer11_relu2 L__self___features_denseblock2_denselayer11_relu2 (l__self___features_denseblock2_denselayer11_norm2,) {} call_module new_features_32 L__self___features_denseblock2_denselayer11_conv2 (l__self___features_denseblock2_denselayer11_relu2,) {} - call_function concated_features_17 ([l__self___features_transition1_pool, new_features_12, new_features_14, new_features_16, new_features_18, new_features_20, new_features_22, new_features_24, new_features_26, new_features_28, new_features_30, new_features_32], 1) {} + call_function concated_features_17 ([l__self___features_transition1_pool, new_features_12, new_features_14, new_features_16, new_features_18, new_features_20, new_features_22, new_features_24, new_features_26, new_features_28, new_features_30, new_features_32], 1) {} call_module l__self___features_denseblock2_denselayer12_norm1 L__self___features_denseblock2_denselayer12_norm1 (concated_features_17,) {} call_module l__self___features_denseblock2_denselayer12_relu1 L__self___features_denseblock2_denselayer12_relu1 (l__self___features_denseblock2_denselayer12_norm1,) {} call_module bottleneck_output_34 L__self___features_denseblock2_denselayer12_conv1 (l__self___features_denseblock2_denselayer12_relu1,) {} call_module l__self___features_denseblock2_denselayer12_norm2 L__self___features_denseblock2_denselayer12_norm2 (bottleneck_output_34,) {} call_module l__self___features_denseblock2_denselayer12_relu2 L__self___features_denseblock2_denselayer12_relu2 (l__self___features_denseblock2_denselayer12_norm2,) {} call_module new_features_34 L__self___features_denseblock2_denselayer12_conv2 (l__self___features_denseblock2_denselayer12_relu2,) {} - call_function cat_19 ([l__self___features_transition1_pool, new_features_12, new_features_14, new_features_16, new_features_18, new_features_20, new_features_22, new_features_24, new_features_26, new_features_28, new_features_30, new_features_32, new_features_34], 1) {} + call_function cat_19 ([l__self___features_transition1_pool, new_features_12, new_features_14, new_features_16, new_features_18, new_features_20, new_features_22, new_features_24, new_features_26, new_features_28, new_features_30, new_features_32, new_features_34], 1) {} call_module l__self___features_transition2_norm L__self___features_transition2_norm (cat_19,) {} call_module l__self___features_transition2_relu L__self___features_transition2_relu (l__self___features_transition2_norm,) {} call_module l__self___features_transition2_conv L__self___features_transition2_conv (l__self___features_transition2_relu,) {} call_module l__self___features_transition2_pool L__self___features_transition2_pool (l__self___features_transition2_conv,) {} - call_function concated_features_18 ([l__self___features_transition2_pool], 1) {} + call_function concated_features_18 ([l__self___features_transition2_pool], 1) {} call_module l__self___features_denseblock3_denselayer1_norm1 L__self___features_denseblock3_denselayer1_norm1 (concated_features_18,) {} call_module l__self___features_denseblock3_denselayer1_relu1 L__self___features_denseblock3_denselayer1_relu1 (l__self___features_denseblock3_denselayer1_norm1,) {} call_module bottleneck_output_36 L__self___features_denseblock3_denselayer1_conv1 (l__self___features_denseblock3_denselayer1_relu1,) {} call_module l__self___features_denseblock3_denselayer1_norm2 L__self___features_denseblock3_denselayer1_norm2 (bottleneck_output_36,) {} call_module l__self___features_denseblock3_denselayer1_relu2 L__self___features_denseblock3_denselayer1_relu2 (l__self___features_denseblock3_denselayer1_norm2,) {} call_module new_features_36 L__self___features_denseblock3_denselayer1_conv2 (l__self___features_denseblock3_denselayer1_relu2,) {} - call_function concated_features_19 ([l__self___features_transition2_pool, new_features_36], 1) {} + call_function concated_features_19 ([l__self___features_transition2_pool, new_features_36], 1) {} call_module l__self___features_denseblock3_denselayer2_norm1 L__self___features_denseblock3_denselayer2_norm1 (concated_features_19,) {} call_module l__self___features_denseblock3_denselayer2_relu1 L__self___features_denseblock3_denselayer2_relu1 (l__self___features_denseblock3_denselayer2_norm1,) {} call_module bottleneck_output_38 L__self___features_denseblock3_denselayer2_conv1 (l__self___features_denseblock3_denselayer2_relu1,) {} call_module l__self___features_denseblock3_denselayer2_norm2 L__self___features_denseblock3_denselayer2_norm2 (bottleneck_output_38,) {} call_module l__self___features_denseblock3_denselayer2_relu2 L__self___features_denseblock3_denselayer2_relu2 (l__self___features_denseblock3_denselayer2_norm2,) {} call_module new_features_38 L__self___features_denseblock3_denselayer2_conv2 (l__self___features_denseblock3_denselayer2_relu2,) {} - call_function concated_features_20 ([l__self___features_transition2_pool, new_features_36, new_features_38], 1) {} + call_function concated_features_20 ([l__self___features_transition2_pool, new_features_36, new_features_38], 1) {} call_module l__self___features_denseblock3_denselayer3_norm1 L__self___features_denseblock3_denselayer3_norm1 (concated_features_20,) {} call_module l__self___features_denseblock3_denselayer3_relu1 L__self___features_denseblock3_denselayer3_relu1 (l__self___features_denseblock3_denselayer3_norm1,) {} call_module bottleneck_output_40 L__self___features_denseblock3_denselayer3_conv1 (l__self___features_denseblock3_denselayer3_relu1,) {} call_module l__self___features_denseblock3_denselayer3_norm2 L__self___features_denseblock3_denselayer3_norm2 (bottleneck_output_40,) {} call_module l__self___features_denseblock3_denselayer3_relu2 L__self___features_denseblock3_denselayer3_relu2 (l__self___features_denseblock3_denselayer3_norm2,) {} call_module new_features_40 L__self___features_denseblock3_denselayer3_conv2 (l__self___features_denseblock3_denselayer3_relu2,) {} - call_function concated_features_21 ([l__self___features_transition2_pool, new_features_36, new_features_38, new_features_40], 1) {} + call_function concated_features_21 ([l__self___features_transition2_pool, new_features_36, new_features_38, new_features_40], 1) {} call_module l__self___features_denseblock3_denselayer4_norm1 L__self___features_denseblock3_denselayer4_norm1 (concated_features_21,) {} call_module l__self___features_denseblock3_denselayer4_relu1 L__self___features_denseblock3_denselayer4_relu1 (l__self___features_denseblock3_denselayer4_norm1,) {} call_module bottleneck_output_42 L__self___features_denseblock3_denselayer4_conv1 (l__self___features_denseblock3_denselayer4_relu1,) {} call_module l__self___features_denseblock3_denselayer4_norm2 L__self___features_denseblock3_denselayer4_norm2 (bottleneck_output_42,) {} call_module l__self___features_denseblock3_denselayer4_relu2 L__self___features_denseblock3_denselayer4_relu2 (l__self___features_denseblock3_denselayer4_norm2,) {} call_module new_features_42 L__self___features_denseblock3_denselayer4_conv2 (l__self___features_denseblock3_denselayer4_relu2,) {} - call_function concated_features_22 ([l__self___features_transition2_pool, new_features_36, new_features_38, new_features_40, new_features_42], 1) {} + call_function concated_features_22 ([l__self___features_transition2_pool, new_features_36, new_features_38, new_features_40, new_features_42], 1) {} call_module l__self___features_denseblock3_denselayer5_norm1 L__self___features_denseblock3_denselayer5_norm1 (concated_features_22,) {} call_module l__self___features_denseblock3_denselayer5_relu1 L__self___features_denseblock3_denselayer5_relu1 (l__self___features_denseblock3_denselayer5_norm1,) {} call_module bottleneck_output_44 L__self___features_denseblock3_denselayer5_conv1 (l__self___features_denseblock3_denselayer5_relu1,) {} call_module l__self___features_denseblock3_denselayer5_norm2 L__self___features_denseblock3_denselayer5_norm2 (bottleneck_output_44,) {} call_module l__self___features_denseblock3_denselayer5_relu2 L__self___features_denseblock3_denselayer5_relu2 (l__self___features_denseblock3_denselayer5_norm2,) {} call_module new_features_44 L__self___features_denseblock3_denselayer5_conv2 (l__self___features_denseblock3_denselayer5_relu2,) {} - call_function concated_features_23 ([l__self___features_transition2_pool, new_features_36, new_features_38, new_features_40, new_features_42, new_features_44], 1) {} + call_function concated_features_23 ([l__self___features_transition2_pool, new_features_36, new_features_38, new_features_40, new_features_42, new_features_44], 1) {} call_module l__self___features_denseblock3_denselayer6_norm1 L__self___features_denseblock3_denselayer6_norm1 (concated_features_23,) {} call_module l__self___features_denseblock3_denselayer6_relu1 L__self___features_denseblock3_denselayer6_relu1 (l__self___features_denseblock3_denselayer6_norm1,) {} call_module bottleneck_output_46 L__self___features_denseblock3_denselayer6_conv1 (l__self___features_denseblock3_denselayer6_relu1,) {} call_module l__self___features_denseblock3_denselayer6_norm2 L__self___features_denseblock3_denselayer6_norm2 (bottleneck_output_46,) {} call_module l__self___features_denseblock3_denselayer6_relu2 L__self___features_denseblock3_denselayer6_relu2 (l__self___features_denseblock3_denselayer6_norm2,) {} call_module new_features_46 L__self___features_denseblock3_denselayer6_conv2 (l__self___features_denseblock3_denselayer6_relu2,) {} - call_function concated_features_24 ([l__self___features_transition2_pool, new_features_36, new_features_38, new_features_40, new_features_42, new_features_44, new_features_46], 1) {} + call_function concated_features_24 ([l__self___features_transition2_pool, new_features_36, new_features_38, new_features_40, new_features_42, new_features_44, new_features_46], 1) {} call_module l__self___features_denseblock3_denselayer7_norm1 L__self___features_denseblock3_denselayer7_norm1 (concated_features_24,) {} call_module l__self___features_denseblock3_denselayer7_relu1 L__self___features_denseblock3_denselayer7_relu1 (l__self___features_denseblock3_denselayer7_norm1,) {} call_module bottleneck_output_48 L__self___features_denseblock3_denselayer7_conv1 (l__self___features_denseblock3_denselayer7_relu1,) {} call_module l__self___features_denseblock3_denselayer7_norm2 L__self___features_denseblock3_denselayer7_norm2 (bottleneck_output_48,) {} call_module l__self___features_denseblock3_denselayer7_relu2 L__self___features_denseblock3_denselayer7_relu2 (l__self___features_denseblock3_denselayer7_norm2,) {} call_module new_features_48 L__self___features_denseblock3_denselayer7_conv2 (l__self___features_denseblock3_denselayer7_relu2,) {} - call_function concated_features_25 ([l__self___features_transition2_pool, new_features_36, new_features_38, new_features_40, new_features_42, new_features_44, new_features_46, new_features_48], 1) {} + call_function concated_features_25 ([l__self___features_transition2_pool, new_features_36, new_features_38, new_features_40, new_features_42, new_features_44, new_features_46, new_features_48], 1) {} call_module l__self___features_denseblock3_denselayer8_norm1 L__self___features_denseblock3_denselayer8_norm1 (concated_features_25,) {} call_module l__self___features_denseblock3_denselayer8_relu1 L__self___features_denseblock3_denselayer8_relu1 (l__self___features_denseblock3_denselayer8_norm1,) {} call_module bottleneck_output_50 L__self___features_denseblock3_denselayer8_conv1 (l__self___features_denseblock3_denselayer8_relu1,) {} call_module l__self___features_denseblock3_denselayer8_norm2 L__self___features_denseblock3_denselayer8_norm2 (bottleneck_output_50,) {} call_module l__self___features_denseblock3_denselayer8_relu2 L__self___features_denseblock3_denselayer8_relu2 (l__self___features_denseblock3_denselayer8_norm2,) {} call_module new_features_50 L__self___features_denseblock3_denselayer8_conv2 (l__self___features_denseblock3_denselayer8_relu2,) {} - call_function concated_features_26 ([l__self___features_transition2_pool, new_features_36, new_features_38, new_features_40, new_features_42, new_features_44, new_features_46, new_features_48, new_features_50], 1) {} + call_function concated_features_26 ([l__self___features_transition2_pool, new_features_36, new_features_38, new_features_40, new_features_42, new_features_44, new_features_46, new_features_48, new_features_50], 1) {} call_module l__self___features_denseblock3_denselayer9_norm1 L__self___features_denseblock3_denselayer9_norm1 (concated_features_26,) {} call_module l__self___features_denseblock3_denselayer9_relu1 L__self___features_denseblock3_denselayer9_relu1 (l__self___features_denseblock3_denselayer9_norm1,) {} call_module bottleneck_output_52 L__self___features_denseblock3_denselayer9_conv1 (l__self___features_denseblock3_denselayer9_relu1,) {} call_module l__self___features_denseblock3_denselayer9_norm2 L__self___features_denseblock3_denselayer9_norm2 (bottleneck_output_52,) {} call_module l__self___features_denseblock3_denselayer9_relu2 L__self___features_denseblock3_denselayer9_relu2 (l__self___features_denseblock3_denselayer9_norm2,) {} call_module new_features_52 L__self___features_denseblock3_denselayer9_conv2 (l__self___features_denseblock3_denselayer9_relu2,) {} - call_function concated_features_27 ([l__self___features_transition2_pool, new_features_36, new_features_38, new_features_40, new_features_42, new_features_44, new_features_46, new_features_48, new_features_50, new_features_52], 1) {} + call_function concated_features_27 ([l__self___features_transition2_pool, new_features_36, new_features_38, new_features_40, new_features_42, new_features_44, new_features_46, new_features_48, new_features_50, new_features_52], 1) {} call_module l__self___features_denseblock3_denselayer10_norm1 L__self___features_denseblock3_denselayer10_norm1 (concated_features_27,) {} call_module l__self___features_denseblock3_denselayer10_relu1 L__self___features_denseblock3_denselayer10_relu1 (l__self___features_denseblock3_denselayer10_norm1,) {} call_module bottleneck_output_54 L__self___features_denseblock3_denselayer10_conv1 (l__self___features_denseblock3_denselayer10_relu1,) {} call_module l__self___features_denseblock3_denselayer10_norm2 L__self___features_denseblock3_denselayer10_norm2 (bottleneck_output_54,) {} call_module l__self___features_denseblock3_denselayer10_relu2 L__self___features_denseblock3_denselayer10_relu2 (l__self___features_denseblock3_denselayer10_norm2,) {} call_module new_features_54 L__self___features_denseblock3_denselayer10_conv2 (l__self___features_denseblock3_denselayer10_relu2,) {} - call_function concated_features_28 ([l__self___features_transition2_pool, new_features_36, new_features_38, new_features_40, new_features_42, new_features_44, new_features_46, new_features_48, new_features_50, new_features_52, new_features_54], 1) {} + call_function concated_features_28 ([l__self___features_transition2_pool, new_features_36, new_features_38, new_features_40, new_features_42, new_features_44, new_features_46, new_features_48, new_features_50, new_features_52, new_features_54], 1) {} call_module l__self___features_denseblock3_denselayer11_norm1 L__self___features_denseblock3_denselayer11_norm1 (concated_features_28,) {} call_module l__self___features_denseblock3_denselayer11_relu1 L__self___features_denseblock3_denselayer11_relu1 (l__self___features_denseblock3_denselayer11_norm1,) {} call_module bottleneck_output_56 L__self___features_denseblock3_denselayer11_conv1 (l__self___features_denseblock3_denselayer11_relu1,) {} call_module l__self___features_denseblock3_denselayer11_norm2 L__self___features_denseblock3_denselayer11_norm2 (bottleneck_output_56,) {} call_module l__self___features_denseblock3_denselayer11_relu2 L__self___features_denseblock3_denselayer11_relu2 (l__self___features_denseblock3_denselayer11_norm2,) {} call_module new_features_56 L__self___features_denseblock3_denselayer11_conv2 (l__self___features_denseblock3_denselayer11_relu2,) {} - call_function concated_features_29 ([l__self___features_transition2_pool, new_features_36, new_features_38, new_features_40, new_features_42, new_features_44, new_features_46, new_features_48, new_features_50, new_features_52, new_features_54, new_features_56], 1) {} + call_function concated_features_29 ([l__self___features_transition2_pool, new_features_36, new_features_38, new_features_40, new_features_42, new_features_44, new_features_46, new_features_48, new_features_50, new_features_52, new_features_54, new_features_56], 1) {} call_module l__self___features_denseblock3_denselayer12_norm1 L__self___features_denseblock3_denselayer12_norm1 (concated_features_29,) {} call_module l__self___features_denseblock3_denselayer12_relu1 L__self___features_denseblock3_denselayer12_relu1 (l__self___features_denseblock3_denselayer12_norm1,) {} call_module bottleneck_output_58 L__self___features_denseblock3_denselayer12_conv1 (l__self___features_denseblock3_denselayer12_relu1,) {} call_module l__self___features_denseblock3_denselayer12_norm2 L__self___features_denseblock3_denselayer12_norm2 (bottleneck_output_58,) {} call_module l__self___features_denseblock3_denselayer12_relu2 L__self___features_denseblock3_denselayer12_relu2 (l__self___features_denseblock3_denselayer12_norm2,) {} call_module new_features_58 L__self___features_denseblock3_denselayer12_conv2 (l__self___features_denseblock3_denselayer12_relu2,) {} - call_function concated_features_30 ([l__self___features_transition2_pool, new_features_36, new_features_38, new_features_40, new_features_42, new_features_44, new_features_46, new_features_48, new_features_50, new_features_52, new_features_54, new_features_56, new_features_58], 1) {} + call_function concated_features_30 ([l__self___features_transition2_pool, new_features_36, new_features_38, new_features_40, new_features_42, new_features_44, new_features_46, new_features_48, new_features_50, new_features_52, new_features_54, new_features_56, new_features_58], 1) {} call_module l__self___features_denseblock3_denselayer13_norm1 L__self___features_denseblock3_denselayer13_norm1 (concated_features_30,) {} call_module l__self___features_denseblock3_denselayer13_relu1 L__self___features_denseblock3_denselayer13_relu1 (l__self___features_denseblock3_denselayer13_norm1,) {} call_module bottleneck_output_60 L__self___features_denseblock3_denselayer13_conv1 (l__self___features_denseblock3_denselayer13_relu1,) {} call_module l__self___features_denseblock3_denselayer13_norm2 L__self___features_denseblock3_denselayer13_norm2 (bottleneck_output_60,) {} call_module l__self___features_denseblock3_denselayer13_relu2 L__self___features_denseblock3_denselayer13_relu2 (l__self___features_denseblock3_denselayer13_norm2,) {} call_module new_features_60 L__self___features_denseblock3_denselayer13_conv2 (l__self___features_denseblock3_denselayer13_relu2,) {} - call_function concated_features_31 ([l__self___features_transition2_pool, new_features_36, new_features_38, new_features_40, new_features_42, new_features_44, new_features_46, new_features_48, new_features_50, new_features_52, new_features_54, new_features_56, new_features_58, new_features_60], 1) {} + call_function concated_features_31 ([l__self___features_transition2_pool, new_features_36, new_features_38, new_features_40, new_features_42, new_features_44, new_features_46, new_features_48, new_features_50, new_features_52, new_features_54, new_features_56, new_features_58, new_features_60], 1) {} call_module l__self___features_denseblock3_denselayer14_norm1 L__self___features_denseblock3_denselayer14_norm1 (concated_features_31,) {} call_module l__self___features_denseblock3_denselayer14_relu1 L__self___features_denseblock3_denselayer14_relu1 (l__self___features_denseblock3_denselayer14_norm1,) {} call_module bottleneck_output_62 L__self___features_denseblock3_denselayer14_conv1 (l__self___features_denseblock3_denselayer14_relu1,) {} call_module l__self___features_denseblock3_denselayer14_norm2 L__self___features_denseblock3_denselayer14_norm2 (bottleneck_output_62,) {} call_module l__self___features_denseblock3_denselayer14_relu2 L__self___features_denseblock3_denselayer14_relu2 (l__self___features_denseblock3_denselayer14_norm2,) {} call_module new_features_62 L__self___features_denseblock3_denselayer14_conv2 (l__self___features_denseblock3_denselayer14_relu2,) {} - call_function concated_features_32 ([l__self___features_transition2_pool, new_features_36, new_features_38, new_features_40, new_features_42, new_features_44, new_features_46, new_features_48, new_features_50, new_features_52, new_features_54, new_features_56, new_features_58, new_features_60, new_features_62], 1) {} + call_function concated_features_32 ([l__self___features_transition2_pool, new_features_36, new_features_38, new_features_40, new_features_42, new_features_44, new_features_46, new_features_48, new_features_50, new_features_52, new_features_54, new_features_56, new_features_58, new_features_60, new_features_62], 1) {} call_module l__self___features_denseblock3_denselayer15_norm1 L__self___features_denseblock3_denselayer15_norm1 (concated_features_32,) {} call_module l__self___features_denseblock3_denselayer15_relu1 L__self___features_denseblock3_denselayer15_relu1 (l__self___features_denseblock3_denselayer15_norm1,) {} call_module bottleneck_output_64 L__self___features_denseblock3_denselayer15_conv1 (l__self___features_denseblock3_denselayer15_relu1,) {} call_module l__self___features_denseblock3_denselayer15_norm2 L__self___features_denseblock3_denselayer15_norm2 (bottleneck_output_64,) {} call_module l__self___features_denseblock3_denselayer15_relu2 L__self___features_denseblock3_denselayer15_relu2 (l__self___features_denseblock3_denselayer15_norm2,) {} call_module new_features_64 L__self___features_denseblock3_denselayer15_conv2 (l__self___features_denseblock3_denselayer15_relu2,) {} - call_function concated_features_33 ([l__self___features_transition2_pool, new_features_36, new_features_38, new_features_40, new_features_42, new_features_44, new_features_46, new_features_48, new_features_50, new_features_52, new_features_54, new_features_56, new_features_58, new_features_60, new_features_62, new_features_64], 1) {} + call_function concated_features_33 ([l__self___features_transition2_pool, new_features_36, new_features_38, new_features_40, new_features_42, new_features_44, new_features_46, new_features_48, new_features_50, new_features_52, new_features_54, new_features_56, new_features_58, new_features_60, new_features_62, new_features_64], 1) {} call_module l__self___features_denseblock3_denselayer16_norm1 L__self___features_denseblock3_denselayer16_norm1 (concated_features_33,) {} call_module l__self___features_denseblock3_denselayer16_relu1 L__self___features_denseblock3_denselayer16_relu1 (l__self___features_denseblock3_denselayer16_norm1,) {} call_module bottleneck_output_66 L__self___features_denseblock3_denselayer16_conv1 (l__self___features_denseblock3_denselayer16_relu1,) {} call_module l__self___features_denseblock3_denselayer16_norm2 L__self___features_denseblock3_denselayer16_norm2 (bottleneck_output_66,) {} call_module l__self___features_denseblock3_denselayer16_relu2 L__self___features_denseblock3_denselayer16_relu2 (l__self___features_denseblock3_denselayer16_norm2,) {} call_module new_features_66 L__self___features_denseblock3_denselayer16_conv2 (l__self___features_denseblock3_denselayer16_relu2,) {} - call_function concated_features_34 ([l__self___features_transition2_pool, new_features_36, new_features_38, new_features_40, new_features_42, new_features_44, new_features_46, new_features_48, new_features_50, new_features_52, new_features_54, new_features_56, new_features_58, new_features_60, new_features_62, new_features_64, new_features_66], 1) {} + call_function concated_features_34 ([l__self___features_transition2_pool, new_features_36, new_features_38, new_features_40, new_features_42, new_features_44, new_features_46, new_features_48, new_features_50, new_features_52, new_features_54, new_features_56, new_features_58, new_features_60, new_features_62, new_features_64, new_features_66], 1) {} call_module l__self___features_denseblock3_denselayer17_norm1 L__self___features_denseblock3_denselayer17_norm1 (concated_features_34,) {} call_module l__self___features_denseblock3_denselayer17_relu1 L__self___features_denseblock3_denselayer17_relu1 (l__self___features_denseblock3_denselayer17_norm1,) {} call_module bottleneck_output_68 L__self___features_denseblock3_denselayer17_conv1 (l__self___features_denseblock3_denselayer17_relu1,) {} call_module l__self___features_denseblock3_denselayer17_norm2 L__self___features_denseblock3_denselayer17_norm2 (bottleneck_output_68,) {} call_module l__self___features_denseblock3_denselayer17_relu2 L__self___features_denseblock3_denselayer17_relu2 (l__self___features_denseblock3_denselayer17_norm2,) {} call_module new_features_68 L__self___features_denseblock3_denselayer17_conv2 (l__self___features_denseblock3_denselayer17_relu2,) {} - call_function concated_features_35 ([l__self___features_transition2_pool, new_features_36, new_features_38, new_features_40, new_features_42, new_features_44, new_features_46, new_features_48, new_features_50, new_features_52, new_features_54, new_features_56, new_features_58, new_features_60, new_features_62, new_features_64, new_features_66, new_features_68], 1) {} + call_function concated_features_35 ([l__self___features_transition2_pool, new_features_36, new_features_38, new_features_40, new_features_42, new_features_44, new_features_46, new_features_48, new_features_50, new_features_52, new_features_54, new_features_56, new_features_58, new_features_60, new_features_62, new_features_64, new_features_66, new_features_68], 1) {} call_module l__self___features_denseblock3_denselayer18_norm1 L__self___features_denseblock3_denselayer18_norm1 (concated_features_35,) {} call_module l__self___features_denseblock3_denselayer18_relu1 L__self___features_denseblock3_denselayer18_relu1 (l__self___features_denseblock3_denselayer18_norm1,) {} call_module bottleneck_output_70 L__self___features_denseblock3_denselayer18_conv1 (l__self___features_denseblock3_denselayer18_relu1,) {} call_module l__self___features_denseblock3_denselayer18_norm2 L__self___features_denseblock3_denselayer18_norm2 (bottleneck_output_70,) {} call_module l__self___features_denseblock3_denselayer18_relu2 L__self___features_denseblock3_denselayer18_relu2 (l__self___features_denseblock3_denselayer18_norm2,) {} call_module new_features_70 L__self___features_denseblock3_denselayer18_conv2 (l__self___features_denseblock3_denselayer18_relu2,) {} - call_function concated_features_36 ([l__self___features_transition2_pool, new_features_36, new_features_38, new_features_40, new_features_42, new_features_44, new_features_46, new_features_48, new_features_50, new_features_52, new_features_54, new_features_56, new_features_58, new_features_60, new_features_62, new_features_64, new_features_66, new_features_68, new_features_70], 1) {} + call_function concated_features_36 ([l__self___features_transition2_pool, new_features_36, new_features_38, new_features_40, new_features_42, new_features_44, new_features_46, new_features_48, new_features_50, new_features_52, new_features_54, new_features_56, new_features_58, new_features_60, new_features_62, new_features_64, new_features_66, new_features_68, new_features_70], 1) {} call_module l__self___features_denseblock3_denselayer19_norm1 L__self___features_denseblock3_denselayer19_norm1 (concated_features_36,) {} call_module l__self___features_denseblock3_denselayer19_relu1 L__self___features_denseblock3_denselayer19_relu1 (l__self___features_denseblock3_denselayer19_norm1,) {} call_module bottleneck_output_72 L__self___features_denseblock3_denselayer19_conv1 (l__self___features_denseblock3_denselayer19_relu1,) {} call_module l__self___features_denseblock3_denselayer19_norm2 L__self___features_denseblock3_denselayer19_norm2 (bottleneck_output_72,) {} call_module l__self___features_denseblock3_denselayer19_relu2 L__self___features_denseblock3_denselayer19_relu2 (l__self___features_denseblock3_denselayer19_norm2,) {} call_module new_features_72 L__self___features_denseblock3_denselayer19_conv2 (l__self___features_denseblock3_denselayer19_relu2,) {} - call_function concated_features_37 ([l__self___features_transition2_pool, new_features_36, new_features_38, new_features_40, new_features_42, new_features_44, new_features_46, new_features_48, new_features_50, new_features_52, new_features_54, new_features_56, new_features_58, new_features_60, new_features_62, new_features_64, new_features_66, new_features_68, new_features_70, new_features_72], 1) {} + call_function concated_features_37 ([l__self___features_transition2_pool, new_features_36, new_features_38, new_features_40, new_features_42, new_features_44, new_features_46, new_features_48, new_features_50, new_features_52, new_features_54, new_features_56, new_features_58, new_features_60, new_features_62, new_features_64, new_features_66, new_features_68, new_features_70, new_features_72], 1) {} call_module l__self___features_denseblock3_denselayer20_norm1 L__self___features_denseblock3_denselayer20_norm1 (concated_features_37,) {} call_module l__self___features_denseblock3_denselayer20_relu1 L__self___features_denseblock3_denselayer20_relu1 (l__self___features_denseblock3_denselayer20_norm1,) {} call_module bottleneck_output_74 L__self___features_denseblock3_denselayer20_conv1 (l__self___features_denseblock3_denselayer20_relu1,) {} call_module l__self___features_denseblock3_denselayer20_norm2 L__self___features_denseblock3_denselayer20_norm2 (bottleneck_output_74,) {} call_module l__self___features_denseblock3_denselayer20_relu2 L__self___features_denseblock3_denselayer20_relu2 (l__self___features_denseblock3_denselayer20_norm2,) {} call_module new_features_74 L__self___features_denseblock3_denselayer20_conv2 (l__self___features_denseblock3_denselayer20_relu2,) {} - call_function concated_features_38 ([l__self___features_transition2_pool, new_features_36, new_features_38, new_features_40, new_features_42, new_features_44, new_features_46, new_features_48, new_features_50, new_features_52, new_features_54, new_features_56, new_features_58, new_features_60, new_features_62, new_features_64, new_features_66, new_features_68, new_features_70, new_features_72, new_features_74], 1) {} + call_function concated_features_38 ([l__self___features_transition2_pool, new_features_36, new_features_38, new_features_40, new_features_42, new_features_44, new_features_46, new_features_48, new_features_50, new_features_52, new_features_54, new_features_56, new_features_58, new_features_60, new_features_62, new_features_64, new_features_66, new_features_68, new_features_70, new_features_72, new_features_74], 1) {} call_module l__self___features_denseblock3_denselayer21_norm1 L__self___features_denseblock3_denselayer21_norm1 (concated_features_38,) {} call_module l__self___features_denseblock3_denselayer21_relu1 L__self___features_denseblock3_denselayer21_relu1 (l__self___features_denseblock3_denselayer21_norm1,) {} call_module bottleneck_output_76 L__self___features_denseblock3_denselayer21_conv1 (l__self___features_denseblock3_denselayer21_relu1,) {} call_module l__self___features_denseblock3_denselayer21_norm2 L__self___features_denseblock3_denselayer21_norm2 (bottleneck_output_76,) {} call_module l__self___features_denseblock3_denselayer21_relu2 L__self___features_denseblock3_denselayer21_relu2 (l__self___features_denseblock3_denselayer21_norm2,) {} call_module new_features_76 L__self___features_denseblock3_denselayer21_conv2 (l__self___features_denseblock3_denselayer21_relu2,) {} - call_function concated_features_39 ([l__self___features_transition2_pool, new_features_36, new_features_38, new_features_40, new_features_42, new_features_44, new_features_46, new_features_48, new_features_50, new_features_52, new_features_54, new_features_56, new_features_58, new_features_60, new_features_62, new_features_64, new_features_66, new_features_68, new_features_70, new_features_72, new_features_74, new_features_76], 1) {} + call_function concated_features_39 ([l__self___features_transition2_pool, new_features_36, new_features_38, new_features_40, new_features_42, new_features_44, new_features_46, new_features_48, new_features_50, new_features_52, new_features_54, new_features_56, new_features_58, new_features_60, new_features_62, new_features_64, new_features_66, new_features_68, new_features_70, new_features_72, new_features_74, new_features_76], 1) {} call_module l__self___features_denseblock3_denselayer22_norm1 L__self___features_denseblock3_denselayer22_norm1 (concated_features_39,) {} call_module l__self___features_denseblock3_denselayer22_relu1 L__self___features_denseblock3_denselayer22_relu1 (l__self___features_denseblock3_denselayer22_norm1,) {} call_module bottleneck_output_78 L__self___features_denseblock3_denselayer22_conv1 (l__self___features_denseblock3_denselayer22_relu1,) {} call_module l__self___features_denseblock3_denselayer22_norm2 L__self___features_denseblock3_denselayer22_norm2 (bottleneck_output_78,) {} call_module l__self___features_denseblock3_denselayer22_relu2 L__self___features_denseblock3_denselayer22_relu2 (l__self___features_denseblock3_denselayer22_norm2,) {} call_module new_features_78 L__self___features_denseblock3_denselayer22_conv2 (l__self___features_denseblock3_denselayer22_relu2,) {} - call_function concated_features_40 ([l__self___features_transition2_pool, new_features_36, new_features_38, new_features_40, new_features_42, new_features_44, new_features_46, new_features_48, new_features_50, new_features_52, new_features_54, new_features_56, new_features_58, new_features_60, new_features_62, new_features_64, new_features_66, new_features_68, new_features_70, new_features_72, new_features_74, new_features_76, new_features_78], 1) {} + call_function concated_features_40 ([l__self___features_transition2_pool, new_features_36, new_features_38, new_features_40, new_features_42, new_features_44, new_features_46, new_features_48, new_features_50, new_features_52, new_features_54, new_features_56, new_features_58, new_features_60, new_features_62, new_features_64, new_features_66, new_features_68, new_features_70, new_features_72, new_features_74, new_features_76, new_features_78], 1) {} call_module l__self___features_denseblock3_denselayer23_norm1 L__self___features_denseblock3_denselayer23_norm1 (concated_features_40,) {} call_module l__self___features_denseblock3_denselayer23_relu1 L__self___features_denseblock3_denselayer23_relu1 (l__self___features_denseblock3_denselayer23_norm1,) {} call_module bottleneck_output_80 L__self___features_denseblock3_denselayer23_conv1 (l__self___features_denseblock3_denselayer23_relu1,) {} call_module l__self___features_denseblock3_denselayer23_norm2 L__self___features_denseblock3_denselayer23_norm2 (bottleneck_output_80,) {} call_module l__self___features_denseblock3_denselayer23_relu2 L__self___features_denseblock3_denselayer23_relu2 (l__self___features_denseblock3_denselayer23_norm2,) {} call_module new_features_80 L__self___features_denseblock3_denselayer23_conv2 (l__self___features_denseblock3_denselayer23_relu2,) {} - call_function concated_features_41 ([l__self___features_transition2_pool, new_features_36, new_features_38, new_features_40, new_features_42, new_features_44, new_features_46, new_features_48, new_features_50, new_features_52, new_features_54, new_features_56, new_features_58, new_features_60, new_features_62, new_features_64, new_features_66, new_features_68, new_features_70, new_features_72, new_features_74, new_features_76, new_features_78, new_features_80], 1) {} + call_function concated_features_41 ([l__self___features_transition2_pool, new_features_36, new_features_38, new_features_40, new_features_42, new_features_44, new_features_46, new_features_48, new_features_50, new_features_52, new_features_54, new_features_56, new_features_58, new_features_60, new_features_62, new_features_64, new_features_66, new_features_68, new_features_70, new_features_72, new_features_74, new_features_76, new_features_78, new_features_80], 1) {} call_module l__self___features_denseblock3_denselayer24_norm1 L__self___features_denseblock3_denselayer24_norm1 (concated_features_41,) {} call_module l__self___features_denseblock3_denselayer24_relu1 L__self___features_denseblock3_denselayer24_relu1 (l__self___features_denseblock3_denselayer24_norm1,) {} call_module bottleneck_output_82 L__self___features_denseblock3_denselayer24_conv1 (l__self___features_denseblock3_denselayer24_relu1,) {} call_module l__self___features_denseblock3_denselayer24_norm2 L__self___features_denseblock3_denselayer24_norm2 (bottleneck_output_82,) {} call_module l__self___features_denseblock3_denselayer24_relu2 L__self___features_denseblock3_denselayer24_relu2 (l__self___features_denseblock3_denselayer24_norm2,) {} call_module new_features_82 L__self___features_denseblock3_denselayer24_conv2 (l__self___features_denseblock3_denselayer24_relu2,) {} - call_function cat_44 ([l__self___features_transition2_pool, new_features_36, new_features_38, new_features_40, new_features_42, new_features_44, new_features_46, new_features_48, new_features_50, new_features_52, new_features_54, new_features_56, new_features_58, new_features_60, new_features_62, new_features_64, new_features_66, new_features_68, new_features_70, new_features_72, new_features_74, new_features_76, new_features_78, new_features_80, new_features_82], 1) {} + call_function cat_44 ([l__self___features_transition2_pool, new_features_36, new_features_38, new_features_40, new_features_42, new_features_44, new_features_46, new_features_48, new_features_50, new_features_52, new_features_54, new_features_56, new_features_58, new_features_60, new_features_62, new_features_64, new_features_66, new_features_68, new_features_70, new_features_72, new_features_74, new_features_76, new_features_78, new_features_80, new_features_82], 1) {} call_module l__self___features_transition3_norm L__self___features_transition3_norm (cat_44,) {} call_module l__self___features_transition3_relu L__self___features_transition3_relu (l__self___features_transition3_norm,) {} call_module l__self___features_transition3_conv L__self___features_transition3_conv (l__self___features_transition3_relu,) {} call_module l__self___features_transition3_pool L__self___features_transition3_pool (l__self___features_transition3_conv,) {} - call_function concated_features_42 ([l__self___features_transition3_pool], 1) {} + call_function concated_features_42 ([l__self___features_transition3_pool], 1) {} call_module l__self___features_denseblock4_denselayer1_norm1 L__self___features_denseblock4_denselayer1_norm1 (concated_features_42,) {} call_module l__self___features_denseblock4_denselayer1_relu1 L__self___features_denseblock4_denselayer1_relu1 (l__self___features_denseblock4_denselayer1_norm1,) {} call_module bottleneck_output_84 L__self___features_denseblock4_denselayer1_conv1 (l__self___features_denseblock4_denselayer1_relu1,) {} call_module l__self___features_denseblock4_denselayer1_norm2 L__self___features_denseblock4_denselayer1_norm2 (bottleneck_output_84,) {} call_module l__self___features_denseblock4_denselayer1_relu2 L__self___features_denseblock4_denselayer1_relu2 (l__self___features_denseblock4_denselayer1_norm2,) {} call_module new_features_84 L__self___features_denseblock4_denselayer1_conv2 (l__self___features_denseblock4_denselayer1_relu2,) {} - call_function concated_features_43 ([l__self___features_transition3_pool, new_features_84], 1) {} + call_function concated_features_43 ([l__self___features_transition3_pool, new_features_84], 1) {} call_module l__self___features_denseblock4_denselayer2_norm1 L__self___features_denseblock4_denselayer2_norm1 (concated_features_43,) {} call_module l__self___features_denseblock4_denselayer2_relu1 L__self___features_denseblock4_denselayer2_relu1 (l__self___features_denseblock4_denselayer2_norm1,) {} call_module bottleneck_output_86 L__self___features_denseblock4_denselayer2_conv1 (l__self___features_denseblock4_denselayer2_relu1,) {} call_module l__self___features_denseblock4_denselayer2_norm2 L__self___features_denseblock4_denselayer2_norm2 (bottleneck_output_86,) {} call_module l__self___features_denseblock4_denselayer2_relu2 L__self___features_denseblock4_denselayer2_relu2 (l__self___features_denseblock4_denselayer2_norm2,) {} call_module new_features_86 L__self___features_denseblock4_denselayer2_conv2 (l__self___features_denseblock4_denselayer2_relu2,) {} - call_function concated_features_44 ([l__self___features_transition3_pool, new_features_84, new_features_86], 1) {} + call_function concated_features_44 ([l__self___features_transition3_pool, new_features_84, new_features_86], 1) {} call_module l__self___features_denseblock4_denselayer3_norm1 L__self___features_denseblock4_denselayer3_norm1 (concated_features_44,) {} call_module l__self___features_denseblock4_denselayer3_relu1 L__self___features_denseblock4_denselayer3_relu1 (l__self___features_denseblock4_denselayer3_norm1,) {} call_module bottleneck_output_88 L__self___features_denseblock4_denselayer3_conv1 (l__self___features_denseblock4_denselayer3_relu1,) {} call_module l__self___features_denseblock4_denselayer3_norm2 L__self___features_denseblock4_denselayer3_norm2 (bottleneck_output_88,) {} call_module l__self___features_denseblock4_denselayer3_relu2 L__self___features_denseblock4_denselayer3_relu2 (l__self___features_denseblock4_denselayer3_norm2,) {} call_module new_features_88 L__self___features_denseblock4_denselayer3_conv2 (l__self___features_denseblock4_denselayer3_relu2,) {} - call_function concated_features_45 ([l__self___features_transition3_pool, new_features_84, new_features_86, new_features_88], 1) {} + call_function concated_features_45 ([l__self___features_transition3_pool, new_features_84, new_features_86, new_features_88], 1) {} call_module l__self___features_denseblock4_denselayer4_norm1 L__self___features_denseblock4_denselayer4_norm1 (concated_features_45,) {} call_module l__self___features_denseblock4_denselayer4_relu1 L__self___features_denseblock4_denselayer4_relu1 (l__self___features_denseblock4_denselayer4_norm1,) {} call_module bottleneck_output_90 L__self___features_denseblock4_denselayer4_conv1 (l__self___features_denseblock4_denselayer4_relu1,) {} call_module l__self___features_denseblock4_denselayer4_norm2 L__self___features_denseblock4_denselayer4_norm2 (bottleneck_output_90,) {} call_module l__self___features_denseblock4_denselayer4_relu2 L__self___features_denseblock4_denselayer4_relu2 (l__self___features_denseblock4_denselayer4_norm2,) {} call_module new_features_90 L__self___features_denseblock4_denselayer4_conv2 (l__self___features_denseblock4_denselayer4_relu2,) {} - call_function concated_features_46 ([l__self___features_transition3_pool, new_features_84, new_features_86, new_features_88, new_features_90], 1) {} + call_function concated_features_46 ([l__self___features_transition3_pool, new_features_84, new_features_86, new_features_88, new_features_90], 1) {} call_module l__self___features_denseblock4_denselayer5_norm1 L__self___features_denseblock4_denselayer5_norm1 (concated_features_46,) {} call_module l__self___features_denseblock4_denselayer5_relu1 L__self___features_denseblock4_denselayer5_relu1 (l__self___features_denseblock4_denselayer5_norm1,) {} call_module bottleneck_output_92 L__self___features_denseblock4_denselayer5_conv1 (l__self___features_denseblock4_denselayer5_relu1,) {} call_module l__self___features_denseblock4_denselayer5_norm2 L__self___features_denseblock4_denselayer5_norm2 (bottleneck_output_92,) {} call_module l__self___features_denseblock4_denselayer5_relu2 L__self___features_denseblock4_denselayer5_relu2 (l__self___features_denseblock4_denselayer5_norm2,) {} call_module new_features_92 L__self___features_denseblock4_denselayer5_conv2 (l__self___features_denseblock4_denselayer5_relu2,) {} - call_function concated_features_47 ([l__self___features_transition3_pool, new_features_84, new_features_86, new_features_88, new_features_90, new_features_92], 1) {} + call_function concated_features_47 ([l__self___features_transition3_pool, new_features_84, new_features_86, new_features_88, new_features_90, new_features_92], 1) {} call_module l__self___features_denseblock4_denselayer6_norm1 L__self___features_denseblock4_denselayer6_norm1 (concated_features_47,) {} call_module l__self___features_denseblock4_denselayer6_relu1 L__self___features_denseblock4_denselayer6_relu1 (l__self___features_denseblock4_denselayer6_norm1,) {} call_module bottleneck_output_94 L__self___features_denseblock4_denselayer6_conv1 (l__self___features_denseblock4_denselayer6_relu1,) {} call_module l__self___features_denseblock4_denselayer6_norm2 L__self___features_denseblock4_denselayer6_norm2 (bottleneck_output_94,) {} call_module l__self___features_denseblock4_denselayer6_relu2 L__self___features_denseblock4_denselayer6_relu2 (l__self___features_denseblock4_denselayer6_norm2,) {} call_module new_features_94 L__self___features_denseblock4_denselayer6_conv2 (l__self___features_denseblock4_denselayer6_relu2,) {} - call_function concated_features_48 ([l__self___features_transition3_pool, new_features_84, new_features_86, new_features_88, new_features_90, new_features_92, new_features_94], 1) {} + call_function concated_features_48 ([l__self___features_transition3_pool, new_features_84, new_features_86, new_features_88, new_features_90, new_features_92, new_features_94], 1) {} call_module l__self___features_denseblock4_denselayer7_norm1 L__self___features_denseblock4_denselayer7_norm1 (concated_features_48,) {} call_module l__self___features_denseblock4_denselayer7_relu1 L__self___features_denseblock4_denselayer7_relu1 (l__self___features_denseblock4_denselayer7_norm1,) {} call_module bottleneck_output_96 L__self___features_denseblock4_denselayer7_conv1 (l__self___features_denseblock4_denselayer7_relu1,) {} call_module l__self___features_denseblock4_denselayer7_norm2 L__self___features_denseblock4_denselayer7_norm2 (bottleneck_output_96,) {} call_module l__self___features_denseblock4_denselayer7_relu2 L__self___features_denseblock4_denselayer7_relu2 (l__self___features_denseblock4_denselayer7_norm2,) {} call_module new_features_96 L__self___features_denseblock4_denselayer7_conv2 (l__self___features_denseblock4_denselayer7_relu2,) {} - call_function concated_features_49 ([l__self___features_transition3_pool, new_features_84, new_features_86, new_features_88, new_features_90, new_features_92, new_features_94, new_features_96], 1) {} + call_function concated_features_49 ([l__self___features_transition3_pool, new_features_84, new_features_86, new_features_88, new_features_90, new_features_92, new_features_94, new_features_96], 1) {} call_module l__self___features_denseblock4_denselayer8_norm1 L__self___features_denseblock4_denselayer8_norm1 (concated_features_49,) {} call_module l__self___features_denseblock4_denselayer8_relu1 L__self___features_denseblock4_denselayer8_relu1 (l__self___features_denseblock4_denselayer8_norm1,) {} call_module bottleneck_output_98 L__self___features_denseblock4_denselayer8_conv1 (l__self___features_denseblock4_denselayer8_relu1,) {} call_module l__self___features_denseblock4_denselayer8_norm2 L__self___features_denseblock4_denselayer8_norm2 (bottleneck_output_98,) {} call_module l__self___features_denseblock4_denselayer8_relu2 L__self___features_denseblock4_denselayer8_relu2 (l__self___features_denseblock4_denselayer8_norm2,) {} call_module new_features_98 L__self___features_denseblock4_denselayer8_conv2 (l__self___features_denseblock4_denselayer8_relu2,) {} - call_function concated_features_50 ([l__self___features_transition3_pool, new_features_84, new_features_86, new_features_88, new_features_90, new_features_92, new_features_94, new_features_96, new_features_98], 1) {} + call_function concated_features_50 ([l__self___features_transition3_pool, new_features_84, new_features_86, new_features_88, new_features_90, new_features_92, new_features_94, new_features_96, new_features_98], 1) {} call_module l__self___features_denseblock4_denselayer9_norm1 L__self___features_denseblock4_denselayer9_norm1 (concated_features_50,) {} call_module l__self___features_denseblock4_denselayer9_relu1 L__self___features_denseblock4_denselayer9_relu1 (l__self___features_denseblock4_denselayer9_norm1,) {} call_module bottleneck_output_100 L__self___features_denseblock4_denselayer9_conv1 (l__self___features_denseblock4_denselayer9_relu1,) {} call_module l__self___features_denseblock4_denselayer9_norm2 L__self___features_denseblock4_denselayer9_norm2 (bottleneck_output_100,) {} call_module l__self___features_denseblock4_denselayer9_relu2 L__self___features_denseblock4_denselayer9_relu2 (l__self___features_denseblock4_denselayer9_norm2,) {} call_module new_features_100 L__self___features_denseblock4_denselayer9_conv2 (l__self___features_denseblock4_denselayer9_relu2,) {} - call_function concated_features_51 ([l__self___features_transition3_pool, new_features_84, new_features_86, new_features_88, new_features_90, new_features_92, new_features_94, new_features_96, new_features_98, new_features_100], 1) {} + call_function concated_features_51 ([l__self___features_transition3_pool, new_features_84, new_features_86, new_features_88, new_features_90, new_features_92, new_features_94, new_features_96, new_features_98, new_features_100], 1) {} call_module l__self___features_denseblock4_denselayer10_norm1 L__self___features_denseblock4_denselayer10_norm1 (concated_features_51,) {} call_module l__self___features_denseblock4_denselayer10_relu1 L__self___features_denseblock4_denselayer10_relu1 (l__self___features_denseblock4_denselayer10_norm1,) {} call_module bottleneck_output_102 L__self___features_denseblock4_denselayer10_conv1 (l__self___features_denseblock4_denselayer10_relu1,) {} call_module l__self___features_denseblock4_denselayer10_norm2 L__self___features_denseblock4_denselayer10_norm2 (bottleneck_output_102,) {} call_module l__self___features_denseblock4_denselayer10_relu2 L__self___features_denseblock4_denselayer10_relu2 (l__self___features_denseblock4_denselayer10_norm2,) {} call_module new_features_102 L__self___features_denseblock4_denselayer10_conv2 (l__self___features_denseblock4_denselayer10_relu2,) {} - call_function concated_features_52 ([l__self___features_transition3_pool, new_features_84, new_features_86, new_features_88, new_features_90, new_features_92, new_features_94, new_features_96, new_features_98, new_features_100, new_features_102], 1) {} + call_function concated_features_52 ([l__self___features_transition3_pool, new_features_84, new_features_86, new_features_88, new_features_90, new_features_92, new_features_94, new_features_96, new_features_98, new_features_100, new_features_102], 1) {} call_module l__self___features_denseblock4_denselayer11_norm1 L__self___features_denseblock4_denselayer11_norm1 (concated_features_52,) {} call_module l__self___features_denseblock4_denselayer11_relu1 L__self___features_denseblock4_denselayer11_relu1 (l__self___features_denseblock4_denselayer11_norm1,) {} call_module bottleneck_output_104 L__self___features_denseblock4_denselayer11_conv1 (l__self___features_denseblock4_denselayer11_relu1,) {} call_module l__self___features_denseblock4_denselayer11_norm2 L__self___features_denseblock4_denselayer11_norm2 (bottleneck_output_104,) {} call_module l__self___features_denseblock4_denselayer11_relu2 L__self___features_denseblock4_denselayer11_relu2 (l__self___features_denseblock4_denselayer11_norm2,) {} call_module new_features_104 L__self___features_denseblock4_denselayer11_conv2 (l__self___features_denseblock4_denselayer11_relu2,) {} - call_function concated_features_53 ([l__self___features_transition3_pool, new_features_84, new_features_86, new_features_88, new_features_90, new_features_92, new_features_94, new_features_96, new_features_98, new_features_100, new_features_102, new_features_104], 1) {} + call_function concated_features_53 ([l__self___features_transition3_pool, new_features_84, new_features_86, new_features_88, new_features_90, new_features_92, new_features_94, new_features_96, new_features_98, new_features_100, new_features_102, new_features_104], 1) {} call_module l__self___features_denseblock4_denselayer12_norm1 L__self___features_denseblock4_denselayer12_norm1 (concated_features_53,) {} call_module l__self___features_denseblock4_denselayer12_relu1 L__self___features_denseblock4_denselayer12_relu1 (l__self___features_denseblock4_denselayer12_norm1,) {} call_module bottleneck_output_106 L__self___features_denseblock4_denselayer12_conv1 (l__self___features_denseblock4_denselayer12_relu1,) {} call_module l__self___features_denseblock4_denselayer12_norm2 L__self___features_denseblock4_denselayer12_norm2 (bottleneck_output_106,) {} call_module l__self___features_denseblock4_denselayer12_relu2 L__self___features_denseblock4_denselayer12_relu2 (l__self___features_denseblock4_denselayer12_norm2,) {} call_module new_features_106 L__self___features_denseblock4_denselayer12_conv2 (l__self___features_denseblock4_denselayer12_relu2,) {} - call_function concated_features_54 ([l__self___features_transition3_pool, new_features_84, new_features_86, new_features_88, new_features_90, new_features_92, new_features_94, new_features_96, new_features_98, new_features_100, new_features_102, new_features_104, new_features_106], 1) {} + call_function concated_features_54 ([l__self___features_transition3_pool, new_features_84, new_features_86, new_features_88, new_features_90, new_features_92, new_features_94, new_features_96, new_features_98, new_features_100, new_features_102, new_features_104, new_features_106], 1) {} call_module l__self___features_denseblock4_denselayer13_norm1 L__self___features_denseblock4_denselayer13_norm1 (concated_features_54,) {} call_module l__self___features_denseblock4_denselayer13_relu1 L__self___features_denseblock4_denselayer13_relu1 (l__self___features_denseblock4_denselayer13_norm1,) {} call_module bottleneck_output_108 L__self___features_denseblock4_denselayer13_conv1 (l__self___features_denseblock4_denselayer13_relu1,) {} call_module l__self___features_denseblock4_denselayer13_norm2 L__self___features_denseblock4_denselayer13_norm2 (bottleneck_output_108,) {} call_module l__self___features_denseblock4_denselayer13_relu2 L__self___features_denseblock4_denselayer13_relu2 (l__self___features_denseblock4_denselayer13_norm2,) {} call_module new_features_108 L__self___features_denseblock4_denselayer13_conv2 (l__self___features_denseblock4_denselayer13_relu2,) {} - call_function concated_features_55 ([l__self___features_transition3_pool, new_features_84, new_features_86, new_features_88, new_features_90, new_features_92, new_features_94, new_features_96, new_features_98, new_features_100, new_features_102, new_features_104, new_features_106, new_features_108], 1) {} + call_function concated_features_55 ([l__self___features_transition3_pool, new_features_84, new_features_86, new_features_88, new_features_90, new_features_92, new_features_94, new_features_96, new_features_98, new_features_100, new_features_102, new_features_104, new_features_106, new_features_108], 1) {} call_module l__self___features_denseblock4_denselayer14_norm1 L__self___features_denseblock4_denselayer14_norm1 (concated_features_55,) {} call_module l__self___features_denseblock4_denselayer14_relu1 L__self___features_denseblock4_denselayer14_relu1 (l__self___features_denseblock4_denselayer14_norm1,) {} call_module bottleneck_output_110 L__self___features_denseblock4_denselayer14_conv1 (l__self___features_denseblock4_denselayer14_relu1,) {} call_module l__self___features_denseblock4_denselayer14_norm2 L__self___features_denseblock4_denselayer14_norm2 (bottleneck_output_110,) {} call_module l__self___features_denseblock4_denselayer14_relu2 L__self___features_denseblock4_denselayer14_relu2 (l__self___features_denseblock4_denselayer14_norm2,) {} call_module new_features_110 L__self___features_denseblock4_denselayer14_conv2 (l__self___features_denseblock4_denselayer14_relu2,) {} - call_function concated_features_56 ([l__self___features_transition3_pool, new_features_84, new_features_86, new_features_88, new_features_90, new_features_92, new_features_94, new_features_96, new_features_98, new_features_100, new_features_102, new_features_104, new_features_106, new_features_108, new_features_110], 1) {} + call_function concated_features_56 ([l__self___features_transition3_pool, new_features_84, new_features_86, new_features_88, new_features_90, new_features_92, new_features_94, new_features_96, new_features_98, new_features_100, new_features_102, new_features_104, new_features_106, new_features_108, new_features_110], 1) {} call_module l__self___features_denseblock4_denselayer15_norm1 L__self___features_denseblock4_denselayer15_norm1 (concated_features_56,) {} call_module l__self___features_denseblock4_denselayer15_relu1 L__self___features_denseblock4_denselayer15_relu1 (l__self___features_denseblock4_denselayer15_norm1,) {} call_module bottleneck_output_112 L__self___features_denseblock4_denselayer15_conv1 (l__self___features_denseblock4_denselayer15_relu1,) {} call_module l__self___features_denseblock4_denselayer15_norm2 L__self___features_denseblock4_denselayer15_norm2 (bottleneck_output_112,) {} call_module l__self___features_denseblock4_denselayer15_relu2 L__self___features_denseblock4_denselayer15_relu2 (l__self___features_denseblock4_denselayer15_norm2,) {} call_module new_features_112 L__self___features_denseblock4_denselayer15_conv2 (l__self___features_denseblock4_denselayer15_relu2,) {} - call_function concated_features_57 ([l__self___features_transition3_pool, new_features_84, new_features_86, new_features_88, new_features_90, new_features_92, new_features_94, new_features_96, new_features_98, new_features_100, new_features_102, new_features_104, new_features_106, new_features_108, new_features_110, new_features_112], 1) {} + call_function concated_features_57 ([l__self___features_transition3_pool, new_features_84, new_features_86, new_features_88, new_features_90, new_features_92, new_features_94, new_features_96, new_features_98, new_features_100, new_features_102, new_features_104, new_features_106, new_features_108, new_features_110, new_features_112], 1) {} call_module l__self___features_denseblock4_denselayer16_norm1 L__self___features_denseblock4_denselayer16_norm1 (concated_features_57,) {} call_module l__self___features_denseblock4_denselayer16_relu1 L__self___features_denseblock4_denselayer16_relu1 (l__self___features_denseblock4_denselayer16_norm1,) {} call_module bottleneck_output_114 L__self___features_denseblock4_denselayer16_conv1 (l__self___features_denseblock4_denselayer16_relu1,) {} call_module l__self___features_denseblock4_denselayer16_norm2 L__self___features_denseblock4_denselayer16_norm2 (bottleneck_output_114,) {} call_module l__self___features_denseblock4_denselayer16_relu2 L__self___features_denseblock4_denselayer16_relu2 (l__self___features_denseblock4_denselayer16_norm2,) {} call_module new_features_114 L__self___features_denseblock4_denselayer16_conv2 (l__self___features_denseblock4_denselayer16_relu2,) {} - call_function cat_61 ([l__self___features_transition3_pool, new_features_84, new_features_86, new_features_88, new_features_90, new_features_92, new_features_94, new_features_96, new_features_98, new_features_100, new_features_102, new_features_104, new_features_106, new_features_108, new_features_110, new_features_112, new_features_114], 1) {} + call_function cat_61 ([l__self___features_transition3_pool, new_features_84, new_features_86, new_features_88, new_features_90, new_features_92, new_features_94, new_features_96, new_features_98, new_features_100, new_features_102, new_features_104, new_features_106, new_features_108, new_features_110, new_features_112, new_features_114], 1) {} call_module features L__self___features_norm5 (cat_61,) {} - call_function out (features,) {'inplace': True} - call_function out_1 (out, (1, 1)) {} - call_function out_2 (out_1, 1) {} + call_function out (features,) {'inplace': True} + call_function out_1 (out, (1, 1)) {} + call_function out_2 (out_1, 1) {} call_module out_3 L__self___classifier (out_2,) {} output output output ((out_3,),) {} @@ -1483,7 +1483,7 @@ data-dependent control flow. Consider the function below, where the line ------------- ------ ------------------------------------------------------ ----------- -------- placeholder l_a_ L_a_ () {} placeholder l_b_ L_b_ () {} - call_function abs_1 (l_a_,) {} + call_function abs_1 (l_a_,) {} call_function add (abs_1, 1) {} call_function x (l_a_, add) {} call_method sum_1 sum (l_b_,) {} @@ -1565,7 +1565,7 @@ We can see where TorchDynamo breaks the graph by using ``torch._dynamo.explain`` Ops per Graph: Ops 1: - + @@ -1574,17 +1574,17 @@ We can see where TorchDynamo breaks the graph by using ``torch._dynamo.explain`` Out Guards: Guard 1: - Name: "L['b']" + Name: "L['a']" Source: local Create Function: TENSOR_MATCH Guard Types: ['TENSOR_MATCH'] - Code List: ["hasattr(L['b'], '_dynamo_dynamic_indices') == False"] - Object Weakref: - Guarded Class Weakref: + Code List: ["hasattr(L['a'], '_dynamo_dynamic_indices') == False"] + Object Weakref: + Guarded Class Weakref: Guard 2: Name: '' Source: global - Create Function: DETERMINISTIC_ALGORITHMS + Create Function: TORCH_FUNCTION_STATE Guard Types: None Code List: None Object Weakref: None @@ -1598,22 +1598,14 @@ We can see where TorchDynamo breaks the graph by using ``torch._dynamo.explain`` Object Weakref: None Guarded Class Weakref: None Guard 4: - Name: '' + Name: "G['torch'].abs" Source: global - Create Function: TORCH_FUNCTION_STATE + Create Function: FUNCTION_MATCH Guard Types: None Code List: None Object Weakref: None Guarded Class Weakref: None Guard 5: - Name: '' - Source: shape_env - Create Function: SHAPE_ENV - Guard Types: None - Code List: None - Object Weakref: None - Guarded Class Weakref: None - Guard 6: Name: '' Source: global Create Function: DEFAULT_DEVICE @@ -1621,31 +1613,23 @@ We can see where TorchDynamo breaks the graph by using ``torch._dynamo.explain`` Code List: ['utils_device.CURRENT_DEVICE == None'] Object Weakref: None Guarded Class Weakref: None - Guard 7: - Name: "G['torch'].abs" - Source: global - Create Function: FUNCTION_MATCH + Guard 6: + Name: '' + Source: shape_env + Create Function: SHAPE_ENV Guard Types: None Code List: None Object Weakref: None Guarded Class Weakref: None - Guard 8: + Guard 7: Name: '' Source: global Create Function: BACKEND_MATCH Guard Types: ['BACKEND_MATCH'] - Code List: ['___check_current_backend(139670084077168)'] + Code List: ['___check_current_backend(139891651057920)'] Object Weakref: None Guarded Class Weakref: None - Guard 9: - Name: "L['a']" - Source: local - Create Function: TENSOR_MATCH - Guard Types: ['TENSOR_MATCH'] - Code List: ["hasattr(L['a'], '_dynamo_dynamic_indices') == False"] - Object Weakref: - Guarded Class Weakref: - Guard 10: + Guard 8: Name: '' Source: global Create Function: GRAD_MODE @@ -1653,28 +1637,44 @@ We can see where TorchDynamo breaks the graph by using ``torch._dynamo.explain`` Code List: None Object Weakref: None Guarded Class Weakref: None - Guard 11: + Guard 9: + Name: '' + Source: global + Create Function: DETERMINISTIC_ALGORITHMS + Guard Types: None + Code List: None + Object Weakref: None + Guarded Class Weakref: None + Guard 10: Name: "L['b']" Source: local Create Function: TENSOR_MATCH Guard Types: ['TENSOR_MATCH'] Code List: ["hasattr(L['b'], '_dynamo_dynamic_indices') == False"] - Object Weakref: - Guarded Class Weakref: - Guard 12: + Object Weakref: + Guarded Class Weakref: + Guard 11: Name: '' Source: global - Create Function: DETERMINISTIC_ALGORITHMS + Create Function: TORCH_FUNCTION_STATE Guard Types: None Code List: None Object Weakref: None Guarded Class Weakref: None + Guard 12: + Name: "L['x']" + Source: local + Create Function: TENSOR_MATCH + Guard Types: ['TENSOR_MATCH'] + Code List: ["hasattr(L['x'], '_dynamo_dynamic_indices') == False"] + Object Weakref: + Guarded Class Weakref: Guard 13: Name: '' Source: global - Create Function: TORCH_FUNCTION_STATE - Guard Types: None - Code List: None + Create Function: DEFAULT_DEVICE + Guard Types: ['DEFAULT_DEVICE'] + Code List: ['utils_device.CURRENT_DEVICE == None'] Object Weakref: None Guarded Class Weakref: None Guard 14: @@ -1686,30 +1686,22 @@ We can see where TorchDynamo breaks the graph by using ``torch._dynamo.explain`` Object Weakref: None Guarded Class Weakref: None Guard 15: - Name: '' - Source: global - Create Function: DEFAULT_DEVICE - Guard Types: ['DEFAULT_DEVICE'] - Code List: ['utils_device.CURRENT_DEVICE == None'] - Object Weakref: None - Guarded Class Weakref: None - Guard 16: Name: '' Source: global Create Function: BACKEND_MATCH Guard Types: ['BACKEND_MATCH'] - Code List: ['___check_current_backend(139670084077168)'] + Code List: ['___check_current_backend(139891651057920)'] Object Weakref: None Guarded Class Weakref: None - Guard 17: - Name: "L['x']" + Guard 16: + Name: "L['b']" Source: local Create Function: TENSOR_MATCH Guard Types: ['TENSOR_MATCH'] - Code List: ["hasattr(L['x'], '_dynamo_dynamic_indices') == False"] - Object Weakref: - Guarded Class Weakref: - Guard 18: + Code List: ["hasattr(L['b'], '_dynamo_dynamic_indices') == False"] + Object Weakref: + Guarded Class Weakref: + Guard 17: Name: '' Source: global Create Function: GRAD_MODE @@ -1717,10 +1709,18 @@ We can see where TorchDynamo breaks the graph by using ``torch._dynamo.explain`` Code List: None Object Weakref: None Guarded Class Weakref: None + Guard 18: + Name: '' + Source: global + Create Function: DETERMINISTIC_ALGORITHMS + Guard Types: None + Code List: None + Object Weakref: None + Guarded Class Weakref: None Compile Times: TorchDynamo compilation metrics: Function Runtimes (s) ------------------------------- -------------- - _compile..compile_inner 0.0111, 0.0067 + _compile..compile_inner 0.0109, 0.0067 OutputGraph.call_user_compiler 0.0001, 0.0000 @@ -1821,13 +1821,13 @@ the model we used above for demonstrating speedups. .. code-block:: none - tensor([[ 0.1344, 0.1976, -0.2056, ..., 0.1861, -0.1632, -0.1642], - [ 0.2680, 0.2349, -0.1903, ..., 0.2170, -0.0055, 0.0309], - [ 0.0164, 0.2180, -0.1115, ..., 0.1709, -0.1681, -0.0635], + tensor([[ 0.1343, 0.1977, -0.2055, ..., 0.1862, -0.1632, -0.1640], + [ 0.2683, 0.2349, -0.1905, ..., 0.2167, -0.0055, 0.0311], + [ 0.0162, 0.2181, -0.1115, ..., 0.1708, -0.1679, -0.0636], ..., - [ 0.0807, 0.2680, -0.1889, ..., 0.0381, -0.2073, -0.1444], - [-0.0209, 0.0858, -0.2457, ..., 0.1862, -0.1280, -0.0282], - [-0.0386, 0.0730, -0.1961, ..., 0.0864, -0.2199, -0.1485]], + [ 0.0805, 0.2680, -0.1888, ..., 0.0382, -0.2072, -0.1445], + [-0.0210, 0.0859, -0.2458, ..., 0.1863, -0.1280, -0.0282], + [-0.0387, 0.0729, -0.1960, ..., 0.0863, -0.2200, -0.1486]], device='cuda:0', grad_fn=) @@ -1855,7 +1855,7 @@ with FX graphs. We hope that you will give ``torch.compile`` a try! .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 6 minutes 11.441 seconds) + **Total running time of the script:** ( 6 minutes 9.834 seconds) .. _sphx_glr_download_intermediate_torch_compile_tutorial.py: diff --git a/_sources/intermediate/torch_export_tutorial.rst.txt b/_sources/intermediate/torch_export_tutorial.rst.txt index e5dee147a5..b6f47bacb9 100644 --- a/_sources/intermediate/torch_export_tutorial.rst.txt +++ b/_sources/intermediate/torch_export_tutorial.rst.txt @@ -98,34 +98,6 @@ We will detail the ``dynamic_shapes`` argument later in the tutorial. - - - -.. rst-class:: sphx-glr-script-out - - .. code-block:: none - - - tensor([[0.8632, 0.8407, 0.0407, 0.0000, 0.4132, 0.0000, 0.0000, 0.1538, 0.6111, - 0.0000], - [0.0000, 0.0000, 0.0273, 0.8057, 0.0000, 1.0162, 0.8042, 0.0000, 0.2660, - 0.0000], - [0.9481, 0.1396, 1.0225, 0.9563, 0.5832, 0.2546, 0.4095, 0.4591, 0.0000, - 2.0053], - [1.1300, 0.4873, 0.0000, 0.9663, 1.2275, 1.4015, 0.0000, 0.9444, 0.0000, - 0.0000], - [0.0000, 0.8724, 1.1648, 0.6867, 0.0000, 0.2833, 0.3202, 0.5848, 0.0000, - 0.0833], - [1.1311, 0.1324, 0.0000, 1.7842, 0.0000, 0.3474, 0.9916, 0.3571, 0.0000, - 0.0000], - [1.4348, 1.0570, 0.1771, 0.0000, 0.9510, 0.0000, 0.0000, 0.0000, 0.2618, - 0.0000], - [0.8853, 0.0000, 0.0000, 0.4486, 0.0000, 0.0000, 0.5841, 0.7604, 0.0000, - 0.0000]], grad_fn=) - - - - .. GENERATED FROM PYTHON SOURCE LINES 79-90 Let's review some attributes of ``ExportedProgram`` that are of interest. @@ -149,42 +121,6 @@ so that it can be ran as a ``torch.nn.Module``. print(exported_mod.graph_module) - - - -.. rst-class:: sphx-glr-script-out - - .. code-block:: none - - ExportedProgram: - class GraphModule(torch.nn.Module): - def forward(self, arg0_1: "f32[10, 100]", arg1_1: "f32[10]", arg2_1: "f32[8, 100]", arg3_1: "f32[8, 100]"): - # File: /var/lib/workspace/intermediate_source/torch_export_tutorial.py:70 in forward, code: return torch.nn.functional.relu(self.lin(x + y), inplace=True) - add: "f32[8, 100]" = torch.ops.aten.add.Tensor(arg2_1, arg3_1); arg2_1 = arg3_1 = None - t: "f32[100, 10]" = torch.ops.aten.t.default(arg0_1); arg0_1 = None - addmm: "f32[8, 10]" = torch.ops.aten.addmm.default(arg1_1, add, t); arg1_1 = add = t = None - relu: "f32[8, 10]" = torch.ops.aten.relu.default(addmm); addmm = None - return (relu,) - - Graph signature: ExportGraphSignature(input_specs=[InputSpec(kind=, arg=TensorArgument(name='arg0_1'), target='lin.weight', persistent=None), InputSpec(kind=, arg=TensorArgument(name='arg1_1'), target='lin.bias', persistent=None), InputSpec(kind=, arg=TensorArgument(name='arg2_1'), target=None, persistent=None), InputSpec(kind=, arg=TensorArgument(name='arg3_1'), target=None, persistent=None)], output_specs=[OutputSpec(kind=, arg=TensorArgument(name='relu'), target=None)]) - Range constraints: {} - - GraphModule() - - - - def forward(self, arg0_1, arg1_1, arg2_1, arg3_1): - add = torch.ops.aten.add.Tensor(arg2_1, arg3_1); arg2_1 = arg3_1 = None - t = torch.ops.aten.t.default(arg0_1); arg0_1 = None - addmm = torch.ops.aten.addmm.default(arg1_1, add, t); arg1_1 = add = t = None - relu = torch.ops.aten.relu.default(addmm); addmm = None - return (relu,) - - # To see more debug info, please use `graph_module.print_readable()` - - - - .. GENERATED FROM PYTHON SOURCE LINES 95-105 The printed code shows that FX graph only contains ATen-level ops (such as ``torch.ops.aten``) @@ -206,18 +142,6 @@ Other attributes of interest in ``ExportedProgram`` include: print(exported_mod.graph_signature) - - - -.. rst-class:: sphx-glr-script-out - - .. code-block:: none - - ExportGraphSignature(input_specs=[InputSpec(kind=, arg=TensorArgument(name='arg0_1'), target='lin.weight', persistent=None), InputSpec(kind=, arg=TensorArgument(name='arg1_1'), target='lin.bias', persistent=None), InputSpec(kind=, arg=TensorArgument(name='arg2_1'), target=None, persistent=None), InputSpec(kind=, arg=TensorArgument(name='arg3_1'), target=None, persistent=None)], output_specs=[OutputSpec(kind=, arg=TensorArgument(name='relu'), target=None)]) - - - - .. GENERATED FROM PYTHON SOURCE LINES 109-111 See the ``torch.export`` `documentation `__ @@ -258,78 +182,6 @@ A graph break is necessary in cases such as: tb.print_exc() - - - -.. rst-class:: sphx-glr-script-out - - .. code-block:: none - - Traceback (most recent call last): - File "/var/lib/workspace/intermediate_source/torch_export_tutorial.py", line 136, in - export(Bad1(), (torch.randn(3, 3),)) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/export/__init__.py", line 174, in export - return _export( - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/export/_trace.py", line 635, in wrapper - raise e - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/export/_trace.py", line 618, in wrapper - ep = fn(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/export/exported_program.py", line 83, in wrapper - return fn(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/export/_trace.py", line 860, in _export - gm_torch_level = _export_to_torch_ir( - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/export/_trace.py", line 347, in _export_to_torch_ir - gm_torch_level, _ = torch._dynamo.export( - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 1311, in inner - result_traced = opt_f(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl - return self._call_impl(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl - return forward_call(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 451, in _fn - return fn(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl - return self._call_impl(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl - return forward_call(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 921, in catch_errors - return callback(frame, cache_entry, hooks, frame_state, skip=1) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 400, in _convert_frame_assert - return _compile( - File "/opt/conda/envs/py_3.10/lib/python3.10/contextlib.py", line 79, in inner - return func(*args, **kwds) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 676, in _compile - guarded_code = compile_inner(code, one_graph, hooks, transform) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 262, in time_wrapper - r = func(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 535, in compile_inner - out_code = transform_code_object(code, transform) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/bytecode_transformation.py", line 1036, in transform_code_object - transformations(instructions, code_options) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 165, in _fn - return fn(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 500, in transform - tracer.run() - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2149, in run - super().run() - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 810, in run - and self.step() - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 773, in step - getattr(self, inst.opname)(inst) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 464, in inner - raise exc.UserError( - torch._dynamo.exc.UserError: Dynamic control flow is not supported at the moment. Please use functorch.experimental.control_flow.cond to explicitly capture the control flow. For more information about this error, see: https://pytorch.org/docs/main/generated/exportdb/index.html#cond-operands - - from user code: - File "/var/lib/workspace/intermediate_source/torch_export_tutorial.py", line 130, in forward - if x.sum() > 0: - - Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information - - - - - .. GENERATED FROM PYTHON SOURCE LINES 141-142 - accessing tensor data with ``.data`` @@ -350,12 +202,6 @@ A graph break is necessary in cases such as: tb.print_exc() - - - - - - .. GENERATED FROM PYTHON SOURCE LINES 154-155 - calling unsupported functions (such as many built-in functions) @@ -376,88 +222,6 @@ A graph break is necessary in cases such as: tb.print_exc() - - - -.. rst-class:: sphx-glr-script-out - - .. code-block:: none - - Traceback (most recent call last): - File "/var/lib/workspace/intermediate_source/torch_export_tutorial.py", line 162, in - export(Bad3(), (torch.randn(3, 3),)) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/export/__init__.py", line 174, in export - return _export( - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/export/_trace.py", line 635, in wrapper - raise e - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/export/_trace.py", line 618, in wrapper - ep = fn(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/export/exported_program.py", line 83, in wrapper - return fn(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/export/_trace.py", line 860, in _export - gm_torch_level = _export_to_torch_ir( - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/export/_trace.py", line 347, in _export_to_torch_ir - gm_torch_level, _ = torch._dynamo.export( - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 1311, in inner - result_traced = opt_f(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl - return self._call_impl(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl - return forward_call(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 451, in _fn - return fn(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl - return self._call_impl(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl - return forward_call(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 921, in catch_errors - return callback(frame, cache_entry, hooks, frame_state, skip=1) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 400, in _convert_frame_assert - return _compile( - File "/opt/conda/envs/py_3.10/lib/python3.10/contextlib.py", line 79, in inner - return func(*args, **kwds) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 676, in _compile - guarded_code = compile_inner(code, one_graph, hooks, transform) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 262, in time_wrapper - r = func(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 535, in compile_inner - out_code = transform_code_object(code, transform) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/bytecode_transformation.py", line 1036, in transform_code_object - transformations(instructions, code_options) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 165, in _fn - return fn(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 500, in transform - tracer.run() - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2149, in run - super().run() - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 810, in run - and self.step() - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 773, in step - getattr(self, inst.opname)(inst) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 489, in wrapper - return inner_fn(self, inst) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1219, in CALL_FUNCTION - self.call_function(fn, args, {}) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 674, in call_function - self.push(fn.call_function(self, args, kwargs)) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/builtin.py", line 687, in call_function - result = handler(tx, *args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/builtin.py", line 1520, in call_id - unimplemented(f"call_id with args {args}") - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/exc.py", line 190, in unimplemented - raise Unsupported(msg) - torch._dynamo.exc.Unsupported: call_id with args (TensorVariable(),) - - from user code: - File "/var/lib/workspace/intermediate_source/torch_export_tutorial.py", line 159, in forward - return x + id(x) - - Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information - - - - - .. GENERATED FROM PYTHON SOURCE LINES 167-168 - unsupported Python language features (e.g. throwing exceptions, match statements) @@ -482,88 +246,6 @@ A graph break is necessary in cases such as: tb.print_exc() - - - -.. rst-class:: sphx-glr-script-out - - .. code-block:: none - - Traceback (most recent call last): - File "/var/lib/workspace/intermediate_source/torch_export_tutorial.py", line 179, in - export(Bad4(), (torch.randn(3, 3),)) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/export/__init__.py", line 174, in export - return _export( - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/export/_trace.py", line 635, in wrapper - raise e - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/export/_trace.py", line 618, in wrapper - ep = fn(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/export/exported_program.py", line 83, in wrapper - return fn(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/export/_trace.py", line 860, in _export - gm_torch_level = _export_to_torch_ir( - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/export/_trace.py", line 347, in _export_to_torch_ir - gm_torch_level, _ = torch._dynamo.export( - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 1311, in inner - result_traced = opt_f(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl - return self._call_impl(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl - return forward_call(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 451, in _fn - return fn(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl - return self._call_impl(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl - return forward_call(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 921, in catch_errors - return callback(frame, cache_entry, hooks, frame_state, skip=1) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 400, in _convert_frame_assert - return _compile( - File "/opt/conda/envs/py_3.10/lib/python3.10/contextlib.py", line 79, in inner - return func(*args, **kwds) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 676, in _compile - guarded_code = compile_inner(code, one_graph, hooks, transform) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 262, in time_wrapper - r = func(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 535, in compile_inner - out_code = transform_code_object(code, transform) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/bytecode_transformation.py", line 1036, in transform_code_object - transformations(instructions, code_options) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 165, in _fn - return fn(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 500, in transform - tracer.run() - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2149, in run - super().run() - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 810, in run - and self.step() - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 773, in step - getattr(self, inst.opname)(inst) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 489, in wrapper - return inner_fn(self, inst) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1219, in CALL_FUNCTION - self.call_function(fn, args, {}) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 674, in call_function - self.push(fn.call_function(self, args, kwargs)) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/builtin.py", line 730, in call_function - return super().call_function(tx, args, kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/base.py", line 349, in call_function - unimplemented(f"call_function {self} {args} {kwargs}") - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/exc.py", line 190, in unimplemented - raise Unsupported(msg) - torch._dynamo.exc.Unsupported: call_function BuiltinVariable(RuntimeError) [ConstantVariable()] {} - - from user code: - File "/var/lib/workspace/intermediate_source/torch_export_tutorial.py", line 173, in forward - raise RuntimeError("bad") - - Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information - - - - - .. GENERATED FROM PYTHON SOURCE LINES 184-202 Non-Strict Export @@ -599,28 +281,6 @@ Looking at some of the previous examples which resulted in graph breaks: print(bad2_nonstrict.module()(torch.ones(3, 3))) - - - -.. rst-class:: sphx-glr-script-out - - .. code-block:: none - - /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/export/_unlift.py:58: UserWarning: - - Attempted to insert a get_attr Node with no underlying reference in the owning GraphModule! Call GraphModule.add_submodule to add the necessary submodule, GraphModule.add_parameter to add the necessary Parameter, or nn.Module.register_buffer to add the necessary buffer - - /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/graph.py:1460: UserWarning: - - Node _lifted_tensor_constant0_1 target _lifted_tensor_constant0 _lifted_tensor_constant0 of does not reference an nn.Module, nn.Parameter, or buffer, which is what 'get_attr' Nodes typically target - - tensor([[3., 1., 1.], - [1., 1., 1.], - [1., 1., 1.]]) - - - - .. GENERATED FROM PYTHON SOURCE LINES 212-216 - Calling unsupported functions (such as many built-in functions) traces @@ -643,31 +303,6 @@ operation is not recorded in the graph. print(bad3_nonstrict.module()(torch.ones(3, 3))) - - - -.. rst-class:: sphx-glr-script-out - - .. code-block:: none - - ExportedProgram: - class GraphModule(torch.nn.Module): - def forward(self, arg0_1: "f32[3, 3]"): - # No stacktrace found for following nodes - add: "f32[3, 3]" = torch.ops.aten.add.Tensor(arg0_1, 1); arg0_1 = None - add_1: "f32[3, 3]" = torch.ops.aten.add.Tensor(add, 139668425840400); add = None - return (add_1,) - - Graph signature: ExportGraphSignature(input_specs=[InputSpec(kind=, arg=TensorArgument(name='arg0_1'), target=None, persistent=None)], output_specs=[OutputSpec(kind=, arg=TensorArgument(name='add_1'), target=None)]) - Range constraints: {} - - tensor([[1.3967e+14, 1.3967e+14, 1.3967e+14], - [1.3967e+14, 1.3967e+14, 1.3967e+14], - [1.3967e+14, 1.3967e+14, 1.3967e+14]]) - - - - .. GENERATED FROM PYTHON SOURCE LINES 227-229 - Unsupported Python language features (such as throwing exceptions, match @@ -692,20 +327,6 @@ statements) now also get traced through. - - - -.. rst-class:: sphx-glr-script-out - - .. code-block:: none - - tensor([[4., 4., 4.], - [4., 4., 4.], - [4., 4., 4.]]) - - - - .. GENERATED FROM PYTHON SOURCE LINES 244-246 However, there are still some features that require rewrites to the original @@ -740,23 +361,6 @@ we can fix the control flow example above using the ``cond`` op, like so: print(exported_bad1_fixed.module()(-torch.ones(3, 3))) - - - -.. rst-class:: sphx-glr-script-out - - .. code-block:: none - - tensor([[0.8415, 0.8415, 0.8415], - [0.8415, 0.8415, 0.8415], - [0.8415, 0.8415, 0.8415]]) - tensor([[0.5403, 0.5403, 0.5403], - [0.5403, 0.5403, 0.5403], - [0.5403, 0.5403, 0.5403]]) - - - - .. GENERATED FROM PYTHON SOURCE LINES 270-281 There are limitations to ``cond`` that one should be aware of: @@ -826,39 +430,6 @@ with a different shape, we get an error: tb.print_exc() - - - -.. rst-class:: sphx-glr-script-out - - .. code-block:: none - - Traceback (most recent call last): - File "/var/lib/workspace/intermediate_source/torch_export_tutorial.py", line 325, in - exported_mod2.module()(torch.randn(10, 100), torch.randn(10, 100)) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/graph_module.py", line 737, in call_wrapped - return self._wrapped_call(self, *args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/graph_module.py", line 317, in __call__ - raise e - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/graph_module.py", line 304, in __call__ - return super(self.cls, obj).__call__(*args, **kwargs) # type: ignore[misc] - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl - return self._call_impl(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1561, in _call_impl - args_kwargs_result = hook(self, args, kwargs) # type: ignore[misc] - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 451, in _fn - return fn(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/external_utils.py", line 36, in inner - return fn(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/export/_unlift.py", line 32, in _check_input_constraints_pre_hook - return _check_input_constraints_for_graph( - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_export/utils.py", line 129, in _check_input_constraints_for_graph - raise RuntimeError( - RuntimeError: Expected input at *args[0].shape[0] to be equal to 8, but got 10 - - - - .. GENERATED FROM PYTHON SOURCE LINES 330-351 We can relax this constraint using the ``dynamic_shapes`` argument of @@ -923,102 +494,6 @@ dimension must be in the interval [4, 18]. tb.print_exc() - - - -.. rst-class:: sphx-glr-script-out - - .. code-block:: none - - tensor([[[0.0000, 0.0000], - [0.8850, 1.2371], - [0.0000, 0.0000]], - - [[0.0000, 0.0000], - [0.0000, 0.3487], - [0.2520, 1.2545]], - - [[0.5863, 0.2831], - [0.0000, 0.4669], - [0.1059, 0.0000]], - - [[0.7833, 0.0000], - [0.4480, 0.0523], - [0.0000, 0.0000]], - - [[0.9306, 0.0000], - [0.0000, 0.7895], - [0.1160, 0.0000]]]) - Traceback (most recent call last): - File "/var/lib/workspace/intermediate_source/torch_export_tutorial.py", line 372, in - exported_dynamic_shapes_example1.module()(torch.randn(8, 1, 2)) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/graph_module.py", line 737, in call_wrapped - return self._wrapped_call(self, *args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/graph_module.py", line 317, in __call__ - raise e - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/graph_module.py", line 304, in __call__ - return super(self.cls, obj).__call__(*args, **kwargs) # type: ignore[misc] - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl - return self._call_impl(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1561, in _call_impl - args_kwargs_result = hook(self, args, kwargs) # type: ignore[misc] - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 451, in _fn - return fn(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/external_utils.py", line 36, in inner - return fn(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/export/_unlift.py", line 32, in _check_input_constraints_pre_hook - return _check_input_constraints_for_graph( - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_export/utils.py", line 117, in _check_input_constraints_for_graph - raise RuntimeError( - RuntimeError: Expected input at *args[0].shape[1] to be >= 4, but got 1 - Traceback (most recent call last): - File "/var/lib/workspace/intermediate_source/torch_export_tutorial.py", line 377, in - exported_dynamic_shapes_example1.module()(torch.randn(8, 20, 2)) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/graph_module.py", line 737, in call_wrapped - return self._wrapped_call(self, *args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/graph_module.py", line 317, in __call__ - raise e - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/graph_module.py", line 304, in __call__ - return super(self.cls, obj).__call__(*args, **kwargs) # type: ignore[misc] - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl - return self._call_impl(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1561, in _call_impl - args_kwargs_result = hook(self, args, kwargs) # type: ignore[misc] - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 451, in _fn - return fn(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/external_utils.py", line 36, in inner - return fn(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/export/_unlift.py", line 32, in _check_input_constraints_pre_hook - return _check_input_constraints_for_graph( - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_export/utils.py", line 123, in _check_input_constraints_for_graph - raise RuntimeError( - RuntimeError: Expected input at *args[0].shape[1] to be <= 18, but got 20 - Traceback (most recent call last): - File "/var/lib/workspace/intermediate_source/torch_export_tutorial.py", line 382, in - exported_dynamic_shapes_example1.module()(torch.randn(8, 8, 3)) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/graph_module.py", line 737, in call_wrapped - return self._wrapped_call(self, *args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/graph_module.py", line 317, in __call__ - raise e - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/graph_module.py", line 304, in __call__ - return super(self.cls, obj).__call__(*args, **kwargs) # type: ignore[misc] - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl - return self._call_impl(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1561, in _call_impl - args_kwargs_result = hook(self, args, kwargs) # type: ignore[misc] - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 451, in _fn - return fn(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/external_utils.py", line 36, in inner - return fn(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/export/_unlift.py", line 32, in _check_input_constraints_pre_hook - return _check_input_constraints_for_graph( - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_export/utils.py", line 129, in _check_input_constraints_for_graph - raise RuntimeError( - RuntimeError: Expected input at *args[0].shape[2] to be equal to 2, but got 3 - - - - .. GENERATED FROM PYTHON SOURCE LINES 387-389 Note that if our example inputs to ``torch.export`` do not satisfy the constraints @@ -1040,132 +515,6 @@ given by ``dynamic_shapes``, then we get an error. tb.print_exc() - - - -.. rst-class:: sphx-glr-script-out - - .. code-block:: none - - Traceback (most recent call last): - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/export/_trace.py", line 347, in _export_to_torch_ir - gm_torch_level, _ = torch._dynamo.export( - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 1354, in inner - raise constraint_violation_error - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 1311, in inner - result_traced = opt_f(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl - return self._call_impl(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl - return forward_call(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 451, in _fn - return fn(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl - return self._call_impl(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl - return forward_call(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 921, in catch_errors - return callback(frame, cache_entry, hooks, frame_state, skip=1) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 400, in _convert_frame_assert - return _compile( - File "/opt/conda/envs/py_3.10/lib/python3.10/contextlib.py", line 79, in inner - return func(*args, **kwds) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 676, in _compile - guarded_code = compile_inner(code, one_graph, hooks, transform) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 262, in time_wrapper - r = func(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 535, in compile_inner - out_code = transform_code_object(code, transform) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/bytecode_transformation.py", line 1036, in transform_code_object - transformations(instructions, code_options) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 165, in _fn - return fn(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 482, in transform - tracer = InstructionTranslator( - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2115, in __init__ - self.symbolic_locals = VariableTracker.apply( - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/base.py", line 217, in apply - result = { - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/base.py", line 218, in - k: cls.apply(fn, v, cache, skip_fn) for k, v in list(value.items()) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/base.py", line 203, in apply - result = fn(update_object_dict(value)) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2116, in - lambda x: x.realize(), self.symbolic_locals - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/lazy.py", line 58, in realize - self._cache.realize(self.parents_tracker) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/lazy.py", line 24, in realize - self.vt = VariableBuilder(tx, self.source)(self.value) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/builder.py", line 269, in __call__ - vt = self._wrap(value) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/builder.py", line 402, in _wrap - return type_dispatch(self, value) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/builder.py", line 1073, in wrap_tensor - tensor_variable = wrap_fx_proxy( - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/builder.py", line 1330, in wrap_fx_proxy - return wrap_fx_proxy_cls(target_cls=TensorVariable, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/builder.py", line 1440, in wrap_fx_proxy_cls - example_value = wrap_to_fake_tensor_and_record( - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/builder.py", line 1880, in wrap_to_fake_tensor_and_record - fake_e = wrap_fake_exception( - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 1190, in wrap_fake_exception - return fn() - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/builder.py", line 1881, in - lambda: tx.fake_mode.from_tensor( - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_subclasses/fake_tensor.py", line 1666, in from_tensor - return self.fake_tensor_converter( - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_subclasses/fake_tensor.py", line 349, in __call__ - return self.from_real_tensor( - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_subclasses/fake_tensor.py", line 306, in from_real_tensor - out = self.meta_converter( - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_subclasses/meta_utils.py", line 967, in __call__ - r = self.meta_tensor( - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_subclasses/meta_utils.py", line 782, in meta_tensor - ) = sym_sizes_strides_storage_offset(t, source, symbolic_context) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_subclasses/meta_utils.py", line 269, in sym_sizes_strides_storage_offset - return shape_env.create_symbolic_sizes_strides_storage_offset( - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/experimental/symbolic_shapes.py", line 2339, in create_symbolic_sizes_strides_storage_offset - return self._create_symbolic_sizes_strides_storage_offset( - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/experimental/recording.py", line 231, in wrapper - return fn(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/experimental/symbolic_shapes.py", line 2395, in _create_symbolic_sizes_strides_storage_offset - size: List[sympy.Expr] = self._produce_dyn_sizes_from_int_tuple(ex_size, source, symbolic_context) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/experimental/symbolic_shapes.py", line 2271, in _produce_dyn_sizes_from_int_tuple - size.append(self.create_symbol( - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/experimental/recording.py", line 231, in wrapper - return fn(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/experimental/symbolic_shapes.py", line 2709, in create_symbol - raise ConstraintViolationError(f"{val} not in range [{vr.lower}, {vr.upper}]") - torch.fx.experimental.symbolic_shapes.ConstraintViolationError: 10 not in range [11, 18] - - Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information - - - During handling of the above exception, another exception occurred: - - Traceback (most recent call last): - File "/var/lib/workspace/intermediate_source/torch_export_tutorial.py", line 396, in - export(DynamicShapesExample1(), (inp1,), dynamic_shapes=dynamic_shapes1_bad) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/export/__init__.py", line 174, in export - return _export( - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/export/_trace.py", line 635, in wrapper - raise e - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/export/_trace.py", line 618, in wrapper - ep = fn(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/export/exported_program.py", line 83, in wrapper - return fn(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/export/_trace.py", line 860, in _export - gm_torch_level = _export_to_torch_ir( - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/export/_trace.py", line 359, in _export_to_torch_ir - raise UserError(UserErrorType.CONSTRAINT_VIOLATION, str(e)) # noqa: TRY200 - torch._dynamo.exc.UserError: 10 not in range [11, 18] - - Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information - - - - - .. GENERATED FROM PYTHON SOURCE LINES 401-403 We can enforce that equalities between dimensions of different tensors @@ -1202,41 +551,6 @@ by using the same ``torch.export.Dim`` object, for example, in matrix multiplica tb.print_exc() - - - -.. rst-class:: sphx-glr-script-out - - .. code-block:: none - - tensor([[-2.9354, -2.2066, -0.2080, 4.6121], - [-0.5658, -0.6108, 0.8887, 1.5908]]) - Traceback (most recent call last): - File "/var/lib/workspace/intermediate_source/torch_export_tutorial.py", line 425, in - exported_dynamic_shapes_example2.module()(torch.randn(4, 8), torch.randn(4, 2)) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/graph_module.py", line 737, in call_wrapped - return self._wrapped_call(self, *args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/graph_module.py", line 317, in __call__ - raise e - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/graph_module.py", line 304, in __call__ - return super(self.cls, obj).__call__(*args, **kwargs) # type: ignore[misc] - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl - return self._call_impl(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1561, in _call_impl - args_kwargs_result = hook(self, args, kwargs) # type: ignore[misc] - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 451, in _fn - return fn(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/external_utils.py", line 36, in inner - return fn(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/export/_unlift.py", line 32, in _check_input_constraints_pre_hook - return _check_input_constraints_for_graph( - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_export/utils.py", line 85, in _check_input_constraints_for_graph - raise RuntimeError( - RuntimeError: Expected input at *args[1].shape[0] to be equal to 8, but got 4 - - - - .. GENERATED FROM PYTHON SOURCE LINES 430-433 We can also describe one dimension in terms of other. There are some @@ -1285,41 +599,6 @@ but generally, those in the form of ``A * Dim + B`` should work. print(derived_dim_example2.module()(torch.randn(7), torch.randn(19))) - - - -.. rst-class:: sphx-glr-script-out - - .. code-block:: none - - tensor([ 0.3007, -1.7282, -0.0729, 0.1139]) - Traceback (most recent call last): - File "/var/lib/workspace/intermediate_source/torch_export_tutorial.py", line 450, in - derived_dim_example1.module()(torch.randn(4), torch.randn(6)) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/graph_module.py", line 737, in call_wrapped - return self._wrapped_call(self, *args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/graph_module.py", line 317, in __call__ - raise e - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/graph_module.py", line 304, in __call__ - return super(self.cls, obj).__call__(*args, **kwargs) # type: ignore[misc] - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl - return self._call_impl(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1561, in _call_impl - args_kwargs_result = hook(self, args, kwargs) # type: ignore[misc] - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 451, in _fn - return fn(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/external_utils.py", line 36, in inner - return fn(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/export/_unlift.py", line 32, in _check_input_constraints_pre_hook - return _check_input_constraints_for_graph( - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_export/utils.py", line 85, in _check_input_constraints_for_graph - raise RuntimeError( - RuntimeError: Expected input at *args[1].shape[0] to be equal to 5, but got 6 - tensor([ 2.5416, -0.2760, 0.9003, -1.7479, -2.9716, 0.1013]) - - - - .. GENERATED FROM PYTHON SOURCE LINES 471-476 We can actually use ``torch.export`` to guide us as to which ``dynamic_shapes`` constraints @@ -1353,96 +632,6 @@ error out. tb.print_exc() - - - -.. rst-class:: sphx-glr-script-out - - .. code-block:: none - - Traceback (most recent call last): - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/export/_trace.py", line 347, in _export_to_torch_ir - gm_torch_level, _ = torch._dynamo.export( - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 1354, in inner - raise constraint_violation_error - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 1311, in inner - result_traced = opt_f(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl - return self._call_impl(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl - return forward_call(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 451, in _fn - return fn(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl - return self._call_impl(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl - return forward_call(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 921, in catch_errors - return callback(frame, cache_entry, hooks, frame_state, skip=1) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 400, in _convert_frame_assert - return _compile( - File "/opt/conda/envs/py_3.10/lib/python3.10/contextlib.py", line 79, in inner - return func(*args, **kwds) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 676, in _compile - guarded_code = compile_inner(code, one_graph, hooks, transform) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 262, in time_wrapper - r = func(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 634, in compile_inner - check_fn = CheckFunctionManager( - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/guards.py", line 1048, in __init__ - guard.create(builder) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_guards.py", line 249, in create - return self.create_fn(builder, self) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/guards.py", line 705, in SHAPE_ENV - guards = output_graph.shape_env.produce_guards( - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/experimental/symbolic_shapes.py", line 3308, in produce_guards - raise ConstraintViolationError( - torch.fx.experimental.symbolic_shapes.ConstraintViolationError: Constraints violated (inp4_dim0, inp5_dim0, inp5_dim1)! For more information, run with TORCH_LOGS="+dynamic". - - The values of inp5_dim0 = L['y'].size()[0] and inp4_dim1 = L['x'].size()[1] must always be equal. - - Not all values of inp5_dim1 = L['y'].size()[1] in the specified range satisfy the generated guard Ne(L['y'].size()[1], 16). - - Not all values of inp4_dim0 = L['x'].size()[0] in the specified range satisfy the generated guard 2 <= L['x'].size()[0] and L['x'].size()[0] <= 16 - - Not all values of inp5_dim1 = L['y'].size()[1] in the specified range satisfy the generated guard 16 <= L['y'].size()[1] and L['y'].size()[1] <= 9223372036854775806 - - Suggested fixes: - inp4_dim0 = Dim('inp4_dim0', max=16) - inp4_dim1 = Dim('inp4_dim1') - inp4_dim1 = Dim('inp4_dim1', min=2, max=9223372036854775806) # 2 <= inp4_dim1 <= 9223372036854775806 - inp5_dim1 = Dim('inp5_dim1', min=16) - inp5_dim0 = inp4_dim1 - - During handling of the above exception, another exception occurred: - - Traceback (most recent call last): - File "/var/lib/workspace/intermediate_source/torch_export_tutorial.py", line 492, in - export(DynamicShapesExample3(), (inp4, inp5), dynamic_shapes=dynamic_shapes3) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/export/__init__.py", line 174, in export - return _export( - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/export/_trace.py", line 635, in wrapper - raise e - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/export/_trace.py", line 618, in wrapper - ep = fn(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/export/exported_program.py", line 83, in wrapper - return fn(*args, **kwargs) - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/export/_trace.py", line 860, in _export - gm_torch_level = _export_to_torch_ir( - File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/export/_trace.py", line 359, in _export_to_torch_ir - raise UserError(UserErrorType.CONSTRAINT_VIOLATION, str(e)) # noqa: TRY200 - torch._dynamo.exc.UserError: Constraints violated (inp4_dim0, inp5_dim0, inp5_dim1)! For more information, run with TORCH_LOGS="+dynamic". - - The values of inp5_dim0 = L['y'].size()[0] and inp4_dim1 = L['x'].size()[1] must always be equal. - - Not all values of inp5_dim1 = L['y'].size()[1] in the specified range satisfy the generated guard Ne(L['y'].size()[1], 16). - - Not all values of inp4_dim0 = L['x'].size()[0] in the specified range satisfy the generated guard 2 <= L['x'].size()[0] and L['x'].size()[0] <= 16 - - Not all values of inp5_dim1 = L['y'].size()[1] in the specified range satisfy the generated guard 16 <= L['y'].size()[1] and L['y'].size()[1] <= 9223372036854775806 - - Suggested fixes: - inp4_dim0 = Dim('inp4_dim0', max=16) - inp4_dim1 = Dim('inp4_dim1') - inp4_dim1 = Dim('inp4_dim1', min=2, max=9223372036854775806) # 2 <= inp4_dim1 <= 9223372036854775806 - inp5_dim1 = Dim('inp5_dim1', min=16) - inp5_dim0 = inp4_dim1 - - - - .. GENERATED FROM PYTHON SOURCE LINES 497-500 We can see that the error message gives us suggested fixes to our @@ -1471,29 +660,6 @@ suggestions may differ slightly): print(exported_dynamic_shapes_example3.module()(torch.randn(4, 32), torch.randn(32, 64))) - - - -.. rst-class:: sphx-glr-script-out - - .. code-block:: none - - tensor([[ 12.5915, -0.7265, -1.8981, -8.0323, -2.1447, 14.4020, 13.7854, - 1.5568, 3.7933, 7.2591, -2.2477, 1.6366, -5.9276, 8.5279, - 7.9349, -1.1328], - [ -5.2210, 9.4576, -0.2372, 9.0035, 6.2572, -8.4716, 6.0191, - 4.8424, -0.4486, 0.1885, -0.1749, 2.4314, 3.8271, 8.1822, - 6.5064, 0.6512], - [ -3.6856, 7.5222, 4.8073, 13.1255, 3.6440, -4.1587, 2.9806, - 0.3689, 1.1133, -1.7169, -2.1537, 1.1841, 6.7619, 9.3401, - -1.1372, -8.9628], - [ -4.3608, 5.1219, -0.6240, -6.8640, 2.3344, -2.0273, 0.2769, - -0.9930, 2.0298, -10.3922, -1.7186, -5.0928, 11.7383, 6.4864, - 8.0827, 0.5863]]) - - - - .. GENERATED FROM PYTHON SOURCE LINES 518-525 Note that in the example above, because we constrained the value of ``x.shape[0]`` in @@ -1517,51 +683,6 @@ or use ``torch._logging.set_logs``. torch._logging.set_logs(dynamic=logging.WARNING, dynamo=logging.WARNING) - - - -.. rst-class:: sphx-glr-script-out - - .. code-block:: none - - I0625 21:38:32.787000 139677001331328 torch/_dynamo/logging.py:55] [16/0] Step 1: torchdynamo start tracing forward /var/lib/workspace/intermediate_source/torch_export_tutorial.py:481 - I0625 21:38:32.790000 139677001331328 torch/fx/experimental/symbolic_shapes.py:2724] [16/0] create_symbol s0 = 8 for L['x'].size()[0] [2, 16] (_dynamo/variables/builder.py:1881 in ) - I0625 21:38:32.791000 139677001331328 torch/fx/experimental/symbolic_shapes.py:2724] [16/0] create_symbol s1 = 16 for L['x'].size()[1] [2, 9223372036854775806] (_dynamo/variables/builder.py:1881 in ) - I0625 21:38:32.793000 139677001331328 torch/fx/experimental/symbolic_shapes.py:2724] [16/0] create_symbol s2 = 16 for L['y'].size()[0] [2, 9223372036854775806] (_dynamo/variables/builder.py:1881 in ) - I0625 21:38:32.794000 139677001331328 torch/fx/experimental/symbolic_shapes.py:2724] [16/0] create_symbol s3 = 32 for L['y'].size()[1] [17, 9223372036854775806] (_dynamo/variables/builder.py:1881 in ) - I0625 21:38:32.802000 139677001331328 torch/fx/experimental/symbolic_shapes.py:3809] [16/0] set_replacement s2 = s1 (solve_backed) ValueRanges(lower=2, upper=9223372036854775806, is_bool=False) - I0625 21:38:32.802000 139677001331328 torch/fx/experimental/symbolic_shapes.py:4035] [16/0] eval Eq(s1, s2) [guard added] at ar/lib/workspace/intermediate_source/torch_export_tutorial.py:483 in forward (_meta_registrations.py:2014 in meta_mm) - I0625 21:38:32.803000 139677001331328 torch/_dynamo/logging.py:55] [16/0] Step 1: torchdynamo done tracing forward (RETURN_VALUE) - I0625 21:38:32.805000 139677001331328 torch/fx/experimental/symbolic_shapes.py:3809] [16/0] set_replacement s2 = s1 (find) ValueRanges(lower=2, upper=9223372036854775806, is_bool=False) - I0625 21:38:32.805000 139677001331328 torch/_dynamo/logging.py:55] [16/0] Step 2: calling compiler function dynamo_normalization_capturing_compiler - I0625 21:38:32.805000 139677001331328 torch/_dynamo/logging.py:55] [16/0] Step 2: done compiler function dynamo_normalization_capturing_compiler - I0625 21:38:32.807000 139677001331328 torch/fx/experimental/symbolic_shapes.py:2806] [16/0] produce_guards - I0625 21:38:32.834000 139677001331328 torch/_dynamo/eval_frame.py:1339] Summary of dimension constraints: - I0625 21:38:32.834000 139677001331328 torch/_dynamo/eval_frame.py:1339] Suggested fixes: - I0625 21:38:32.834000 139677001331328 torch/_dynamo/eval_frame.py:1339] inp4_dim0 = Dim('inp4_dim0', max=16) - I0625 21:38:32.834000 139677001331328 torch/_dynamo/eval_frame.py:1339] inp5_dim1 = Dim('inp5_dim1', min=17) - I0625 21:38:32.834000 139677001331328 torch/_dynamo/eval_frame.py:1339] shared_dim = Dim('shared_dim') - I0625 21:38:32.834000 139677001331328 torch/_dynamo/eval_frame.py:1363] Dynamo captured graph: - I0625 21:38:32.834000 139677001331328 torch/_dynamo/eval_frame.py:1363] - I0625 21:38:32.834000 139677001331328 torch/_dynamo/eval_frame.py:1363] class GraphModule(torch.nn.Module): - I0625 21:38:32.834000 139677001331328 torch/_dynamo/eval_frame.py:1363] def forward(self, L_x_ : torch.Tensor, L_y_ : torch.Tensor): - I0625 21:38:32.834000 139677001331328 torch/_dynamo/eval_frame.py:1363] l_x_ = L_x_ - I0625 21:38:32.834000 139677001331328 torch/_dynamo/eval_frame.py:1363] l_y_ = L_y_ - I0625 21:38:32.834000 139677001331328 torch/_dynamo/eval_frame.py:1363] - I0625 21:38:32.834000 139677001331328 torch/_dynamo/eval_frame.py:1363] # File: /var/lib/workspace/intermediate_source/torch_export_tutorial.py:482 in forward, code: if x.shape[0] <= 16: - I0625 21:38:32.834000 139677001331328 torch/_dynamo/eval_frame.py:1363] size = l_x_.size() - I0625 21:38:32.834000 139677001331328 torch/_dynamo/eval_frame.py:1363] getitem = size[0]; size = None - I0625 21:38:32.834000 139677001331328 torch/_dynamo/eval_frame.py:1363] le = getitem <= 16; getitem = None - I0625 21:38:32.834000 139677001331328 torch/_dynamo/eval_frame.py:1363] - I0625 21:38:32.834000 139677001331328 torch/_dynamo/eval_frame.py:1363] # File: /var/lib/workspace/intermediate_source/torch_export_tutorial.py:483 in forward, code: return x @ y[:, :16] - I0625 21:38:32.834000 139677001331328 torch/_dynamo/eval_frame.py:1363] getitem_2 = l_y_[(slice(None, None, None), slice(None, 16, None))]; l_y_ = None - I0625 21:38:32.834000 139677001331328 torch/_dynamo/eval_frame.py:1363] matmul = l_x_ @ getitem_2; l_x_ = getitem_2 = None - I0625 21:38:32.834000 139677001331328 torch/_dynamo/eval_frame.py:1363] return (matmul,) - I0625 21:38:32.834000 139677001331328 torch/_dynamo/eval_frame.py:1363] - - - - .. GENERATED FROM PYTHON SOURCE LINES 534-536 We can view an ``ExportedProgram``'s symbolic shape ranges using the @@ -1575,18 +696,6 @@ We can view an ``ExportedProgram``'s symbolic shape ranges using the print(exported_dynamic_shapes_example3.range_constraints) - - - -.. rst-class:: sphx-glr-script-out - - .. code-block:: none - - {s0: ValueRanges(lower=2, upper=16, is_bool=False), s1: ValueRanges(lower=2, upper=oo, is_bool=False), s3: ValueRanges(lower=17, upper=oo, is_bool=False)} - - - - .. GENERATED FROM PYTHON SOURCE LINES 540-549 Custom Ops @@ -1616,12 +725,6 @@ Currently, the steps to register a custom op for use by ``torch.export`` are: return torch.relu(x) - - - - - - .. GENERATED FROM PYTHON SOURCE LINES 562-564 - Define a ``"Meta"`` implementation of the custom op that returns an empty @@ -1637,12 +740,6 @@ Currently, the steps to register a custom op for use by ``torch.export`` are: return torch.empty_like(x) - - - - - - .. GENERATED FROM PYTHON SOURCE LINES 570-571 - Call the custom op from the code you want to export using ``torch.ops`` @@ -1660,12 +757,6 @@ Currently, the steps to register a custom op for use by ``torch.export`` are: return x - - - - - - .. GENERATED FROM PYTHON SOURCE LINES 580-581 - Export the code as before @@ -1680,33 +771,6 @@ Currently, the steps to register a custom op for use by ``torch.export`` are: print(exported_custom_op_example.module()(torch.randn(3, 3))) - - - -.. rst-class:: sphx-glr-script-out - - .. code-block:: none - - class GraphModule(torch.nn.Module): - def forward(self, arg0_1: "f32[3, 3]"): - # File: /var/lib/workspace/intermediate_source/torch_export_tutorial.py:574 in forward, code: x = torch.sin(x) - sin: "f32[3, 3]" = torch.ops.aten.sin.default(arg0_1); arg0_1 = None - - # File: /var/lib/workspace/intermediate_source/torch_export_tutorial.py:575 in forward, code: x = torch.ops.my_custom_library.custom_op(x) - custom_op: "f32[3, 3]" = torch.ops.my_custom_library.custom_op.default(sin); sin = None - - # File: /var/lib/workspace/intermediate_source/torch_export_tutorial.py:576 in forward, code: x = torch.cos(x) - cos: "f32[3, 3]" = torch.ops.aten.cos.default(custom_op); custom_op = None - return (cos,) - - custom_op called! - tensor([[0.6387, 0.5722, 1.0000], - [0.8776, 0.9846, 0.8223], - [0.9983, 1.0000, 0.9979]]) - - - - .. GENERATED FROM PYTHON SOURCE LINES 587-594 Note in the above outputs that the custom op is included in the exported graph. @@ -1739,19 +803,6 @@ and you can inspect if an operator is functional by checking print(torch.ops.aten.add_.Tensor._schema.is_mutable) - - - -.. rst-class:: sphx-glr-script-out - - .. code-block:: none - - False - True - - - - .. GENERATED FROM PYTHON SOURCE LINES 611-633 By default, the environment in which you want to run the exported graph @@ -1797,31 +848,6 @@ which consists of only ~180 operators. print(core_ir_ep.graph) - - - -.. rst-class:: sphx-glr-script-out - - .. code-block:: none - - graph(): - %arg0_1 : [num_users=1] = placeholder[target=arg0_1] - %arg1_1 : [num_users=1] = placeholder[target=arg1_1] - %arg2_1 : [num_users=1] = placeholder[target=arg2_1] - %t : [num_users=1] = call_function[target=torch.ops.aten.t.default](args = (%arg0_1,), kwargs = {}) - %addmm : [num_users=1] = call_function[target=torch.ops.aten.addmm.default](args = (%arg1_1, %arg2_1, %t), kwargs = {}) - return (addmm,) - graph(): - %arg0_1 : [num_users=1] = placeholder[target=arg0_1] - %arg1_1 : [num_users=1] = placeholder[target=arg1_1] - %arg2_1 : [num_users=1] = placeholder[target=arg2_1] - %permute : [num_users=1] = call_function[target=torch.ops.aten.permute.default](args = (%arg0_1, [1, 0]), kwargs = {}) - %addmm : [num_users=1] = call_function[target=torch.ops.aten.addmm.default](args = (%arg1_1, %arg2_1, %permute), kwargs = {}) - return (addmm,) - - - - .. GENERATED FROM PYTHON SOURCE LINES 649-661 Notice that after running ``run_decompositions`` the @@ -1859,31 +885,6 @@ decomposition implementations. print(core_ir_ep.graph) - - - -.. rst-class:: sphx-glr-script-out - - .. code-block:: none - - graph(): - %arg0_1 : [num_users=1] = placeholder[target=arg0_1] - %arg1_1 : [num_users=1] = placeholder[target=arg1_1] - %arg2_1 : [num_users=1] = placeholder[target=arg2_1] - %t : [num_users=1] = call_function[target=torch.ops.aten.t.default](args = (%arg0_1,), kwargs = {}) - %addmm : [num_users=1] = call_function[target=torch.ops.aten.addmm.default](args = (%arg1_1, %arg2_1, %t), kwargs = {}) - return (addmm,) - graph(): - %arg0_1 : [num_users=1] = placeholder[target=arg0_1] - %arg1_1 : [num_users=1] = placeholder[target=arg1_1] - %arg2_1 : [num_users=1] = placeholder[target=arg2_1] - %permute : [num_users=1] = call_function[target=torch.ops.aten.permute.default](args = (%arg0_1, [1, 0]), kwargs = {}) - %addmm : [num_users=1] = call_function[target=torch.ops.aten.addmm.default](args = (%arg1_1, %arg2_1, %permute), kwargs = {}) - return (addmm,) - - - - .. GENERATED FROM PYTHON SOURCE LINES 679-682 If there is no existing decomposition function for an ATen operator that you would @@ -1924,12 +925,6 @@ We can look at the example called ``cond_predicate``, which has a ``torch.cond`` return cond(pred, lambda x: x.cos(), lambda y: y.sin(), [x]) - - - - - - .. GENERATED FROM PYTHON SOURCE LINES 711-719 More generally, ExportDB can be used as a reference when one of the following occurs: @@ -1980,23 +975,6 @@ or `TensorRT `__. print(res) - - - -.. rst-class:: sphx-glr-script-out - - .. code-block:: none - - tensor([[ 1.3676, -0.4303, -0.2113], - [-0.5053, -0.0877, 0.5134]], device='cuda:0', - grad_fn=) - tensor([[ 1.3676, -0.4303, -0.2113], - [-0.5053, -0.0877, 0.5134]], device='cuda:0', - grad_fn=) - - - - .. GENERATED FROM PYTHON SOURCE LINES 753-767 .. code-block:: python @@ -2026,7 +1004,7 @@ and considerations (control flow ops, constraints, etc.) that need to be made in .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 0 minutes 1.776 seconds) + **Total running time of the script:** ( 0 minutes 0.000 seconds) .. _sphx_glr_download_intermediate_torch_export_tutorial.py: diff --git a/_sources/intermediate/torchvision_tutorial.rst.txt b/_sources/intermediate/torchvision_tutorial.rst.txt index b387bda675..9293743f1d 100644 --- a/_sources/intermediate/torchvision_tutorial.rst.txt +++ b/_sources/intermediate/torchvision_tutorial.rst.txt @@ -157,7 +157,7 @@ Here is one example of a pair of images and segmentation masks .. code-block:: none - + @@ -321,10 +321,10 @@ way of doing it: Downloading: "https://download.pytorch.org/models/fasterrcnn_resnet50_fpn_coco-258fb6c6.pth" to /var/lib/ci-user/.cache/torch/hub/checkpoints/fasterrcnn_resnet50_fpn_coco-258fb6c6.pth 0%| | 0.00/160M [00:00 + @@ -851,7 +849,7 @@ the torchvision repository. .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 0 minutes 49.792 seconds) + **Total running time of the script:** ( 0 minutes 49.338 seconds) .. _sphx_glr_download_intermediate_torchvision_tutorial.py: diff --git a/_sources/prototype/sg_execution_times.rst.txt b/_sources/prototype/sg_execution_times.rst.txt index 26e91b8e4b..8f90959997 100644 --- a/_sources/prototype/sg_execution_times.rst.txt +++ b/_sources/prototype/sg_execution_times.rst.txt @@ -5,7 +5,7 @@ Computation times ================= -**00:00.029** total execution time for **prototype** files: +**00:00.028** total execution time for **prototype** files: +-----------------------------------------------------------------------------------------------------------+-----------+--------+ | :ref:`sphx_glr_prototype_maskedtensor_advanced_semantics.py` (``maskedtensor_advanced_semantics.py``) | 00:00.025 | 0.0 MB | @@ -14,10 +14,10 @@ Computation times +-----------------------------------------------------------------------------------------------------------+-----------+--------+ | :ref:`sphx_glr_prototype_maskedtensor_overview.py` (``maskedtensor_overview.py``) | 00:00.001 | 0.0 MB | +-----------------------------------------------------------------------------------------------------------+-----------+--------+ -| :ref:`sphx_glr_prototype_numeric_suite_tutorial.py` (``numeric_suite_tutorial.py``) | 00:00.001 | 0.0 MB | -+-----------------------------------------------------------------------------------------------------------+-----------+--------+ | :ref:`sphx_glr_prototype_maskedtensor_adagrad.py` (``maskedtensor_adagrad.py``) | 00:00.001 | 0.0 MB | +-----------------------------------------------------------------------------------------------------------+-----------+--------+ +| :ref:`sphx_glr_prototype_numeric_suite_tutorial.py` (``numeric_suite_tutorial.py``) | 00:00.001 | 0.0 MB | ++-----------------------------------------------------------------------------------------------------------+-----------+--------+ | :ref:`sphx_glr_prototype_maskedtensor_sparsity.py` (``maskedtensor_sparsity.py``) | 00:00.000 | 0.0 MB | +-----------------------------------------------------------------------------------------------------------+-----------+--------+ | :ref:`sphx_glr_prototype_fx_graph_mode_ptq_dynamic.py` (``fx_graph_mode_ptq_dynamic.py``) | 00:00.000 | 0.0 MB | diff --git a/_sources/recipes/compiling_optimizer_lr_scheduler.rst.txt b/_sources/recipes/compiling_optimizer_lr_scheduler.rst.txt index e95d8cb824..3dded62004 100644 --- a/_sources/recipes/compiling_optimizer_lr_scheduler.rst.txt +++ b/_sources/recipes/compiling_optimizer_lr_scheduler.rst.txt @@ -167,31 +167,31 @@ LR in a tensor. .. code-block:: none - V0625 22:03:41.119000 139677001331328 torch/_dynamo/guards.py:1425] [__recompiles] Recompiling function step in /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/optim/adam.py:135 - V0625 22:03:41.119000 139677001331328 torch/_dynamo/guards.py:1425] [__recompiles] triggered by the following guard failure(s): - V0625 22:03:41.119000 139677001331328 torch/_dynamo/guards.py:1425] [__recompiles] - ___key_to_id(L['self'].state) == [139670086608880,139670086608960,139670086601920,139670086603360,139670086600320,139670086599760,139670086606880,139670086609760,139670086612080,139670086602480] - V0625 22:03:44.430000 139677001331328 torch/_dynamo/guards.py:1425] [__recompiles] Recompiling function step in /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/optim/adam.py:135 - V0625 22:03:44.430000 139677001331328 torch/_dynamo/guards.py:1425] [__recompiles] triggered by the following guard failure(s): - V0625 22:03:44.430000 139677001331328 torch/_dynamo/guards.py:1425] [__recompiles] - L['self'].param_groups[0]['lr'] == 0.003333333333333333 - V0625 22:03:44.430000 139677001331328 torch/_dynamo/guards.py:1425] [__recompiles] - G['__optimizer_139668517417792_139670031288176_c79']() is not None - V0625 22:03:46.979000 139677001331328 torch/_dynamo/guards.py:1425] [__recompiles] Recompiling function step in /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/optim/adam.py:135 - V0625 22:03:46.979000 139677001331328 torch/_dynamo/guards.py:1425] [__recompiles] triggered by the following guard failure(s): - V0625 22:03:46.979000 139677001331328 torch/_dynamo/guards.py:1425] [__recompiles] - L['self'].param_groups[0]['lr'] == 0.004666666666666667 - V0625 22:03:46.979000 139677001331328 torch/_dynamo/guards.py:1425] [__recompiles] - L['self'].param_groups[0]['lr'] == 0.003333333333333333 - V0625 22:03:46.979000 139677001331328 torch/_dynamo/guards.py:1425] [__recompiles] - G['__optimizer_139668517417792_139670031288176_c79']() is not None - V0625 22:03:49.513000 139677001331328 torch/_dynamo/guards.py:1425] [__recompiles] Recompiling function step in /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/optim/adam.py:135 - V0625 22:03:49.513000 139677001331328 torch/_dynamo/guards.py:1425] [__recompiles] triggered by the following guard failure(s): - V0625 22:03:49.513000 139677001331328 torch/_dynamo/guards.py:1425] [__recompiles] - L['self'].param_groups[0]['lr'] == 0.006000000000000001 - V0625 22:03:49.513000 139677001331328 torch/_dynamo/guards.py:1425] [__recompiles] - L['self'].param_groups[0]['lr'] == 0.004666666666666667 - V0625 22:03:49.513000 139677001331328 torch/_dynamo/guards.py:1425] [__recompiles] - L['self'].param_groups[0]['lr'] == 0.003333333333333333 - V0625 22:03:49.513000 139677001331328 torch/_dynamo/guards.py:1425] [__recompiles] - G['__optimizer_139668517417792_139670031288176_c79']() is not None - V0625 22:03:52.060000 139677001331328 torch/_dynamo/guards.py:1425] [__recompiles] Recompiling function step in /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/optim/adam.py:135 - V0625 22:03:52.060000 139677001331328 torch/_dynamo/guards.py:1425] [__recompiles] triggered by the following guard failure(s): - V0625 22:03:52.060000 139677001331328 torch/_dynamo/guards.py:1425] [__recompiles] - L['self'].param_groups[0]['lr'] == 0.007333333333333335 - V0625 22:03:52.060000 139677001331328 torch/_dynamo/guards.py:1425] [__recompiles] - L['self'].param_groups[0]['lr'] == 0.006000000000000001 - V0625 22:03:52.060000 139677001331328 torch/_dynamo/guards.py:1425] [__recompiles] - L['self'].param_groups[0]['lr'] == 0.004666666666666667 - V0625 22:03:52.060000 139677001331328 torch/_dynamo/guards.py:1425] [__recompiles] - L['self'].param_groups[0]['lr'] == 0.003333333333333333 - V0625 22:03:52.060000 139677001331328 torch/_dynamo/guards.py:1425] [__recompiles] - G['__optimizer_139668517417792_139670031288176_c79']() is not None + V0625 22:40:10.944000 139900085105280 torch/_dynamo/guards.py:1425] [__recompiles] Recompiling function step in /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/optim/adam.py:135 + V0625 22:40:10.944000 139900085105280 torch/_dynamo/guards.py:1425] [__recompiles] triggered by the following guard failure(s): + V0625 22:40:10.944000 139900085105280 torch/_dynamo/guards.py:1425] [__recompiles] - ___key_to_id(L['self'].state) == [139893161731440,139893161724240,139893161722720,139893161727120,139893161732960,139893161726480,139893161722640,139893161724800,139893161731680,139893161724960] + V0625 22:40:14.236000 139900085105280 torch/_dynamo/guards.py:1425] [__recompiles] Recompiling function step in /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/optim/adam.py:135 + V0625 22:40:14.236000 139900085105280 torch/_dynamo/guards.py:1425] [__recompiles] triggered by the following guard failure(s): + V0625 22:40:14.236000 139900085105280 torch/_dynamo/guards.py:1425] [__recompiles] - L['self'].param_groups[0]['lr'] == 0.003333333333333333 + V0625 22:40:14.236000 139900085105280 torch/_dynamo/guards.py:1425] [__recompiles] - G['__optimizer_139893146295488_139893161731840_c59']() is not None + V0625 22:40:16.768000 139900085105280 torch/_dynamo/guards.py:1425] [__recompiles] Recompiling function step in /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/optim/adam.py:135 + V0625 22:40:16.768000 139900085105280 torch/_dynamo/guards.py:1425] [__recompiles] triggered by the following guard failure(s): + V0625 22:40:16.768000 139900085105280 torch/_dynamo/guards.py:1425] [__recompiles] - L['self'].param_groups[0]['lr'] == 0.004666666666666667 + V0625 22:40:16.768000 139900085105280 torch/_dynamo/guards.py:1425] [__recompiles] - L['self'].param_groups[0]['lr'] == 0.003333333333333333 + V0625 22:40:16.768000 139900085105280 torch/_dynamo/guards.py:1425] [__recompiles] - G['__optimizer_139893146295488_139893161731840_c59']() is not None + V0625 22:40:19.304000 139900085105280 torch/_dynamo/guards.py:1425] [__recompiles] Recompiling function step in /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/optim/adam.py:135 + V0625 22:40:19.304000 139900085105280 torch/_dynamo/guards.py:1425] [__recompiles] triggered by the following guard failure(s): + V0625 22:40:19.304000 139900085105280 torch/_dynamo/guards.py:1425] [__recompiles] - L['self'].param_groups[0]['lr'] == 0.006000000000000001 + V0625 22:40:19.304000 139900085105280 torch/_dynamo/guards.py:1425] [__recompiles] - L['self'].param_groups[0]['lr'] == 0.004666666666666667 + V0625 22:40:19.304000 139900085105280 torch/_dynamo/guards.py:1425] [__recompiles] - L['self'].param_groups[0]['lr'] == 0.003333333333333333 + V0625 22:40:19.304000 139900085105280 torch/_dynamo/guards.py:1425] [__recompiles] - G['__optimizer_139893146295488_139893161731840_c59']() is not None + V0625 22:40:21.833000 139900085105280 torch/_dynamo/guards.py:1425] [__recompiles] Recompiling function step in /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/optim/adam.py:135 + V0625 22:40:21.833000 139900085105280 torch/_dynamo/guards.py:1425] [__recompiles] triggered by the following guard failure(s): + V0625 22:40:21.833000 139900085105280 torch/_dynamo/guards.py:1425] [__recompiles] - L['self'].param_groups[0]['lr'] == 0.007333333333333335 + V0625 22:40:21.833000 139900085105280 torch/_dynamo/guards.py:1425] [__recompiles] - L['self'].param_groups[0]['lr'] == 0.006000000000000001 + V0625 22:40:21.833000 139900085105280 torch/_dynamo/guards.py:1425] [__recompiles] - L['self'].param_groups[0]['lr'] == 0.004666666666666667 + V0625 22:40:21.833000 139900085105280 torch/_dynamo/guards.py:1425] [__recompiles] - L['self'].param_groups[0]['lr'] == 0.003333333333333333 + V0625 22:40:21.833000 139900085105280 torch/_dynamo/guards.py:1425] [__recompiles] - G['__optimizer_139893146295488_139893161731840_c59']() is not None @@ -219,7 +219,7 @@ See also: .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 0 minutes 16.936 seconds) + **Total running time of the script:** ( 0 minutes 16.831 seconds) .. _sphx_glr_download_recipes_compiling_optimizer_lr_scheduler.py: diff --git a/_sources/recipes/recipes/changing_default_device.rst.txt b/_sources/recipes/recipes/changing_default_device.rst.txt index 19d81b73b7..f34b71539e 100644 --- a/_sources/recipes/recipes/changing_default_device.rst.txt +++ b/_sources/recipes/recipes/changing_default_device.rst.txt @@ -128,7 +128,7 @@ is causing problems for you, please comment on .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 0 minutes 0.099 seconds) + **Total running time of the script:** ( 0 minutes 0.101 seconds) .. _sphx_glr_download_recipes_recipes_changing_default_device.py: diff --git a/_sources/recipes/recipes/module_load_state_dict_tips.rst.txt b/_sources/recipes/recipes/module_load_state_dict_tips.rst.txt index 9b6abf59a9..9905a0ebc3 100644 --- a/_sources/recipes/recipes/module_load_state_dict_tips.rst.txt +++ b/_sources/recipes/recipes/module_load_state_dict_tips.rst.txt @@ -157,7 +157,7 @@ loaded into CPU RAM, which can be undesirable when: .. code-block:: none - loading time without mmap=0.021975040435791016 + loading time without mmap=0.022372722625732422 @@ -189,7 +189,7 @@ storages will be memory-mapped. .. code-block:: none - loading time with mmap=0.0016951560974121094 + loading time with mmap=0.0015723705291748047 @@ -352,7 +352,7 @@ be used to aid when loading a model from a checkpoint. .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 0 minutes 0.387 seconds) + **Total running time of the script:** ( 0 minutes 0.391 seconds) .. _sphx_glr_download_recipes_recipes_module_load_state_dict_tips.py: diff --git a/_sources/recipes/recipes/reasoning_about_shapes.rst.txt b/_sources/recipes/recipes/reasoning_about_shapes.rst.txt index 5633207423..09a3c9df77 100644 --- a/_sources/recipes/recipes/reasoning_about_shapes.rst.txt +++ b/_sources/recipes/recipes/reasoning_about_shapes.rst.txt @@ -60,7 +60,7 @@ of a layer without materializing any data. .. code-block:: none tensor(..., device='meta', size=(2, 5, 9, 9), grad_fn=) - Time taken: 0.00018522299978940282 + Time taken: 0.00019496499953675084 @@ -94,7 +94,7 @@ inputs will not significantly alter the time taken for shape computation. tensor(..., device='meta', size=(1024, 5, 65535, 65535), grad_fn=) - Time taken: 0.00011191299927304499 + Time taken: 0.00011925200124096591 @@ -193,7 +193,7 @@ forward hook to each layer that prints the shape of the output. .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 0 minutes 0.015 seconds) + **Total running time of the script:** ( 0 minutes 0.016 seconds) .. _sphx_glr_download_recipes_recipes_reasoning_about_shapes.py: diff --git a/_sources/recipes/recipes/swap_tensors.rst.txt b/_sources/recipes/recipes/swap_tensors.rst.txt index 47c47fbf60..07494e3f10 100644 --- a/_sources/recipes/recipes/swap_tensors.rst.txt +++ b/_sources/recipes/recipes/swap_tensors.rst.txt @@ -217,8 +217,8 @@ of the subclass' payload (``elem``) does not change. .. code-block:: none - Before: id(m.weight)=139960653111168, id(m.bias)=139960653111328 - After: id(m.weight)=139960653111168, id(m.bias)=139960653111328 + Before: id(m.weight)=139895005243936, id(m.bias)=139895005241616 + After: id(m.weight)=139895005243936, id(m.bias)=139895005241616 m.weight.dtype: torch.bfloat16 m.weight.elem.dtype: torch.float32 m.bias.dtype: torch.bfloat16 @@ -259,8 +259,8 @@ the ``dtype`` of the payload is properly converted. .. code-block:: none - Before: id(m.weight)=139960653106848, id(m.bias)=139960652981136 - After: id(m.weight)=139960653106848, id(m.bias)=139960652981136 + Before: id(m.weight)=139895005239936, id(m.bias)=139896021365792 + After: id(m.weight)=139895005239936, id(m.bias)=139896021365792 m.weight.dtype: torch.bfloat16 m.weight.elem.dtype: torch.bfloat16 m.bias.dtype: torch.bfloat16 @@ -392,7 +392,7 @@ for biases, we want to preserve the properties of the tensor in the ``state_dict .. code-block:: none - Before: id(weight)=139960653108048, id(bias)=139960653117168 + Before: id(weight)=139895005242736, id(bias)=139895005237136 m.state_dict() before load_state_dict(): OrderedDict([('weight', MyQuantizedLinearWeight(tensor(..., device='meta', size=(5, 3)), scale=0.5)), ('bias', tensor(..., device='meta', size=(5,)))]) state_dict: @@ -401,7 +401,7 @@ for biases, we want to preserve the properties of the tensor in the ``state_dict [ 0.2932, -0.3519, -0.5715], [-0.2231, -0.4428, 0.4737], [ 0.1663, 0.2391, 0.1826]])), ('bias', tensor([-0.0100, 0.4518, -0.4102, 0.0364, -0.3941]))]) - After: id(weight)=139960653108048, id(bias)=139960653117168 + After: id(weight)=139895005242736, id(bias)=139895005237136 m.state_dict() after load_state_dict(): OrderedDict([('weight', MyQuantizedLinearWeight(tensor([[ 0.2430, 0.5155, 0.3337], [-0.2524, 0.3333, 0.1033], @@ -431,7 +431,7 @@ use the two new extension points that are gated by .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 0 minutes 0.018 seconds) + **Total running time of the script:** ( 0 minutes 0.017 seconds) .. _sphx_glr_download_recipes_recipes_swap_tensors.py: diff --git a/_sources/recipes/torch_compile_user_defined_triton_kernel_tutorial.rst.txt b/_sources/recipes/torch_compile_user_defined_triton_kernel_tutorial.rst.txt index 87577ee551..08be9ee54d 100644 --- a/_sources/recipes/torch_compile_user_defined_triton_kernel_tutorial.rst.txt +++ b/_sources/recipes/torch_compile_user_defined_triton_kernel_tutorial.rst.txt @@ -249,7 +249,7 @@ See Also .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 0 minutes 1.471 seconds) + **Total running time of the script:** ( 0 minutes 1.478 seconds) .. _sphx_glr_download_recipes_torch_compile_user_defined_triton_kernel_tutorial.py: diff --git a/_sources/recipes/torch_logs.rst.txt b/_sources/recipes/torch_logs.rst.txt index 471482909f..ef69bc630a 100644 --- a/_sources/recipes/torch_logs.rst.txt +++ b/_sources/recipes/torch_logs.rst.txt @@ -147,7 +147,7 @@ For more information on torch.compile, see the `torch.compile tutorial`_. .. rst-class:: sphx-glr-timing - **Total running time of the script:** ( 0 minutes 0.002 seconds) + **Total running time of the script:** ( 0 minutes 0.003 seconds) .. _sphx_glr_download_recipes_torch_logs.py: diff --git a/advanced/coding_ddpg.html b/advanced/coding_ddpg.html index a08919fbaf..1b9d69b0f4 100644 --- a/advanced/coding_ddpg.html +++ b/advanced/coding_ddpg.html @@ -1559,26 +1559,26 @@

Time to train the policy
  0%|          | 0/10000 [00:00<?, ?it/s]
-  8%|8         | 800/10000 [00:00<00:08, 1065.00it/s]
- 16%|#6        | 1600/10000 [00:03<00:20, 401.41it/s]
- 24%|##4       | 2400/10000 [00:04<00:13, 543.48it/s]
- 32%|###2      | 3200/10000 [00:05<00:10, 655.82it/s]
- 40%|####      | 4000/10000 [00:06<00:08, 737.72it/s]
- 48%|####8     | 4800/10000 [00:06<00:06, 799.24it/s]
- 56%|#####6    | 5600/10000 [00:07<00:05, 846.72it/s]
-reward: -2.56 (r0 = -2.20), reward eval: reward: -0.01, reward normalized=-3.21/6.37, grad norm= 44.83, loss_value= 395.90, loss_actor= 18.20, target value: -18.76:  56%|#####6    | 5600/10000 [00:08<00:05, 846.72it/s]
-reward: -2.56 (r0 = -2.20), reward eval: reward: -0.01, reward normalized=-3.21/6.37, grad norm= 44.83, loss_value= 395.90, loss_actor= 18.20, target value: -18.76:  64%|######4   | 6400/10000 [00:09<00:05, 635.05it/s]
-reward: -0.10 (r0 = -2.20), reward eval: reward: -0.01, reward normalized=-2.70/6.05, grad norm= 81.33, loss_value= 343.52, loss_actor= 14.75, target value: -16.15:  64%|######4   | 6400/10000 [00:10<00:05, 635.05it/s]
-reward: -0.10 (r0 = -2.20), reward eval: reward: -0.01, reward normalized=-2.70/6.05, grad norm= 81.33, loss_value= 343.52, loss_actor= 14.75, target value: -16.15:  72%|#######2  | 7200/10000 [00:12<00:05, 486.42it/s]
-reward: -1.82 (r0 = -2.20), reward eval: reward: -0.01, reward normalized=-3.04/5.69, grad norm= 203.44, loss_value= 302.93, loss_actor= 15.25, target value: -20.36:  72%|#######2  | 7200/10000 [00:13<00:05, 486.42it/s]
-reward: -1.82 (r0 = -2.20), reward eval: reward: -0.01, reward normalized=-3.04/5.69, grad norm= 203.44, loss_value= 302.93, loss_actor= 15.25, target value: -20.36:  80%|########  | 8000/10000 [00:14<00:04, 418.73it/s]
-reward: -4.83 (r0 = -2.20), reward eval: reward: -0.01, reward normalized=-2.87/5.11, grad norm= 241.11, loss_value= 259.33, loss_actor= 16.64, target value: -19.13:  80%|########  | 8000/10000 [00:15<00:04, 418.73it/s]
-reward: -4.83 (r0 = -2.20), reward eval: reward: -0.01, reward normalized=-2.87/5.11, grad norm= 241.11, loss_value= 259.33, loss_actor= 16.64, target value: -19.13:  88%|########8 | 8800/10000 [00:17<00:03, 382.18it/s]
-reward: -5.14 (r0 = -2.20), reward eval: reward:  0.53, reward normalized=-2.44/4.86, grad norm= 150.89, loss_value= 188.20, loss_actor= 18.66, target value: -16.22:  88%|########8 | 8800/10000 [00:20<00:03, 382.18it/s]
-reward: -5.14 (r0 = -2.20), reward eval: reward:  0.53, reward normalized=-2.44/4.86, grad norm= 150.89, loss_value= 188.20, loss_actor= 18.66, target value: -16.22:  96%|#########6| 9600/10000 [00:21<00:01, 284.56it/s]
-reward: -5.13 (r0 = -2.20), reward eval: reward:  0.53, reward normalized=-2.81/5.41, grad norm= 125.30, loss_value= 268.30, loss_actor= 16.89, target value: -19.89:  96%|#########6| 9600/10000 [00:22<00:01, 284.56it/s]
-reward: -5.13 (r0 = -2.20), reward eval: reward:  0.53, reward normalized=-2.81/5.41, grad norm= 125.30, loss_value= 268.30, loss_actor= 16.89, target value: -19.89: : 10400it [00:25, 264.83it/s]
-reward: -3.58 (r0 = -2.20), reward eval: reward:  0.53, reward normalized=-3.82/5.60, grad norm= 87.65, loss_value= 267.84, loss_actor= 23.16, target value: -27.36: : 10400it [00:26, 264.83it/s]
+  8%|8         | 800/10000 [00:00<00:08, 1070.76it/s]
+ 16%|#6        | 1600/10000 [00:03<00:20, 411.40it/s]
+ 24%|##4       | 2400/10000 [00:04<00:13, 557.10it/s]
+ 32%|###2      | 3200/10000 [00:05<00:10, 672.12it/s]
+ 40%|####      | 4000/10000 [00:05<00:07, 756.22it/s]
+ 48%|####8     | 4800/10000 [00:06<00:06, 819.44it/s]
+ 56%|#####6    | 5600/10000 [00:07<00:05, 863.13it/s]
+reward: -2.33 (r0 = -2.76), reward eval: reward:  0.00, reward normalized=-2.94/6.04, grad norm= 39.94, loss_value= 309.89, loss_actor= 16.81, target value: -18.49:  56%|#####6    | 5600/10000 [00:08<00:05, 863.13it/s]
+reward: -2.33 (r0 = -2.76), reward eval: reward:  0.00, reward normalized=-2.94/6.04, grad norm= 39.94, loss_value= 309.89, loss_actor= 16.81, target value: -18.49:  64%|######4   | 6400/10000 [00:09<00:05, 638.43it/s]
+reward: -0.12 (r0 = -2.76), reward eval: reward:  0.00, reward normalized=-2.67/5.59, grad norm= 90.14, loss_value= 235.72, loss_actor= 14.76, target value: -16.42:  64%|######4   | 6400/10000 [00:10<00:05, 638.43it/s]
+reward: -0.12 (r0 = -2.76), reward eval: reward:  0.00, reward normalized=-2.67/5.59, grad norm= 90.14, loss_value= 235.72, loss_actor= 14.76, target value: -16.42:  72%|#######2  | 7200/10000 [00:12<00:05, 491.19it/s]
+reward: -2.76 (r0 = -2.76), reward eval: reward:  0.00, reward normalized=-2.44/5.94, grad norm= 201.55, loss_value= 271.25, loss_actor= 13.79, target value: -15.96:  72%|#######2  | 7200/10000 [00:12<00:05, 491.19it/s]
+reward: -2.76 (r0 = -2.76), reward eval: reward:  0.00, reward normalized=-2.44/5.94, grad norm= 201.55, loss_value= 271.25, loss_actor= 13.79, target value: -15.96:  80%|########  | 8000/10000 [00:14<00:04, 423.48it/s]
+reward: -4.90 (r0 = -2.76), reward eval: reward:  0.00, reward normalized=-2.51/4.70, grad norm= 34.93, loss_value= 161.79, loss_actor= 15.98, target value: -16.57:  80%|########  | 8000/10000 [00:15<00:04, 423.48it/s]
+reward: -4.90 (r0 = -2.76), reward eval: reward:  0.00, reward normalized=-2.51/4.70, grad norm= 34.93, loss_value= 161.79, loss_actor= 15.98, target value: -16.57:  88%|########8 | 8800/10000 [00:16<00:03, 388.29it/s]
+reward: -4.41 (r0 = -2.76), reward eval: reward: -5.39, reward normalized=-2.43/5.13, grad norm= 72.99, loss_value= 241.28, loss_actor= 13.02, target value: -16.27:  88%|########8 | 8800/10000 [00:19<00:03, 388.29it/s]
+reward: -4.41 (r0 = -2.76), reward eval: reward: -5.39, reward normalized=-2.43/5.13, grad norm= 72.99, loss_value= 241.28, loss_actor= 13.02, target value: -16.27:  96%|#########6| 9600/10000 [00:21<00:01, 288.47it/s]
+reward: -4.65 (r0 = -2.76), reward eval: reward: -5.39, reward normalized=-2.92/5.14, grad norm= 147.22, loss_value= 251.85, loss_actor= 11.03, target value: -20.73:  96%|#########6| 9600/10000 [00:22<00:01, 288.47it/s]
+reward: -4.65 (r0 = -2.76), reward eval: reward: -5.39, reward normalized=-2.92/5.14, grad norm= 147.22, loss_value= 251.85, loss_actor= 11.03, target value: -20.73: : 10400it [00:24, 268.73it/s]
+reward: -4.72 (r0 = -2.76), reward eval: reward: -5.39, reward normalized=-3.49/4.19, grad norm= 133.24, loss_value= 169.25, loss_actor= 14.39, target value: -24.45: : 10400it [00:25, 268.73it/s]
@@ -1622,7 +1622,7 @@

Next Steps[Feature] Distpatch IQL loss module.)

  • Allowing flexible TensorDict keys.

    • -

    Total running time of the script: ( 0 minutes 41.549 seconds)

    +

    Total running time of the script: ( 0 minutes 40.906 seconds)