Merge branch 'master' into master

microsoft · Nov 25, 2024 · 41d09b5 · 41d09b5
2 parents c8f2faa + f743fec
commit 41d09b5
Show file tree

Hide file tree

Showing 20 changed files with 207 additions and 7 deletions.
diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml
@@ -7,7 +7,7 @@ on:
 
 jobs:
   deploy:
-    runs-on: ubuntu-24.04
+    runs-on: ubuntu-22.04
     environment: release-env
 
     steps:

diff --git a/blogs/deepspeed-domino/README.md b/blogs/deepspeed-domino/README.md
diff --git a/blogs/deepspeed-domino/images/design-base.png b/blogs/deepspeed-domino/images/design-base.png
diff --git a/blogs/deepspeed-domino/images/design-column.png b/blogs/deepspeed-domino/images/design-column.png
diff --git a/blogs/deepspeed-domino/images/design-hybrid.png b/blogs/deepspeed-domino/images/design-hybrid.png
diff --git a/blogs/deepspeed-domino/images/design-row.png b/blogs/deepspeed-domino/images/design-row.png
diff --git a/blogs/deepspeed-domino/images/domino-hero.png b/blogs/deepspeed-domino/images/domino-hero.png
diff --git a/blogs/deepspeed-domino/images/domino-logo.png b/blogs/deepspeed-domino/images/domino-logo.png
diff --git a/blogs/deepspeed-domino/images/gpt3-scale.png b/blogs/deepspeed-domino/images/gpt3-scale.png
diff --git a/blogs/deepspeed-domino/images/implement-bwd.png b/blogs/deepspeed-domino/images/implement-bwd.png
diff --git a/blogs/deepspeed-domino/images/implement-fwd.png b/blogs/deepspeed-domino/images/implement-fwd.png
diff --git a/blogs/deepspeed-domino/images/tp-ar.png b/blogs/deepspeed-domino/images/tp-ar.png
diff --git a/blogs/deepspeed-domino/images/tp-comm-overhead.png b/blogs/deepspeed-domino/images/tp-comm-overhead.png
diff --git a/docs/code-docs/source/index.rst b/docs/code-docs/source/index.rst
@@ -71,7 +71,7 @@ Optimizers
    optimizers
 
 Learning Rate Schedulers
---------------------
+------------------------
 .. toctree::
    :maxdepth: 2
 

diff --git a/docs/code-docs/source/inference-engine.rst b/docs/code-docs/source/inference-engine.rst
@@ -1,5 +1,5 @@
 Inference API
-============
+=============
 
 :func:`deepspeed.init_inference` returns an *inference engine*
 of type :class:`InferenceEngine`.

diff --git a/docs/code-docs/source/initialize.rst b/docs/code-docs/source/initialize.rst
@@ -38,7 +38,7 @@ Example usage:
 .. autofunction:: deepspeed.initialize
 
 Distributed Initialization
------------------------
+--------------------------
 Optional distributed backend initialization separate from ``deepspeed.initialize()``. Useful in scenarios where the user wants to use torch distributed calls before calling ``deepspeed.initialize()``, such as when using model parallelism, pipeline parallelism, or certain data loader scenarios.
 
 .. autofunction:: deepspeed.init_distributed
diff --git a/docs/code-docs/source/moe.rst b/docs/code-docs/source/moe.rst
@@ -1,5 +1,5 @@
 Mixture of Experts (MoE)
-====================
+========================
 
 Layer specification
 --------------------

diff --git a/docs/code-docs/source/schedulers.rst b/docs/code-docs/source/schedulers.rst
@@ -1,5 +1,5 @@
 Learning Rate Schedulers
-===================
+=================================
 
 DeepSpeed offers implementations of ``LRRangeTest``, ``OneCycle``, ``WarmupLR``, ``WarmupDecayLR``, ``WarmupCosineLR`` learning rate schedulers. When using a DeepSpeed's learning rate scheduler (specified in the `ds_config.json` file), DeepSpeed calls the `step()` method of the scheduler at every training step (when `model_engine.step()` is executed). When not using a DeepSpeed's learning rate scheduler:
   * if the schedule is supposed to execute at every training step, then the user can pass the scheduler to `deepspeed.initialize` when initializing the DeepSpeed engine and let DeepSpeed manage it for update or save/restore.

diff --git a/docs/code-docs/source/zero3.rst b/docs/code-docs/source/zero3.rst
@@ -310,6 +310,7 @@ DeepSpeed can automatically detect the following external parameter scenarios:
 
 
 .. `Module.apply <https://pytorch.org/docs/stable/generated/torch.nn.Module.html?highlight=module+apply#torch.nn.Module.apply>`_
+
 Overriding Module.apply
 ===============================
 A convenient mechanism for customizing model initialization is `Module.apply <https://pytorch.org/docs/stable/generated/torch.nn.Module.html?highlight=module+apply#torch.nn.Module.apply>`_.

diff --git a/version.txt b/version.txt
@@ -1 +1 @@
-0.15.5
+0.16.1
-Original file line number
+Diff line change
@@ Expand Up / @@ -7,7 +7,7 @@ on: @@
     jobs:
       deploy:
-        runs-on: ubuntu-24.04
+        runs-on: ubuntu-22.04
         environment: release-env
         steps:
@@ Expand Down @@