Skip to content

Commit

Permalink
Merge pull request cyclus#227 from bam241/cep3_update
Browse files Browse the repository at this point in the history 
WIP: update CEP3 to actual release process.
  • Loading branch information
@scopatz
scopatz authored Feb 13, 2017
2 parents 52435e7 + caa84db commit a630b02
Show file tree
Hide file tree
Showing 2 changed files with 108 additions and 52 deletions.
121 changes: 87 additions & 34 deletions source/cep/cep3.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -110,8 +110,6 @@ Release Candidate Process

#. Review the current state of documentation and make approriate updates.

#. Update any new database types in ``cyclus/share/dbtypes.json``

#. Finish the release candidate process

- make sure all commits in the ``release`` branch also are in ``develop``
Expand All @@ -120,12 +118,14 @@ Release Process
---------------

#. Make sure every local |cyclus| project repository is up to date with its
``master``, ``develop``, and ``vX.X.X-release`` branches on ``upstream``
``master``, ``develop``, and ``vX.X.X-release`` branches on ``upstream``.

#. Bump the version in ``cyclus/src/version.h``, ``cycamore/src/version.h``, and
#. Bump the version in ``cyclus/src/version.h.in``,
``cycamore/src/cycamore_version.h.in``, and
``cymetric/setup.py``; commit the changes

#. Perform maintainence tasks for all projects
#. Perform maintenance tasks for all projects. The maintenance depends on `PyNE
<https://github.com/pyne/pyne.git>`_and Doxygen.

- they are described in detail below, *but* the ``maintenence.sh`` utility
in ``release/utils`` will do this automatically for you
Expand All @@ -139,15 +139,37 @@ Release Process
$ export CYCAMORE_DIR=/path/to/cycamore
$ ./maintenence.sh -r -v X.X.X # X.X.X is *this* version
#. Commit all changes for all projects
.. note::

If maintenance script fails because of an ABI failure that is caused by
a compiler update (or other similar change caused by reasons other
than code changes), you might want to accept them and procceed with the
release with those. To do so you need to generate the new symbols and
commit them:

#. First make sure those changes can be ignored by emailing for
discussion/approval the dev-list

#. if the dev-list agrees to those changes, update the symbols and
commit the new one:

.. code-block:: bash
$ cd $CYCLUS_DIR/release
$ ./smbchk.py --update -t X.X.X # X.X.X is *this* version
$ git add symbols.json
$ git commit -m "Describe here all the change on the ABI"
#. Commit all changes for all projects.

.. code-block:: bash
$ cd /path/to/project
$ git checkout vX.X.X-release
$ git commit -am "final release commit after maintenence"
#. Update all develop branches
#. Update all develop branches.

.. code-block:: bash
Expand All @@ -156,7 +178,7 @@ Release Process
$ git merge --no-ff vX.X.X-release
$ git push upstream develop
#. *Locally* tag the repository for *each* of the projects
#. *Locally* tag the repository for *each* of the projects.

.. code-block:: bash
Expand All @@ -165,7 +187,7 @@ Release Process
$ git merge --no-ff vX.X.X-release
$ git tag -a -m "Cyclus project release X.X.X, see http://fuelcycle.org/previous/vX.X.X.html for release notes" X.X.X
#. Draft release notes
#. Draft release notes.

- the ``make_release_notes.sh`` utility in ``release/utils`` will help
provide a template
Expand All @@ -181,7 +203,7 @@ Release Process
- add the release notes as ``cyclus.github.com/source/previous/vX.X.X.rst``
with appropriate updates to ``index.rst`` in that directory

#. Update the API docs
#. Update the API docs.

- the ``api_docs.sh`` utility in ``release/utils`` will do this
automatically for you
Expand All @@ -193,58 +215,89 @@ Release Process
$ export CYCAMORE_DIR=/path/to/cycamore
$ ./api_docs.sh X.X.X # X.X.X is *this* version
#. Upload the conda packages

- the ``upload_conda.sh`` utility in ``release/utils`` will do this
automatically for you

.. note::

You must be associated with the account at http://binstar.org/cyclus/cyclus.

.. note::

You may be prompted for your binstar user name and password.

.. code-block:: bash
$ cd /path/to/release/utils
$ export CYCLUS_DIR=/path/to/cyclus
$ export CYCAMORE_DIR=/path/to/cycamore
$ export CYMETRIC_DIR=/path/to/cymetric
$ ./conda_upload.sh X.X.X # X.X.X is *this* version
#. Update the ``master`` branch of all projects and clean up
#. Update the ``master`` branch of all projects and clean up.

.. code-block:: bash
$ cd /path/to/project
$ git push upstream X.X.X master
$ git push upstream --delete vX.X.X-release
#. Manually visit the github.com page for each project and mark the tags as releases
#. Manually visit the github.com page for each project and mark the tags as releases.

- This can be updated one day to use the Github `release API
<https://developer.github.com/v3/repos/releases/#create-a-release>`_


#. Update Conda-forge

- For each project, find the corresponding feedstock repository in the
conda-forge organization on github. For example, cyclus' feedstock is at
https://github.com/conda-forge/cyclus-feedstock

- In each project's feedstok, open up a PR which updates the
`recipe/meta.yaml` file with the new version number and the new SHA-256
value of the new version's tarball. See conda-forge documentation for more
or ask the feedstock maintainers for help.

- Note that each feedstock must be accepted and the package uploaded to
anaconda.org (automatic) prior to accepting updates for the next feedstock
dependency. For example, cyclus must be fully updated before cycamore.


#. Create a DOI. See :doc:`CEP4 <./cep4>` for details.

- This can be updated one day to use the Figshare `API
<http://api.figshare.com/docs/intro.html>`_

#. Update website release information
#. Update website release information.

- on the front page (``source/index.rst``)
- DOIs (``source/cite/index.rst``)
- release notes (``source/previous/index.rst``), remember both the release
notes and the zip/tar URLs!
- layout template (``source/atemplates/layout.html``) of the website
- install from tarball instruction
(``source/user/install_from_tarball.rst``)


#. Commit all changes to ``cyclus.github.com`` and ``make gh-publish``

#. Send out an email to `cyclus-dev` and `cyclus-users` to announce the release!


.. This part has been commented, as it is required for the website, but the
person in charge of the release might not have the proper access to update the
Dory worker. This should be automated when a merge is done on the master branch
a CI-hook should update the dory cloudlus server and relaunch the worker.
Moreover the cloudlus server is not directly related to Cyclus and depend on
the UW-Madison community but the website relies on it to host the
online cyclus calculation... (see
https://github.com/cyclus/cyclus.github.com/pull/227#pullrequestreview-21589660
discussion for more details.)
.. #. Update ``Dory``'s ``Cyclus``/``Cycamore`` version and relaunch ``Dory`` worker.
To do this you need a acces to the ``Dory`` server (if you don't please
contact an administrator), the ``go`` toolchain as well as ``cde`` installed
on your computer. Also, the release version of ``Cyclus`` and ``Cycamore``
have to be compiled on you system and both executable and lib have to be on
the default paths. Please refer to the :doc:`source installation
<../user/install_from_git>` if you need.
.. .. code-block:: bash
$ git clone https://github.com/rwcarlsen/cloudlus.git
$ cd cloudlus
$ go install ./cmd/cloudlus
$ cd misc/fuelcycle.org
$ make
$ ssh dory.fuelcycle.org 'mv cyc-cde.tar.gz cyc-cde.tar.gz_bkp'
$ scp cyc-cde.tar.gz dory:fuelcycle.org:./
$ ssh dory.fuelcycle.org
$ ps -fe | grep cloudlus | grep work | grep ':80' | cut -d" " -f6 | xargs kill -9
$ rm -rf worker-*
$ ./launch.sh 2
Maintainence Tasks
==================

Expand Down
39 changes: 21 additions & 18 deletions source/user/dbdoc.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -29,10 +29,10 @@ the same. |Cyclus| ships supporting two database formats:
Below are a few sections that describe the data tables found in the database.

Table Descriptions
+++++++++++++++++++
++++++++++++++++++

Resources Table
----------------
---------------

Because resources are tracked as immutable objects, every time a resource is
changed in the simulation (split, combined, transmuted, decayed, etc.), it
Expand Down Expand Up @@ -83,7 +83,7 @@ tables.
.. _explicit-inv-table:

ExplicitInventory Table
------------------------
-----------------------

This is an optional table that can be activated by setting
``<explicit_inventory>true</explicit_inventory>`` in the ``<control>`` section
Expand Down Expand Up @@ -144,7 +144,7 @@ this data as an xml string from a boost-serialized ``std::map<int, double>``
double>`` object.

Compositions Table
--------------------
------------------

A composition consists of one or more nuclides and their respective mass
fractions. Each nuclide for a composition gets its own row and have the same
Expand All @@ -160,7 +160,7 @@ QualId.
* **MassFrac** (double): Mass fraction for the nuclide in this composition.

Recipes Table
-------------------
-------------

* **SimId** (uuid)

Expand All @@ -170,7 +170,7 @@ Recipes Table
``Compositions`` table.

Products Table
----------------
--------------

* **SimId** (uuid)

Expand All @@ -181,7 +181,7 @@ Products Table
etc.)

ResCreators Table
-------------------
-----------------

Every time an agent creates a new resource from scratch, that event is
recorded in this table.
Expand All @@ -195,7 +195,7 @@ recorded in this table.
the ResourceId.

AgentEntry Table
-------------------
----------------

Each agent that enters and participates in a simulation gets a row in this
table.
Expand Down Expand Up @@ -223,7 +223,7 @@ table.
simulation.

AgentExit Table
------------------
---------------

Due to implementation details in the |cyclus| kernel, this table is separate
from the ``AgentEntry`` table. If this table doesn't exist, then no agents
Expand Down Expand Up @@ -254,7 +254,7 @@ each simulation.
* **Version** (string): The version string provided by the archetype.

Transactions Table
-------------------
------------------

Every single resource transfer between two agents is recorded as a row
in this table.
Expand All @@ -276,7 +276,7 @@ in this table.
* **Time** (int): The time step at which the resource transfer took place.

Info Table
-------------------
----------

Each simulation gets a single row in this table describing global simulation
parameters and |cyclus| dependency version information.
Expand Down Expand Up @@ -342,7 +342,8 @@ Each simulation gets one row in this table.


Finish Table
-------------------
------------

Each simulation gets one row/entry in this table.

* **SimId** (uuid)
Expand All @@ -354,14 +355,16 @@ Each simulation gets one row/entry in this table.


InputFiles Table
-------------------
----------------

* **SimId** (uuid)

* **Data** (blob): A dump of the entire input file used for this simulation.


DecomSchedule Table
--------------------
-------------------

When agents are scheduled to be decommissioned in the simulation, the details
are recorded in this table. Note that this table contains an entry for each
scheduling regardless of whether or not it actually occurred; if a simulation
Expand All @@ -379,7 +382,7 @@ decommissioned.
been) decommissioned.

BuildSchedule Table
--------------------
-------------------

When agents are scheduled to be built in the simulation, the details are
recorded in this table. Note that this table contains an entry for each
Expand All @@ -402,7 +405,7 @@ built.
been) built and deployed into the simulation.

Snapshots Table
-------------------
---------------

Every snapshot made during the simulation gets an entry in this table. All
times in this table are candidates for simulation restart/branching.
Expand All @@ -412,7 +415,7 @@ times in this table are candidates for simulation restart/branching.
* **Time** (int): The time step a snapshot was taken for this simulation.

Debugging
----------
---------

If |Cyclus| was run in debugging mode then the database will then contain
the following two extra tables:
Expand Down Expand Up @@ -440,7 +443,7 @@ the following two extra tables:


Post Processing
+++++++++++++++++
+++++++++++++++

We are currently working on developing a post-process step for the database
that creates a few new tables to assist data analysis and visualization.
Expand Down

0 comments on commit a630b02

Please sign in to comment.