Recent changes and conda updates (#70)

* Change simulation to be more intuitive and always have the desired range of simulation

* Remove pdb statements

* Incredibly hacky version of masking

* Some simulation changes

* Some partial changes wrt output class

* Simple record instance to store trees before converting to tables

* First stab at functional tables, not much in terms of inferring them from the data

* Recursion to find paths

* Find MRC node

* Working tree updates, need to do some QC

* Remove internal nodes

* Test case working except for one edge

* Same MRC recomb and reciever causing havoc in node creation

* Producing sensible trees up to time

* move

* Simple addition to get migration segments

* Bump version to 1.0.2 and have working migration segment inference.

* Still working

* Conda changes for 1.0.2, and generalisation of simulation protocol.

* Get rid of pedantic warnings

* Fix RPATH during native cmake and amend local conda-build instructions with env file

* update changelog

* move things around a little

* Edit simulations and reporting of scrm approximation

* Flexible conda building, version bump, and small edits.

* Edit readme

* Edit installation

CHANGELOG.md

@@ -0,0 +1,19 @@
+# Changelog
+
+This file contains notable updates to `smcsmc` partitioned by version.
+
+## [Unreleased]
+
+- Fixed a bug where the `scrm` approximation window was not being properly set.
+
+## [1.0.2] - 16/10/19
+### Added
+- Scripts to convert from the ARGs output by `smcsmc` to a partial `tskit` tree-sequence. Notably, this is used to infer better migration tracts.
+- Build environment for local `conda` building.
+
+### Changed
+- Default parameters of simulations, in order to correctly bound the proportions to the correct epochs.
+- Format of the simulations to be more expressive.
+
+### Fixed
+- `rpath` of `smcsmc` when installed now automatically prefers `conda` libraries. 

CMakeLists.txt

@@ -62,7 +62,7 @@ string(CONCAT CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS}"
                               "-DCOMPILEDATE=\\\"${COMPILEDATE}\\\" "
                               "-DVERSION=\\\"${VERSION}\\\" ")
 
-find_package(Boost 1.69.0 COMPONENTS iostreams regex)
+find_package(Boost COMPONENTS iostreams regex)
 find_package(ZLIB)
 include_directories(${Boost_INCLUDE_DIRS})
 include_directories(${CONDA_PREFIX}/include)
@@ -74,6 +74,7 @@ target_link_libraries(smcsmc ${Boost_LIBRARIES})
 message(STATUS "Boost_LIBRARIES: ${Boost_LIBRARIES}")
 target_link_libraries(smcsmc ${ZLIB_LIBRARIES})
 target_include_directories(smcsmc PRIVATE src/scrm/src src/scrm/src/random)
+#SET(CMAKE_INSTALL_RPATH "${CMAKE_INSaTALL_PREFIX}/lib")
 
 install(TARGETS smcsmc scrm
 	DESTINATION bin)

README.md

@@ -12,17 +12,18 @@ This repository contains two components, and both must be installed to properly
 
 ### Recommended Installation via `conda`
 
+> NOTE: We currently only support `conda` installation on **64 bit Linux**.  If you are using a different operating system you must install manually -- see below. 
+
 We have automated this process in a `conda` package, and we highly recommend installing it this way.
 
 ```sh
-conda config --add channels conda-forge
-conda config --add channels terhorst
-conda install -c luntergroup smcsmc
+conda install -c conda-forge -c luntergroup smcsmc
 ```
 
-> NOTE: We currently only support `conda` installation on **64 bit Linux**.  If you are using a different operating system you must install manually -- see below. 
 
-### Manual installation
+We must add `conda-forge` as a channel (with the `-c` flag) because the Boost version there is more current than default channels. 
+
+### Installation from Source
 
 Alternatively, a combination of `cmake` and `pip` can be used to install the python and core components:
 

docs/source/introduction.rst

@@ -14,14 +14,11 @@ Installation
 
 We *highly* recommend installing SMCSMC from :code:`conda`, as it comes packaged with all necessary dependencies. A seperate guide for manual compilation may be found `in the developer reference <https::github.com>`_\ .  `See here <https://docs.conda.io/projects/conda/en/latest/user-guide/getting-started.html>`_ for a helpful guide to installing and using :code:`conda` to manage programs. 
 
-First add both :code:`conda-forge` and :code:`terhorst` to your channel lists (if they are not there already), then install :code:`smcsmc`. 
+To install from :code:`conda` you must add both the :code:`luntergroup` and :code:`conda-forge` channels. The first contains :code:`smcsmc`, while the second is required for a more recent version of Boost.
 
 .. code-block:: bash
 
-   conda config --add channels conda-forge
-   conda config --add channels terhorst
-
-   conda install -c luntergroup smcsmc
+   conda install -c conda-forge -c luntergroup smcsmc
 
 
 Basic Usage

recipe/README.md

@@ -1,25 +1,23 @@
 # Locally building via `conda-build`
 
-Use this recipe for locally creating your own `conda` package. 
-
-## Operating Systems:
-
-- **Linux**: Tested, verified.
-- **MacOSX**: Work in progress, but follow [this](https://docs.conda.io/projects/conda-build/en/latest/resources/compiler-tools.html#macos-sdk) guide for installing the SDK which is *not* packaged with `conda-build`. 
-- **Windows**: Currently no support, nor is it planned.
+Use this recipe for locally creating your own `conda` package. It only works on Linux right now.
 
 ## Usage
 
-To build your package (after cloning the repository and installing `conda-build`)
+You can build the package with 
 
 ```sh
-conda build . -c terhorst
+conda build .
 ```
 
-To install:
+To install from the local build files:
 
 ```sh
-conda install -c ${CONDA_PREFIX}/conda-bld smcsmc
+conda install -c conda-forge -c ${CONDA_PREFIX}/conda-bld smcsmc
 ```
 
 This will install both the python package and `smcsmc`/`scrm` binaries. 
+
+## Building for other Python versions
+
+Build variants are included in the `conda_build_config.yaml`. The only real constraint on the python versions that we are able to build for is the `boost` package. Note that the version on `conda-forge` is more current than the default channels, so this should be set preferentially on the command line.

recipe/build.sh

@@ -1,3 +1,4 @@
+#$PYTHON -m pip install --ignore-installed --verbose .
 $PYTHON setup.py install
 rm -rf build
 mkdir -p build 

recipe/conda_build_config.yaml

@@ -0,0 +1,3 @@
+python:
+    - 3.6
+    - 3.7

recipe/meta.yaml

@@ -1,50 +1,57 @@
 package:
   name: smcsmc
-  version: 1.0.1
+  version: 1.0.2b
 
 source:
   path: ..
 
-python:
-   - 3.6.8
-   - 3.7.3
-
 build:
-  number: 0
+  number: 2
   entry_points:
     - smc2=smcsmc.cli:smcsmc_main
 
 requirements:
   build:
     - {{ compiler('cxx') }}
     - gperftools 
+    - python {{ python }}
     - cmake
     - boost
     - zlib
-    - msprime
+    - tqdm
 
   host:
-    - python
-    - msprime
+    - python {{ python }}
     - numpy
     - pandas
+    - tqdm
+    - msprime
 
   run:
     - gperftools
     - numpy
     - pandas
-    - msprime
-    - boost 
-    - python
     - boost
+    - python
     - zlib 
     - matplotlib
+    - tqdm
+    - msprime
+    - tskit
 
 test:
   imports:
     - smcsmc
+  commands:
+    - smc2 -h
+    - smcsmc -h
+    - scrm -h
 
 about:
   home: https://github.com/luntergroup/smcsmc
   doc_url: https://smcsmc.readthedocs.io
   summary: 'A particle filter for demographic inference.'
+
+extra:
+  recipe-maintainers:
+    - Chris1221

setup.py

@@ -1,16 +1,17 @@
 from setuptools import setup
 
 setup(name='smcsmc',
-      version='0.1',
+      version='1.0.2',
       description='Demographic Inference using Particle Filters',
       url='https://github.com/luntergroup/smcsmc',
       author='Chris Cole, Donna Henderson, Sha (Joe) Zhu, Gerton Lunter',
       author_email='[email protected], [email protected], [email protected], [email protected]',
       license='CC-BY-4.0',
       packages=['smcsmc'],
-      setup_requires=[
+      install_requires=[
           'numpy',
           'pandas',
-          'msprime'
+          'msprime',
+          'tqdm'
           ],
       zip_safe=False)

smcsmc/__init__.py

@@ -1,6 +1,7 @@
 from .model import Smcsmc
-from .square_model import *
+from .simulations import *
 from .populationmodels import Population
 from .plot import *
 from .utils import ts_to_seg, run_smcsmc, vcf_to_seg
 from .generate_smcsmcinput import run_multihetsep
+from .output import Output, Event

smcsmc/model.py

@@ -3,6 +3,8 @@
 import sys
 import os
 import logging
+import subprocess
+import stat
 import gzip
 import subprocess
 import time
@@ -31,9 +33,6 @@ def listitems(d):
 
 logger = logging.getLogger(__name__)
 logger.setLevel(logging.INFO)
-
-
-#
 # Todo: recomb.gz is emitted from position 1 rather than startpos; is it expected to start from 1?
 # If so, leave for now?
 #