Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

blocks 2020 and lodes 8 #372

Merged
merged 8 commits into from
Oct 2, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .ci/310.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@ dependencies:
- pandas
- giddy >=2.2.1
- libpysal
- cenpy
- geopandas >=0.9
- hdbscan
- matplotlib
Expand Down
1 change: 0 additions & 1 deletion .ci/311.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@ dependencies:
- pandas
- giddy >=2.2.1
- libpysal
- cenpy
- geopandas >=0.9
- hdbscan
- matplotlib
Expand Down
1 change: 0 additions & 1 deletion .ci/39.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@ dependencies:
- pandas
- giddy >=2.2.1
- libpysal
- cenpy
- geopandas >=0.9
- matplotlib
- scikit-learn
Expand Down
3 changes: 3 additions & 0 deletions .github/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,9 @@ changelog:
- title: Enhancements
labels:
- enhancement
- title: Maintenance
labels:
- maintenance
- title: Other Changes
labels:
- "*"
150 changes: 60 additions & 90 deletions .github/workflows/unittests.yml
Original file line number Diff line number Diff line change
@@ -1,91 +1,61 @@
name: Continuous Integration

on:
push:
branches:
- '*'
pull_request:
branches:
- '*'
schedule:
- cron: '59 23 * * *'
name: Tests

jobs:
unittests:
name: CI (${{ matrix.os }}-${{ matrix.environment-file }})
runs-on: ${{ matrix.os }}
continue-on-error: true
timeout-minutes: 90
strategy:
matrix:
os: ['ubuntu-latest', 'macos-latest', 'windows-latest']
environment-file: [.ci/39.yml, .ci/310.yml, .ci/311.yml]

steps:
- name: checkout repo
uses: actions/checkout@v2

- name: setup micromamba
uses: mamba-org/provision-with-micromamba@main
with:
environment-file: ${{ matrix.environment-file }}
micromamba-version: 'latest'
mamba-version: "*"
channels: conda-forge
channel-priority: true


- name: install geosnap - bash
shell: bash -l {0}
run: pip install -e . --no-deps --force-reinstall
if: matrix.os != 'windows-latest'

- name: install geosnap - powershell
shell: powershell
run: pip install -e . --no-deps --force-reinstall
if: matrix.os == 'windows-latest'


- name: download data - bash
shell: bash -l {0}
run: python geosnap/tests/_dl_data.py
env:
COMBO_DATA: ${{ secrets.COMBO_DATA }}
if: matrix.os != 'windows-latest'

- name: download data - powershell
shell: powershell
run: python geosnap/tests/_dl_data.py
env:
COMBO_DATA: ${{ secrets.COMBO_DATA }}
if: matrix.os == 'windows-latest'


- name: run tests - bash
shell: bash -l {0}
run: pytest -v geosnap --cov=geosnap --cov-report=xml
env:
LTDB_SAMPLE: ${{ secrets.COMBO_DATA }} # check whether we can pull secrets
LTDB_FULL: ${{ secrets.COMBO_DATA }}
NCDB: ${{ secrets.COMBO_DATA }}
if: matrix.os != 'windows-latest'

- name: run tests - powershell
shell: powershell
run: pytest -v geosnap
env:
LTDB_SAMPLE: ${{ secrets.COMBO_DATA }} # check whether we can pull secrets
LTDB_FULL: ${{ secrets.COMBO_DATA }}
NCDB: ${{ secrets.COMBO_DATA }}
if: matrix.os == 'windows-latest'


- name: codecov
uses: codecov/codecov-action@v2
with:
token: ${{ secrets.CODECOV_TOKEN }}
file: ./coverage.xml
flags: unittests # optional
name: geosnap-codecov
verbose: true
if: matrix.os != 'windows-latest'
on:
push:
branches: [main]
pull_request:
branches:
- "*"
schedule:
- cron: "0 0 * * 1,4"

jobs:
Test:
name: ${{ matrix.os }}, ${{ matrix.environment-file }}
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest]
environment-file:
- .ci/39.yml
- .ci/310.yml
- .ci/311.yml
include:
- environment-file: .ci/310.yml
os: macos-latest
- environment-file: .ci/310.yml
os: windows-latest
defaults:
run:
shell: bash -l {0}

steps:
- uses: actions/checkout@v4

- name: setup micromamba
uses: mamba-org/provision-with-micromamba@main
with:
environment-file: ${{ matrix.environment-file }}
micromamba-version: "latest"

- name: Install geosnap
run: pip install . ;python geosnap/tests/_dl_data.py;
env:
COMBO_DATA: ${{ secrets.COMBO_DATA }}

- name: Test geosnap
run: |
pytest -v --color yes --cov geosnap --cov-append --cov-report term-missing --cov-report xml .

- uses: codecov/codecov-action@v3

- name: Generate and publish the report
if: |
failure()
&& steps.status.outcome == 'failure'
&& github.event_name == 'schedule'
&& github.repository_owner == 'pysal'
uses: xarray-contrib/issue-from-pytest-log@v1
with:
log-path: pytest-log.jsonl
6 changes: 4 additions & 2 deletions docs/api.rst
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ available quickly with no configuration by accessing methods on the class.
DataStore.acs
DataStore.blocks_2000
DataStore.blocks_2010
DataStore.blocks_2020
DataStore.codebook
DataStore.counties
DataStore.ejscreen
Expand Down Expand Up @@ -59,6 +60,7 @@ files instead of streaming over the web.
io.store_census
io.store_blocks_2000
io.store_blocks_2010
io.store_blocks_2020
io.store_ejscreen
io.store_ltdb
io.store_ncdb
Expand Down Expand Up @@ -108,9 +110,9 @@ Model neighborhood change using optimal-matching algorithms or spatial discrete
.. autosummary::
:toctree: generated/

analyze.draw_sequence_from_gdf
analyze.linc
analyze.lincs_from_gdf
analyze.predict_markov_labels
analyze.sequence
analyze.transition

Expand Down Expand Up @@ -141,7 +143,7 @@ Compute shortest path distance along a network using pandana, and visualize trav

analyze.compute_travel_cost_adjlist
analyze.isochrone
analyze.isocrones
analyze.isochrones

The ModelResults Class
'''''''''''''''''''''''''''''''''''''''''''''
Expand Down
3 changes: 3 additions & 0 deletions docs/data.rst
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,9 @@ Geo Data
| Blocks | https://www2.census.gov/geo/tiger/TIGER2018/TABBLOCK/ |
| 2010 | |
+---------+------------------------------------------------------------+
| Blocks | https://www2.census.gov/geo/tiger/TIGER2021/TABBLOCK/ |
| 2020 | |
+---------+------------------------------------------------------------+
| Tracts | https://github.co |
| 1990 | m/loganpowell/census-geojson/tree/master/GeoJSON/500k/1990 |
+---------+------------------------------------------------------------+
Expand Down
11 changes: 3 additions & 8 deletions geosnap/__init__.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@

r"""

geosnap: Geospatial Neighborhood Analysis Package.
Expand All @@ -25,12 +24,8 @@
import contextlib
from importlib.metadata import PackageNotFoundError, version

from . import analyze
from . import io
from . import util
from . import visualize
from . import harmonize
from ._data import _Map, DataStore
from . import analyze, harmonize, io, util, visualize
from ._data import DataStore, _Map

with contextlib.suppress(PackageNotFoundError):
__version__ = version("geosnap")
__version__ = version("geosnap")
38 changes: 38 additions & 0 deletions geosnap/_data.py
Original file line number Diff line number Diff line change
Expand Up @@ -358,6 +358,44 @@ def blocks_2010(self, states=None, fips=None):

return blocks

def blocks_2020(self, states=None, fips=None):
"""Census blocks for 2020.

Parameters
----------
states : list-like
list of state fips codes to return as a datafrrame.

Returns
-------
type
pandas.DataFrame or geopandas.GeoDataFrame
2010 blocks as a geodataframe or as a dataframe with geometry
stored as well-known binary on the 'wkb' column.

"""
msg = (
"Unable to locate local census 2020 block data. Streaming instead.\n"
"If you plan to use census data repeatedly you can store it locally "
"with the io.store_blocks_2010 function for better performance"
)
if isinstance(states, (str, int)):
states = [states]
blks = {}
for state in states:
local = pathlib.Path(self.data_dir, "blocks_2020", f"{state}.parquet")
remote = f"s3://spatial-ucr/census/blocks_2020/{state}.parquet"
blks[state] = _fetcher(local, remote, msg)

if fips:
blks[state] = blks[state][blks[state]["geoid"].str.startswith(fips)]

blks[state]["year"] = 2020
blocks = list(blks.values())
blocks = gpd.GeoDataFrame(pd.concat(blocks, sort=True))

return blocks

def tracts_1990(self, states=None):
"""Nationwide Census Tracts as drawn in 1990 (cartographic 500k).

Expand Down
Loading
Loading