Skip to content

Zenhub.io extraction tool for the Singer.io data warehouse

License

Notifications You must be signed in to change notification settings

chillu/singer-tap-zenhub

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

16 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

tap-zenhub

Overview

This is a Singer tap that produces JSON-formatted data following the Singer spec.

This tap:

  • Pulls raw data from Zenhub
  • Extracts the following resources:
  • Outputs the schema for each resource
  • Incrementally pulls data based on the input state

Due to limitations in the Zenhub API, issue events are pulled for all existing issues - there is no way to tell when an issue was last updated with Zenhub.

Quick start

  1. Install through pipenv

    pipenv install git+https://github.com/chillu/singer-tap-zenhub.git@master#egg=tap_zenhub
    
  2. Create a GitHub access token

    Login to your GitHub account, go to the Personal Access Tokens settings page, and generate a new token with at least the repo scope. Save this access token, you'll need it for the next step.

  3. Create a Zenhub API token.

  4. Create the config file

    Create a JSON file containing the access token you just created and the path to the repository. The repo paths are relative to https://github.com/. For example the path for this repository is singer-io/tap-github.

    {
        "github_token": "",
        "zenhub_token": "",
        "repos": [
            "some-org/some-repo",
            "some-org/some-other-repo"
        ]
    }
  5. Run the application

    pipenv run tap-zenhub --config config.json --state state.json
  6. Using state

Supplying a state.json helps the tap to reduce API calls, and only fetch closed issues since the last run. It will also re-start partial imports on a repo, and prioritise repos with the least recent import runs.

First you'll need to create an empty state file:

echo '{"bookmarks": {"issues": {}}}' > state.json

Now you can run it with that empty state:

pipenv run tap-zenhub --config config.json --state state.json

In order to write new state, you'll need to send to a target. Here's an example with the Stitch Target.

pipenv run tap-zenhub --state state.json --config config.json | target-stitch --config target-config.json -v  > state.tmp.json && rm state.json && mv state.tmp.json state.json

Development

Install as an editable dependency

git clone https://github.com/chillu/singer-tap-zenhub.git
pipenv install --dev -e .

About

Zenhub.io extraction tool for the Singer.io data warehouse

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages