Skip to content

Latest commit

 

History

History
90 lines (73 loc) · 5.06 KB

README.md

File metadata and controls

90 lines (73 loc) · 5.06 KB
Raw

stock-watcher

Simple python script to get quotes and minute bars from Alpaca and publish them. Details:

  • Quotes are retrieved by polling
  • Bars are delivered aynchronously through websockets
  • Quotes and bars are printed to stdout (log output goes to stderr)
  • A FastAPI server is started on port 8004 (configurable), and prometheus metrics are accessible at http://localhost:8004/metrics/
  • Quote and bar price/volume/size info is published via prometheus guages with symbol as label
  • Prometheus counters for quote/bar/trade/error counts are also available
  • If kafka is enabled (KAFKA_DISABLE unset), quotes and bars are published to kafka (set KAFKA_BOOTSTRAP)
  • Only new quotes are published (i.e. if unchanged it is not published)

To run:

First obtain API credentials for your alpaca account and set ALPACA_SECRET and ALPACA_KEY environment variables to your secret and key values respectively

Local script

As a local pythons script

  • [optional] create a venv and activate it
  • pip3 install -r requirements.txt
  • Configure STOCK_SYMBOLS, KAFKA_BOOTSTRAP or KAFKA_DISABLE and ALPACA_POLL_SECONDS environment variables if required (see below)
  • python3 main.py

Docker

$ docker pull drpump/stock-watcher
$ docker run -it -p 8004:8004 -e ALPACA_KEY=${ALPACA_KEY} -e ALPACA_SECRET=${ALPACA_SECRET} -e KAFKA_DISABLE='' drpump/stock-watcher

k8s + kafka/prometheus

Suggest running locally first to confirm your credentials and connectivity are OK.

  1. Kafka (optional)

  2. Install strimzi kubernetes operator for kakfa and create a kafka cluster: see the quickstarts, use cluster name dev-cluster if you want to minimise editing of manifests

  3. kubectl apply -f manifests/kafk-topics.yaml, modifying cluster name if desired

  4. Prometheus (recommended but sometimes finicky to install)

  5. Install the prometheusstack, suggest using the helm chart

  6. kubectl applf -f manifests/stock-watcher

  7. Watcher app

  8. Create a k8s secret with these credentials: kubectl create secret generic alpaca-creds --from-literal=ALPACA_KEY=${ALPACA_KEY} --from-literal=ALPACA_SECRET=${ALPACA_SECRET}

  9. Edit manifests/stock-watcher.yaml and set your STOCK_SYMBOLS, KAFKA_BOOTSTRAP and ALPACA_POLL_INTERVAL environment variables. If not using kafka, add a KAFKA_DISABLE variable.

  10. `kubectl apply -f manifests/stock-watcher.yaml

  11. To see kafka output (replace boostrap url if required): kubectl -n kafka run kafka-consumer -ti --image=quay.io/strimzi/kafka:0.40.0-kafka-3.7.0 --rm=true --restart=Never -- bin/kafka-console-consumer.sh --bootstrap-server dev-cluster-kafka-bootstrap:9092 --topic stock-quotes

  12. For prometheus/grafana:

  13. Open grafana and choose Explore

  14. Choose prometheus as the source

  15. In the query dialogue, set the metric to quote_bid and optionally filter by symbol

  16. Hey presto, you'll get a graph of the bid prices

  17. Optionally add another query for the quote_ask metric and see them plotted together

Configuration

The app uses the following environment variables for config:

  • STOCK_SYMBOLS Comma-separated list of symbols to poll/watch, no spaces permitted, default is RMD,AAPL (ResMed and Apple)
  • ALPACA_KEY Your ALPACA API key
  • ALPACA_SECRET Your ALPACA API secret
  • KAFKA_BOOTSTRAP Kafka bootstrap host+port for publishing quotes to Kafka. Default is dev-cluster-kafka-bootstrap.kafka.svc.cluster.local:9092 (strimzi k8s cluster called dev-cluster in kafka namespace)
  • KAFKA_DISABLE If set to any value, kafka will not be used. Quotes will be printed to stdout in JSON format and price + count metrics will still be accessible on the FastAPI endpoint.
  • ALPACA_POLL_SECONDS Polling interval in seconds, default is 60s (same as Alpaca bars interval)
  • SERVICE_PORT Port to use for HTTP serve prometheus metrics and livez/readyz (health) checks. Default is 8004. Retrieve metrics from http://localhost:${SERVICE_PORT}/metrics/.

Implementation and behaviour notes

  • Alpaca free accounts can only access the IEX exchange. Quotes, bars and trades from this exchange are considerably less frequent than for other exchanges. For example, RMD gets a new bar every 1-5 minutes on average trading days.
  • Assuming you run a Prometheus server, you will configure the server to poll the price gauges every N seconds If there is no update since the last poll it gets the same value again. If there were multiple updates, it only gets the last one in the period.
  • If you want to better reflect all updates in your analytics, Kafka + Flink will allow you to do near-real-time analytics on all messages the data streams. You could also add the received messages to a time-series database for more static analysis.
  • A zero price for ask/bid in the quote data means that there are no sellers/buyers, not that the price is zero, so these values are not added to the corresponding prometheus price guages.
  • The trades push feed is currently untested and has no prometheus guages.