basic setup for trading stock and crypto assets and backtesting trading algo ideas, and gathering stock data
copy the file api_keys_TEMPLATE.json into a new file and name it api_keys.json
go to the website(s) for the exchange(s) you plan to use, create an account, and get whatever api keys, endpoints, etc. that you need
open the file and enter the keys into their corresponding location (based on exchanges and stuff)
(the .gitignore file keeps api_keys.json from getting put online)
Create Webull Setup
they have an API and everything!
Create Yahoo Finance
if theres data you can't find on webull
Stock Pup:
NOW:
use the stockpup code to create a scrapper too other sites also
which ones to use:
write the definitions of each CSV column in the DESCRIPTION of this file.
they're all on stockpup
http://www.stockpup.com/data/
EVENTUALLY:
make a script to plot average earnings of all companies over time
has it decreased in the last few years?
make a script to plot the values of all the fundamentals of a given company over time
sort the stocks by their stability in price and long term growth
what do their fundamentals look like in the beginning when they first went public?
what is differnent about their fundamentals from the stocks that are volitile and don't grow?
poloniex - crypto price data
doesn't service US citizens anymore though :(
0.25% trading fee
has margin trading with max 2x leverage
does provide an API
kraken - crypto price data
services US citizens!
0.26% trading fee
has margin trading with max 5x leverage
does provide an API
SOURCES:
API Docs:
https://www.kraken.com/en-us/features/api
Github python wrapper:
https://github.com/veox/python3-krakenex
coinbase pro - crypto price data
services US citizens!
0.50% trading fee
doesnt have margin trading
does provide an API
alpaca - stock price and fundamental data
services US citizens!
has no trading fee, except $0.02 SEC fee
has margin trading with max 4x leverage
does provide an API
https://alpaca.markets/
whats the volume?
can you short - yes
is there leverage - yes 2x (4x if you don't hold over night, cause its stocks, not 24/7)
can US citizens use it - yes, only US citizens can use it
https://www.reddit.com/r/algotrading/comments/9qqnwx/alpaca_just_released_their_commissionfree_trading/
https://medium.com/automation-generation/quick-review-of-setting-up-alpaca-api-with-python-dd31f3ac045f
https://docs.alpaca.markets/trading-on-alpaca/margin-and-shorting/
search "Margin Interest Calculation"
https://www.investopedia.com/ask/answers/07/margin_interest.asp
to see how 3.75% annual margin interest is prooooobably no big deal
stuff i signed:
https://app.alpaca.markets/internal/accounts/c518ee6d-9da7-492c-bd18-ef21c488c94a/agreements/nasdaq/preview
https://app.alpaca.markets/internal/accounts/c518ee6d-9da7-492c-bd18-ef21c488c94a/agreements/nyse/preview
SOURCES:
Step by Step Tutorial Videos - Paper Trading with the Alpaca API
https://alpaca.markets/docs/get-started-with-alpaca/tutorial-videos/
alpaca-trade-api-python
https://github.com/alpacahq/alpaca-trade-api-python
StockPup
DESCRIPTION:
This script will download the quarterly financial data and stock price of 765 companies
over the past 30 years from stockpup.com using the BeautifulSoup python library and
save all the data to the ./data/quarterly_financial_data/ folder as .csv files.
The 41 CSV columns are:
Quarter end
Shares
Shares split adjusted
Split factor
Assets .............................................. net value (in USD) of all money and goods the company owns
Current Assets
Liabilities ......................................... net value (in USD) of all debts the company owes
Current Liabilities
Shareholders equity
Non-controlling interest
Preferred equity
Goodwill & intangibles
Long-term debt
Revenue ............................................. net value (in USD) of all money the company made the previous quarter
Earnings
Earnings available for common stockholders
EPS basic
EPS diluted
Dividend per share
Cash from operating activities
Cash from investing activities
Cash from financing activities
Cash change during period
Cash at end of period
Capital expenditures
Capital expenditures (CapEx) are funds used by a company to acquire, upgrade, and maintain physical assets such as property, plants, buildings, technology, or equipment.
https://www.investopedia.com/terms/c/capitalexpenditure.asp
Price
Price high
Price low
ROE
ROA
Book value of equity per share
P/B ratio
P/E ratio = Price / Earnings
Cumulative dividends per share
Dividend payout ratio ............................... total money given in dividends to shareholders / net income
Long-term debt to equity ratio
Equity to assets ratio .............................. Equity / Assets = (Assets - Liabilities) / Assets
Net margin
Asset turnover ...................................... Revenue / Assets
Free cash flow per share
Current ratio
For the following 765 companies:
A
AAN
AAP
AAPL
ABBV
ABC
ABI_0
ABMD
ABT
ACAS
ACC
ACIW
ACM
ACN
ACS
ACV
ACXM
ADBE
ADCT
ADI
ADM
ADP
ADS
ADSK
ADT
AEE
AEO
AEP
AES
AET
AFG
AFL
AGCO
AGN
WPI
AHL
AIG
AIV
AIZ
AJG
AKAM
AKRX
ALB
ALEX
ALGN
ALK
ALL
ALLE
ALTR
ALXN
AMAT
ABK
AMCX
AMD
AME
AMG
AMGN
AMP
AMSC
AMT
AMZN
AN
TSO
ANDW
ANF
ANR
ANSS
WLP
AOC
AOS
APA
APC
APD
APH
APOL
APU
ARE
ARG
AA
ARRS
ARW
ASH
DV
ATI
ATVI
AVB
AVGO
AVP
AVY
AW
AWK
AXP
AYE
AYI
AZO
BA
BAC
BAX
BBBY
BBT
BBY
BC
BCR
BDK
BDX
FO
BEN
BF.B
BHF
BHI
BIG
BIIB
BJS
BK
BKE
BLK
BLL
BMC
BMS
BMY
BNI
BRCM
BRK.A
BRL
BRLI
BSC
BSX
BTU
BUD
BWA
BXP
C
CA
CAG
CAH
CAM
CAT
ACE
CB
CBE
CBG
CBOE
CBS
CCE
CCE_1
CCI
CCL
CCU
CDNS
CEG
CELG
CERN
CF
CFC
CFG
CFN
CHD
CHK
CHRW
CI
CIEN
CINF
CIT
CL
CLF
CLX
CMA
CMCSA
CME
CMG
CMI
CMS
CMVT
CNC
CNO
CNP
CNX
COF
COG
COH
COL
COO
COP
COST
COTY
COV
CPB
CPN
CPWR
CR
CRA
CRM
CRVL
CSCO
CSRA
CSX
CTAS
CTL
CTSH
CTX
CTXS
CVC
CVG
CVH
CVS
CVX
CXO
D
DAL
DCI
DD
DDR
DDS
DE
DELL
DF
DFS
DG_1
DGX
DHI
DHR
DIS
DISCA
DISH
DLPH
DLR
DLTR
DNB
DNR
DO
DOV
DPS
DRE
DRI
DTE
DTV
DUK
DVA
DVN
DOW
CSC
DYN
ERTS
EBAY
ECL
ED
EDS
EFX
EIX
EL
EMC
EMN
EMR
ENDP
EOG
EP
EQ
EQIX
EQR
EQT
NU
ESRX
ESS
ESV
ETFC
ETN
ETR
EW
EXC
EXPD
EXPE
EXR
F
FAST
FB
FBHS
FCX
FDO
FDS
FDX
FE
FFIV
FHN
FII
FIS
FISV
FITB
FL
FLIR
FLR
FLS
FMC
FRE
FNM
FOSL
NWS.A
FRT
FRX
FSLR
FTI
CZN
FTV
GAS
GAS_0
GCI_0
GD
GE
GENZ
GGP
GGP_0
WPO
GILD
GIS
GLAD
GLW
GM
GM_0
GME
GNW
GOOG
GPC
GPN
GPS
GR
GRA
GRMN
GS
GT
GWW
HAL
HAR
HAS
HBAN
HBI
HCA
HCBK
HCN
HCP
HD
HES
HIBB
HIG
HLT
HNZ
HOG
HOLX
HON
HOT
HP
HPC
HPE
HPQ
HRB
HRL
HRS
SLE
HSIC
HSP
HST
HSY
HUM
IACI
IBM
ICE
NYX
IDXX
IFF
IGT
ILMN
INCY
INFO
INTC
INTU
IP
IPG
IR
IRM
ISRG
IT
ITT
ITW
IVZ
JAVA
JBHT
JBL
JCI
JCP
JEC
JNJ
JNPR
JNS
JNY
JPM
JWN
K
LIZ
KBH
KEY
KG
KHC
KIM
KLAC
KMB
KMI
KMX
KO
EK
KORS
KR
KRFT
KSS
KSU
LTR
LTD
SAI
LEG
LEH_0
LEN
LH
LIFE
LKQ
LLL
LLTC
LLY
LM
LMT
LNC
LNKD
LNT
LOW
LRCX
LSI
LSTR
LUK
LUV
LVLT
LXK
LYB
M
MA
MAA
MAC
MAR
MAS
MAT
MBI
MCD
MCHP
MCK
MCO
KFT
MDP
MDT
MED
MER
MET
MFE
MGM
MHK
MHS
MI
MIL
MJN
MKC
MLM
MMC
MMM
MNK
MNST
MO
MOLX
MON
MOS
MPC
MRK
MRO
MS
MSFT
MOT
MTB
MTD
MTG
MTW
MU
MUR
MWV
MNST_0
MYL
NAV
NAVI
NBL
NBR
NCC
NCLH
NDAQ
NE
FPL
NEM
NFLX
NFX
NI
NILE
NKE
NLSN
NOC
NOV
NOVL
NRG
NSC
NSM
NTAP
NTRS
NUE
NUS
NVDA
NVLS
NWL
NWS
NYT
O
ODP
OFLX
OI
OKE
OMC
OMX
ORCL
ORLY
OXY
PAYX
PBCT
PBG
PBI
PCAR
PCG
PCL
PCLN
PCP
PDCO
PEG
PEP
PETM
PETS
PFE
PFG
PFIE
PG
PGN
PGR
PH
PHM
PKG
PKI
PLD
PLD_0
PLL
PM
PNC
PNR
PNW
POM
PPG
PPL
PRGO
PRU
PSA
PSX
PMTC
PTV
PVH
PWR
PX
PXD
PYPL
Q
Q_1
QCOM
QLGC
QRVO
QSII
R
RAD
RAI
RAVN
RCL
RDC
RE
REG
REGN
RF
RHI
RHT
RIG
RJF
RL
RMD
ROH
ROK
ROP
ROST
RRC
RRD
RSG
RSH
RTN
RX
S
SAF
SAM
SANM
SAPE
SBAC
SBL
SBUX
SCG
SCHW
SE
SEE
SGP
SHLD
SHW
SIAL
SIG
SII
SJM
SLB
SLG
SLM
SNA
SNDK
SNI
SNPS
SO
SOV
SPG
MHP
SPLS
SRCL
SRE
SSP
STI
STJ
STR
STT
STX
STZ
SUN
WFR
SVU
SWK
SWKS
SWN
SWY
SYF
SYK
SYMC
SYNT
SYY
T
TAP
TDC
TDG
TE
TEG
TEL
TER
TEX
GCI
TGT
THC
TIE
TIF
TIN
TJX
TKR
TLAB
TMK
TMO
TRIP
TROW
TRV
TSCO
TSLA
TSN
TSS
TT
TWC
TWTR
TWX
TXN
TXT
TYC
UA
UAL
UDR
UG
UHS
UIS
ULTA
UNH
UNM
UNP
UPS
URBN
URI
USB
UST
UTX
V
VAR
VFC
VIA.B
JDSU
VIVO
VLO
VMC
VNO
VRSK
VRSN
VRTX
VTR
VZ
WAG
WAT
WB
WBA
WDC
WDR
WEC
WEN
WFC
WFMI
WFT
WHR
WIN
WINA
WLTW
WM
WM_0
WMB
WMT
WPX
WRK
WU
WWY
WY
WYE
WYN
WYNN
X
XEC
XEL
XL
XLNX
XOM
XRAY
XRX
XTO
XYL
YHOO
YUM
ZMH
ZION
ZTS
USAGE:
Open trading-setup/src/data_sources/stocks/stockpup.py in your favorite text editor.
In main function there is:
assets = stockpup.get_data_of_all_assets('web')
stockpup.plot_data_quality_report()
one or both lines might be commented out. If you want to just look at the data quality report,
comment out the first line and run stockpup.py. If you want to append new data from stockpup
to the data you currently have, uncomment the first line, comment out the 2nd line, and run
stockpup.py. If you want to view the new data report, go into the plot_data_quality_report
function and find the lines:
# data_coverage_dct, data_coverage_2D_array = calculate_data_coverage(assets, quarters)
data_coverage_dct, data_coverage_2D_array = get_data_coverage_from_file()
One of them will be commented out. Use the one, calculate_data_coverage(assets, quarters) and
run stockpup.py.
SOURCES:
data source
http://www.stockpup.com/data/
SimFin:
DESCRIPTION:
This script will download the quarterly financial data and stock price of 2083 companies
over the past 10 years from stockpup.com using the BeautifulSoup python library and
save all the data to the ./data/quarterly_financial_data/ folder as .csv files.
The 61 CSV columns are:
Revenue
money generated from normal business operations
https://www.investopedia.com/terms/r/revenue.asp
Cost of Revenue
total cost of manufacturing and delivering a product or service to consumers
investopedia.com/terms/c/cost-of-revenue.asp
Gross Profit
Gross profit is the profit a company makes after deducting the costs associated with making and selling its products, or the costs associated with providing its services. Gross profit will appear on a company's income statement and can be calculated by subtracting the cost of goods sold (COGS) from revenue (sales). These figures can be found on a company's income statement. Gross profit may also be referred to as sales profit or gross income.
https://www.investopedia.com/terms/g/grossprofit.asp
Operating Expenses
An operating expense is an expense a business incurs through its normal business operations. Often abbreviated as OPEX, operating expenses include rent, equipment, inventory costs, marketing, payroll, insurance, step costs, and funds allocated for research and development. By contrast, a non-operating expense is an expense incurred by a business that is unrelated to the business's core operations.
https://www.investopedia.com/terms/o/operating_expense.asp
"Selling, General & Administrative"
Selling, general and administrative expense (SG&A) is reported on the income statement as the sum of all direct and indirect selling expenses and all general and administrative expenses (G&A) of a company. SG&A, also known as SGA, includes all the costs not directly tied to making a product or performing a service. That is, SG&A includes the costs to sell and deliver products and services and the costs to manage the company.
https://www.investopedia.com/terms/s/sga.asp
Research & Development
Research and development (R&D) expenses are associated directly with the research and development of a company's goods or services and any intellectual property generated in the process.
https://www.investopedia.com/terms/r/research-and-development-expenses.asp
Operating Income (Loss)
Non-Operating Income (Loss)
"Interest Expense, Net"
"Pretax Income (Loss), Adj."
Abnormal Gains (Losses)
Pretax Income (Loss)
"Income Tax (Expense) Benefit, Net"
Income (Loss) from Continuing Operations
Net Extraordinary Gains (Losses)
Net Income
Net Income (Common)
Close Price
"Cash, Cash Equivalents & Short Term Investments"
Accounts & Notes Receivable
Inventories
Total Current Assets
"Property, Plant & Equipment, Net"
Long Term Investments & Receivables
Other Long Term Assets
Total Noncurrent Assets
Total Assets
Total assets refers to the total amount of assets owned by a person or entity
https://www.accountingtools.com/articles/2017/5/15/total-assets
Payables & Accruals
Short Term Debt
Total Current Liabilities
Long Term Debt
Total Noncurrent Liabilities
Total Liabilities
Total liabilities are the combined debts and obligations that an individual or company owes to outside parties. ... On the balance sheet, total assets minus total liabilities equals equity.
https://www.investopedia.com/terms/t/total-liabilities.asp
Share Capital & Additional Paid-In Capital
Treasury Stock
Retained Earnings
Total Equity
Total Liabilities & Equity
Quarter end
Report Date
Publish Date,
Shares (Basic)
Shares (Diluted)
Net Income/Starting Line
Depreciation & Amortization
Non-Cash Items
Change in Working Capital
Change in Accounts Receivable
Change in Inventories
Change in Accounts Payable
Change in Other
Net Cash from Operating Activities
Change in Fixed Assets & Intangibles
Net Change in Long Term Investment
Net Cash from Acquisitions & Divestitures
Net Cash from Investing Activities
Dividends Paid
Cash from (Repayment of) Debt
Cash from (Repurchase of) Equity
Net Cash from Financing Activities
Net Change in Cash
For the following 2083 companies:
A
AA
AAC
AAL
AAME
AAN
AAOI
AAON
AAP
AAPL
AAWW
AAXN
AB
ABAX
ABBV
ABC
ABCD
ABG
ABM
ABMC
ABMD
ABT
ABTL
AC
ACA
ACAD
ACAT
ACCO
ACET
ACF
ACGL
ACHC
ACIA
ACLS
ACLZ
ACM
ACN
ACRX
ACT
ACW
ACY
ADBE
ADES
ADI
ADIL
ADM
ADNT
ADP
ADPT
ADRO
ADS
ADSK
ADT
ADVM
AE
AEE
AEHR
AEIS
AEO
AEP
AEPI
AES
AET
AFG
AFI
AFL
AGCO
AGHI
AGI
AGIO
AGLE
AGN
AGNC
AGN_old
AGR
AGX
AGYS
AHPI
AHS
AIG
AIMC
AIR
AIRI
AIT
AJBI
AJG
AJRD
AJX
AKAM
AKER
AKR
AKRX
AKS
AL
ALB
ALCO
ALE
ALEX
ALG
ALGN
ALGT
ALJ
ALJJ
ALK
ALKS
ALL
ALLE
ALNY
ALPN
ALR
ALRM
ALRN
ALSK
ALSN
ALT
ALTR
ALV
ALX
ALXN
AMAT
AMBA
AMCI
AMCX
AMD
AMDA
AME
AMED
AMG
AMGN
AMH
AMKR
AMNL
AMOT
AMP
AMRK
AMSC
AMSWA
AMT
AMTD
AMWD
AMZN
AN
ANDE
ANDV
ANDX
ANET
ANF
ANGI
ANIK
ANSS
ANTM
AOBC
AON
AOS
AOSL
APA
APC
APD
APH
APIC
APOG
APPF
APPS
APRN
APTI
APTV
APU
APVO
AR
ARA
ARAY
ARCB
ARCI
ARCW
ARE
AREX
ARG
ARI
ARKR
ARLP
ARMK
ARMK_old
ARNA
ARNC
ARO
ARQL
ARRS
ARRY
ARS
ARTX
ARW
ASCMA
ASH
ASNA
ASNB
ASPN
ASPU
ASTC
ASYS
ATHN
ATI
ATKR
ATNX
ATO
ATR
ATRO
ATU
ATVI
ATW
AVA
AVAV
AVB
AVEO
AVGO
AVGO_old
AVGR
AVIR
AVNW
AVP
AVT
AVXL
AVY
AWI
AWK
AWRE
AWX
AXGN
AXN
AXON
AXP
AXTA
AXTI
AYI
AYR
AYX
AZO
AZPN
AZZ
BA
BABY
BAC
BAH
BAX
BBBY
BBGI
BBRG
BBSI
BBT
BBY
BC
BCO
BCOR
BCPC
BCR
BDX
BERY
BFAM
BFS
BG
BGCP
BGG
BGNE
BGS
BGSF
BHE
BHF
BHGE
BHI
BID
BIG
BIIB
BIO
BJRI
BKE
BKH
BKI
BKNG
BKS
BL
BLCM
BLDR
BLFS
BLK
BLKB
BLL
BLOX
BLUE
BMI
BMRA
BMRN
BMS
BMY
BNED
BNFT
BOJA
BOKF
BONT
BOOM
BOOT
BOX
BPL
BPMC
BR
BRC
BRCD
BRCM
BREW
BRID
BRKA
BRKS
BRO
BRRE
BRS
BRSS
BRX
BSET
BSPM
BSQR
BSTC
BSTG
BSX
BTAI
BUDZ
BURL
BV
BVSN
BWA
BWLA
BWXT
BX
BXC
BXP
BYND
BZH
C
CA
CACI
CAG
CAGU
CAH
CAKE
CAL
CALA
CALD
CALL
CALM
CAMP
CARG
CARS
CASI
CASY
CAT
CATS
CAVM
CB
CBB
CBG
CBI
CBK
CBKM
CBLK
CBM
CBOE
CBRL
CBS
CBSH
CC
CCE
CCF
CCH
CCI
CCK
CCL
CCMP
CCOI
CCS
CCXI
CDE
CDI
CDNA
CDNS
CDR
CDW
CE
CEI
CELG
CENT
CERN
CETX
CF
CFG
CFI
CFMS
CFR
CGA
CGNX
CHD
CHDN
CHE
CHEF
CHJI
CHK
CHKE
CHRS
CHRW
CHSP
CHTR
CI
CIDM
CIE
CIEN
CIK
CINF
CIT
CKP
CL
CLAR
CLBS
CLC
CLCT
CLDR
CLF
CLFD
CLH
CLNE
CLNT
CLOW
CLR
CLRB
CLX
CMA
CMCO
CMCSA
CMD
CME
CMG
CMI
CMLS
CMND
CMP
CMPR
CMRO
CMS
CMTL
CNA
CNC
CNDT
CNFR
CNK
CNND
CNNX
CNP
CNS
CNSL
CNVR
CNXR
CNXT
CODA
COF
COG
COHR
COKE
COL
COLM
COMM
CONN
COO
COOL
COP
COR
CORT
COT
COTV
COTY
COUP
COVS
CP
CPB
CPN
CPRI
CPRT
CPS
CPSI
CPST
CPT
CPXX
CQH
CQP
CR
CRAY
CRC
CREE
CRIS
CRL
CRM
CROX
CRR
CRSP
CRTO
CRUS
CRVS
CRWS
CRY
CRZO
CSC
CSCO
CSGP
CSGS
CSII
CSL
CSLT
CSOD
CSRA
CSS
CST
CSU
CSWI
CSX
CTAS
CTIC
CTL
CTMX
CTOP
CTSH
CTXS
CUB
CUBE
CUO
CUZ
CVCO
CVGI
CVNA
CVS
CVSI
CVX
CWEN
CWH
CWRL
CXO
CXW
CY
CYAN
CYCC
CYIG
CZR
Cost
D
DAL
DATA
DBD
DBX
DCI
DCPH
DDD
DDE
DDS
DE
DELL
DEST
DF
DFS
DFT
DG
DGX
DHR
DHX
DIN
DIS
DISCA
DISH
DK
DKS
DLB
DLPH
DLTH
DLTR
DLX
DMRC
DNB
DNKN
DNOW
DOCU
DOMO
DORM
DOV
DOW
DPLO
DPS
DPZ
DRAD
DRE
DRI
DRQ
DRTT
DS
DSW
DTE
DTRM
DUK
DVA
DVAX
DVN
DWDP
DXC
DXCM
DXLG
DXPE
DY
DYN
DYNT
EA
EAF
EB
EBAY
EBF
EBIX
ECA
ECL
ECYT
ED
EEFT
EEI
EFH
EFOI
EFX
EGAN
EGHT
EGLT
EGP
EGY
EHTH
EIGI
EIX
EL
ELMD
ELS
ELY
EMC
EME
EMN
EMR
ENDP
ENDP_old
ENG
ENPH
ENR
ENS
ENV
ENVA
ENZ
EOG
EPAM
EPD
EPE
EPM
EPZM
EQIX
EQR
EQT
ERIE
ERS
ES
ESCA
ESI
ESL
ESP
ESRX
ESS
ESTC
ESV
ES_old
ETE
ETEC
ETFC
ETM
ETN
ETR
ETSY
EV
EVA
EVBG
EVC
EVH
EVI
EVLO
EVRI
EW
EWBC
EXA
EXAS
EXEL
EXP
EXPD
EXPE
EXPR
EXR
EXTR
F
FARO
FAST
FB
FBHS
FC
FCEL
FCNCA
FCPT
FCS
FCX
FDC
FDS
FDX
FEIM
FELE
FET
FEYE
FFIV
FGEN
FGP
FHB
FICO
FII
FIS
FISV
FIT
FITB
FIVE
FIVN
FIZZ
FKWL
FL
FLEX
FLIR
FLLZ
FLO
FLR
FLS
FLT
FLUG
FLWS
FLXS
FMBH
FMC
FMSA
FNB
FND
FNJN
FNKO
FNSR
FOE
FOLD
FONR
FORD
FORM
FORR
FOSL
FOX
FOX_old
FPRX
FRAN
FRD
FRGI
FRLF
FRP
FRPT
FRT
FRX
FRZT
FSCT
FSI
FSLR
FSS
FTDR
FTEK
FTI
FTI_old
FTNT
FTR
FTV
FUL
Fred
Fuel
G
GAMZ
GAS
GBT
GBX
GCI
GCO
GD
GDDY
GDI
GDOT
GE
GEL
GENC
GEO
GEOS
GERN
GES
GFF
GGG
GGP_old
GH
GHC
GHDX
GIDYL
GIII
GILD
GIMO
GIS
GLBL
GLGI
GLP
GLPI
GLUU
GLW
GLYC
GM
GMAN
GME
GMED
GNTX
GNW
GOOD
GOOG
GOOG_old
GOV
GPC
GPN
GPOR
GPRE
GPRO
GPS
GRA
GRC
GRMM
GRMN
GRPN
GRUB
GS
GSEG
GSIT
GT
GTHX
GTN
GTXI
GUID
GWGH
GWR
GWRE
GWRS
GWW
GXP
Golf
H
HABT
HAL
HALO
HAR
HAS
HAYN
HBAN
HBI
HCA
HCCI
HCHC
HCOM
HCP
HCSG
HD
HDP
HDS
HDSN
HEAR
HEI
HELE
HEMA
HFC
HGEN
HGR
HHC
HI
HIBB
HIG
HII
HIL
HLF
HLI
HLS
HLT
HMHC
HMTV
HNI
HNNA
HNRG
HOG
HOLX
HOMB
HOME
HON
HOT
HP
HPE
HPJ
HPQ
HQY
HRB
HRC
HRI
HRL
HRS
HRTG
HSIC
HSII
HST
HSTM
HSY
HTGM
HTH
HTM
HTZ
HUBB
HUBS
HUM
HUN
HURC
HURN
HWC
HXL
HYH
IAC
IAIC
IART
IBKC
IBKR
IBM
IBOC
ICE
ICHR
ICPT
ICPW
ICUI
IDA
IDCC
IDSY
IDT
IDXX
IEHC
IEP
IEX
IFF
IFON
IGT
IHRT
IHT
IIN
ILMN
IMDZ
IMGN
IMMY
IMO
IMPV
IMSCQ
INCR
INCY
INFN
INFO
INFU
INGR
INOV
INT
INTC
INTL
INTT
INTU
INVA
IO
IONS
IOTS
IP
IPAR
IPDN
IPG
IPGP
IPHI
IPI
IPXL
IQV
IR
IRBT
IRG
IRM
ISBC
ISIL
ISRG
ISSC
IT
ITCI
ITG
ITGR
ITI
ITRI
ITW
IVAC
IVDN
IVTY
IVZ
JACK
JAKK
JAZZ
JBCT
JBHT
JBL
JBLU
JCI
JCOM
JCP
JCS
JEC
JILL
JKHY
JLL
JMBA
JNJ
JNPR
JOUT
JOY
JPM
JSDA
JUVF
JVA
JWA
JWN
Joe
K
KANP
KAR
KATE
KE
KEM
KEY
KEYS
KHC
KIM
KIRK
KITE
KKD
KKR
KLAC
KLIC
KLXI
KMB
KMG
KMI
KMX
KND
KO
KONA
KOOL
KOPN
KPTI
KR
KRO
KSS
KSU
KTEC
KTOS
KTWO
KURA
KWR
L
LABL
LAKE
LAMR
LANC
LARK
LAYN
LAZ
LB
LBTYA
LC
LCI
LCUT
LDOS
LDOS_old
LE
LEA
LEDS
LEE
LEG
LEVB
LEVI
LFGR
LFUS
LGCY
LGIH
LGL
LGND
LH
LHCG
LII
LITE
LIVE
LIVN
LKQ
LKSD
LL
LLL
LLTC
LLY
LMB
LMT
LNC
LNG
LNGG
LNN
LNTH
LOGI
LOGM
LOPE
LOV
LPG
LPLA
LPTH
LPX
LQDT
LRCX
LRDC
LSI
LSI_old
LSTR
LSXMA
LSYN
LTHM
LTRPA
LTRX
LUK
LULU
LUNA
LUV
LVGO
LVLT
LXRX
LYB
LYFT
LYV
LZB
M
MA
MAA
MACK
MAMS
MAN
MANH
MAR
MARA
MAS
MASI
MAT
MATX
MAXD
MB
MBFI
MBOT
MBWM
MC
MCBC
MCD
MCFT
MCHP
MCK
MCO
MD
MDB
MDLZ
MDP
MDR
MDRX
MDSO
MDT
MDXG
MECK
MED
MEET
MEI
MEIP
MELI
MENT
MET
MFON
MGCD
MGEE
MGLN
MGM
MGNX
MHH
MHK
MIC
MICR
MICT
MIK
MIND
MITK
MIXX
MJCO
MJN
MJTK
MKC
MKL
MKSI
MKTX
MLAB
MLM
MLP
MMC
MMEG
MMI
MMM
MMMB
MMP
MMS
MN
MNI
MNK
MNKD
MNST
MNTM
MO
MOBL
MON
MORN
MOS
MOV
MOXC
MPC
MPWR
MPX
MRAM
MRC
MRCY
MRK
MRNA
MRO
MRSN
MRTX
MRVL
MS
MSA
MSCC
MSCI
MSFT
MSGO
MSI
MSM
MTB
MTBC
MTCH
MTD
MTEM
MTN
MTOR
MTRN
MTRX
MTSI
MTZ
MU
MUR
MVIS
MWW
MXIM
MXMG
MXPT
MXWL
MYCC
MYGN
MYL
MYL_old
Mac
Mark
NAII
NANO
NATI
NATL
NATR
NAV
NAVI
NBIX
NBL
NC
NCLH
NCMI
NCR
NDAQ
NDLS
NDSN
NE
NEE
NEM
NEO
NEOG
NEOS
NEP
NEPH
NERV
NEWR
NFG
NFLX
NFX
NGL
NGVC
NGVT
NHTC
NI
NICK
NJR
NK
NKE
NKSH
NKTR
NLN
NLNK
NLSN
NLST
NMBL
NNN
NOC
NOV
NOVT
NOW
NP
NPTN
NRG
NSC
NSPR
NSSC
NTAP
NTGR
NTNX
NTRA
NTRI
NTRS
NUAN
NUE
NUTR
NUVA
NVAX
NVCR
NVDA
NVEC
NWE
NWGI
NWL
NWS
NWY
NX
NXST
NYT
O
OA
OCC
OCN
ODFL
ODP
ODT
OFLX
OGS
OHI
OI
OII
OKE
OKTA
OLED
OLLI
OLN
OMC
OMCL
OMED
OMF
OMGB
OMI
OMP
ON
ONS
ONVO
OOMA
OPHT
OPRX
OPY
ORCL
ORLY
OSIS
OSK
OSTK
OTEX
OTTR
OUT
OXFD
OXM
OXY
OZRK
P
PACB
PACW
PAH
PAHC
PANW
PATK
PAYC
PAYS
PAYX
PBCT
PBH
PBI
PBYI
PCG
PCMI
PCP
PCTI
PDCO
PDEX
PDFS
PDLI
PEB
PEDH
PEG
PEGA
PEN
PEP
PERF
PESI
PETM
PETS
PFE
PFG
PFGC
PG
PGC
PGNX
PGR
PGRE
PGTI
PH
PHH
PHM
PHX
PI
PII
PINC
PINS
PIR
PJT
PK
PKE
PKG
PKI
PLAB
PLAN
PLAY
PLCE
PLD
PLNT
PLPC
PM
PMD
PNC
PNFP
PNR
PNRA
PNW
PODD
POOL
POST
PPC
PPG
PPIH
PPL
PPSI
PRCP
PRGB
PRGO
PRGS
PRKA
PRLB
PRLE
PROM
PRSC
PRTS
PRTX
PRU
PRXL
PSA
PSDV
PSSR
PSTG
PSTI
PSX
PTE
PTEN
PTN
PTON
PUB
PURE
PVAC
PVH
PVHO
PWR
PX
PXD
PYPL
PYX
PZZA
QCOM
QCP
QEP
QLGC
QNST
QRVO
QSII
QSIM
QSR
QTNA
QUOT
R
RA
RAD
RAI
RAIL
RATE
RAVE
RAVN
RAX
RCII
RCL
RCUS
RDC
RDFN
RDNT
RE
REAL
RECN
REED
REGN
REI
REMI
REN
REPR
RES
RESI
REV
REXR
REZI
RF
RFIL
RFP
RGA
RGC
RGEN
RGLS
RGNX
RGS
RH
RHE
RHI
RHT
RICK
RIG
RIGL
RIHT
RKDA
RL
RLGY
RLJE
RLOC
RMBL
RMCF
RMD
RNG
ROCK
ROK
ROKU
ROL
ROP
ROST
RP
RPM
RRC
RS
RSG
RSLS
RSPP
RSYS
RT
RTN
RUBI
RUN
RUSHA
RWC
RWLK
RYAM
RYN
S
SABR
SAFM
SAFT
SAIA
SAIC
SAIL
SAM
SANM
SANP
SANW
SATS
SAVE
SBAC
SBGI
SBH
SBUX
SCG
SCHL
SCHW
SCND
SCOR
SCRH
SCSC
SCVL
SCWX
SCX
SDPI
SE
SEAS
SEB
SEDG
SEE
SEGN
SEIC
SERV
SEV
SFBC
SFD
SFIX
SFLY
SFM
SFS
SGA
SGEN
SGH
SGM
SGMA
SGMO
SGY
SHAK
SHLD
SHLM
SHLO
SHLX
SHO
SHOR
SHOS
SHPGF
SHW
SIG
SIGI
SIGM
SIGN
SINO
SIRI
SITE
SIVB
SIX
SJI
SJM
SKX
SKY
SLB
SLCA
SLG
SLM
SLNM
SMA
SMCI
SMED
SMG
SMID
SMIT
SMLP
SMP
SMRT
SMSI
SMTC
SNA
SNAK
SNAP
SNBR
SNCR
SNDK
SNH
SNHY
SNI
SNOW
SNPS
SNX
SO
SOFO
SOHO
SOHU
SONC
SONO
SONS
SORL
SOTK
SPAN
SPB
SPCE
SPG
SPGI
SPLK
SPLS
SPNC
SPNE
SPPI
SPRT
SPXC
SQ
SQBG
SRCE
SRCI
SRCL
SRDX
SRE
SRNE
SRPT
SRRA
SRT
SSC
SSI
SSNC
SSNI
SSNT
SSP
SSTK
SSYS
ST
STAY
STBA
STE
STI
STJ
STLD
STLR
STML
STMP
STOR
STRM
STRT
STT
STX
STZ
SUP
SUPN
SURF
SVT
SVU
SWI
SWK
SWKS
SWM
SWN
SWX
SWY
SXC
SXT
SYF
SYK
SYMC
SYNA
SYNT
SYRS
SYUT
SYY
T
TA
TAP
TAST
TAYD
TBI
TBIO
TBTC
TCMD
TCNT
TCX
TDC
TDG
TDOC
TDW
TECD
TECH
TEL
TEN
TESO
TESS
TEVA
TFSL
TFX
TGI
TGNA
TGT
TGTX
THO
THS
TIF
TIS
TITN
TIVO
TJX
TLCC
TLND
TLRA
TLRD
TLRY
TLYS
TMED
TMK
TMO
TMUS
TNDM
TNET
TOCA
TOL
TOWR
TPB
TPCS
TPIC
TPR
TPX
TR
TRC
TREE
TRGP
TRHC
TRIP
TRMB
TRN
TRNC
TRNS
TROV
TROW
TROX
TRT
TRU
TRUE
TRUP
TRV
TSCO
TSE
TSLA
TSN
TSQ
TSRI
TSRO
TSS
TSSI
TST
TTC
TTD
TTGT
TTMI
TTOO
TTWO
TUES
TUP
TUSK
TVPT
TVTY
TWI
TWIN
TWLO
TWMC
TWOU
TWTR
TWX
TXMD
TXN
TXRH
TYL
TYME
TYPE
TZOO
Time
UA
UAL
UBER
UBNT
UDR
UFI
UFS
UG
UHAL
UHS
UIHC
UIS
ULBI
ULH
ULTA
ULTI
UMBF
UMRX
UNAM
UNFI
UNH
UNM
UNP
UOLI
UPL
UPLD
UPS
UPWK
URBN
URI
USAP
USAT
USB
USCR
USFD
USG
USM
USNA
UTHR
UTL
UTMD
UTX
UVE
V
VAL
VALU
VAPE
VAR
VDSI
VEC
VEEV
VER
VERU
VFC
VGR
VIAB
VIDE
VIRC
VISI
VKTX
VLGEA
VLO
VLRX
VMC
VMI
VMW
VNCE
VNO
VOXX
VRA
VRNS
VRNT
VRS
VRSK
VRSN
VRTU
VRTX
VRX
VSLR
VSM
VSTM
VSTO
VTR
VZ
W
WAB
WAIR
WAL
WAT
WATT
WBA
WBC
WBMD
WCG
WCIC
WDAY
WDC
WDDD
WEB
WEC
WELL
WEN
WESC
WFC
WFM
WFT
WFT_old
WGL
WGO
WGP
WHG
WHR
WIFI
WIN
WINA
WINH
WIRE
WK
WLB
WLK
WLTW
WM
WMAR
WMB
WMK
WMT
WNC
WNDW
WOOF
WOR
WPTH
WPX
WR
WRI
WRK
WSBC
WSCI
WSCO
WSM
WSTC
WSTL
WTR
WTS
WTW
WU
WWD
WWE
WWW
WY
WYN
WYNN
X
XBIT
XCO
XCRA
XEC
XEL
XLNX
XNCR
XOG
XOM
XONE
XOXO
XPEL
XPER
XPO
XRAY
XRX
XSPY
XYL
Y
YELP
YEXT
YGYI
YHOO
YOGA
YUM
ZAGG
ZAYO
ZBH
ZEDG
ZEN
ZEUS
ZG
ZGNX
ZION
ZIOP
ZNGA
ZS
ZTS
ZUMZ
ZVO
ZYNE
ZYXI
USAGE:
Open trading-setup/src/data_sources/stocks/simfin_scrapper.py in your favorite text editor.
In main function there is:
assets = simfin_scrapper.get_data_of_all_assets('web')
simfin_scrapper.plot_data_quality_report()
one or both lines might be commented out. If you want to just look at the data quality report,
comment out the first line and run simfin_scrapper.py. If you want to append new data from
simfin to the data you currently have, uncomment the first line, comment out the 2nd line, and run
simfin_scrapper.py. If you want to view the new data report, go into the plot_data_quality_report
function and find the lines:
# data_coverage_dct, data_coverage_2D_array = calculate_data_coverage(assets, quarters)
data_coverage_dct, data_coverage_2D_array = get_data_coverage_from_file()
One of them will be commented out. Use the one, calculate_data_coverage(assets, quarters) and
run simfin_scrapper.py.
SOURCES:
Github:
https://github.com/SimFin/simfin
Docs:
https://simfin.readthedocs.io/en/latest/hubs.html