README.md

💹 justETF Scraping

Scrape the justETF.

🛠️ Installation

To use justETF scraping package in your project, install the actual version from GitHub:

pip install git+https://github.com/druzsan/justetf-scraping.git

If you are going to play notebooks through, use the following installation:

pip install justetf-scraping[all]@git+https://github.com/druzsan/justetf-scraping.git

🚀 Usage

📋 Scrape the justETF Screener

Load overviews for all available (over 3400 at the moment) ETFs (requires a request for all ETF type: long-only, active, short & leveraged):

import justetf_scraping

df = justetf_scraping.load_overview()
df

	wkn	ticker	valor	name	inception_date	age_in_days	age_in_years	strategy	domicile_country	currency	hedged	securities_lending	dividends	ter	replication	size	is_sustainable	number_of_holdings	yesterday	last_week	last_month	last_three_months	last_six_months	last_year	last_three_years	last_five_years	2023	2022	2021	2020	last_dividends	last_year_dividends	last_year_volatility	last_three_years_volatility	last_five_years_volatility	last_year_return_per_risk	last_three_years_return_per_risk	last_five_years_return_per_risk	max_drawdown	last_year_max_drawdown	last_three_years_max_drawdown	last_five_years_max_drawdown	asset_class	instrument	region	exchange
isin
IE00B4L5Y983	A0RPWH	EUNL	10608388	iShares Core MSCI World UCITS ETF USD (Acc)	2009-09-25	5555	15.219178	Long-only	Ireland	USD	False	True	Accumulating	0.20	Optimized sampling	88335	False	1409	28.62	0.14	3.48	13.61	14.62	30.54	36.24	90.45	19.55	-12.96	32.10	6.13	NaN	NaN	11.83	15.29	18.35	2.57	0.71	0.75	-33.91	-8.75	-16.88	-33.91	Equity	ETF	World	gettex, XETRA, London, Stuttgart, SIX Swiss Exchange, Borsa Italiana, Euronext Amsterdam
IE00B3XXRP09	A1JX53	VUSA	18575508	Vanguard S&P 500 UCITS ETF (USD) Distributing	2012-05-22	4585	12.561644	Long-only	Ireland	USD	False	True	Distributing	0.07	Full replication	43952	False	487	34.27	-0.32	3.16	16.07	17.53	35.75	44.35	116.12	21.54	-13.29	39.08	8.05	0.95	1.27	13.67	18.25	21.76	2.61	0.71	0.76	-33.70	-9.09	-17.28	-33.70	Equity	ETF		gettex, XETRA, London, Euronext Paris, Stuttgart, SIX Swiss Exchange, Borsa Italiana, Euronext Amsterdam
IE00B3YCGJ38	A1CYW7	P500	11358996	Invesco S&P 500 UCITS ETF	2010-05-20	5318	14.569863	Long-only	Ireland	USD	False	False	Accumulating	0.05	Swap based Unfunded	27500	False	<NA>	34.71	-0.16	3.34	16.30	17.82	36.20	45.49	118.77	21.80	-13.10	39.37	8.31	NaN	NaN	13.66	18.16	21.75	2.64	0.73	0.78	-33.69	-9.09	-17.20	-33.69	Equity	ETF		gettex, XETRA, London, SIX Swiss Exchange, Borsa Italiana, Euronext Amsterdam

Further enrich the data with additional information (asset class, region, exchanges and instrument, it requires further requests):

df = justetf_scraping.load_overview(enrich=True)

Load long-only ETFs (only requires one single request):

df = justetf_scraping.load_overview(strategy="epg-longOnly")

Load MSCI World ETFs:

df = justetf_scraping.load_overview(strategy="epg-longOnly", index="MSCI World")

📈 Scrape ETF Chart Data from justETF (e.g.)

Load the whole history of a chosen ETF by its ISIN:

df = justetf_scraping.load_chart("IE00B0M62Q58")
df

	quote	relative	dividends	cumulative_dividends	quote_with_dividends	relative_with_dividends	reinvested_dividends	quote_with_reinvested_dividends	relative_with_reinvested_dividends
date
2005-10-28	20.60	0.000000	0.0	0.00	20.60	0.000000	0.000000	20.600000	0.000000
2005-10-29	20.60	0.000000	0.0	0.00	20.60	0.000000	0.000000	20.600000	0.000000
2005-10-30	20.60	0.000000	0.0	0.00	20.60	0.000000	0.000000	20.600000	0.000000
2005-10-31	20.99	1.893204	0.0	0.00	20.99	1.893204	0.000000	20.990000	1.893204
2005-11-01	21.03	2.087379	0.0	0.00	21.03	2.087379	0.000000	21.030000	2.087379
...	...	...	...	...	...	...	...	...	...
2024-12-05	77.86	277.961165	0.0	10.04	87.90	326.699029	24.922629	102.782629	398.944801
2024-12-06	77.70	277.184466	0.0	10.04	87.74	325.922330	24.871414	102.571414	397.919484
2024-12-07	77.70	277.184466	0.0	10.04	87.74	325.922330	24.871414	102.571414	397.919484
2024-12-08	77.70	277.184466	0.0	10.04	87.74	325.922330	24.871414	102.571414	397.919484
2024-12-09	77.60	276.699029	0.0	10.04	87.64	325.436893	24.839404	102.439404	397.278661

6983 rows × 9 columns

Compare

df = justetf_scraping.compare_charts(
    {
        "IE00B0M62Q58": justetf_scraping.load_chart("IE00B0M62Q58"),
        "IE00B0M63177": justetf_scraping.load_chart("IE00B0M63177"),
    },
    input_value="quote_with_dividends"
)
df

	IE00B0M62Q58	IE00B0M63177
date
2005-11-18	0.000000	0.000000
2005-11-19	0.000000	0.000000
2005-11-20	0.000000	0.000000
2005-11-21	-0.539811	-0.887436
2005-11-22	0.629780	-0.934143
...	...	...
2024-12-05	295.411606	143.390939
2024-12-06	294.691858	143.110696
2024-12-07	294.691858	143.110696
2024-12-08	294.691858	143.110696
2024-12-09	294.242015	146.987389

6962 rows × 2 columns

For further exploration examples, see Jupyter Notebooks

⚒️ Development Setup

To setup locally cloned project, first install Poetry:

pip install poetry

In the local project folder, install all dependencies and extras:

poetry install --all-extras

Activate local Poetry environment:

poetry shell

Setup pre-commit hooks:

pre-commit install

Similar Projects

• https://github.com/AshNL/justETF-overview-scraper
• https://github.com/SimonMandlik/etf_filter

Thanks

This project was inspired by this Stack Overflow question.