Skip to content

Commit

Permalink
Change to_csv param 'get_params' for **kwargs
Browse files Browse the repository at this point in the history
Also adds `to_csv` to `other_prices_methods.ipynb` tutorial.
  • Loading branch information
maread99 committed Feb 19, 2024
1 parent 81f2aad commit ee40de4
Show file tree
Hide file tree
Showing 4 changed files with 132 additions and 30 deletions.
116 changes: 111 additions & 5 deletions docs/tutorials/other_prices_methods.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,8 @@
"source": [
"#### Sections\n",
"* [`request_all_prices`](#request_all_prices)\n",
"* [`prices_for_symbols`](#prices_for_symbols)"
"* [`prices_for_symbols`](#prices_for_symbols)\n",
"* [`to_csv`](#to_csv)"
]
},
{
Expand Down Expand Up @@ -135,7 +136,7 @@
"source": [
"Notice that no prices are stored for the H1 interval. This is because prices include an equity listed on the New York Stock Exchange and one listed on the London Stock Exchange. The opening times of these exchanges are such that sessions often overlap although the indices at H1 are not aligned. Consequently, it's not possible to evaluate common hourly indices (at least not using H1 data).\n",
"\n",
"The rest of this subsection demonstrates some under-the-bonnet behaviour that can be safely skipped - move onto [`prices_for_symbols`](#prices_for_symbols) if you're not interested.\n",
"The rest of this section demonstrates some under-the-bonnet behaviour that can be safely skipped - move onto [`prices_for_symbols`](#prices_for_symbols) if you're not interested.\n",
"\n",
"Srictly speaking, valid hourly data will be available for any session where only one of these two exchanges is open, or when one or both have irregular opening hours such that they do not overlap (or align if they do). If intraday prices are requested for just such a session then a request for hourly data will be sent to the provider if intraday prices for that session are not available at an interval smaller than one hour."
]
Expand Down Expand Up @@ -729,13 +730,118 @@
"source": [
"new_prices._pdata[new_prices.bis.T5]._table"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## `to_csv`\n",
"`to_csv` provides for exporting price data to .csv files. The method has one required argument 'path' which takes a path to an **existing** directory to which the .csv files should be written. If no further arguments are passed then the default implementation will export all available price data as one csv file per aligned base interval per symbol, such that if there are 3 base intervals and 5 symbols then 15 .csv files will be created (the base intervals for a prices instance can be inspected with `prices.bis.__members__`).\n",
"\n",
"Optional arguments provide for defining the `intervals` for which price data should be exported together with which symbols to `include` or `exclude`. The period over which price data is to be exported and the configuration of that data can be defined by passing any kwargs that are accepted by the `get` method (with the exception of 'interval'). For example 'start', 'days', 'anchor', 'priority', 'strict' etc are all valid keyword arguments.\n",
"\n",
"Files exported with `to_csv` can be retrieved with the default implementation of the `PricesCsv` class. (NB this requires that the exported data conforms with the requirements of the `PricesCsv` class, for example that prices are anchored on the 'open' and have an interval no higher than daily. Files exported with the default implementation will always be retrievable via the `PricesCsv` class.)\n",
"\n",
"See the method doc for further information..."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"prices.to_csv?"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"```\n",
"Signature:\n",
"prices.to_csv(\n",
" path: 'Annotated[Union[str, Path], Coerce(Path), Parser(parsing.verify_directory)]',\n",
" intervals: 'Optional[Union[str, pd.Timedelta, datetime.timedelta, list[str], list[pd.Timedelta], list[datetime.timedelta]]]' = None,\n",
" include: 'Optional[mptypes.Symbols]' = None,\n",
" exclude: 'Optional[mptypes.Symbols]' = None,\n",
" **kwargs,\n",
") -> 'list[Path]'\n",
"Docstring:\n",
"Export price data to .csv file(s).\n",
"\n",
"Note: Exported price data can be retrieved with the default\n",
"implementation of the `PricesCsv` class (requires that the\n",
"exported data conforms with the requirements of the\n",
"`PricesCsv` class, for example that prices are anchored on\n",
"the 'open' and have an interval no higher than daily).\n",
"\n",
"Price data will be exported by symbol by interval, such that if\n",
"data is requested for 3 intervals and 5 symbols then 15 .csv\n",
"files will be created.\n",
"\n",
".csv filenames will follow the format:\n",
" <SYMBOL>_<INTERVAL>_<YYMMDD>_<YYMMDD>.csv\n",
" For example:\n",
" MSFT_5T_240122_240215.csv\n",
" This file would hold '5T' (i.e. 5 minute) price data for\n",
" the symbol MSFT covering the period from 2024-01-22 through\n",
" 2024-02-15. Note: for intraday intervals the dates will\n",
" represent the earliest and latest sessions for which at least\n",
" some price data is included.\n",
"\n",
"Parameters\n",
"----------\n",
"path\n",
" Directory to which .csv files should be written. This path\n",
" must exist.\n",
"\n",
"intervals\n",
" Intervals for which price data is to be exported. To define\n",
" a single interval pass as for the 'interval` parameter of the\n",
" `.get` method. To define multiple intervals pass as a list\n",
" of one of the types that's acceptable input to the 'interval`\n",
" parameter of the `.get` method.\n",
"\n",
" By default (None) .csv files are exported for all available\n",
" base intervals.\n",
"\n",
"include : list[str] | str | None\n",
" Symbol or symbols to include in export. All other symbols will\n",
" be excluded. If passed, do not pass `exclude`.\n",
"\n",
" By default, if neither include nor exclude are passed then data\n",
" will be exported for all symbols.\n",
"\n",
"exclude : list[str] | str | None\n",
" Symbol or symbols to exclude from export. Data will be exported\n",
" for all other symbols. If passed, do not pass `include`.\n",
"\n",
" By default, if neither exclude nor include are passed then data\n",
" will be exported for all symbols.\n",
"\n",
"kwargs\n",
" All other kwargs will be passed on to the `.get` method to\n",
" define the period over which prices are to be exported. Can\n",
" include other options, for example 'anchor', 'priority',\n",
" 'strict' etc.\n",
"\n",
" If no other kwargs are not passed then by default all available\n",
" data will be exported for each requested symbol / interval.\n",
"\n",
"Returns\n",
"-------\n",
"paths\n",
" List of Path objects to which data exported.\n",
"```"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "mkt_prices 3.8.2",
"display_name": "market_prices_ve_39",
"language": "python",
"name": "mkt_prices"
"name": "market_prices_ve_39"
},
"language_info": {
"codemirror_mode": {
Expand All @@ -747,7 +853,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.2"
"version": "3.9.13"
},
"widgets": {
"application/vnd.jupyter.widget-state+json": {
Expand Down
27 changes: 11 additions & 16 deletions src/market_prices/prices/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -5122,7 +5122,7 @@ def to_csv(
] = None,
include: Optional[mptypes.Symbols] = None,
exclude: Optional[mptypes.Symbols] = None,
get_params: Optional[dict] = None,
**kwargs,
) -> list[Path]:
"""Export price data to .csv file(s).
Expand Down Expand Up @@ -5176,23 +5176,20 @@ def to_csv(
By default, if neither exclude nor include are passed then data
will be exported for all symbols.
get_params
Dictionary of parameters to be passed on to the `.get` method
to define the period over which prices are to be exported. Can
kwargs
All other kwargs will be passed on to the `.get` method to
define the period over which prices are to be exported. Can
include other options, for example 'anchor', 'priority',
'strict' etc.
If not passed then by default all available data will be
exported for each requested symbol / interval.
If no other kwargs are not passed then by default all available
data will be exported for each requested symbol / interval.
Returns
-------
paths
List of Path objects to which data exported.
"""
# NOTE If / when valimp supports **kwargs then the 'get_params'
# parameter could be changed to **kwargs (more convenient for client).

if TYPE_CHECKING:
assert isinstance(path, Path)

Expand All @@ -5204,18 +5201,16 @@ def to_csv(
else:
intervals_ = [to_ptinterval(intervals)]

if get_params is not None and "lose_single_symbol" in get_params:
get_params["lose_single_symbol"] = False
if kwargs.get("lose_single_symbol", False):
kwargs["lose_single_symbol"] = False

dfs = {}
for intrvl in intervals_:
if get_params is not None:
if kwargs:
try:
df = self.get(
intrvl, include=include, exclude=exclude, **get_params
)
df = self.get(intrvl, include=include, exclude=exclude, **kwargs)
except Exception as err:
raise errors.PricesUnavailableForExport(intrvl, get_params) from err
raise errors.PricesUnavailableForExport(intrvl, kwargs) from err
else:
try:
df = self.get(
Expand Down
18 changes: 9 additions & 9 deletions tests/test_base_prices.py
Original file line number Diff line number Diff line change
Expand Up @@ -6368,18 +6368,18 @@ def assert_output_as_original_files(paths: list[Path]):
df_reloaded = df_reloaded[df_reloaded.columns.sort_values()]
assert_frame_equal(df, df_reloaded)

# verify can pass get_params
# verify can pass kwargs
clean_temp_test_dir()
get_params = {"start": df.index[7].right, "end": df.index[-7].left}
paths = prices.to_csv(temp_dir, "5T", get_params=get_params)
kwargs = {"start": df.index[7].right, "end": df.index[-7].left}
paths = prices.to_csv(temp_dir, "5T", **kwargs)
assert len(paths) == 3
prices_reloaded = csv.PricesCsv(temp_dir, symbols, calendars)
df_reloaded = prices_reloaded.get("5T")
assert df_reloaded.pt.interval == prices.bis.T5
assert df_reloaded.pt.first_ts == get_params["start"]
assert df_reloaded.pt.last_ts == get_params["end"]
assert df_reloaded.pt.first_ts == kwargs["start"]
assert df_reloaded.pt.last_ts == kwargs["end"]

# verify raises when no get_params
# verify raises when no kwargs
clean_temp_test_dir()
match = re.escape(
"It was not possible to export prices as an error was raised when prices were"
Expand All @@ -6390,8 +6390,8 @@ def assert_output_as_original_files(paths: list[Path]):
prices.to_csv(temp_dir, include=["NOT_A_SYMBOL"])
assert not list(temp_dir.iterdir())

# verify raises when pass get_params
get_params = {"start": df.pt.first_ts - (one_day * 7)}
# verify raises when pass kwargs
kwargs = {"start": df.pt.first_ts - (one_day * 7)}
match = re.escape(
"It was not possible to export prices as an error was raised when"
f" prices were requested for interval {TDInterval.T5}. The error is included at"
Expand All @@ -6400,5 +6400,5 @@ def assert_output_as_original_files(paths: list[Path]):
"\nNB prices have not been exported for any interval."
)
with pytest.raises(errors.PricesUnavailableForExport, match=match):
prices.to_csv(temp_dir, "5T", get_params=get_params)
prices.to_csv(temp_dir, "5T", **kwargs)
assert not list(temp_dir.iterdir())
1 change: 1 addition & 0 deletions tests/test_yahoo.py
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@
# ...sessions that yahoo temporarily fails to return prices for if (seemingly)
# send a high frequency of requests for prices from the same IP address.
_flakylist = (
pd.Timestamp("2024-01-21"),
pd.Timestamp("2023-09-08"),
pd.Timestamp("2023-09-01"),
pd.Timestamp("2023-07-17"),
Expand Down

0 comments on commit ee40de4

Please sign in to comment.