Fixed: news + trends, basic_data -> basic_info.

CHANGELOG.rst

@@ -0,0 +1,103 @@
+CHANGELOG
+=========
+
+0.4.10
+------
+- fixed top news fetch function
+- fixed Google trends fetch funtction
+- ``get_basic_data()`` -> ``get_basic_info()``
+
+0.4.9
+-----
+- new retry policy
+
+0.4.8
+-----
+- new ``fetch_crypto_live_data()``
+
+0.4.7.
+------
+- dependencies updated
+- news with non UTF-8 chars handled and dropped
+- fixed code formatting
+
+0.4.6
+-----
+- new ``price_change_24`` and ``price_change_24_percents`` properties for ``get_basic_data()``
+- new ``get_quick_search_data()``
+
+0.4.5
+-----
+- fixed dependencies
+
+0.4.4
+-----
+- removed obsolete parts of the code and some dependencies
+
+0.4.3
+-----
+- fixed ``get_basic_data()`` method (different data source)
+- new property in ``get_basic_data()`` return dict - ``rank``
+- tests enhanced
+
+0.4.2
+-----
+- fixed minor bugs
+
+0.4.1
+-----
+- new data in ``get_basic_data()`` method - ``year_low``, ``year_high``, ``yoy_change``
+
+0.4
+---
+- new method ``get_basic_data()``
+
+0.3.3
+-----
+- removed twitter integration - twitterscraper library is not up to date
+- fixed news fetching
+
+
+0.3.2
+-----
+- new method ``get_coin_ids()``
+- method ``fetch_crypto_historical_data()`` has ``id`` param now
+
+0.3.1
+-----
+- migrated to coingecko.com API (no API key needed anymore)
+
+0.3
+---
+- migrated to cryptocompare.com API (you need an API key now)
+- requirements are now managed by Poetry
+
+0.2.5
+-----
+- added ``fetch_top_news()`` method for top crypto news separated in 2 categories
+
+0.2.4
+-----
+- ``fetch_news()`` adds new "description" item and renames "image_url" to "image"
+- all ``fetch_news()`` item properties are now presented even if they are ``None``
+
+0.2.3
+-----
+- simplified import from ``from karpet.karpet import Karpet`` to ``from karpet import Karpet``
+
+0.2.2
+-----
+- added ``fetch_news()`` method for retrieving crypto news
+
+0.2.1
+-----
+- added ``fetch_exchanges()`` method for retrieving symbol exchange list
+- removed obsolete library dependency
+
+0.2
+---
+- twitter scraping added
+
+0.1
+---
+- initial release

README.rst

@@ -44,7 +44,7 @@ Usage
 .. code-block:: bash
 
    pip install karpet  # Basics only
-   pip install karpet[google]  # For Google trends
+   pip install karpet[google]  # With Google trends
 
 2. Import the library class first.
 
@@ -239,101 +239,7 @@ Retrieves live market data.
 
 Changelog
 ---------
-
-0.4.9
-~~~~~
-- new retry policy
-
-0.4.8
-~~~~~
-- new ``fetch_crypto_live_data()``
-
-0.4.7.
-~~~~~~
-- dependencies updated
-- news with non UTF-8 chars handled and dropped
-- fixed code formatting
-
-0.4.6
-~~~~~
-- new ``price_change_24`` and ``price_change_24_percents`` properties for ``get_basic_data()``
-- new ``get_quick_search_data()``
-
-0.4.5
-~~~~~
-- fixed dependencies
-
-0.4.4
-~~~~~
-- removed obsolete parts of the code and some dependencies
-
-0.4.3
-~~~~~
-- fixed ``get_basic_data()`` method (different data source)
-- new property in ``get_basic_data()`` return dict - ``rank``
-- tests enhanced
-
-0.4.2
-~~~~~
-- fixed minor bugs
-
-0.4.1
-~~~~~
-- new data in ``get_basic_data()`` method - ``year_low``, ``year_high``, ``yoy_change``
-
-0.4
-~~~
-- new method ``get_basic_data()``
-
-0.3.3
-~~~~~
-- removed twitter integration - twitterscraper library is not up to date
-- fixed news fetching
-
-
-0.3.2
-~~~~~
-- new method ``get_coin_ids()``
-- method ``fetch_crypto_historical_data()`` has ``id`` param now
-
-0.3.1
-~~~~~
-- migrated to coingecko.com API (no API key needed anymore)
-
-0.3
-~~~
-- migrated to cryptocompare.com API (you need an API key now)
-- requirements are now managed by Poetry
-
-0.2.5
-~~~~~
-- added ``fetch_top_news()`` method for top crypto news separated in 2 categories
-
-0.2.4
-~~~~~
-- ``fetch_news()`` adds new "description" item and renames "image_url" to "image"
-- all ``fetch_news()`` item properties are now presented even if they are ``None``
-
-0.2.3
-~~~~~
-- simplified import from ``from karpet.karpet import Karpet`` to ``from karpet import Karpet``
-
-0.2.2
-~~~~~
-- added ``fetch_news()`` method for retrieving crypto news
-
-0.2.1
-~~~~~
-- added ``fetch_exchanges()`` method for retrieving symbol exchange list
-- removed obsolete library dependency
-
-0.2
-~~~
-- twitter scraping added
-
-0.1
-~~~
-- initial release
+[here](./CHANGELOG.md)
 
 Credits
 -------

karpet/core.py

@@ -16,7 +16,6 @@
 
 
 class Karpet:
-
     quick_search_data = None
     req_retries = 4
     req_backoff_factor = 3
@@ -34,7 +33,6 @@ def __init__(self, start=None, end=None):
         self.req_ses = self.get_session()
 
     def get_session(self):
-
         # Waits for 1.5s, 3s, 6s, 12s, 24s between requests.
         status_forcelist = (500, 502, 503, 504, 429)
 
@@ -44,7 +42,7 @@ def get_session(self):
             connect=self.req_retries,
             backoff_factor=self.req_backoff_factor,
             status_forcelist=status_forcelist,
-            method_whitelist=False,
+            # method_whitelist=False,
         )
         adapter = HTTPAdapter(max_retries=retry)
 
@@ -293,7 +291,6 @@ def fetch_google_trends(
 
         # Fetch other batches.
         for date in trend_dates[1:]:
-
             time.sleep(sleeptime)
             pytrends.build_payload(
                 kw_list, cat=cat, timeframe=date, geo=geo, gprop=gprop
@@ -435,14 +432,13 @@ def parse_news(news_items):
                 news = []
 
                 for i in news_items:
-
                     try:
                         href = i.find("a")["href"]
                     except AttributeError:
                         # A without href attribute.
                         continue
 
-                    if not href.startswith("https://") or not href.startswith("//"):
+                    if not href.startswith("https://"):
                         href = "https://cointelegraph.com" + href
 
                     news.append({"url": href})
@@ -489,7 +485,7 @@ def get_coin_ids(self, symbol):
 
         return found_ids
 
-    def get_basic_data(self, symbol=None, id=None):
+    def get_basic_info(self, symbol=None, id=None):
         """
         Fetches coin/token basic data like:
 
@@ -616,7 +612,6 @@ async def fetch_one(session, news):
 
             try:
                 async with session.get(news["url"]) as response:
-
                     html = await response.text()
                     dom = BeautifulSoup(html, features="lxml")