Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added Chrome Web Store Scrapper #1136

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
88 changes: 88 additions & 0 deletions dev-documentation.md
Original file line number Diff line number Diff line change
Expand Up @@ -2019,4 +2019,92 @@ cwars.get_profile()
| ----------------- | ---------------------------------------------------------------------------------- |
| `.get_profile()` | Returns the user data in json format. |

# Chrome Web Store

```python
from scrape_up import chromewebstore.ChromeWebStore
```
### Scraping Details about Chrome extensions

First create a object for class `ChromeWebStore`

```python
store = ChromeWebStore()
```
### Methods Details
| Methods | Details |
|----|----|
| `.search(keywords,filter(optional))` | returns a list of dictionaries containing extension details based on keyword and optional search filter and statues message |
| `.get_details(id,overview(optional))` | returns a dict containing details about given extension id |

### Notes
### **.search()**<br>
`.search()` returns message "success" if scraping is successful.
`.search()` uses an optional argument `filter` to filter the extension results
```python
filter = {
'itemTypes': 'EXTENSION' | 'THEME',
'filterBy': 'featured' | 'establishedPublisher' | 'featured,establishedPublisher',
'minimalRating': rating out of 5
}
```
the list of dictionaries returned by `.search()` has the following structure
```python
{
"title": "Elephantastic Theme",
"id": "abcdefg1234567890",
"link": "https://chromewebstore.google.com/webstore/detail/elephantastic-theme/abcdefg1234567890",
"thumbnail": "https://...",
"rating": "4.5",
"no_of_rating": "1000",
"description": "A delightful theme featuring playful elephant illustrations.",
"featured": True,
"verified_publisher": False
}
```

### **.get_details()**
`.get_details()` returns message "success" if scraping is sucessfull.
`.get_details()` uses optional argument `overview` which if `True` returns the overview or descripton of the extension along with the details.
the dicionary returned by `.get_details()` has following structure
```python
{
"title": "Video Speed Controller",
"thumbnail": "https:/...",
"rating": "4.6",
"featured": True,
"verified_publisher": False,
"tags": ["Extension", "Functionality & UI"],
"no_of_users": "3,000,000",
"overview": "Speed up, slow down, advance and rewind HTML5 audio/video with shortcuts",
"Updated": "May 24, 2024",
"Offered by": "igrigorik",
"Size": "72.43KiB",
"Languages": [""],
"Developer": "N/A708 Long Bridge Street\nSan Francisco, CA 94158-2512\nUS Email [email protected]",
"Non-trader": True,
}

```
## Example Usage
`.search()` without `filter`
```python
from scrape_up import chromewebstore.ChromeWebStore
store = ChromeWebStore()
extensions,message=store.search("video downloader")
```
`.search()` with `filter`
```python
from scrape_up import chromewebstore.ChromeWebStore
store = ChromeWebStore()
filter = { 'itemTypes': 'THEME','filterBy': 'featured','minimalRating': 3 }
extensions,message=store.search( "elephant", filter)
```
`.get_details()`
```python
store = ChromeWebStore()
extensions,message=store.get_details("nffaoalbilbmmfgbnbgppjihopabppdk", overview=True)
```


---
86 changes: 86 additions & 0 deletions docs/modules/chromewebstore.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
## Chrome Web Store

```python
from scrape_up import chromewebstore.ChromeWebStore
```
### Scraping Details about Chrome extensions

First create a object for class `ChromeWebStore`

```python
store = ChromeWebStore()
```
### Methods Details
| Methods | Details |
|----|----|
| `.search(keywords,filter(optional))` | returns a list of dictionaries containing extension details based on keyword and optional search filter and statues message |
| `.get_details(id,overview(optional))` | returns a dict containing details about given extension id |

### Notes
### **.search()**<br>
`.search()` returns message "success" if scraping is successful.
`.search()` uses an optional argument `filter` to filter the extension results
```python
filter = {
'itemTypes': 'EXTENSION' | 'THEME',
'filterBy': 'featured' | 'establishedPublisher' | 'featured,establishedPublisher',
'minimalRating': rating out of 5
}
```
the list of dictionaries returned by `.search()` has the following structure
```python
{
"title": "Elephantastic Theme",
"id": "abcdefg1234567890",
"link": "https://chromewebstore.google.com/webstore/detail/elephantastic-theme/abcdefg1234567890",
"thumbnail": "https://...",
"rating": "4.5",
"no_of_rating": "1000",
"description": "A delightful theme featuring playful elephant illustrations.",
"featured": True,
"verified_publisher": False
}
```

### **.get_details()**
`.get_details()` returns message "success" if scraping is sucessfull.
`.get_details()` uses optional argument `overview` which if `True` returns the overview or descripton of the extension along with the details.
the dicionary returned by `.get_details()` has following structure
```python
{
"title": "Video Speed Controller",
"thumbnail": "https:/...",
"rating": "4.6",
"featured": True,
"verified_publisher": False,
"tags": ["Extension", "Functionality & UI"],
"no_of_users": "3,000,000",
"overview": "Speed up, slow down, advance and rewind HTML5 audio/video with shortcuts",
"Updated": "May 24, 2024",
"Offered by": "igrigorik",
"Size": "72.43KiB",
"Languages": [""],
"Developer": "N/A708 Long Bridge Street\nSan Francisco, CA 94158-2512\nUS Email [email protected]",
"Non-trader": True,
}

```
## Example Usage
`.search()` without `filter`
```python
from scrape_up import chromewebstore.ChromeWebStore
store = ChromeWebStore()
extensions,message=store.search("video downloader")
```
`.search()` with `filter`
```python
from scrape_up import chromewebstore.ChromeWebStore
store = ChromeWebStore()
filter = { 'itemTypes': 'THEME','filterBy': 'featured','minimalRating': 3 }
extensions,message=store.search( "elephant", filter)
```
`.get_details()`
```python
store = ChromeWebStore()
extensions,message=store.get_details("nffaoalbilbmmfgbnbgppjihopabppdk", overview=True)
```
85 changes: 85 additions & 0 deletions documentation.md
Original file line number Diff line number Diff line change
Expand Up @@ -733,3 +733,88 @@ boxoffice = imdb.BoxOffice()
| Methods | Details |
| --------------- | ------------------------------------------------------------------------------- |
| `.top_movies()` | Returns the top box office movies, weekend and total gross, and weeks released. |


### Chrome Web Store

```python
from scrape_up import chromewebstore.ChromeWebStore
```
#### Scraping Details about Chrome extensions

First create a object for class `ChromeWebStore`

```python
store = ChromeWebStore()
```
#### Methods Details
| Methods | Details |
|----|----|
| `.search(keywords,filter(optional))` | returns a list of dictionaries containing extension details based on keyword and optional search filter and statues message |
| `.get_details(id,overview(optional))` | returns a dict containing details about given extension id |

#### Notes
`.search()` returns message "success" if scraping is successful.
`.search()` uses an optional argument `filter` to filter the extension results
```python
filter = {
'itemTypes': 'EXTENSION' | 'THEME',
'filterBy': 'featured' | 'establishedPublisher' | 'featured,establishedPublisher',
'minimalRating': rating out of 5
}
```
the list of dictionaries returned by `.search()` has the following structure
```python
{
"title": "Elephantastic Theme",
"id": "abcdefg1234567890",
"link": "https://chromewebstore.google.com/webstore/detail/elephantastic-theme/abcdefg1234567890",
"thumbnail": "https://...",
"rating": "4.5",
"no_of_rating": "1000",
"description": "A delightful theme featuring playful elephant illustrations.",
"featured": True,
"verified_publisher": False
}
```
`.get_details()` returns message "success" if scraping is sucessfull.
`.get_details()` uses optional argument `overview` which if `True` returns the overview or descripton of the extension along with the details.
the dicionary returned by `.get_details()` has following structure
```python
{
"title": "Video Speed Controller",
"thumbnail": "https:/...",
"rating": "4.6",
"featured": True,
"verified_publisher": False,
"tags": ["Extension", "Functionality & UI"],
"no_of_users": "3,000,000",
"overview": "Speed up, slow down, advance and rewind HTML5 audio/video with shortcuts",
"Updated": "May 24, 2024",
"Offered by": "igrigorik",
"Size": "72.43KiB",
"Languages": [""],
"Developer": "N/A708 Long Bridge Street\nSan Francisco, CA 94158-2512\nUS Email [email protected]",
"Non-trader": True,
}

```
#### Example Usage
`.search()` without `filter`
```python
from scrape_up import chromewebstore.ChromeWebStore
store = ChromeWebStore()
extensions,message=store.search("video downloader")
```
`.search()` with `filter`
```python
from scrape_up import chromewebstore.ChromeWebStore
store = ChromeWebStore()
filter = { 'itemTypes': 'THEME','filterBy': 'featured','minimalRating': 3 }
extensions,message=store.search( "elephant", filter)
```
`.get_details()`
```python
store = ChromeWebStore()
extensions,message=store.get_details("nffaoalbilbmmfgbnbgppjihopabppdk", overview=True)
```
3 changes: 3 additions & 0 deletions src/scrape_up/chromewebstore/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
from .chromewebstore import ChromeWebStore

__all__ = ['ChromeWebStore']
Loading