Skip to content

Commit

Permalink
Add a Readme.md
Browse files Browse the repository at this point in the history
  • Loading branch information
NV4RE committed Dec 17, 2022
1 parent d32206f commit 89b28c4
Show file tree
Hide file tree
Showing 2 changed files with 71 additions and 1 deletion.
67 changes: 67 additions & 0 deletions Readme.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
## tiktok-scraper-chrome-headless-go

TikTok's user profile scraper, if Captcha is detected, you have to resolve it manually, and press enter in terminal to continue.

### Basic use
```shell
./tiktok-scraper-linux-amd64 -profile-url https://www.tiktok.com/@username_here
```

### With video duration
```shell
./tiktok-scraper-linux-amd64 -profile-url https://www.tiktok.com/@username_here -video-duration
```

### If you have problem starting the scraper
```shell
./tiktok-scraper-linux-amd64 -profile-url https://www.tiktok.com/@username_here -disable-gpu
```

### All arguments
```shell
Usage of ./main:
-debug-log
Enable debug logging
-disable-gpu
Disable GPU
-exec-path string
Path to Chrome/Chromium or Brave executable
-headless
Run browser in headless mode
-max-page-wait-sec int
Maximum time to wait for page to load (default 20)
-output string
Output file (default "output.json")
-profile-url string
URL to the profile to scrape, e.g. https://www.tiktok.com/@username
-video-duration
Get video duration
```

### Example Result
```json
{
"ProfileUrl": "https://www.tiktok.com/@xxxx",
"StartedAt": "0001-01-01T00:00:00Z",
"VideoStats": [
{
"Url": "https://www.tiktok.com/@xxxx/video/123412312?is_copy_url=1\u0026is_from_webapp=v1",
"Description": "Foo bar",
"Likes": "6.9K",
"Comments": "699",
"UploadAt": "11h ago",
"DurationSeconds": 194.466667,
"Date": "0001-01-01T00:00:00Z"
},
{
"Url": "https://www.tiktok.com/@xxxx/video/123412312?is_copy_url=1\u0026is_from_webapp=v1",
"Description": "Foo bar 2",
"Likes": "96K",
"Comments": "699",
"UploadAt": "11h ago",
"DurationSeconds": 69.69,
"Date": "0001-01-01T00:00:00Z"
}
]
}
```
5 changes: 4 additions & 1 deletion main.go
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,7 @@ func main() {
result := Result{
ProfileUrl: *profileUrl,
VideoStats: make([]VideoStat, 0),
StartedAt: time.Now(),
}

err = chromedp.Run(ctx,
Expand Down Expand Up @@ -184,7 +185,9 @@ func checkCaptcha(ctx context.Context) error {
}

func getVideoStat(ctx context.Context, maxPageWait time.Duration, videoDuration bool) (VideoStat, error) {
vStat := VideoStat{}
vStat := VideoStat{
Date: time.Now(),
}
ctx, cancel := context.WithTimeout(ctx, maxPageWait)
defer cancel()

Expand Down

0 comments on commit 89b28c4

Please sign in to comment.