Підтримка проксі

[OleksandrBlack]

#389

Завжди актуальний список проксі який використовується для спільної мети.
https://github.com/porthole-ascend-cinnamon/mhddos_proxy/blob/820defda23342fdaf90f2149d94b3464099d155b/runner.py#L15

Розробник
https://github.com/porthole-ascend-cinnamon

[SomalianCoolhacker]

За власним спостереженням додам, що переважна частина спільнот перейшла на цю програму саме завдяки відсутності мороки з VPN.
Звісно, ті хто тямить прикрутять OpenVPN чи ротацію проксі прямо в Docker і т.п., але, гадаю, для більшості це буде складно, навіть з гуглом під рукою.
А ще якщо враховувати, що машин десятки, то взагалі завал все налаштовувати

[arriven]

For anyone who want's to use this proxylist you can use following commandline flag:
--proxy '{{ join (split (get_url "https://raw.githubusercontent.com/porthole-ascend-cinnamon/proxy_scraper/main/proxies.txt") "\n") "," }}'

[arriven]

You can also pass that argument via SYSTEM_PROXY environment variable, i.e. SYSTEM_PROXY='{{ join (split (get_url "https://raw.githubusercontent.com/porthole-ascend-cinnamon/proxy_scraper/main/proxies.txt") "\n") "," }}' go run main.go

[arriven]

Please note that it only works starting from v0.8.12 release as split <string> "\n" was a bit broken before that

[OleksandrBlack]

dbd53bf

--proxy '{{ join (split (get_url "https://raw.githubusercontent.com/porthole-ascend-cinnamon/proxy_scraper/main/proxies.txt") "\n") "," }}'

SYSTEM_PROXY='{{ join (split (get_url "https://raw.githubusercontent.com/porthole-ascend-cinnamon/proxy_scraper/main/proxies.txt") "\n") "," }}' go run main.go

[OleksandrBlack]

Arriven
Doesn't work for Windows

[arriven]

hmm, might be something to do with how powershell/cmd handle string arguments. Can you check if that approach works for you if you put everything into a docker-compose file like this?

version: "3.9"

services:
  programm:
    image: ghcr.io/arriven/db1000n
    restart: unless-stopped
    environment:
      SYSTEM_PROXY: '{{ join (split (get_url "https://raw.githubusercontent.com/porthole-ascend-cinnamon/proxy_scraper/main/proxies.txt") "\n") "," }}'
      DEBUG: "true"

[OleksandrBlack]

hmm, might be something to do with how powershell/cmd handle string arguments. Can you check if that approach works for you if you put everything into a docker-compose file like this?

version: "3.9"

services:
  programm:
    image: ghcr.io/arriven/db1000n
    restart: unless-stopped
    environment:
      SYSTEM_PROXY: '{{ join (split (get_url "https://raw.githubusercontent.com/porthole-ascend-cinnamon/proxy_scraper/main/proxies.txt") "\n") "," }}'
      DEBUG: "true"

[arriven]

Yeah, proxies get down often. In mhddos it first tries to check if the proxy is working and only then starts using it for DDoS. My implementation just uses the proxy straight away and falls back to a different one if it fails (well, at least it should do that, there's a chance that there's a bug there). It does report the error but it's just to make it easier to detect broken proxies. Also compared to mhddos it doesn't track broken proxies and can return back to the one it has already detected as down but unless majority of proxies are down it might actually be better that way

[arriven]

To reiterate: those last error messages (error sending request {"error": "socks connect ..."}) indicate that the app has successfully parsed provided proxylist and uses it. Errors are there just because it fails to connect to some of the proxies but those errors are temporary and removing them might actually reduce efficiency

[industral]

autohealer service is not needed in case of using proxy, am I right?

[arriven]

yeah, it shouldn't be needed

[industral]

few things here. We need ability to:

1. reconnect to another IP after some time, in case IP will be blocked (either via ENV parameter with time or via docker exec command to send to container to reconnect)
2. determine that selected IP is working IP, if no - reconnect

Do we have any of such abilities? Would be very useful

[industral]

Also, I'm not sure if proxy works...

I can't see it from logs

2022/03/29 18:05:32.403233 main.go:50: DB1000n [Version: v0.8.13][PID=1]
2022/03/29 18:05:32.403446 countrychecker.go:52: Checking IP address, attempt #1
2022/03/29 18:05:32.541111 countrychecker.go:52: Checking IP address, attempt #2
2022/03/29 18:05:32.631694 countrychecker.go:52: Checking IP address, attempt #3
2022/03/29 18:05:32.726110 countrychecker.go:69: Current country: Canada (134.xxx.xxx.xxx)
2022/03/29 18:05:32.726391 config.go:74: Loading config from "/ko-app/config/config.json"

Can you add some logs to be sure that proxy really works?

[arriven]

@industral I've added proxy to countrychecker (should be available in 0.8.14) but be aware that with proxylist like this you might need a lot of retries for countrychecker to find working proxy (default is 3 but I didn't manage to get a result with this list even with 15 retries). The app won't crash and continue working even if it doesn't find working proxy in time as actual attack jobs do it independently and without any limit (it would be unwise to replicate that behavior in countrychecker). I'm also sure that proxy related code works properly - tested it out with my local tor proxy

[localdotcom]

I added SYSTEM_PROXY variable to the k8s deployment manifest

envVars:
- name: "SYSTEM_PROXY"
  value: '{{ join (split (get_url "https://raw.githubusercontent.com/porthole-ascend-cinnamon/proxy_scraper/main/proxies.txt") "\n") "," }}'

And getting following errors in container log

Is value '{{ join (split (get_url "https://raw.githubusercontent.com/porthole-ascend-cinnamon/proxy_scraper/main/proxies.txt") "\n") "," }}' correct?

[industral]

yeah, have the same after pull latest docker

{"level":"warn","ts":1648652837.6963356,"caller":"templates/templates.go:175","msg":"error executing template","error":"template: tpl:1:16: executing \"tpl\" at <get_url \"https://raw.githubusercontent.com/porthole-ascend-cinnamon/proxy_scraper/main/proxies.txt\">: error calling get_url: context deadline exceeded"}

[arriven]

Context deadline exceeded usually means that it has timed out while fetching url. Can you check if the pod has access to the internet?

[localdotcom]

@arriven I've tested it again with 1 pod replica, the error is gone.
Before I ran 15 pods with scale_factor=25. But I'm not sure if it's related to amount of pods.

[arriven]

if I were to guess I'd put my money on scale factor rather than the amount of pods. could be that the amount of connections used by the app exceeds the configured limit of file descriptors allowed for an app (not sure how that works in k8s, I never had that problem at all before this project)