Merge branch 'Guovin:master' into master

s71557 · Nov 5, 2024 · 04d9568 · 04d9568
2 parents 570ac40 + b86a241
commit 04d9568
Show file tree

Hide file tree

Showing 17 changed files with 151 additions and 74 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,5 +1,45 @@
 # 更新日志（Changelog）
 
+## v1.5.1
+
+### 2024/11/5
+
+- ✨ 新增频道接口白名单：不参与测速，永远保留在结果最前面（#470）
+  使用方法：
+  1. 模板频道接口地址后添加$!即可实现（如：广东珠江,http://xxx.m3u$! ）
+  2. 额外信息补充（如：广东珠江,http://xxx.m3u$!额外信息 ），更多接口白名单请至https://github.com/Guovin/TV/issues/514 讨论
+- ✨ 新增 🈳 无结果频道分类：无结果频道默认归类至该底部分类下（#473）
+- ✨ 接口地址增加来源类型说明
+- ✨ 默认模板增加广东民生（#481）、广州综合（#504）
+- 🪄 优化偏好结果输出
+- 🪄 重构配置读取与增加全局常量
+- 🐛 修复部分接口匹配失败问题
+- 🐛 修复更新结果为空等问题（#464，#467）
+- 🐛 修复接口地址复制空格问题（#472 by:@haohaitao）
+- 🐛 修复结果日志 unpack error
+- 🐛 修复结果接口信息为空问题（#505）
+- 🗑️ 移除仓库根目录 txt 结果文件，请至 output 目录下查看结果文件
+
+<details>
+  <summary>English</summary>
+
+- ✨ Added channel interface whitelist: Not participating in speed testing, always kept at the very front of the results. (#470)
+  Usage:
+  1. Add $! after the template channel interface address (e.g., Guangdong Pearl River, http://xxx.m3u$!).
+  2. Additional information can be appended (e.g., Guangdong Pearl River, http://xxx.m3u$! Additional Information) (#470). For more interface whitelists, please discuss at https://github.com/Guovin/TV/issues/514.
+- ✨ Added 🈳 No Results Channel Category: Channels without results are categorized under this bottom category by default (#473).
+- ✨ Interface addresses now include source type descriptions.
+- ✨ Default templates now include Guangdong People's Livelihood (#481) and Guangzhou Comprehensive (#504).
+- 🪄 Optimized preferred result output.
+- 🪄 Refactored configuration reading and added global constants.
+- 🐛 Fixed issues with partial interface matching failures.
+- 🐛 Fixed problems with empty update results, etc. (#464, #467).
+- 🐛 Fixed the issue of spaces being copied with the interface address (#472 by:@haohaitao).
+- 🐛 Fixed the unpack error in result logs.
+- 🐛 Fixed the issue of empty interface information in results (#505).
+- 🗑️ Removed txt result files from the repository root directory. Please check the result files in the output directory.
+</details>
+
 ## v1.5.0
 
 ### 2024/10/25

diff --git a/README.md b/README.md
@@ -21,7 +21,7 @@
   </div>
   <br>
   <div>
-  🏠广东频道: 广东珠江, 广东体育, 广东新闻, 广东卫视, 大湾区卫视, 广州影视, 广州竞赛, 江门综合, 江门侨乡生活, 佛山综合, 深圳卫视, 汕头综合, 汕头经济, 汕头文旅, 茂名综合, 茂名公共
+  🏠广东频道: 广东珠江, 广东体育, 广东新闻, 广东民生, 广东卫视, 大湾区卫视, 广州综合, 广州影视, 广州竞赛, 江门综合, 江门侨乡生活, 佛山综合, 深圳卫视, 汕头综合, 汕头经济, 汕头文旅, 茂名综合, 茂名公共
   </div>
   <br>
   <div>
@@ -70,23 +70,27 @@
   <a href="https://hub.docker.com/repository/docker/guovern/tv-requests">
     <img src="https://img.shields.io/docker/pulls/guovern/tv-requests?label=docker:requests" />
   </a>
-   <a href="https://hub.docker.com/repository/docker/guovern/tv-driver">
+  <a href="https://hub.docker.com/repository/docker/guovern/tv-driver">
     <img src="https://img.shields.io/docker/pulls/guovern/tv-driver?label=docker:driver" />
   </a>
+  <a href="https://github.com/Guovin/TV/fork">
+    <img src="https://img.shields.io/github/forks/guovin/tv" />
+  </a>
 </p>
 
 [English](./README_en.md) | 中文
 
-## 特点
+## ✅ 特点
 
-- 自定义模板，生成您想要的频道
-- 支持多种获取源方式：组播源、酒店源、订阅源、关键字搜索
-- 接口测速验效，响应时间、分辨率优先级，过滤无效接口
-- 定时执行，北京时间每日 6:00 与 18:00 执行更新
-- 支持多种运行方式：工作流、命令行、GUI 软件、Docker(amd64/arm64)
-- 更多功能请见[配置参数](./docs/config.md)
+- ✅ 自定义模板，生成您想要的频道
+- ✅ 支持多种获取源方式：组播源、酒店源、订阅源、关键字搜索
+- ✅ 接口测速验效，响应时间、分辨率优先级，过滤无效接口
+- ✅ 偏好设置：IPv6、接口来源排序优先级与数量配置、接口白名单
+- ✅ 定时执行，北京时间每日 6:00 与 18:00 执行更新
+- ✅ 支持多种运行方式：工作流、命令行、GUI 软件、Docker(amd64/arm64)
+- ✨ 更多功能请见[配置参数](./docs/config.md)
 
-## 最新结果：
+## ✨ 最新结果
 
 - 接口源：
 
@@ -104,11 +108,11 @@ https://ghproxy.net/raw.githubusercontent.com/Guovin/TV/gd/output/result.txt
 https://ghproxy.net/raw.githubusercontent.com/Guovin/TV/gd/source.json
 ```
 
-## 配置
+## ⚙️ 配置
 
 [配置参数](./docs/config.md)
 
-## 快速上手
+## 🪄 快速上手
 
 ### 方式一：工作流
 
@@ -117,8 +121,14 @@ Fork 本项目并开启工作流更新，具体步骤请见[详细教程](./docs
 ### 方式二：命令行
 
 ```python
-pip3 install pipenv
+pip install pipenv
+```
+
+```python
 pipenv install
+```
+
+```python
 pipenv run build
 ```
 
@@ -168,22 +178,22 @@ docker run -d -p 8000:8000 guovern/tv-requests 或 tv-driver
 
 #### 注：方式一至三更新完成后的结果文件链接：http://本地 ip:8000 或 http://localhost:8000
 
-## 更新日志
+## 🗓️ 更新日志
 
 [更新日志](./CHANGELOG.md)
 
-## 许可证
+## 📄 许可证
 
 [MIT](./LICENSE) License &copy; 2024-PRESENT [Govin](https://github.com/guovin)
 
-## 赞赏
+## 💰️ 赞赏
 
 <div>请我喝杯咖啡☕️吧~</div>
 
 | 支付宝                                    | 微信                                        |
 | ----------------------------------------- | ------------------------------------------- |
 | ![支付宝扫码](./static/images/alipay.jpg) | ![微信扫码](./static/images/appreciate.jpg) |
 
-## 免责声明
+## 📣 免责声明
 
 本项目仅供学习交流用途，接口数据均来源于网络，如有侵权，请联系删除
diff --git a/README_en.md b/README_en.md
@@ -21,7 +21,7 @@
   </div>
   <br>
   <div>
-  🏠Guangdong Channel: 广东珠江, 广东体育, 广东新闻, 广东卫视, 大湾区卫视, 广州影视, 广州竞赛, 江门综合, 江门侨乡生活, 佛山综合, 深圳卫视, 汕头综合, 汕头经济, 汕头文旅, 茂名综合, 茂名公共
+  🏠Guangdong Channel: 广东珠江, 广东体育, 广东新闻, 广东民生, 广东卫视, 大湾区卫视, 广州综合, 广州影视, 广州竞赛, 江门综合, 江门侨乡生活, 佛山综合, 深圳卫视, 汕头综合, 汕头经济, 汕头文旅, 茂名综合, 茂名公共
   </div>
   <br>
   <div>
@@ -70,23 +70,27 @@
   <a href="https://hub.docker.com/repository/docker/guovern/tv-requests">
     <img src="https://img.shields.io/docker/pulls/guovern/tv-requests?label=docker:requests" />
   </a>
-   <a href="https://hub.docker.com/repository/docker/guovern/tv-driver">
+  <a href="https://hub.docker.com/repository/docker/guovern/tv-driver">
     <img src="https://img.shields.io/docker/pulls/guovern/tv-driver?label=docker:driver" />
   </a>
+  <a href="https://github.com/Guovin/TV/fork">
+    <img src="https://img.shields.io/github/forks/guovin/tv" />
+  </a>
 </p>
 
 [中文](./README.md) | English
 
-## Features
+## ✅ Features
 
-- Customize the template to generate the channel you want
-- Supports multiple source acquisition methods: multicast source, hotel source, subscription source, keyword search
-- Interface speed testing and verification, with priority on response time and resolution, filtering out ineffective interfaces
-- Scheduled execution at 6:00 AM and 18:00 PM Beijing time daily
-- Supports various execution methods: workflows, command line, GUI software, Docker(amd64/arm64)
-- For more features, see [Config parameter](./docs/config_en.md)
+- ✅ Customize the template to generate the channel you want
+- ✅ Supports multiple source acquisition methods: multicast source, hotel source, subscription source, keyword search
+- ✅ Interface speed testing and verification, with priority on response time and resolution, filtering out ineffective interfaces
+- ✅ Preferences: IPv6, priority and quantity of interface source sorting, and interface whitelist
+- ✅ Scheduled execution at 6:00 AM and 18:00 PM Beijing time daily
+- ✅ Supports various execution methods: workflows, command line, GUI software, Docker(amd64/arm64)
+- ✨ For more features, see [Config parameter](./docs/config_en.md)
 
-## Latest results：
+## ✨ Latest results
 
 - Interface source:
 
@@ -104,11 +108,11 @@ https://ghproxy.net/raw.githubusercontent.com/Guovin/TV/gd/output/result.txt
 https://ghproxy.net/raw.githubusercontent.com/Guovin/TV/gd/source.json
 ```
 
-## Config
+## ⚙️ Config
 
 [Config parameter](./docs/config_en.md)
 
-## Quick Start
+## 🪄 Quick Start
 
 ### Method 1: Workflow
 
@@ -117,8 +121,14 @@ Fork this project and initiate workflow updates, detailed steps are available at
 ### Method 2: Command Line
 
 ```python
-pip3 install pipenv
+pip install pipenv
+```
+
+```python
 pipenv install
+```
+
+```python
 pipenv run build
 ```
 
@@ -168,22 +178,22 @@ For example: docker run -v /etc/docker/config:/tv-requests/config -v /etc/docker
 
 #### Note: Link to the result file after updates of methods one to three: http://local ip:8000 or http://localhost:8000
 
-## Changelog
+## 🗓️ Changelog
 
 [Changelog](./CHANGELOG.md)
 
-## License
+## 📄 License
 
 [MIT](./LICENSE) License &copy; 2024-PRESENT [Govin](https://github.com/guovin)
 
-## Appreciate
+## 💰️ Appreciate
 
 <div>Please buy me a cup of coffee☕️~</div>
 
 | Alipay                                | Wechat                                    |
 | ------------------------------------- | ----------------------------------------- |
 | ![Alipay](./static/images/alipay.jpg) | ![Wechat](./static/images/appreciate.jpg) |
 
-## Disclaimer
+## 📣 Disclaimer
 
 This project is for learning and communication purposes only. All interface data comes from the internet. If there is any infringement, please contact us for removal.
diff --git a/docs/tutorial.md b/docs/tutorial.md
@@ -174,7 +174,7 @@ https://mirror.ghproxy.com/raw.githubusercontent.com/您的github用户名/仓
 
 2. 运行更新
 项目目录下打开终端 CMD 依次运行以下命令：
-pip3 install pipenv
+pip install pipenv
 pipenv install
 pipenv run build
 ```

diff --git a/docs/tutorial_en.md b/docs/tutorial_en.md
@@ -171,7 +171,7 @@ Please download and install Python from the official site. During installation,
 
 2. Run Update
 Open a CMD terminal in the project directory and run the following commands in sequence:
-pip3 install pipenv
+pip install pipenv
 pipenv install
 pipenv run build
 ```

diff --git a/updates/fofa/fofa_hotel_region_result.pkl b/updates/fofa/fofa_hotel_region_result.pkl
diff --git a/updates/fofa/request.py b/updates/fofa/request.py
@@ -6,6 +6,7 @@
 from driver.setup import setup_driver
 import re
 from utils.config import config
+import utils.constants as constants
 from utils.retry import retry_func
 from utils.channel import format_channel_name
 from utils.tools import merge_objects, get_pbar_remaining, add_url_info, resource_path
@@ -91,9 +92,10 @@ async def get_channels_by_fofa(urls=None, multicast=False, callback=None):
         test_url = fofa_urls[0][0]
         proxy = await get_proxy(test_url, best=True, with_test=True)
     cancel_event = threading.Event()
+    hotel_name = constants.origin_map["hotel"]
 
     def process_fofa_channels(fofa_info):
-        nonlocal proxy, fofa_urls_len, open_driver, open_sort, cancel_event
+        nonlocal proxy
         if cancel_event.is_set():
             return {}
         fofa_url = fofa_info[0]
@@ -116,7 +118,7 @@ def process_fofa_channels(fofa_info):
                 page_source = retry_func(
                     lambda: get_source_requests(fofa_url), name=fofa_url
                 )
-            if "资源访问每天限制" in page_source:
+            if "禁止访问" in page_source or "资源访问每天限制" in page_source:
                 cancel_event.set()
                 raise ValueError("Limited access to fofa page")
             fofa_source = re.sub(r"<!--.*?-->", "", page_source, flags=re.DOTALL)
@@ -130,7 +132,11 @@ def process_fofa_channels(fofa_info):
                 with ThreadPoolExecutor(max_workers=100) as executor:
                     futures = [
                         executor.submit(
-                            process_fofa_json_url, url, fofa_info[1], open_sort
+                            process_fofa_json_url,
+                            url,
+                            fofa_info[1],
+                            open_sort,
+                            hotel_name,
                         )
                         for url in urls
                     ]
@@ -184,7 +190,7 @@ def process_fofa_channels(fofa_info):
     return fofa_results
 
 
-def process_fofa_json_url(url, region, open_sort):
+def process_fofa_json_url(url, region, open_sort, hotel_name="酒店源"):
     """
     Process the FOFA json url
     """
@@ -208,11 +214,11 @@ def process_fofa_json_url(url, region, open_sort):
                                 total_url = (
                                     add_url_info(
                                         f"{url}{item_url}",
-                                        f"{region}酒店源|cache:{url}",
+                                        f"{region}{hotel_name}|cache:{url}",
                                     )
                                     if open_sort
                                     else add_url_info(
-                                        f"{url}{item_url}", f"{region}酒店源"
+                                        f"{url}{item_url}", f"{region}{hotel_name}"
                                     )
                                 )
                                 if item_name not in channels:

diff --git a/updates/hotel/request.py b/updates/hotel/request.py
@@ -41,7 +41,7 @@ async def get_channels_by_hotel(callback=None):
     start_time = time()
 
     def process_region_by_hotel(region):
-        nonlocal proxy, open_driver, page_num
+        nonlocal proxy
         name = f"{region}"
         info_list = []
         driver = None

diff --git a/updates/multicast/request.py b/updates/multicast/request.py
@@ -53,7 +53,7 @@ async def get_channels_by_multicast(names, callback=None):
         merge_objects(search_region_type_result, fofa_result)
 
     def process_channel_by_multicast(region, type):
-        nonlocal proxy, open_driver, page_num, start_time
+        nonlocal proxy
         name = f"{region}{type}"
         info_list = []
         driver = None

diff --git a/updates/online_search/request.py b/updates/online_search/request.py
@@ -1,5 +1,6 @@
 from asyncio import create_task, gather
 from utils.config import config
+import utils.constants as constants
 from utils.speed import get_speed
 from utils.channel import (
     format_channel_name,
@@ -11,6 +12,7 @@
     get_pbar_remaining,
     get_soup,
     format_url_with_cache,
+    add_url_info,
 )
 from updates.proxy import get_proxy, get_proxy_next
 from time import time
@@ -61,9 +63,10 @@ async def get_channels_by_online_search(names, callback=None):
     if open_proxy:
         proxy = await get_proxy(pageUrl, best=True, with_test=True)
     start_time = time()
+    online_search_name = constants.origin_map["online_search"]
 
     def process_channel_by_online_search(name):
-        nonlocal proxy, open_proxy, open_driver, page_num
+        nonlocal proxy
         info_list = []
         driver = None
         try:
@@ -166,6 +169,7 @@ def process_channel_by_online_search(name):
                             for result in results:
                                 url, date, resolution = result
                                 if url and check_url_by_patterns(url):
+                                    url = add_url_info(url, online_search_name)
                                     url = format_url_with_cache(url)
                                     info_list.append((url, date, resolution))
                             break

diff --git a/updates/proxy/request.py b/updates/proxy/request.py
@@ -28,7 +28,6 @@ def get_proxy_list(page_count=1):
     pbar = tqdm(total=len(urls), desc="Getting proxy list")
 
     def get_proxy(url):
-        nonlocal open_driver
         proxys = []
         try:
             if open_driver: