节点&任务状态优化 #7066

ZhuoZhuoCrayon · 2023-09-13T02:35:29Z

No description provided.

hanshuaikang · 2023-10-11T03:06:51Z

gcloud/core/apis/drf/viewsets/taskflow.py

+                project_id=taskflow_instance.project_id,
+            )
+            get_task_status_result: typing.Dict[str, typing.Any] = dispatcher.get_task_status(with_ex_data=False)
+            if get_task_status_result.get("result"):


关于状态是否为 pending_process的逻辑可以直接在这里处理，在这一步就可以筛选出来 pending_process_taskflow_ids

关于状态是否为 pending_process的逻辑可以直接在这里处理，在这一步就可以筛选出来 pending_process_taskflow_ids

不行，get_task_status 相对于 DB 查询来说，是一个耗时操作，需要先 DB 过滤出 RUNNING 的流程，再根据 RUNNING 判断最终状态是「等待处理」的流程，这样走 get_task_status 的任务数量在一个项目下就非常可控。

hanshuaikang · 2023-10-11T03:13:12Z

gcloud/core/apis/drf/viewsets/taskflow.py

-            .values("root_id")
-            .distinct()
+        return self.queryset.exclude(
+            pipeline_instance_id__in=self._fetch_pipeline_instance_ids(statuses=[states.FAILED, states.SUSPENDED])


看我的注释，是未完成的任务 -（暂停 + 失败的任务)，这里改的好像有问题

看我的注释，是未完成的任务 -（暂停 + 失败的任务)，这里改的好像有问题

已确认，没有问题

hanshuaikang · 2023-10-11T03:20:16Z

gcloud/core/apis/drf/viewsets/taskflow.py

+        # selected_related 只能在 objects 最前面加，此处先查处 ID 列表
+        taskflow_instance_ids: typing.List[int] = list(
+            self.queryset.exclude(
+                pipeline_instance_id__in=self._fetch_pipeline_instance_ids(statuses=[states.FAILED])


等待处理应该只存在running状态下的任务，这里可以只拿处于running状态的taskflow_instance

等待处理应该只存在running状态下的任务，这里可以只拿处于running状态的taskflow_instance

逻辑就是这样，exclude

hanshuaikang · 2023-10-11T03:30:31Z

gcloud/taskflow3/domains/dispatchers/task.py

+
+                    # Raise StopIteration if not found
+                    task_id: typing.Optional[int] = next(
+                        (node_output["value"] for node_output in node_outputs if node_output["key"] == "task_id")


这里是取第一个task吗

这里是取第一个task吗

不是，因为输出是 k-v，这里是遍历取第一个名为 task_id 的键

normal-wls · 2023-10-12T07:43:24Z

gcloud/core/apis/drf/serilaziers/template.py

+            return (
+                notify_type
+                if isinstance(notify_type, dict)
+                else {"success": notify_type, "fail": notify_type, "pending_processing": []}


{"success": notify_type, "fail": notify_type, "pending_processing": notify_type}
是不是应该跟上面保持一致

{"success": notify_type, "fail": notify_type, "pending_processing": notify_type} 是不是应该跟上面保持一致

已修改

normal-wls · 2023-10-12T08:26:38Z

gcloud/taskflow3/signals/handlers.py

@@ -189,6 +191,13 @@ def bamboo_engine_eri_post_set_state_handler(sender, node_id, to_state, version,

        _finish_taskflow_and_send_signal(root_id, taskflow_finished, True)

+    elif to_state == bamboo_engine_states.SUSPENDED and node_id == root_id:
+        # TODO 发送通知，向上找到根流程，发送通知


这块是否需要确认完再合入

这块是否需要确认完再合入

已去除，按之前讨论的：独立子流程继承父流程通知配置

normal-wls · 2023-10-12T08:30:17Z

pipeline_plugins/components/collections/sites/open/bk/approve/v1_0.py

@@ -83,6 +85,15 @@ def execute(self, data, parent_data):
            return False

        data.outputs.sn = result["data"]["sn"]
+
+        task_id: int = parent_data.get_one_of_inputs("task_id")
+        send_taskflow_message.delay(


目前发送通知的进程配置较低，需要看看这块加上之后是否需要调整进程资源

目前发送通知的进程配置较低，需要看看这块加上之后是否需要调整进程资源

后续关注

normal-wls · 2023-10-12T08:33:45Z

gcloud/taskflow3/celery/tasks.py

+
+        if skip_if_not_status:
+            # 满足某个具体状态才发通知
+            dispatcher = TaskCommandDispatcher(


个人认为这个地方不应该有查询任务状态的操作，这个函数的功能抽象未发送任务消息，至于发送什么消息和配置，理论上应该在外部确认好传入

个人认为这个地方不应该有查询任务状态的操作，这个函数的功能抽象未发送任务消息，至于发送什么消息和配置，理论上应该在外部确认好传入

讨论结论：不做「等待处理」的整体状态判断，节点走到便发通知。
背景：并行网关场景下一分支有失败节点，另一分支有审批/人工确认场景下，任务状态是失败，如果这种场景发通知是否合理。

normal-wls · 2023-10-12T08:46:33Z

gcloud/core/apis/drf/viewsets/taskflow.py

+            else:
+                return {"id": taskflow_instance.id, "state": None}
+
+        task_status_infos: typing.List[typing.Dict[str, typing.Any]] = concurrent.batch_call(


这块的逻辑可能对 db 有比较大的请求，需要考虑下这块风险

这块的逻辑可能对 db 有比较大的请求，需要考虑下这块风险

先 DB 过滤出 RUNNING 的流程，再根据 RUNNING 判断最终状态是「等待处理」的流程，这样走 get_task_status 的任务数量在一个项目下就非常可控。

…ture/issue_6653_bird # Conflicts: # frontend/desktop/src/assets/fonts/bksops-icon.eot # frontend/desktop/src/assets/fonts/bksops-icon.svg # frontend/desktop/src/assets/fonts/bksops-icon.ttf # frontend/desktop/src/assets/fonts/bksops-icon.woff # frontend/desktop/src/config/i18n/cn.js # frontend/desktop/src/config/i18n/en.js # frontend/desktop/src/pages/task/TaskExecute/TaskOperation.vue # frontend/desktop/src/scss/iconfont.scss # gcloud/core/apis/drf/viewsets/taskflow.py

…ture/issue_6653

* optimization: 任务状态联调 * optimization: 任务状态联调

* fix: 节点详情是否允许操作逻辑优化 # Reviewed, transaction id: 591 * fix: 失败后跳过节点展示报错信息 # Reviewed, transaction id: 592 * fix: 任务列表【等待处理】传参优化 # Reviewed, transaction id: 594 * fix: 失败后跳过展示异常修复 # Reviewed, transaction id: 596 * fix: 节点详情状态的icon对齐节点右上角icon # Reviewed, transaction id: 602

# Reviewed, transaction id: 621

* optimization: 更换icon&&任务详情中画布连线取消hover效果 # Reviewed, transaction id: 624 * optimization: 设置了自动重试的节点，点击 "暂停执行" 后，不给节点后面的线条加样式 # Reviewed, transaction id: 628

# Reviewed, transaction id: 650

# Reviewed, transaction id: 654

# Reviewed, transaction id: 667

feat: 任务状态优化

ZhuoZhuoCrayon requested review from ywywZhou and hanshuaikang and removed request for ywywZhou September 13, 2023 07:00

ZhuoZhuoCrayon linked an issue Sep 26, 2023 that may be closed by this pull request

节点&任务状态优化 #6653

Open

ZhuoZhuoCrayon force-pushed the feature/issue_6653 branch 3 times, most recently from 350768c to 787475e Compare October 11, 2023 01:11

ZhuoZhuoCrayon changed the base branch from release_horse to release_humming_bird October 11, 2023 02:12

ZhuoZhuoCrayon requested a review from normal-wls October 11, 2023 02:12

ZhuoZhuoCrayon self-assigned this Oct 11, 2023

ZhuoZhuoCrayon changed the title ~~执行中任务状态展示及过滤优化~~ 节点&任务状态优化 Oct 11, 2023

hanshuaikang reviewed Oct 11, 2023

View reviewed changes

normal-wls requested changes Oct 12, 2023

View reviewed changes

ZhuoZhuoCrayon force-pushed the feature/issue_6653 branch from 6937a9e to 2e0a59e Compare October 18, 2023 09:19

ZhuoZhuoCrayon added 5 commits October 19, 2023 19:27

feature: 执行中任务状态展示及过滤优化

5ed420e

feature: 等待处理通知

1854e11

minor: unit test fix

b2d94ac

minor: review fix

e1817da

minor: review fix

b0cdb91

ZhuoZhuoCrayon force-pushed the feature/issue_6653 branch from 2e0a59e to b0cdb91 Compare October 19, 2023 11:28

ywywZhou and others added 3 commits October 30, 2023 10:50

optimization: 节点&任务状态优化

2a27b2e

optimization: 删除暂停标签逻辑调整

d7e15a8

feature: 等待继续状态展示 & 任务实例补充元数据

87b798e

ZhuoZhuoCrayon closed this Nov 7, 2023

ZhuoZhuoCrayon reopened this Nov 7, 2023

ZhuoZhuoCrayon closed this Nov 7, 2023

ZhuoZhuoCrayon reopened this Nov 7, 2023

ZhuoZhuoCrayon and others added 3 commits November 7, 2023 12:07

optimization: 任务状态联调

c656e73

feature: 等待继续状态收敛为等待处理 & batch_status 接口支持返回节点自动重试信息

29c182d

ZhuoZhuoCrayon and others added 20 commits November 22, 2023 09:39

Merge remote-tracking branch 'upstream/release_humming_bird' into fea…

9eca538

…ture/issue_6653

feature: 优化子流程存在节点自动重试的场景

34670d0

任务状态联调 (#7198)

7cecc9a

* optimization: 任务状态联调 * optimization: 任务状态联调

optimization: 任务状态测试问题修复

e0312da

bugfix: 修复节点状态联调问题

5a4d9ab

bugfix: 任务状态体验问题修复

4fe6479

optimization: 任务状态联调问题修复

10edaa1

bugfix: 修改子流程暂停时搜索暂停状态会搜索到等待处理的问题

299a96c

bugfix: 等待处理状态允许修改任务参数

35ae20c

minor: 任务列表等待处理状态icon调整

ba2f215

bugfix: 修复节点暂停状态查询速度过慢的问题

74364f8

optimization: 任务状态体验问题修复

7257a64

# Reviewed, transaction id: 621

optimization: 添加【节点等待任务继续】角标

1a87d8c

# Reviewed, transaction id: 650

minor: 修改节点操作【确认】按钮文案

5bb8ea9

# Reviewed, transaction id: 654

bugfix: 任务暂停时进行节点操作后取消线上暂停样式&&再次执行添加hover

4b813df

# Reviewed, transaction id: 667

minor: merge hb

61cd6df

minor: merge task sttus

66f6c3d

Merge pull request #7254 from hanshuaikang/merged_hb

a3f2385

feat: 任务状态优化

hanshuaikang merged commit 8cd814e into release_humming_bird Dec 20, 2023
5 of 6 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

节点&任务状态优化 #7066

节点&任务状态优化 #7066

ZhuoZhuoCrayon commented Sep 13, 2023

hanshuaikang Oct 11, 2023

ZhuoZhuoCrayon Oct 12, 2023

hanshuaikang Oct 11, 2023

ZhuoZhuoCrayon Oct 12, 2023

hanshuaikang Oct 11, 2023

ZhuoZhuoCrayon Oct 12, 2023

hanshuaikang Oct 11, 2023

ZhuoZhuoCrayon Oct 12, 2023

normal-wls Oct 12, 2023

ZhuoZhuoCrayon Oct 12, 2023

normal-wls Oct 12, 2023

ZhuoZhuoCrayon Oct 12, 2023

normal-wls Oct 12, 2023

ZhuoZhuoCrayon Oct 12, 2023

normal-wls Oct 12, 2023

ZhuoZhuoCrayon Oct 12, 2023

normal-wls Oct 12, 2023

ZhuoZhuoCrayon Oct 12, 2023

节点&任务状态优化 #7066

节点&任务状态优化 #7066

Conversation

ZhuoZhuoCrayon commented Sep 13, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment