Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Performance telemetry API #2490

Draft
wants to merge 3 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions tcms/settings/common.py
Original file line number Diff line number Diff line change
Expand Up @@ -326,6 +326,12 @@
(_("TestCase health"), reverse_lazy("test-case-health")),
],
),
(
_("Management"),
[
(_("Performance"), reverse_lazy("management-performance")),
],
),
(
"More coming soon",
"http://kiwitcms.org/blog/kiwi-tcms-team/2019/03/03/legacy-reports-become-telemetry/",
Expand Down
37 changes: 37 additions & 0 deletions tcms/telemetry/api.py
Original file line number Diff line number Diff line change
Expand Up @@ -247,3 +247,40 @@ def _get_count_for(test_executions):
.annotate(count=Count("case_id"))
.order_by("case_id")
)


@http_basic_auth_login_required
@rpc_method(name="Management.performance")
def performance_telemetry(query=None):

if query is None:
query = {}

executions = (
TestExecution.objects.filter(**query)
# count only execution which are finished
.exclude(status__weight=0)
.order_by("run_id")
.values("assignee_id", "run_id", "assignee__username")
)

result = {}
run_id = 0
for execution in executions:
if run_id == 0:
run_id = execution["run_id"]

assignee_count = {}
assignee = execution["assignee__username"] or ""
if assignee in assignee_count:
assignee_count[assignee] = assignee_count[assignee] + 1
else:
assignee_count[assignee] = 1

if execution["run_id"] != run_id:
result[run_id] = assignee_count
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can either be "TR-123" directly here or adjust the JS code to render the coordinates on the X axis as TR-123.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done. I changed it on the UI level.


assignee_count = {}
run_id = execution["run_id"]

return result
134 changes: 134 additions & 0 deletions tcms/telemetry/static/telemetry/js/management/performance.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
$(document).ready(() => {
$('.selectpicker').selectpicker()
$('[data-toggle="tooltip"]').tooltip()

loadInitialProduct(reloadCharts)

$('#id_after').on('dp.change', reloadCharts)
$('#id_before').on('dp.change', reloadCharts)
document.getElementById('id_test_plan').onchange = reloadCharts
document.getElementById('id_product').onchange = () => {
updateTestPlanSelectFromProduct(reloadCharts)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When changing the product I get error

Internal error: Cannot resolve keyword 'category' into field. Choices are: assignee, assignee_id, bugs, build, build_id, case, case_id, case_text_version, id, linkreference, run, run_id, sortkey, start_date, status, status_id, stop_date, tested_by, tested_by_id

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed.

}
})

function reloadCharts () {
const query = {}

const testPlanIds = $('#id_test_plan').val()
const productIds = $('#id_product').val()

if (testPlanIds.length) {
query.plan__in = testPlanIds
} else if (productIds.length) {
query.case__category__product_id__in = productIds
}

const dateBefore = $('#id_before')
if (dateBefore.val()) {
query.create_date__lte = dateBefore.data('DateTimePicker').date().format('YYYY-MM-DD 23:59:59')
}

const dateAfter = $('#id_after')
if (dateAfter.val()) {
query.create_date__gte = dateAfter.data('DateTimePicker').date().format('YYYY-MM-DD 00:00:00')
}

jsonRPC('Management.performance', query, result => {
console.log(result)

// the actual result is in the same format, only it can be much bigger
// and the chart may break
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to have an understanding of why & how this chart breaks. If you are seeing breakage locally in devel, that's not a terribly large dataset.

Both Status matrix & Execution trends pages seem to have a larger dataset, at least in production and they seem to load just fine. On tcms.kiwitcms.org Execution trends takes a few seconds but it contains info from 5564 executions across around 1000 test runs grouped by statuses.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I meant this comment for here - #2490 (comment)

// const r = {
// 1: {
// "asankov": 1,
// "atodorov": 2
// },
// 2: {
// "asankov": 1,
// "atodorov": 2
// },
// 3: {
// "asankov": 1,
// "atodorov": 1,
// "": 1
// },
// 4: {
// "asankov": 1,
// "atodorov": 1
// },
// 5: {
// "asankov": 5,
// "atodorov": 2,
// "bot": 1
// }
// }

drawChart(result)
}, true)
}

function drawChart (data) {
// the X axis of the chart - run IDs
const groupedCategories = []
// map of username -> table column. we use map here for faster lookup by username.
const groupedColumnsDataMap = {}
const usernames = new Set()

// collect all the testers so that we know how much columns we will have
Object.entries(data).forEach(([_testRunId, asigneeCount]) => {
Object.entries(asigneeCount).forEach(([username, _executionCount]) => {
// filter empty users
// TODO: maybe we can do that on the API level
if (username) {
usernames.add(username)
}
})
})

usernames.forEach(username => (groupedColumnsDataMap[username] = [username]))

Object.entries(data).forEach(([testRunId, _asigneeCount]) => {
groupedCategories.push(`TR-${testRunId}`)

const asigneesCount = data[testRunId]

// for each user in the groupedColumnsDataMap check if that user
// is assigned any executions for this run.
Object.entries(groupedColumnsDataMap).forEach(([username, data]) => {
const count = asigneesCount[username]
if (count) {
data.push(count)
} else {
// TODO: find a way to hide the 0 valued-columns
data.push(0)
}
})
})

// C3 does not need a map, but an array of values
// get rid of the keys, because we do not need them anymore
const groupedColumnsData = Object.values(groupedColumnsDataMap)

const chartConfig = $().c3ChartDefaults().getDefaultGroupedBarConfig()
chartConfig.bindto = '#performance-chart'
chartConfig.axis = {
x: {
categories: groupedCategories,
type: 'category'
},
y: {
tick: {
format: showOnlyRoundNumbers
}
}
}
chartConfig.data = {
type: 'bar',
columns: groupedColumnsData
}
chartConfig.zoom = {
enabled: true
}
c3.generate(chartConfig)
}
3 changes: 3 additions & 0 deletions tcms/telemetry/static/telemetry/js/testing/breakdown.js
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,9 @@ function drawChart(data, type, selector) {
chartData.push(dataEntry);
});

console.log('data', data)
console.log('groups', groups)
console.log('chartData', chartData)
let chartConfig = $().c3ChartDefaults().getDefaultStackedBarConfig();
chartConfig.bindto = selector;
chartConfig.axis = {
Expand Down
28 changes: 28 additions & 0 deletions tcms/telemetry/templates/telemetry/management/performance.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
{% extends "base.html" %}
{% load i18n %}
{% load static %}

{% block title %}{% trans "Performance" %}{% endblock %}

{% block contents %}

<div class="container-fluid container-cards-pf">
{% include "telemetry/include/filters.html" %}

<div class="col-md-12" style="text-align: center">
<div style="text-align: center; font-weight: bold">{% trans "Performance" %}</div>
<div id="performance-chart"></div>
</div>
</div>

<script src="{% static 'c3/c3.min.js' %}"></script>
<script src="{% static 'd3/d3.min.js' %}"></script>

<script src="{% static 'js/utils.js' %}"></script>
<script src="{% static 'js/jsonrpc.js' %}"></script>

<script src="{% static 'telemetry/js/testing/utils.js' %}"></script>
<script src="{% static 'telemetry/js/management/performance.js' %}"></script>

<!-- <link rel="stylesheet" type="text/css" href="{% static 'telemetry/css/testing/test-case-health.css' %}"/> -->
{% endblock %}
5 changes: 5 additions & 0 deletions tcms/telemetry/urls.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,4 +23,9 @@
views.TestingTestCaseHealth.as_view(),
name="test-case-health",
),
re_path(
r"^management/performance/$",
views.ManagementPerformance.as_view(),
name="management-performance",
),
]
7 changes: 7 additions & 0 deletions tcms/telemetry/views.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,3 +33,10 @@ class TestingExecutionTrendsView(TemplateView):
class TestingTestCaseHealth(TemplateView):

template_name = "telemetry/testing/test-case-health.html"

@method_decorator(
login_required, name="dispatch"
) # pylint: disable=missing-permission-required
class ManagementPerformance(TemplateView):

template_name = "telemetry/management/performance.html"