has_totals not working with querysets

[oberix]

As highlighted in issue #6

Negative indexing is not supported is raised when using attribute has_totals with an aggregate returning a queryset.

[SoulRaven]

a quick fix for this is:
def _split_totals(self, results): if self.has_totals and (len(results) > 0): if pnd and isinstance(results, DataFrame): self._results = results.iloc[:len(results.iloc) - 1] self._totals = results.iloc[len(results.iloc) - 1] else: self._results = results[:len(results) -1 ] self._totals = results[len(results) -1] else: self._results = results self._totals = {}

but after this another error is raised, "'list' object has no attribute 'order_by'"
https://dpaste.de/u1Yq

and this error is because my data is a list of dict not a qs, and the app it things is a queryset

[oberix]

This should be fixed in master.

The tricky part is that you probably are returning a ValuesQuerySet which __repr__ itself as a list of dictionaries (and mostly behave like one) but is actually a subclass of QuerySet, so the code failed to identify and treat it correctly.

[SoulRaven]

so, how we can fix this? because still not working right

[SoulRaven]

After the last updates from the git, now the "has_totals" is not throw any errors, but the last field is the last field from the results,not the totals row

[oberix]

That's odd, it's working for me. Can you check what data type you are actually returning in your aggregate?

[oberix]

I mean check it with type() not looking at the output...

[oberix]

Ok sorry, I answered before reading your last post.

After the last updates from the git, now the "has_totals" is not throw any errors, but the last field is the last field from the results,not the totals row

That's the way it works. You have to take care that the total row is the last one. At the moment I don't see other simple way to achieve this.

Calculating totals is not always plain, think for example if you have a column of percentage (%), you can't simply sum them.

[SoulRaven]

But why you don't create a new row and SUM all the columns?
Normally you have all the raw columns data, and only needs to make a sum with all the rows from each column

Example: in my case, each row is a different record with a different pk

https://s21.postimg.org/sa0rh7ajr/Screenshot_2016_10_13_21_44_24.png

[oberix]

Actually this is a very simple case you are describing, but yes, I think I may add a new attribute for the Report class to automatically add a SUM row to the result of aggregate.

This would be a little bit arbitrary in my opinion, should it SUM any numeric value? What if the numbers are database IDs or codes of any other kind? That's up to the programmer to know

[SoulRaven]

I quess you can make something similar with the formatting attribute and the user select what columns to make sums or avg or diff

[SoulRaven]

have you have any success with this feature?

[oberix]

This is not as simple as formatting as it does not apply per value and thus can not be executed at rendering time.

I implemented a basic idea which works only in "list of dict" case, queryset and DataFrames are faster but will have to think about it a little bit more. Will push this on a separate branch as soon as I can and let you know, would be great if you can test it.

[SoulRaven]

i will test it first hand, now i am try to add a custom menu to the admin menus and append the report to that menu

[oberix]

Pushed 'eval_totals' branch.

You can define the attribute auto_totals as a dict of field_names: function to calculate totals, for the fields not present in these dict an empty string will be used. has_totals has to be true to show the totals row.

As said, at the moment it works only with list of dicts as return value of aggregate() (which includes ValuesQuerySet as well).

[SoulRaven]

for 1.9 is not working, because ValuesQuerySet has been removed
https://docs.djangoproject.com/en/1.9/releases/1.9/#miscellaneous

[oberix]

You are right, e8277bf should fix it.

[SoulRaven]

also there is a problem, when fields are methods name, auto_totals is not working, also if the columns contains 2 or more fields, is impossible to make total formating
My suggestion is to make somehow content aware if the columns are method name or not, and if so to return full record as argument for that method