Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add guppy backend #256

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Conversation

dxe4
Copy link

@dxe4 dxe4 commented Nov 2, 2019

related to my comment here #236 (comment)
with implementation of guppy3 we can get the right values.

note this implementation does self.prevlines[-1] == self.prev_lineno: for guppy backend only, because repeating [i for i in range(0, 10000000)] makes the guppy backend extremely slow. although this may not be ideal or acceptable i thought it's worth sharing the code.


@profile
def myfn_py():
    ones = [1] * NBR_ITEMS
    twos = [i + 1 for i in ones]
    return twos


@profile
def myfn_np():
    ones_np = np.ones(NBR_ITEMS)
    twos_np = ones_np+1
    return twos_np


myfn_np()
myfn_py()


python -m memory_profiler --backend guppy prof.py 


Line #    Mem usage    Increment   Line Contents
================================================
    11   14.191 MiB   14.191 MiB   @profile
    12                             def myfn_py():
    13   18.006 MiB    3.815 MiB       ones = [1] * NBR_ITEMS
    14   22.098 MiB    4.091 MiB       twos = [i + 1 for i in ones]
    15                                 return twos


Filename: prof.py

Line #    Mem usage    Increment   Line Contents
================================================
    18   14.189 MiB   14.189 MiB   @profile
    19                             def myfn_np():
    20   18.005 MiB    3.817 MiB       ones_np = np.ones(NBR_ITEMS)
    21   21.820 MiB    3.815 MiB       twos_np = ones_np+1
    22   21.820 MiB    0.000 MiB       return twos_np

Charalampos.Papaloizou added 2 commits November 2, 2019 17:46
if self.backend == 'guppy':
if self.prevlines[-1] == self.prev_lineno:
# the reason for this is because for code like [i for i in range(0, 1000000)] guppy will be extremely slow
_run = False
Copy link
Author

@dxe4 dxe4 Nov 2, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this can cause issues if you want to measure the memory of a side effect eg

[foo() for i in range(0, 500)]
def foo():
     make_big_array = [i for i in random.randint(0, int(1e9))]

i haven't tested it.
although it will give correct results for this case #236 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant