pylance is too slow when using sklearn. Checking and Analyzing are repeated every time when the file saved ("Long operation: checking:") #3978
Comments
PurpleSand123
changed the title
|
Is this a duplicate of #3908? |
|
can you set this "python.analysis.useLibraryCodeForTypes: false" and see whether it improves performance? |
judej
added
waiting for user response
|
After I set "python.analysis.useLibraryCodeForTypes: false", the performance became normal. |
|
ya, we still need to figure out which part of |
|
see how to generate some local stubs for sklearn https://github.com/microsoft/pyright/blob/main/docs/type-stubs.md#generating-type-stubs |
|
Simpler reproduction:
|
|
@wangtz0607 I don't believe it is the same issue. your case is just parsing/binding taking time when a module is used the first time. once those are done, it is fast second time and after. if that is not the case, please open a new issue with repro steps and logs as the issue creator did. |
|
@erictraut can you take a look? it looks like some of expressions are taking long time to analyze. I modified code a bit to see logs and for some expressions, this log is from this option with some custom logging to get |
|
@heejaechang, can you provide a self-contained minimal repro in text form? |
|
@erictraut the issue creator already provided one. import numpy as np
from sklearn.manifold import TSNE
from sklearn.decomposition import PCA
class sklearn_test(object):
def process(self, reduction_method = 'pca'):
if reduction_method == 'pca' :
self.reduction_method = PCA(n_components=3, random_state=2023)
elif reduction_method == 'tsne' :
self.reduction_method = TSNE(n_components=3, random_state=2023)
value = np.random.random((10,10))
reduced_values = self.reduction_method.fit_transform(value)
return reduced_values
if __name__ == "__main__":
a = sklearn_test()
reduced_values = a.process()
print(reduced_values)
b = dict()
for key, value in b.items():
print("hi")once you have |
|
from the log, I can reduce it to this. from sklearn.manifold import TSNE
method = TSNE(n_components=3, random_state=2023)
reduced_values = method.fit_transform(None) since the log says tsne's fit_transform is the one that shows long operation. |
|
OK, thanks. I'm able to repro the issue. I'll investigate further. |
heejaechang
added
bug
|
I've spent some time looking at this issue. There is nothing that stands out as unexpected other than the fact that Pylance already ships partial stubs for both |
|
Longer term, the right answer is to convince the maintainers of |
|
@gramster has been working on scikit-learn stubs. assigning to Graham to experiment with his stubs. |
|
according to logs from 3 sciki-learn issues. one common source of perf issues is |
|
I'm not seeing this slow performance in pylance v2024.3.1, but maybe i had a newer version of sklearn |
|
Could not repro on 2024.7.1. Closing. |
Environment data
Code Snippet
Repro Steps
Expected behavior
Actual behavior
Logs
The text was updated successfully, but these errors were encountered: