Skip to content

Speedup import time with lazy import of scipy.stats #1268

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Mar 11, 2025

Conversation

ricardoV94
Copy link
Member

@ricardoV94 ricardoV94 commented Mar 5, 2025

This reduces pytensor import time from average 0.9 to 0.45s on my machine

Before:

python -m benchmark_imports pytensor
0.8324 root       pytensor                                
0.3692 project    pytensor.tensor.random                  
0.3689 project    pytensor.tensor.random.rewriting        
0.3649 project    pytensor.tensor.random.rewriting.jax    
0.3640 project    pytensor.tensor.random.basic            
0.3616 dependency scipy.stats                             
0.2857 transitive scipy.stats._stats_py                    from scipy.stats
0.2108 project    pytensor.scalar                         
0.1785 project    pytensor.scalar.math                    
0.1615 dependency scipy.special                           
0.1484 transitive scipy.stats.distributions                from scipy.stats._stats_py
0.1056 project    pytensor.configdefaults                 
0.1020 transitive scipy.stats._continuous_distns           from scipy.stats.distributions
0.0888 dependency numpy                                   
0.0794 transitive scipy.special._support_alternative_backends from scipy.special
0.0791 transitive scipy._lib._array_api                    from scipy.special._support_alternative_backends
0.0767 transitive scipy._lib.array_api_compat.numpy        from scipy._lib._array_api
0.0760 project    pytensor.tensor                         
0.0711 transitive scipy.optimize                           from scipy.stats._stats_py
0.0656 transitive scipy.special._orthogonal                from scipy.special
0.0649 transitive scipy.linalg                             from scipy.special._orthogonal
0.0645 project    pytensor.tensor.rewriting               
0.0477 transitive scipy.linalg._sketches                   from scipy.linalg
0.0474 transitive numpy.f2py                               from scipy._lib.array_api_compat.numpy
0.0470 transitive numpy.f2py.f2py2e                        from numpy.f2py

After:

python -m benchmark_imports pytensor
0.4396 root       pytensor                                
0.1776 project    pytensor.scalar                         
0.1490 project    pytensor.scalar.math                    
0.1348 dependency scipy.special                           
0.0836 project    pytensor.configdefaults                 
0.0766 project    pytensor.tensor                         
0.0733 transitive scipy.special._support_alternative_backends from scipy.special
0.0730 transitive scipy._lib._array_api                    from scipy.special._support_alternative_backends
0.0708 transitive scipy._lib.array_api_compat.numpy        from scipy._lib._array_api
0.0670 project    pytensor.tensor.rewriting               
0.0647 dependency numpy                                   
0.0466 transitive scipy.special._orthogonal                from scipy.special
0.0460 transitive scipy.linalg                             from scipy.special._orthogonal
0.0417 transitive numpy.f2py                               from scipy._lib.array_api_compat.numpy
0.0413 transitive numpy.f2py.f2py2e                        from numpy.f2py
0.0312 transitive scipy.linalg._sketches                   from scipy.linalg
0.0305 transitive scipy.sparse                             from scipy.linalg._sketches
0.0302 transitive numpy.__config__                         from numpy
0.0297 transitive numpy._core                              from numpy.__config__
0.0290 transitive numpy.f2py.crackfortran                  from numpy.f2py.f2py2e
0.0282 project    pytensor.scalar.basic                   
0.0262 project    pytensor.tensor.rewriting.basic         
0.0244 project    pytensor.tensor.random                  
0.0242 transitive numpy.lib                                from numpy
0.0241 project    pytensor.tensor.random.rewriting        

📚 Documentation preview 📚: https://pytensor--1268.org.readthedocs.build/en/1268/

Copy link

codecov bot commented Mar 5, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 81.99%. Comparing base (bf628c9) to head (5df5bdc).
Report is 6 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##             main    #1268   +/-   ##
=======================================
  Coverage   81.99%   81.99%           
=======================================
  Files         188      188           
  Lines       48608    48610    +2     
  Branches     8688     8689    +1     
=======================================
+ Hits        39857    39859    +2     
  Misses       6586     6586           
  Partials     2165     2165           
Files with missing lines Coverage Δ
pytensor/tensor/random/basic.py 98.86% <100.00%> (+<0.01%) ⬆️
🚀 New features to boost your workflow:
  • Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@jessegrabowski jessegrabowski merged commit b3da2a4 into pymc-devs:main Mar 11, 2025
74 checks passed
@ricardoV94 ricardoV94 deleted the lazy_import_stats branch March 11, 2025 16:19
@ricardoV94 ricardoV94 changed the title Speedup pytensor import time with lazy import of scipy.stats Speedup import time with lazy import of scipy.stats Mar 18, 2025
@ricardoV94 ricardoV94 changed the title Speedup import time with lazy import of scipy.stats Speedup import time with lazy import of scipy.stats Mar 18, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants