gh-132917: Use /proc/self/status for mem usage info. #133544
Conversation
Using smaps_rollup is quite a lot slower and we can get the similar info from /proc/self/status.
nascheme
added
skip news
🔨 test-with-buildbots
|
🤖 New build scheduled with the buildbot fleet by @nascheme for commit 5b3621d 🤖 Results will be shown at: https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F133544%2Fmerge If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again. |
bedevere-bot
removed
the
🔨 test-with-buildbots
|
Some more precise run time comparison. With the script from GH-132917 running in 10 parallel threads, the I also tested the MacOS version and it takes 4.5 us/per call. The full GC pass is taking roughly 70 ms. |
|
Comparing the info from /proc/self/smaps_rollup vs /proc/self/status. This is on a Linux 6.1.0 kernel with 16 GB of RAM and 16 GB of swap. I ran a Python program that just allocates a bunch of memory in a loop while printing the proc info.
|
There was a problem hiding this comment.
This looks like a more efficient approach and the change is pretty simple... But unless @hugovk wants to reopen the release branch I think it should just go into beta 2. (Also I still have the same concerns I mentioned before, but they're not pressing enough to worry about for b1.)
|
Let's keep this for b2, thanks! |
|
Thanks @nascheme for the PR 🌮🎉.. I'm working now to backport this PR to: 3.14. |
…133544) On Linux, use /proc/self/status for mem usage info. Using smaps_rollup is quite a lot slower and we can get the similar info from /proc/self/status. (cherry picked from commit 751db4e) Co-authored-by: Neil Schemenauer <[email protected]>
|
GH-133718 is a backport of this pull request to the 3.14 branch. |
… (gh-133718) On Linux, use /proc/self/status for mem usage info. Using smaps_rollup is quite a lot slower and we can get the similar info from /proc/self/status. (cherry picked from commit 751db4e) Co-authored-by: Neil Schemenauer <[email protected]>
Using smaps_rollup is slower and we can get the similar info from /proc/self/status. We don't need the extra accuracy that smaps_rollup is giving.
Profiling is showing that reading the
smaps_rollupfile is taking on the order of 30 ms. Readingstatusis much faster. Some background detail on this difference:https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/issues/10966#note_410194443