Skip to content

Expose a method to return the memory usage of all dynamic atoms. #294

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -21,11 +21,13 @@ name = "string_cache"
[features]
serde_support = ["serde"]
default = ["serde_support"]
malloc_size_of = ["dep:malloc_size_of", "malloc_size_of_derive"]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use of dep: syntax requires Rust 1.71, so we should bump the rust-version if we're going to do this. I think this should be fine for string_cache? But perhaps worth noting I had to revert to the older feature syntax for crates like euclid that have lower MSRVs.


[dependencies]
precomputed-hash = "0.1"
serde = { version = "1", optional = true }
malloc_size_of = { version = "0.1", default-features = false, optional = true }
malloc_size_of_derive = { version = "0.1", optional = true }
phf_shared = "0.11"
new_debug_unreachable = "1.0.2"
parking_lot = "0.12"
Expand Down
16 changes: 16 additions & 0 deletions src/dynamic_set.rs
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@
// option. This file may not be copied, modified, or distributed
// except according to those terms.

use malloc_size_of_derive::MallocSizeOf;
use malloc_size_of::{MallocSizeOf as _, MallocSizeOfOps, MallocShallowSizeOf};
Comment on lines +10 to +11
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These imports need to be cfgd behind the feature flag

use parking_lot::Mutex;
use std::borrow::Cow;
use std::mem;
Expand All @@ -22,6 +24,20 @@ pub(crate) struct Set {
buckets: Box<[Mutex<Option<Box<Entry>>>]>,
}

impl Set {
#[cfg(feature = "malloc_size_of")]
pub(crate) fn size_of(&self, ops: &mut MallocSizeOfOps) -> usize {
self.buckets.shallow_size_of(ops) +
self.buckets
.iter()
.map(|bucket| {
bucket.lock().as_ref().map_or(0, |entry| entry.size_of(ops))
})
.sum::<usize>()
}
}

#[cfg_attr(feature = "malloc_size_of", derive(MallocSizeOf))]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would personally be tempted to do a manual impl here to avoid the dep on the derive crate (but I suspect my opinion will be a minority one here so this is more of a "noted dissenting opinion" rather than a request for a change).

pub(crate) struct Entry {
pub(crate) string: Box<str>,
pub(crate) hash: u32,
Expand Down
6 changes: 6 additions & 0 deletions src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,12 @@ pub use static_sets::{EmptyStaticAtomSet, PhfStrSet, StaticAtomSet};
/// Use this if you don’t care about static atoms.
pub type DefaultAtom = Atom<EmptyStaticAtomSet>;

/// Measure the memory usage of the set of dynamic atoms.
#[cfg(feature = "malloc_size_of")]
pub fn size_of_dynamic_atoms(ops: &mut malloc_size_of::MallocSizeOfOps) -> usize {
dynamic_set::dynamic_set().size_of(ops)
}

// Some minor tests of internal layout here.
// See ../integration-tests for much more.

Expand Down
Loading