Skip to content

Fix up caching, especially fragment_cache #1781

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jun 9, 2016

Conversation

bf4
Copy link
Member

@bf4 bf4 commented Jun 7, 2016

Follows from #1766 and #1684

Benchmark results look good:. Fragment cache went from in #1766
1729 objs/iteration to 1382, and from 716 ips to 767.

It's still not as good as caching or non-caching, but at least we're
seeing some actual performance improvements now.

  • To consider, is step 2 actually slower than step 1 or is the number of ips chosen by the lib just different? need a realtime, i think

Related:

step 1 removing the non caching fragment cache class

bin/bench

Benchmark results:
{
  "commit_hash": "0ef8165",
  "version": "0.10.0",
  "rails_version": "4.2.6",
  "benchmark_run[environment]": "2.2.3p173",
  "runs": [
    {
      "benchmark_type[category]": "caching on: caching serializers: gc off",
      "benchmark_run[result][iterations_per_second]": 1026.37,
      "benchmark_run[result][total_allocated_objects_per_iteration]": 1277
    },
    {
      "benchmark_type[category]": "caching on: fragment caching serializers: gc off",
      "benchmark_run[result][iterations_per_second]": 850.766,
      "benchmark_run[result][total_allocated_objects_per_iteration]": 1442
    },
    {
      "benchmark_type[category]": "caching on: non-caching serializers: gc off",
      "benchmark_run[result][iterations_per_second]": 969.674,
      "benchmark_run[result][total_allocated_objects_per_iteration]": 1243
    },
    {
      "benchmark_type[category]": "caching off: caching serializers: gc off",
      "benchmark_run[result][iterations_per_second]": 880.624,
      "benchmark_run[result][total_allocated_objects_per_iteration]": 1277
    },
    {
      "benchmark_type[category]": "caching off: fragment caching serializers: gc off",
      "benchmark_run[result][iterations_per_second]": 693.928,
      "benchmark_run[result][total_allocated_objects_per_iteration]": 1473
    },
    {
      "benchmark_type[category]": "caching off: non-caching serializers: gc off",
      "benchmark_run[result][iterations_per_second]": 795.081,
      "benchmark_run[result][total_allocated_objects_per_iteration]": 1243
    }
  ]
}

step 2 removing the caching fragment serializer class

Benchmark results:
{
  "commit_hash": "3ed82d7",
  "version": "0.10.0",
  "rails_version": "4.2.6",
  "benchmark_run[environment]": "2.2.3p173",
  "runs": [
    {
      "benchmark_type[category]": "caching on: caching serializers: gc off",
      "benchmark_run[result][iterations_per_second]": 957.072,
      "benchmark_run[result][total_allocated_objects_per_iteration]": 1277
    },
    {
      "benchmark_type[category]": "caching on: fragment caching serializers: gc off",
      "benchmark_run[result][iterations_per_second]": 767.103,
      "benchmark_run[result][total_allocated_objects_per_iteration]": 1382
    },
    {
      "benchmark_type[category]": "caching on: non-caching serializers: gc off",
      "benchmark_run[result][iterations_per_second]": 838.858,
      "benchmark_run[result][total_allocated_objects_per_iteration]": 1243
    },
    {
      "benchmark_type[category]": "caching off: caching serializers: gc off",
      "benchmark_run[result][iterations_per_second]": 772.016,
      "benchmark_run[result][total_allocated_objects_per_iteration]": 1277
    },
    {
      "benchmark_type[category]": "caching off: fragment caching serializers: gc off",
      "benchmark_run[result][iterations_per_second]": 780.055,
      "benchmark_run[result][total_allocated_objects_per_iteration]": 1382
    },
    {
      "benchmark_type[category]": "caching off: non-caching serializers: gc off",
      "benchmark_run[result][iterations_per_second]": 810.992,
      "benchmark_run[result][total_allocated_objects_per_iteration]": 1243
    }
  ]
}

@@ -167,7 +167,7 @@ def serializable_hash(adapter_options = nil, options = {}, adapter_instance = se
adapter_options ||= {}
options[:include_directive] ||= ActiveModel::Serializer.include_directive_from_options(adapter_options)
cached_attributes = adapter_options[:cached_attributes] ||= {}
resource = cached_attributes(options[:fields], cached_attributes, adapter_instance)
resource = fetch_attributes(options[:fields], cached_attributes, adapter_instance)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

otherwise we have a method that takes an argument (a hash) of the same name

@bf4 bf4 force-pushed the serializer_cleanup_6 branch from 0ef8165 to 35a7c81 Compare June 7, 2016 05:52
self.class._cache_options ||= {}
self.class._cache_options[:key] = self.class._cache_key if self.class._cache_key

cached_serializer = _get_or_create_fragment_cached_serializer(serializer_class_name)
cached_serializer = _get_or_create_fragment_cached_serializer
cached_hash = ActiveModelSerializers::SerializableResource.new(
object,
serializer: cached_serializer,
adapter: adapter_instance.class
).serializable_hash
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

first pass at trying to remove the extra class failed, so I left it here

something like

        cached_hash = cached_attributes.fetch(key) do
          self.class.cache_store.fetch(key) do
            attributes(fields, true)
          end
        end

(added cached_attributes to the method params and removed all references to _fragmented since there would be no more special classes)

@bf4
Copy link
Member Author

bf4 commented Jun 7, 2016

It's funny when code coverage drops because there's less code :)
https://travis-ci.org/rails-api/active_model_serializers/jobs/135781269


******************** SimpleCov Results ********************
Group: Libraries
========================================
/home/travis/build/rails-api/active_model_serializers/lib/action_controller/serialization.rb (coverage: 100.0%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model/serializable_resource.rb (coverage: 100.0%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model/serializer.rb (coverage: 100.0%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model/serializer/adapter.rb (coverage: 100.0%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model/serializer/adapter/attributes.rb (coverage: 100.0%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model/serializer/adapter/base.rb (coverage: 88.89%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model/serializer/adapter/json.rb (coverage: 100.0%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model/serializer/adapter/json_api.rb (coverage: 100.0%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model/serializer/adapter/null.rb (coverage: 100.0%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model/serializer/array_serializer.rb (coverage: 100.0%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model/serializer/association.rb (coverage: 100.0%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model/serializer/associations.rb (coverage: 100.0%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model/serializer/attribute.rb (coverage: 100.0%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model/serializer/attributes.rb (coverage: 96.97%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model/serializer/belongs_to_reflection.rb (coverage: 80.0%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model/serializer/caching.rb (coverage: 97.64%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model/serializer/collection_reflection.rb (coverage: 100.0%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model/serializer/collection_serializer.rb (coverage: 100.0%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model/serializer/configuration.rb (coverage: 100.0%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model/serializer/error_serializer.rb (coverage: 100.0%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model/serializer/errors_serializer.rb (coverage: 100.0%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model/serializer/field.rb (coverage: 100.0%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model/serializer/fieldset.rb (coverage: 93.75%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model/serializer/has_many_reflection.rb (coverage: 80.0%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model/serializer/has_one_reflection.rb (coverage: 80.0%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model/serializer/links.rb (coverage: 100.0%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model/serializer/lint.rb (coverage: 100.0%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model/serializer/meta.rb (coverage: 100.0%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model/serializer/null.rb (coverage: 66.67%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model/serializer/reflection.rb (coverage: 100.0%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model/serializer/singular_reflection.rb (coverage: 100.0%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model/serializer/type.rb (coverage: 100.0%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model_serializers.rb (coverage: 100.0%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model_serializers/adapter.rb (coverage: 97.78%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model_serializers/adapter/attributes.rb (coverage: 100.0%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model_serializers/adapter/base.rb (coverage: 100.0%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model_serializers/adapter/json.rb (coverage: 100.0%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model_serializers/adapter/json_api.rb (coverage: 92.04%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model_serializers/adapter/json_api/deserialization.rb (coverage: 94.12%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model_serializers/adapter/json_api/error.rb (coverage: 100.0%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model_serializers/adapter/json_api/jsonapi.rb (coverage: 100.0%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model_serializers/adapter/json_api/link.rb (coverage: 100.0%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model_serializers/adapter/json_api/meta.rb (coverage: 100.0%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model_serializers/adapter/json_api/pagination_links.rb (coverage: 100.0%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model_serializers/adapter/json_api/relationship.rb (coverage: 100.0%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model_serializers/adapter/json_api/resource_identifier.rb (coverage: 100.0%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model_serializers/adapter/null.rb (coverage: 100.0%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model_serializers/callbacks.rb (coverage: 100.0%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model_serializers/deprecate.rb (coverage: 100.0%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model_serializers/deserialization.rb (coverage: 85.71%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model_serializers/json_pointer.rb (coverage: 100.0%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model_serializers/key_transform.rb (coverage: 100.0%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model_serializers/logging.rb (coverage: 100.0%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model_serializers/model.rb (coverage: 91.3%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model_serializers/railtie.rb (coverage: 88.46%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model_serializers/serializable_resource.rb (coverage: 100.0%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model_serializers/serialization_context.rb (coverage: 100.0%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model_serializers/test.rb (coverage: 100.0%)
/home/travis/build/rails-api/active_model_serializers/lib/generators/rails/resource_override.rb (coverage: 100.0%)
/home/travis/build/rails-api/active_model_serializers/lib/generators/rails/serializer_generator.rb (coverage: 100.0%)
/home/travis/build/rails-api/active_model_serializers/lib/grape/active_model_serializers.rb (coverage: 100.0%)
/home/travis/build/rails-api/active_model_serializers/lib/grape/formatters/active_model_serializers.rb (coverage: 100.0%)
/home/travis/build/rails-api/active_model_serializers/lib/grape/helpers/active_model_serializers.rb (coverage: 100.0%)
Group: Long files
========================================
/home/travis/build/rails-api/active_model_serializers/lib/active_model/serializer.rb (coverage: 100.0%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model/serializer/caching.rb (coverage: 97.64%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model/serializer/lint.rb (coverage: 100.0%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model/serializer/reflection.rb (coverage: 100.0%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model_serializers/adapter/json_api.rb (coverage: 92.04%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model_serializers/adapter/json_api/deserialization.rb (coverage: 94.12%)
/home/travis/build/rails-api/active_model_serializers/lib/active_model_serializers/logging.rb (coverage: 100.0%)
Group: Short files
========================================

@@ -208,46 +199,44 @@ def object_cache_key(serializer, adapter_instance)
end
end

def cached_attributes(fields, cached_attributes, adapter_instance)
### INSTANCE METHODS
def fetch_attributes(fields, cached_attributes, adapter_instance)
if self.class.cache_enabled?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we could have klass = self.class here since we use it many times.

Copy link
Member Author

@bf4 bf4 Jun 7, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah. Ultimately, I'd like to move a lot of this into a class that lives outside of the serializer. I started to write that and had a serializer_class attribute

cache_check(adapter_instance) do
attributes(fields)
self.class.cache_store.fetch(key, self.class._cache_options) do
attributes(fields, true)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

passing true since we always want to reload the attributes in a cache block

@bf4 bf4 force-pushed the serializer_cleanup_6 branch from 5e5bd7b to 784799e Compare June 8, 2016 01:00
- on association, fix up assocation logic
- on attribute
@bf4 bf4 force-pushed the serializer_cleanup_6 branch from 784799e to 0f01597 Compare June 8, 2016 01:28
bf4 added a commit to bf4/active_model_serializers that referenced this pull request Jun 8, 2016
@bf4 bf4 force-pushed the serializer_cleanup_6 branch from 0f01597 to b599360 Compare June 8, 2016 01:29
bf4 added a commit to bf4/active_model_serializers that referenced this pull request Jun 8, 2016
@bf4 bf4 merged commit 3d9ab37 into rails-api:master Jun 9, 2016
@bf4 bf4 deleted the serializer_cleanup_6 branch June 9, 2016 04:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants