Performance between wiredTiger mongo and mmapv1

[Moumouls]

New Issue Checklist

• I am not disclosing a vulnerability.
• I am not just asking a question.
• I have searched through existing issues.
• I can reproduce the issue with the latest version of Parse Server.

Issue Description

Slow tests on wiredTiger, we need to check why it's so slow on this db engine.

Steps to reproduce

Run npm test with standalone mmap, time 278sec (MBP 16")
Run npm test with standalone wiredTiger time 657.318 (same pc)

Actual Outcome

Slow tests

Expected Outcome

Get better or similar time

Failing Test Case / Pull Request

• 🤩 I submitted a PR with a fix and a test case.
• 🧐 I submitted a PR with a failing test case.

Environment

master

Server

• Parse Server version: FILL_THIS_OUT
• Operating system: FILL_THIS_OUT
• Local or remote host (AWS, Azure, Google Cloud, Heroku, Digital Ocean, etc): FILL_THIS_OUT

Database

• System (MongoDB or Postgres): Mongo
• Database version: 4.0.4 WiredTiger
• Local or remote host (MongoDB Atlas, mLab, AWS, Azure, Google Cloud, etc): FILL_THIS_OUT

Client

• SDK (iOS, Android, JavaScript, PHP, Unity, etc): FILL_THIS_OUT
• SDK version: FILL_THIS_OUT

Logs

// MMAP
2701 specs, 0 failures, 72 pending specs
Finished in 278.188 seconds

// Wiredtiger
2701 specs, 0 failures, 72 pending specs
Finished in 657.318 seconds

[dplewis]

I have noticed this recently also. I haven’t had time to look into it.

[Moumouls]

Yes no problem, i will try to take a look next week ?
@uzaysan, since you have dived deeply into performance testing are you open to help us to resolve this one ? 😃

[uzaysan]

@Moumouls I'm not a professional. But I will try my best. What kind of tests are we talking about?

[Moumouls]

@uzaysan i found a test file where the performance loss is huge and can be a good entry point here to start inspecting.

MONGODB_STORAGE_ENGINE=mmapv1 npm test spec/MongoStorageAdapter.spec.js -> 1.3sec

MONGODB_STORAGE_ENGINE=wiredTiger npm test spec/MongoStorageAdapter.spec.js -> 28.749 seconds

Good news: If the issue is at the Adapter level it will easily fixable i think

And also in the open source world, professional or not, it doesn't matter, contributing and helping is always welcome and will always be a step forward :)

[dplewis]

@Moumouls I reduced the testing time in my branch

https://github.com/parse-community/parse-server/tree/improve-schema-cache

The reason why MongoStorageAdapter.spec.js is slow is because its over deleting the database between tests. The over deleting the entire database between tests also occurs in other test specs files.

I have a few ideas for improving test speeds.

[Moumouls]

@dplewis what time difference do you have between mma and wired ?

[Moumouls]

You are right @dplewis, delete calls on wiredtiger seems to be super slow

I have modified before with

 beforeEach(done => {
    console.time('perf');
    new MongoStorageAdapter({ uri: databaseURI }).deleteAllClasses().then(() => {
      console.timeEnd('perf');
      done();
    }, fail);
  });

MMA log

Started
perf: 8.118ms
.perf: 8.118ms
.perf: 6.268ms
.perf: 6.516ms
.perf: 10.218ms
.perf: 7.256ms
.perf: 10.502ms
.perf: 7.048ms
.perf: 7.569ms
.perf: 6.7ms
.perf: 7.362ms
.perf: 7.585ms
.perf: 6.12ms
.perf: 6.409ms
.perf: 5.209ms
.perf: 5.952ms
.

Wired Log

perf: 389.629ms
.perf: 374.266ms
.perf: 378.7ms
.perf: 381.291ms
.perf: 426.412ms
.perf: 442.377ms
.perf: 415.312ms
.perf: 412.833ms
.perf: 395.817ms
.perf: 437.409ms
.perf: 452.292ms
.perf: 428.49ms
.perf: 430.66ms
.perf: 380.998ms
.perf: 456.02ms
.perf: 431.247ms
.

[Moumouls]

A simple create find seems super slow too

fit('find succeeds when query is within maxTimeMS', done => {
    const maxTimeMS = 250;
    const adapter = new MongoStorageAdapter({
      uri: databaseURI,
      mongoOptions: { maxTimeMS },
    });
    console.time('perf');
    adapter
      .createObject('Foo', { fields: {} }, { objectId: 'abcde' })
      .then(() => adapter._rawFind('Foo', { $where: `sleep(${maxTimeMS / 2})` }))
      .then(
        () => {
          console.timeEnd('perf');
          done();
        },
        err => {
          done.fail(`maxTimeMS should not affect fast queries ${err}`);
        }
      );
  });

MMA : 158.848ms
Wired: 335.838ms

Only testing createObject:
MMA: 4.269ms
Wired: 149.538ms

Only testing rawFind:
MMA: 157.602ms
Wired: 155.504ms

So class create/deletion seems to have issue

[dplewis]

@Moumouls I recently improved the JS SDK test suite and I found that if you use beforeEach in the helper.js and in your specs the tests run slower or they may run out of sync. Can you try changing beforeEach to beforeAll in the helper.js file and run your perf tests?

https://github.com/parse-community/Parse-SDK-JS/pull/1292/files#diff-d20c72e61c31d8c8f138c8d2d0966d5a24a99959227d55edcd105fca781529c0R118

I used your example and got for wired.

perf: 460.846ms
.perf: 6.841ms
.perf: 3.854ms
.perf: 3.453ms
.perf: 54.001ms
.perf: 56.075ms
.perf: 54.925ms
.perf: 55.055ms
.perf: 55.091ms
.perf: 54.957ms
.perf: 55.085ms
.perf: 55.478ms
.perf: 55.626ms
.perf: 6.603ms
.perf: 183.378ms
.perf: 129.419ms

[Moumouls]

Thanks for the investigation @dplewis, I'll try to take a look asap :)