Full Text Search Support #3904

dplewis · 2017-06-03T05:24:18Z

Adds Full Text Support for mongo and postgres

seriph · 2017-06-03T06:00:28Z

Does this support the textScore from MongoDB? I don't see a test for it or any of the code, though everything else is pretty similar to my PR

dplewis · 2017-06-03T06:02:13Z

Working on a test case now. Unless you want to do it.

flovilmart · 2017-06-03T12:15:59Z

Yeah try to coordinate into a single PR if possible @dplewis, you seem well versed into Postgres, why don't you add support to it on @seriph's PR?

codecov · 2017-06-03T16:13:13Z

Codecov Report

Merging #3904 into master will increase coverage by 0.02%.
The diff coverage is 96.29%.

@@            Coverage Diff             @@
##           master    #3904      +/-   ##
==========================================
+ Coverage   90.26%   90.28%   +0.02%     
==========================================
  Files         114      114              
  Lines        7588     7641      +53     
==========================================
+ Hits         6849     6899      +50     
- Misses        739      742       +3

Impacted Files	Coverage Δ
src/Adapters/Storage/Mongo/MongoStorageAdapter.js	`92.98% <100%> (+0.08%)`	⬆️
src/RestQuery.js	`94.94% <100%> (+0.02%)`	⬆️
src/Adapters/Storage/Mongo/MongoCollection.js	`95.12% <100%> (+0.38%)`	⬆️
src/Controllers/DatabaseController.js	`94.35% <100%> (+0.01%)`	⬆️
src/Adapters/Storage/Mongo/MongoTransform.js	`83.5% <95.23%> (+0.51%)`	⬆️
...dapters/Storage/Postgres/PostgresStorageAdapter.js	`95.7% <95.83%> (ø)`	⬆️
src/RestWrite.js	`93.12% <0%> (-0.2%)`	⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5f991e9...c54865f. Read the comment docs.

dplewis · 2017-06-03T16:17:27Z

@flovilmart I would do that but I don't like how @seriph copy and pasted my PR an hour after I posted it.

He had over a month to do a PR.

Also I announced that I was going to do a PR so something like this wouldn't happen.

flovilmart · 2017-06-03T16:18:56Z

@seriph's code is way old, and that's open source, let's work together to be more efficient, it's not a time for disputes guys.

flovilmart · 2017-06-03T16:40:41Z

You gotta treat people with respect man, the way you're acting and telling @seriph down is not acceptable, not matter how many commits you have in the project. @seriph had a fork with the feature in months ago, and I expect he's now pissed and not likely to contribute to the project again. Read our CODE_OF_CONDUCT. I can not tolerate your behaviour with other contributors.

dplewis · 2017-06-03T16:43:39Z

@flovilmart I understand it won't happen again. @seriph I'll help you get this PR in

flovilmart · 2017-06-03T16:46:09Z

Also, on the code side, $text should not be a special key, but the REST format should be:

{
  "myKey": {
     "$text": {
        "$search": ...
      }
  }
}

seriph · 2017-06-03T16:47:01Z

Haha, to tell you the truth @dplewis I suspected you did the same when I saw you'd posted in the thread where someone had asked me about my fork, then your PR shows up with almost identical code. The textScore code has been live in production on an ecommerce site for months, so surprised you're not thinking it works. Thanks for the professionalism @flovilmart, happy to contribute in the future. I have matching code for the JS API if it helps.

flovilmart · 2017-06-03T16:49:37Z

So we're all good then, if you guys can coordinate, and work together to get the best of that PR, I'll happily create a new full-text-search branch, where you can both merge your PR's before we merge the full-text-search in master. What do you think?

dplewis · 2017-06-03T16:52:53Z

@flovilmart since you can only have 1 Text Index "myKey" doesn't matter.

@seriph I just added a commit for textScore and a test for it. Look at it and you'll see why I thought your textScore didn't work. I didn't see a keyword 'textScore' in any of the docs.

@flovilmart thanks for creating that branch.

dplewis · 2017-06-03T16:57:34Z

@flovilmart do we commit to that branch and move the conversation over to @seriph PR.

seriph · 2017-06-03T17:00:12Z

Happy either way

…

On Sat, Jun 3, 2017 at 10:57 AM, Diamond Lewis ***@***.***> wrote: @flovilmart <https://github.com/flovilmart> do we commit to that branch and move the conversation over to @seriph <https://github.com/seriph> PR. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3904 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABUl43VlTwcs8XH9V7DCSLcrfCeY4GT3ks5sAZCIgaJpZM4Nu-0F> .

flovilmart · 2017-06-03T17:03:00Z

Is that true as well for Postgres? (The single index?)

dplewis · 2017-06-03T17:04:48Z

Postgres doesn't need an index but can only have 1 index

dplewis · 2017-06-03T17:08:49Z

Mongo throws an error if index doesn't exist. I'm writing a test case for it now.

flovilmart · 2017-06-03T17:11:25Z

And on Postgres, there's no need either to specify which key of the table will be looked up for search?

dplewis · 2017-06-03T17:17:50Z

I'm still doing research on it. But we'll know soon enough.

From the looks of it were going to use SELECT [mykey] FROM [className] or SELECT to_tsvector()

dplewis · 2017-06-03T17:34:58Z

I still have 4 test cases to write then we can merge.

Index Exists
Multiple Index Error
Space Delimiter
Phrases

SebC99 · 2017-06-03T18:25:50Z

@dplewis are you sure you need to to change the find function of the mongoStorageAdapter? @seriph's version with an update of mongoCollection seems simpler, but I'm not good enough in JS to judge :)
I just feel adding those 16 lines in mongoStorageAdapter is less clean than the rest of the code :)
My 2 cents

dplewis · 2017-06-03T18:43:37Z

@SebC99 For some reason I couldn't get Mongocollection.find(query...) to handle 2 projections mongoWhere, {{score...}} Which I found to be neccessary after research.

@seriph can you run the test cases against your code? You may have to change the REST format.

dplewis · 2017-06-09T00:56:00Z

@flovilmart I've added support for Postgres. Users have to keep in mind that this will have unexpected consequences if their indexes aren't set up properly

Also is travis not running because of the parse-community:full-text-search branch?

flovilmart · 2017-06-13T14:41:49Z

closing to force travis to pick up the build

flovilmart · 2017-06-13T14:42:27Z

spec/ParseQuery.spec.js

+    }, done.fail);
+  });
+
+  it_exclude_dbs(['postgres'])('fullTextSearch: $caseSensitive', (done) => {


why still excluding PG?

Or can you group together the tests related to mongo (excluding PG) in a sub describe?

There are 3 excludes here. I've added comments in the postgres adapter for $caseSensitive and $diraticSensitive.

No Index... You don't need index to do PG
Case Sensitive... Full text doesn't support case sensitive on PG. You could use regex or make a second column for lower case string.
Diratic Sensitive... Can be automated on a per table / class basis based on setup and index on PG.

Should I replace the comments with error instructions in postgresAdapter or leave it be?

Probably you should Error the query and point to instructions / have a clear way to resolution

flovilmart · 2017-06-13T14:45:05Z

It's looking super great IMHO, @seriph what do you think?

dplewis · 2017-06-13T17:09:38Z

I separated the tests and added the error messages

flovilmart · 2017-06-13T22:11:23Z

Sorry to ask for something more, could we put all the tests in a new file? The ParseQuery.spec.js is already huge :) perhaps ParseQuery.FullTextSearch.spec.js?

flovilmart

This is looking really amazing!!!! Thanks!
Would you be keen to take care of the docs as well?

dplewis · 2017-06-13T23:40:06Z

@flovilmart Sure no problem. Glad I could help. How should we format this?
query.fullTextSearch(key, options)?

@seriph can you work on updating the javascript sdk?

flovilmart · 2017-06-14T01:42:12Z

For now let's start with the REST docs, we'll see the SDK's after right?

dplewis · 2017-06-14T02:15:52Z

Right

srameshr · 2018-01-08T10:06:57Z

Any chance that an API for this will come into the JS SDK?

NicksonYap · 2018-09-10T18:24:41Z

https://docs.parseplatform.org/js/guide/#full-text-search

It seems there is a query.fullText()

but returns empty array

@flovilmart

dplewis added 3 commits June 2, 2017 23:23

Full Text Support

a0af789

invalid input test

a6945d9

Merge branch 'master' into full-text

9b8a1e3

dplewis changed the title ~~Full Text Support for #3747~~ Full Text Support Jun 3, 2017

dplewis changed the title ~~Full Text Support~~ Full Text Search Support Jun 3, 2017

Support for sort

acde6dc

flovilmart changed the base branch from master to full-text-search June 3, 2017 16:51

index exist test

22e58c7

flovilmart closed this Jun 13, 2017

flovilmart reopened this Jun 13, 2017

flovilmart changed the base branch from full-text-search to master June 13, 2017 14:44

flovilmart suggested changes Jun 13, 2017

View reviewed changes

dplewis added 3 commits June 13, 2017 12:02

error instructions for $diacritic and $case sensitivity

5c905f5

nit

638f224

nit

70e4072

dplewis added 2 commits June 13, 2017 14:58

nit

91160b6

Merge branch 'master' into full-text

29007c3

separate test for full text

c54865f

flovilmart approved these changes Jun 13, 2017

View reviewed changes

flovilmart merged commit 8b21d5a into parse-community:master Jun 14, 2017

dplewis deleted the full-text branch June 14, 2017 04:21

dplewis mentioned this pull request Jun 14, 2017

Rest Doc for Full Text Search and withinPolygon parse-community/docs#444

Merged

flovilmart modified the milestone: 2.5.0 Jun 25, 2017

dplewis mentioned this pull request Jul 6, 2017

Full Text Search parse-community/parse-php-sdk#333

Merged

dplewis mentioned this pull request Oct 1, 2017

Full Text Search Query parse-community/Parse-SDK-iOS-OSX#1196

Merged

cbaker6 mentioned this pull request Jan 1, 2022

feat: add score property, withCount, and matchesText options parse-community/Parse-Swift#306

Merged

7 tasks

parseplatformorg mentioned this pull request Jan 3, 2024

refactor: Upgrade mongodb from 4.10.0 to 6.3.0 #8866

Closed

Full Text Search Support #3904

Full Text Search Support #3904

Conversation

dplewis commented Jun 3, 2017 • edited Loading

seriph commented Jun 3, 2017

dplewis commented Jun 3, 2017 • edited Loading

flovilmart commented Jun 3, 2017

codecov bot commented Jun 3, 2017 • edited Loading

Codecov Report

dplewis commented Jun 3, 2017 • edited Loading

flovilmart commented Jun 3, 2017

flovilmart commented Jun 3, 2017

dplewis commented Jun 3, 2017 • edited Loading

flovilmart commented Jun 3, 2017

seriph commented Jun 3, 2017

flovilmart commented Jun 3, 2017 • edited Loading

dplewis commented Jun 3, 2017 • edited Loading

dplewis commented Jun 3, 2017

seriph commented Jun 3, 2017 via email

flovilmart commented Jun 3, 2017

dplewis commented Jun 3, 2017 • edited Loading

dplewis commented Jun 3, 2017

flovilmart commented Jun 3, 2017

dplewis commented Jun 3, 2017 • edited Loading

dplewis commented Jun 3, 2017

SebC99 commented Jun 3, 2017

dplewis commented Jun 3, 2017 • edited Loading

dplewis commented Jun 9, 2017 • edited Loading

flovilmart commented Jun 13, 2017

flovilmart Jun 13, 2017

Choose a reason for hiding this comment

flovilmart Jun 13, 2017

Choose a reason for hiding this comment

dplewis Jun 13, 2017

Choose a reason for hiding this comment

flovilmart Jun 13, 2017

Choose a reason for hiding this comment

flovilmart commented Jun 13, 2017

dplewis commented Jun 13, 2017

flovilmart commented Jun 13, 2017

flovilmart left a comment

Choose a reason for hiding this comment

dplewis commented Jun 13, 2017 • edited Loading

flovilmart commented Jun 14, 2017

dplewis commented Jun 14, 2017

srameshr commented Jan 8, 2018

NicksonYap commented Sep 10, 2018

dplewis commented Jun 3, 2017 •

edited

Loading

dplewis commented Jun 3, 2017 •

edited

Loading

codecov bot commented Jun 3, 2017 •

edited

Loading

dplewis commented Jun 3, 2017 •

edited

Loading

dplewis commented Jun 3, 2017 •

edited

Loading

flovilmart commented Jun 3, 2017 •

edited

Loading

dplewis commented Jun 3, 2017 •

edited

Loading

dplewis commented Jun 3, 2017 •

edited

Loading

dplewis commented Jun 3, 2017 •

edited

Loading

dplewis commented Jun 3, 2017 •

edited

Loading

dplewis commented Jun 9, 2017 •

edited

Loading

dplewis commented Jun 13, 2017 •

edited

Loading