"Cannot parse Firebase url" on snapshot reference access when using RTDB via firebase functions.

[filipesilva]

[REQUIRED] Describe your environment

• Operating System version: Windows 10
• Browser version: Chrome 84
• Firebase SDK version: 7.16.1
• Firebase Product: database

[REQUIRED] Describe the problem

It's currently not possible to access the data snapshot reference when using RTDB via firebase functions.

I believe this is due to the check in https://github.com/firebase/firebase-js-sdk/blob/6af4c27743372ba531e8ce3d046ae2f81e8f5be1/packages/database/src/core/util/libs/parser.ts#L83-L91

That piece of code checks that parsedUrl.domain is localhost, but it seems to be emulator-test-1.localhost instead.

Steps to reproduce:

git clone https://github.com/filipesilva/firebase-emulator-parse-url
cd firebase-emulator-parse-url
yarn
yarn emulators
# open http://localhost:5001/emulator-test-1/us-central1/posts in the browser

You should see the following error log in the console:

i  functions: Beginning execution of "postsPushHandler"
>  [2020-07-22T15:49:53.227Z]  @firebase/database: FIREBASE FATAL ERROR: Cannot parse Firebase url. Please use https://<YOUR FIREBASE>.firebaseio.com
!  functions: Error: FIREBASE FATAL ERROR: Cannot parse Firebase url. Please use https://<YOUR FIREBASE>.firebaseio.com
    at fatal (D:\sandbox\firebase-emulator-parse-url\node_modules\@firebase\database\dist\index.node.cjs.js:341:11)
    at parseRepoInfo (D:\sandbox\firebase-emulator-parse-url\node_modules\@firebase\database\dist\index.node.cjs.js:1296:9)
    at RepoManager.databaseFromApp (D:\sandbox\firebase-emulator-parse-url\node_modules\@firebase\database\dist\index.node.cjs.js:14990:25)
    at Object.initStandalone (D:\sandbox\firebase-emulator-parse-url\node_modules\@firebase\database\dist\index.node.cjs.js:15389:45)
    at DatabaseService.getDatabase (D:\sandbox\firebase-emulator-parse-url\node_modules\firebase-admin\lib\database\database.js:67:23)
    at FirebaseApp.database (D:\sandbox\firebase-emulator-parse-url\node_modules\firebase-admin\lib\firebase-app.js:232:24)
    at DataSnapshot.get ref [as ref] (D:\sandbox\firebase-emulator-parse-url\node_modules\firebase-functions\lib\providers\database.js:293:34)
    at D:\sandbox\firebase-emulator-parse-url\functions\index.js:17:24
    at cloudFunction (D:\sandbox\firebase-emulator-parse-url\node_modules\firebase-functions\lib\cloud-functions.js:132:23)
    at D:\sandbox\firebase-emulator-parse-url\node_modules\firebase-tools\lib\emulator\functionsEmulatorRuntime.js:573:20
!  Your function was killed because it raised an unhandled error.

Relevant Code:

exports.postsPushHandler = functions.database.ref('/posts/{postId}').onCreate(snapshot => {
  console.log(snapshot.ref)
});

[filipesilva]

cc @samtstern (because I've reported other emulator related issues to you)

[samtstern]

@filipesilva what version of firebase-functions are you using? This should be fixed in version 3.8.0 (assuming newest version of the CLI as well)

[filipesilva]

The repro (https://github.com/filipesilva/firebase-emulator-parse-url) is using firebase-functions@3.8.0. Everything there is the latest as of today, I believe.

[samtstern]

@filipesilva sorry I should have clicked into that

[samtstern]

Ok a few things here:

1. I was able to reproduce this
2. If I change the initialization block to just admin.initializeApp() this error goes away.

[samtstern]

It looks like parseRepoInfo() is being called with a URL like:

https://emulator-test-1.localhost

This is a firebase-functions issue, transferring.

[samtstern]

Ok so I added some logging and extractInstanceAndPath is being called like this:

>  extractInstanceAndPath(projects/_/instances/emulator-test-1/refs/posts/-MCvh_klBc9LYP7JIlsJ, localhost)
>  return [https://emulator-test-1.localhost,/posts/-MCvh_klBc9LYP7JIlsJ]

The dataConstructor in onCreate is being given this raw.context:

{
  "eventType": "google.firebase.database.ref.create",
  "params": {
    "postId": "-MCviHJhohdsNE5uAJWZ"
  },
  "domain": "localhost",
  "resource": {
    "service": "firebaseio.com",
    "name": "projects/_/instances/emulator-test-1/refs/posts/-MCviHJhohdsNE5uAJWZ"
  },
  "timestamp": "2020-07-23T12:49:58.150Z",
  "eventId": "5/3OSuz1TP5pWvc/Lt0o9RDcN2M=",
  "authType": "ADMIN"
}

So the port is lost ... gonna have to think about this one.

[samtstern]

@filipesilva question for you: when you're running inside the Functions emulator and you also have the RTDB emulator running, but you do this:

admin.initializeApp({
   databaseURL: "https://emulator-test-1.firebaseio.com/",
   credential: admin.credential.applicationDefault()
});

const db = admin.database()

Do you still expect db to point to the emulator? To me it looks like you're explicitly trying to access production, otherwise you could do admin.initializeApp() to accept the defaults.

So I'm trying to decide if the actual bug is that your posts function writes to the emulators at all.

[filipesilva]

@samtstern I don't quite care about setting credential and would be happy enough if that is just defaulted.

I do care about setting databaseURL because I test a sharded RTDB and it is important that functions I have can be routed to the correct shard, and that the shard is using the emulators. I think this is what happens today because, in my particular work setup, when I save security rules, it will update rules for all 100 shards I use, and they show up in the emulator UI. I could use a URL that's pointing to the emulators by encoding that as a build time variable.

I also care about setting databaseAuthVariableOverride as well because I want my writes from firebase functions to still obey validation rules.

Attempting to set it with no databaseURL (e.g. admin.initializeApp({databaseAuthVariableOverride: "something"}); ) results in a runtime error:

i  functions: Beginning execution of "posts"
!  functions: Error: Can't determine Firebase Database URL.
    at FirebaseDatabaseError.FirebaseError [as constructor] (D:\sandbox\firebase-emulator-parse-url\node_modules\firebase-admin\lib\utils\error.js:43:28)       
    at new FirebaseDatabaseError (D:\sandbox\firebase-emulator-parse-url\node_modules\firebase-admin\lib\utils\error.js:204:23)
    at DatabaseService.ensureUrl (D:\sandbox\firebase-emulator-parse-url\node_modules\firebase-admin\lib\database\database.js:89:15)
    at DatabaseService.getDatabase (D:\sandbox\firebase-emulator-parse-url\node_modules\firebase-admin\lib\database\database.js:56:26)
    at FirebaseApp.database (D:\sandbox\firebase-emulator-parse-url\node_modules\firebase-admin\lib\firebase-app.js:232:24)
    at Proxy.fn (D:\sandbox\firebase-emulator-parse-url\node_modules\firebase-admin\lib\firebase-namespace.js:280:45)
    at D:\sandbox\firebase-emulator-parse-url\functions\index.js:8:20
    at D:\sandbox\firebase-emulator-parse-url\node_modules\firebase-tools\lib\emulator\functionsEmulatorRuntime.js:583:20
    at D:\sandbox\firebase-emulator-parse-url\node_modules\firebase-tools\lib\emulator\functionsEmulatorRuntime.js:558:19
    at Generator.next (<anonymous>)
!  Your function was killed because it raised an unhandled error.

Attempting to set it with databaseURL but no credential also results in a runtime error:

i  functions: Beginning execution of "posts"
!  functions: Error: Only objects are supported for option databaseAuthVariableOverride
    at new Repo (D:\sandbox\firebase-emulator-parse-url\node_modules\@firebase\database\dist\index.node.cjs.js:12513:27)
    at RepoManager.createRepo (D:\sandbox\firebase-emulator-parse-url\node_modules\@firebase\database\dist\index.node.cjs.js:15049:16)
    at RepoManager.databaseFromApp (D:\sandbox\firebase-emulator-parse-url\node_modules\@firebase\database\dist\index.node.cjs.js:15014:25)
    at Object.initStandalone (D:\sandbox\firebase-emulator-parse-url\node_modules\@firebase\database\dist\index.node.cjs.js:15389:45)
    at DatabaseService.getDatabase (D:\sandbox\firebase-emulator-parse-url\node_modules\firebase-admin\lib\database\database.js:67:23)
    at FirebaseApp.database (D:\sandbox\firebase-emulator-parse-url\node_modules\firebase-admin\lib\firebase-app.js:232:24)
    at Proxy.fn (D:\sandbox\firebase-emulator-parse-url\node_modules\firebase-admin\lib\firebase-namespace.js:280:45)
    at D:\sandbox\firebase-emulator-parse-url\functions\index.js:10:20
    at D:\sandbox\firebase-emulator-parse-url\node_modules\firebase-tools\lib\emulator\functionsEmulatorRuntime.js:583:20
    at D:\sandbox\firebase-emulator-parse-url\node_modules\firebase-tools\lib\emulator\functionsEmulatorRuntime.js:558:19
!  Your function was killed because it raised an unhandled error.

So in my concrete work example it does not seem possible to just go with the defaults. I must set credential and databaseURL in order to set databaseAuthVariableOverride. In some cases I must also set databaseURL to access shards.

[samtstern]

@filipesilva thanks for clarifying all of that ... there's a lot going on here and I'll have to think of the best way to solve it.

[filipesilva]

@samtstern heya, did you have time to think about what a resolution for this looks like?