Using restart=false with webdriverio backend leaves browser instances running

[hubidu]

What are you trying to achieve?

I am setting restart to false in order to speed up overall test execution by not restarting browsers between tests.

What do you get instead?

Browsers are not restarted between tests but after the test run the used browser instance is not terminated. There seems to be even a new one with a blank page to be opened.

Details

node_modules\.bin\codeceptjs run --debug --verbose
CodeceptJS v1.2.0
Using test root "C:\Users\stefan.huber\projects\e2e-dashboard-client\examples\codeceptjs"

Failing test scenarios --
[1] Starting recording promises
Emitted | suite.before ([object Object])

[Session] Starting singleton browser session
The test will fail when an element on the page can not be found
Emitted | test.before ([object Object])
Emitted | test.start ([object Object])
Emitted | step.before (I am on page "http://www.check24.de")
Emitted | step.after (I am on page "http://www.check24.de")
Emitted | step.before (I click "#this-selector-does-not-exist")
Emitted | step.after (I click "#this-selector-does-not-exist")
Emitted | step.start (I am on page "http://www.check24.de")

• I am on page "http://www.check24.de"
  Emitted | step.passed (I am on page "http://www.check24.de")
  Step finished in 1.347 sec
  Emitted | step.finish (I am on page "http://www.check24.de")
  Emitted | step.start (I click "#this-selector-does-not-exist")
• I click "#this-selector-does-not-exist"

[SmartWait (2000ms)] Locating #this-selector-does-not-exist in 2000
[1] Error | Error: Clickable element #this-selector-does-not-exist was not found by text|CSS|XPath
Emitted | step.failed (I click "#this-selector-does-not-exist")
Step finished in 2.069 sec
Emitted | step.finish (I click "#this-selector-does-not-exist")
[1] Error | Error: Clickable element #this-selector-does-not-exist was not found by text|CSS|XPath
[1] Starting session
Emitted | test.failed ([object Object])
Emitted | test.finish ([object Object])
[1] Stopping recording promises
Screenshot has been saved to C:\Users\stefan.huber\projects\e2e-dashboard-client\examples\codeceptjs__out\The_test_will_fail_when_an_element_on_the_page_can_not_be_found.failed.png
× FAILED in 3855ms

[2] Starting recording promises
Emitted | test.after ([object Object])

[Session] cleaning cookies and localStorage
Emitted | suite.after ([object Object])
Login Scenarios --
Emitted | suite.before ([object Object])
When I login with an invalid username and password then I will see an error message
Emitted | test.before ([object Object])
Emitted | test.start ([object Object])
Emitted | step.before (I am on page "http://www.check24.de")
Emitted | step.after (I am on page "http://www.check24.de")
Emitted | step.before (I click "#c24-meinkonto")
Emitted | step.after (I click "#c24-meinkonto")
Emitted | step.before (I click "Anmelden")
Emitted | step.after (I click "Anmelden")
Emitted | step.before (I fill field "#email", "Foo")
Emitted | step.after (I fill field "#email", "Foo")
Emitted | step.before (I fill field "#password", "Bar")
Emitted | step.after (I fill field "#password", "Bar")
Emitted | step.before (I click "#c24-kb-register-btn")
Emitted | step.after (I click "#c24-kb-register-btn")
Emitted | step.before (I see "Bitte geben Sie", ".error-desc")
Emitted | step.after (I see "Bitte geben Sie", ".error-desc")
Emitted | step.start (I am on page "http://www.check24.de")

• I am on page "http://www.check24.de"
  Emitted | step.passed (I am on page "http://www.check24.de")
  Step finished in 0.788 sec
  Emitted | step.finish (I am on page "http://www.check24.de")
  Emitted | step.start (I click "#c24-meinkonto")
• I click "#c24-meinkonto"

[SmartWait (2000ms)] Locating #c24-meinkonto in 2000
Emitted | step.passed (I click "#c24-meinkonto")
Step finished in 0.074 sec
Emitted | step.finish (I click "#c24-meinkonto")
Emitted | step.start (I click "Anmelden")

• I click "Anmelden"
  Emitted | step.passed (I click "Anmelden")
  Step finished in 0.336 sec
  Emitted | step.finish (I click "Anmelden")
  Emitted | step.start (I fill field "#email", "Foo")
• I fill field "#email", "Foo"

[SmartWait (2000ms)] Locating #email in 2000
Emitted | step.passed (I fill field "#email", "Foo")
Step finished in 0.11 sec
Emitted | step.finish (I fill field "#email", "Foo")
Emitted | step.start (I fill field "#password", "Bar")

• I fill field "#password", "Bar"

[SmartWait (2000ms)] Locating #password in 2000
Emitted | step.passed (I fill field "#password", "Bar")
Step finished in 0.074 sec
Emitted | step.finish (I fill field "#password", "Bar")
Emitted | step.start (I click "#c24-kb-register-btn")

• I click "#c24-kb-register-btn"

[SmartWait (2000ms)] Locating #c24-kb-register-btn in 2000
Emitted | step.passed (I click "#c24-kb-register-btn")
Step finished in 0.079 sec
Emitted | step.finish (I click "#c24-kb-register-btn")
Emitted | step.start (I see "Bitte geben Sie", ".error-desc")

• I see "Bitte geben Sie", ".error-desc"

[SmartWait (2000ms)] Locating .error-desc in 2000
Emitted | step.passed (I see "Bitte geben Sie", ".error-desc")
Step finished in 0.479 sec
Emitted | step.finish (I see "Bitte geben Sie", ".error-desc")
Emitted | test.passed ([object Object])
Emitted | test.finish ([object Object])
√ OK in 1959ms

Emitted | test.after ([object Object])

[Session] cleaning cookies and localStorage
Emitted | suite.after ([object Object])
Search for Handytarife --
Emitted | suite.before ([object Object])
When I search for "Handytarife" without specifying any details Then I will get a list of various tariffs
Emitted | test.before ([object Object])
Emitted | test.start ([object Object])
Emitted | step.before (I am on page "https://www.check24.de/handytarife")
Emitted | step.after (I am on page "https://www.check24.de/handytarife")
Emitted | step.before (I click "jetzt vergleichen", "button")
Emitted | step.after (I click "jetzt vergleichen", "button")
Emitted | step.before (I wait in url "/handytarife/vergleich")
Emitted | step.after (I wait in url "/handytarife/vergleich")
Emitted | step.before (I see "Handytarife im Vergleich", "h1")
Emitted | step.after (I see "Handytarife im Vergleich", "h1")
Emitted | step.before (I see element "//filter")
Emitted | step.after (I see element "//filter")
Emitted | step.before (I see element "product-item-anniversary")
Emitted | step.after (I see element "product-item-anniversary")
Emitted | step.start (I am on page "https://www.check24.de/handytarife")

• I am on page "https://www.check24.de/handytarife"
  Emitted | step.passed (I am on page "https://www.check24.de/handytarife")
  Step finished in 0.695 sec
  Emitted | step.finish (I am on page "https://www.check24.de/handytarife")
  Emitted | step.start (I click "jetzt vergleichen", "button")
• I click "jetzt vergleichen", "button"

[SmartWait (2000ms)] Locating button in 2000
Emitted | step.passed (I click "jetzt vergleichen", "button")
Step finished in 4.22 sec
Emitted | step.finish (I click "jetzt vergleichen", "button")
Emitted | step.start (I wait in url "/handytarife/vergleich")

• I wait in url "/handytarife/vergleich"
  Emitted | step.passed (I wait in url "/handytarife/vergleich")
  Step finished in 0.779 sec
  Emitted | step.finish (I wait in url "/handytarife/vergleich")
  Emitted | step.start (I see "Handytarife im Vergleich", "h1")
• I see "Handytarife im Vergleich", "h1"

[SmartWait (2000ms)] Locating h1 in 2000
Emitted | step.passed (I see "Handytarife im Vergleich", "h1")
Step finished in 0.632 sec
Emitted | step.finish (I see "Handytarife im Vergleich", "h1")
Emitted | step.start (I see element "//filter")

• I see element "//filter"

[SmartWait (2000ms)] Locating //filter in 2000
Emitted | step.passed (I see element "//filter")
Step finished in 0.181 sec
Emitted | step.finish (I see element "//filter")
Emitted | step.start (I see element "product-item-anniversary")

• I see element "product-item-anniversary"

[SmartWait (2000ms)] Locating product-item-anniversary in 2000
Emitted | step.passed (I see element "product-item-anniversary")
Step finished in 0.52 sec
Emitted | step.finish (I see element "product-item-anniversary")
Emitted | test.passed ([object Object])
Emitted | test.finish ([object Object])
√ OK in 7044ms

Emitted | test.after ([object Object])

[Session] cleaning cookies and localStorage
Emitted | suite.after ([object Object])

-- FAILURES:

1. Failing test scenarios
   The test will fail when an element on the page can not be found:
   Clickable element #this-selector-does-not-exist was not found by text|CSS|XPath

Scenario Steps:

• I.click("#this-selector-does-not-exist") at Test.Scenario (failing_test.js:6:7)
• I.amOnPage("http://www.check24.de") at Test.Scenario (failing_test.js:5:7)

Error: Clickable element #this-selector-does-not-exist was not found by text|CSS|XPath
at new ElementNotFound (node_modules\codeceptjs\lib\helper\errors\ElementNotFound.js:14:11)
at assertElementExists (node_modules\codeceptjs\lib\helper\WebDriverIO.js:2096:11)
at WebDriverIO.click (node_modules\codeceptjs\lib\helper\WebDriverIO.js:553:7)
at
at process._tickDomainCallback (internal/process/next_tick.js:228:7)

FAIL | 2 passed, 1 failed // 16s
Emitted | global.result ([object Object])
Emitted | global.after ([object Object])

{
  "output": "./__out",
  "helpers": {
    "WebDriverIO": {
      "url": "http://localhost",
      "browser": "chrome",
      "restart": false,
      "keepBrowserState": false,
      "keepCookies": false,
      "smartWait": 2000,
      "waitForTimeout": 10000
    }
  },
  "multiple": {
    "parallel": {
      "chunks": 2,
      "browsers": ["chrome"]
    }
  },  
  "include": {
    "I": "./custom-steps.js",
    "loginPage": "./pages/login.page.js"
  },
  "mocha": {},
  "bootstrap": false,
  "teardown": null,
  "hooks": [],
  "tests": "./*_test.js",
  "timeout": 10000,
  "name": "codeceptjs"
}

[DavertMik]

There seems to be even a new one with a blank page to be opened.

That's strange. Did you notice when a new window is opened?
Could you provide a reproducable example. That would be very helpful

[elukoyanov]

I have the similar issue with tests.

Tests works with 1.1.8, but with 1.2.0 (and 1.2.1) browser does not quit after tests, and new one with empty page opens.

codecept.json

{
  "output": "../../../testing/Codeceptjs",
  "helpers": {
    "WebDriverIO": {
      "url": "https://rambler.ru",
      "browser": "chrome",
      "desiredCapabilities": {
        "acceptInsecureCerts": true
      },
      "restart": false
    }
  },
  "mocha": {},
  "hooks": [],
  "tests": "../../../testing/Acceptance/*.Spec.js",
  "timeout": 10000,
  "smartWait": 5000,
  "name": "capirs"
}

openPage.Spec.js

"use strict";
Feature("Open page");
Scenario("should open", async (I) => {
    I.amOnPage("/");
});

[elukoyanov]

I added error throwing and this was logged

Open page --
   [1] Starting recording promises
   Emitted | suite.before ([object Object])
   Emitted | test.before ([object Object])
   Emitted | test.start ([object Object])
   Emitted | step.before (I am on page "/")
   Emitted | step.after (I am on page "/")
   Emitted | step.start (I am on page "/")
 • I am on page "/"
   Emitted | step.passed (I am on page "/")
   Step finished in 2.267 sec
   Emitted | step.finish (I am on page "/")
   Emitted | test.passed ([object Object])
   Emitted | test.finish ([object Object])
 ✓ OK in 2269ms

   Emitted | global.result ([object Object])
   Emitted | global.after ([object Object])
Error: error
    at WebDriverIO._startBrowser (/Users/e.lukoyanov/workbase/capirs-js/node_modules/codeceptjs/lib/helper/WebDriverIO.js:296:39)
    at WebDriverIO._finishTest (/Users/e.lukoyanov/workbase/capirs-js/node_modules/codeceptjs/lib/helper/WebDriverIO.js:360:44)
    at recorder.add (/Users/e.lukoyanov/workbase/capirs-js/node_modules/codeceptjs/lib/listener/helpers.js:21:69)
    at <anonymous>
    at process._tickDomainCallback (internal/process/next_tick.js:228:7)

[elukoyanov]

I think,
_finishTest should use something like if (this.options.restart) return this._stopBrowser();

And startBrowser should be mived to _before event

I can try to refactor, but only after May, 5-6th.
If anybody could do it before, please do it)

[jsparks19]

I am also hitting this issue after updating my codeceptjs version today. As a temporary workaround, I added a custom helper to close the browser after my suite completes. I'm not sure how to prevent the new blank browser window from opening though. Any ideas?

[jsparks19]

This issue is resolved after upgrading to 1.3.0.