Travis CI fails occasionally due to GemFire sample integration test failure

[vpavic]

Build output:

:samples:httpsession-gemfire-boot:runGemFireServer
STARTING GEMFIRE SERVER...
:samples:httpsession-gemfire-boot:integrationTest

sample.AttributeTests > first visit no attributes FAILED
    java.lang.IllegalStateException
        Caused by: org.springframework.context.ApplicationContextException
            Caused by: org.springframework.boot.context.embedded.EmbeddedServletContainerException
                Caused by: org.springframework.beans.factory.UnsatisfiedDependencyException
                    Caused by: org.springframework.beans.factory.UnsatisfiedDependencyException
                        Caused by: org.springframework.beans.factory.BeanCreationException
                            Caused by: com.gemstone.gemfire.cache.NoSubscriptionServersAvailableException
                                Caused by: com.gemstone.gemfire.cache.NoSubscriptionServersAvailableException
    java.lang.NullPointerException

2 tests completed, 1 failed, 1 skipped
:samples:httpsession-gemfire-boot:integrationTest FAILED

FAILURE: Build failed with an exception.

Here are the samples of such jobs:

• https://travis-ci.org/spring-projects/spring-session/builds/175189705
• https://travis-ci.org/spring-projects/spring-session/builds/175888993
• https://travis-ci.org/spring-projects/spring-session/builds/176169898

/cc @jxblum

[jxblum]

It's possible the GemFire server in the Travis CI environment is taking too long to startup. My default timeout is 20 seconds, which should be more than enough time for the server to start (the server often times takes far less than 20 seconds).

All SSDG tests use no more than a 20 second timeout to wait for the server to start and for the client to connect and the run without incident in other CI environments, like build.spring.io, for example. I could find no instances of build failures in build.spring.io related to Subscription Servers available.

It is also possible that prior build failures left orphaned processes on the Travis CI server that are possibly conflicting with subsequent build runs that need to be cleaned up, though this less likely than the former.

[rwinch]

@jxblum I noticed these failures are happening on our Jenkins build too. See

• https://jenkins.spring.io/job/spring-session/131/testReport/junit/sample/AttributeTests/first_visit_no_attributes/
• https://jenkins.spring.io/job/spring-session/123/testReport/junit/sample/AttributeTests/first_visit_no_attributes/

What should we do to fix this?

[jxblum]

@rwinch

As discussed in chat, I am disabling the JMX-based GemFire Management Service in the Spring Boot-based GemFire Server. This should help improve the server's startup time and reduce overhead in starting extraneous services during automated test runs.

Additionally, I am increasing the timeout used by the GemFire cache client to connect to the server from 20 seconds up to 60 seconds. I have also increased the read timeout from 2 seconds up to 15 seconds in waiting for a server's response to a client request (message).

Unfortunately, there is no other/better means than what is already in-place to determine when a GemFire Server has successfully started and, and is "listening" for and "accepting" client connections, and thus be able to avoid the timeout. I know; I have tried.

I have used this pattern, reliably in the SDG test suite for quite some time and all the tests pass both locally and in the Bamboo CI environment without incident between the client and server. I also never have a problem running the Spring Session (Data GemFire) test suite locally, unless there is a legitimate issue. I have not encountered this test failure yet and I have not observed a Spring Session Bamboo CI build environment run with this failure.

Anyway, I hope this addresses any environmental issues. If not, we can reopen this ticket and I will revisit this.

[rwinch]

This is still failing. See https://build.spring.io/browse/SESSION-B13X-159