You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/reference/content/reference/faq/index.md
+90-50Lines changed: 90 additions & 50 deletions
Original file line number
Diff line number
Diff line change
@@ -9,43 +9,77 @@ title = "Frequently Asked Questions"
9
9
+++
10
10
11
11
# What is the difference between connectTimeoutMS, socketTimeoutMS and maxTimeMS ?
12
-
A lof of people run into a similar issue wich is what is the difference between `connectTimeoutMS`, `socketTimeoutMS` and `maxTimeMS` and what values should be used for the different ones. Let's first explain each setting individually setting before discussing the values they should be set to.
13
12
14
13
| Setting | Default Value MongoClient.connect | Description |
15
14
| :----------| :------------- | :------------- |
16
-
| connectTimeoutMS | 30000 | The connectTimeoutMS sets the number of milliseconds a socket will stay inactive before closing during the connection phase of the driver. That is to say when the application initiates a connection or when a Replicaset conntects to new members or re-connect to new members. A value of `10000` milliseconds would mean the driver would wait up to 10 seconds for a response from a MongoDB server.|
17
-
| socketTimeoutMS | 30000 | The socketTimeoutMS sets the number of milliseconds a socket will stay inactive after the driver has successfully connected before closing. That is to say that if the value was set to `30000` milliseconds the socket would close if there was no activity during a 30 seconds window.|
18
-
| maxTimeMS | N/A | The maxTimeMS setting specifies how long MongoDB should run an operation before cancelling it. If you set the the maxTimeMS to `10000` milliseconds, any operation that ran over that limit would return an timeout error|
19
-
20
-
Now let's look at how the different settings affect your experience of using the driver using some example scenarios and what resonable values might be for the settings outlined.
15
+
| connectTimeoutMS | 30000 | The connectTimeoutMS sets the number of milliseconds a socket stays inactive before closing during the connection phase of the driver. That is to say, when the application initiates a connection, when a replica set connects to new members, or when a replica set reconnects to members. A value of 10000 milliseconds would mean the driver would wait up to 10 seconds for a response from a MongoDB server.|
16
+
| socketTimeoutMS | 30000 | The socketTimeoutMS sets the number of milliseconds a socket stays inactive after the driver has successfully connected before closing. If the value is set to 30000 milliseconds, the socket closes if there is no activity during a 30 seconds window.|
17
+
| maxTimeMS | N/A | The maxTimeMS setting specifies how long MongoDB should run an operation before cancelling it. If the maxTimeMS is set to 10000 milliseconds, any operation that runs over that limit returns a timeout error.|
21
18
22
19
#### Fail fast during connection
23
-
We want to ensure that the driver does not hang during the connection phase or spend an unnecessarily long time attempting to connect to Replicaset members who are not reachable.
20
+
In this scenario, the developer wants to ensure that the driver does not
21
+
hang during the connection phase or spend an unnecessarily long time
22
+
attempting to connect to replica set members who are not reachable.
24
23
25
-
As a general rule you need to ensure that the `connectTimeoutMS` setting is not lower than the largest network latency you have to a member of the set. Say one of the `secondary` members is on the other side of the planet and has a latency of `10000` milliseconds setting the `connectTimeoutMS` to anything lower will ensure the driver can never correctly connect to that member.
24
+
As a general rule you should ensure that the `connectTimeoutMS` setting
25
+
is not lower than the longest network latency you have to a member of
26
+
the set. If one of the `secondary` members is on the other side of the
27
+
planet and has a latency of 10000 milliseconds, setting the
28
+
`connectTimeoutMS` to anything lower will prevent the driver from ever
29
+
connecting to that member.
26
30
27
31
### socketTimeoutMS as a way to abort operations
28
-
One of the main ways people use socketTimeoutMS is to abort operations. This is in general a very bad idea. And there are a couple of reasons.
29
-
30
-
1. Closing the socket, will force a reconnect of the connection pool of the driver and introduce latency to any other operations queued up. Chronically slow operations will thus cause a reconnect storm impacting throughput and performance.
31
-
2. Closing the socket does not terminate the operation, it will still be running on MongoDB. This could cause data inconsistencies if your application retries the operation on failure for example.
32
-
33
-
That said the there is a very important usage for `socketTimeoutMS`. If the MongoDB process dies or a misconfigured `firewall` closes socket connections without sending a `FIN` packet dropping all subsequent packets on the floor there is no way for the driver to detect if the connection has died. In this case the socketTimeoutMS is essential to ensure the sockets are closed correctly.
34
-
35
-
A general rule of thumb is to set `socketTimeoutMS` to `2-3x` the time of the slowest operation run through the driver.
36
-
37
-
### socketTimeoutMS and big connection pools
38
-
One of the gotchas around socketTimeoutMS and a big pool seems to be experienced by a lot of people. Say you are performing a backend batch operation and storing the data in MongoDB. You `pool` size if 5 sockets and you have set `socketTimeoutMS` to `5000` milliseconds. You have an operation happening on average every `3000` milliseconds. You still get constant reconnects. Why ?
39
-
40
-
Well each socket will timeout after `5000` milliseconds. That means that all sockets must be exercised during that `5000` milliseconds period to avoid them closing. One message every `3000` milliseconds is not enough to keep the sockets active, meaning several of the sockets will timeout after `5000` milliseconds.
41
-
42
-
In this case you should reduce the pool size to `1` to get the desired effect.
32
+
Developers sometimes try to use ``socketTimeoutMS``
33
+
to end operations which may run for too long and slow
34
+
down the application, but doing so may not achieve the intended result.
35
+
36
+
Closing the socket forces a reconnect of the driver's connection pool
37
+
and introduces latency to any other operations which are queued up.
38
+
Chronically slow operations will therefore cause a large number of
39
+
reconnect requests, negatively impacting throughput and performance.
40
+
41
+
Also, closing the socket does not terminate the operation; it will continue
42
+
to run on the MongoDB server, which could cause data inconsistencies
43
+
if the application retries the operation on failure.
44
+
45
+
That said, there are some important use cases for `socketTimeoutMS`. It's
46
+
possible that a MongoDB process may error out, or that a misconfigured
47
+
firewall may close a socket connection without sending a `FIN` packet.
48
+
In these cases there is no way for the driver to detect that the
49
+
connection has died, and `socketTimeoutMS` is essential to ensure that the
50
+
sockets are closed correctly.
51
+
52
+
A good rule of thumb is to set `socketTimeoutMS` to two to three times the
53
+
length of the slowest operation which runs through the driver.
54
+
55
+
### socketTimeoutMS and large connection pools
56
+
Having a large connection pool does not always reduce reconnection
57
+
requests. Consider the following example: an application has
58
+
a connection pool size of 5 sockets and has `socketTimeoutMS` set
59
+
to 5000 milliseconds. Operations occur, on average, every 3000
60
+
milliseconds, and reconnection requests are frequent.
61
+
Each socket times out after 5000 milliseconds, which means that all
62
+
sockets must do something during that 5000 millisecond period to
63
+
avoid closing. One message every 3000 milliseconds is not enough to
64
+
keep the sockets active, so several of the sockets will time out
65
+
after 5000 milliseconds.
66
+
67
+
Reducing the pool size to 1 will fix the problem.
43
68
44
69
### The special meaning of 0
45
-
Setting `connectTimeoutMS` and `socketTimeoutMS` to the value `0` has a special meaning. On the face of it, it means never timeout. However this is a truth with some modifications. Setting it to `0` actually means apply the operating system default socket timeout value.
70
+
Setting `connectTimeoutMS` and `socketTimeoutMS` to the value 0 has
71
+
a special meaning. It causes the application to use the operating
72
+
system's default socket timeout value.
46
73
47
74
### maxTimeMS is the option you are looking for
48
-
Most people try to set a low `socketTimeoutMS` value to abort server operations. As we have proved above this does not work. To work correctly you want to use the `maxTimeMS` setting on server operations. This will make MongoDB itself abort the operation if it runs for more than `maxTimeMS` milliseconds. A simple example is below performing a `find` operation.
75
+
Many developers set a low `socketTimeoutMS` value, intending
76
+
to prevent long-running server operations from slowing down
77
+
the application. `maxTimeMS` is usually a better choice; it allows
78
+
MongoDB itself to cancel operations which run for more than `maxTimeMS`
79
+
milliseconds.
80
+
81
+
The following example demonstrates how to use `MaxTimeMS` with a `find`
Keep alive is a setting on the sockets available from Node.js that in theory will keep a socket alive by sending probes every once in a while to MongoDB keeping the connection alive.
60
-
61
-
However this only works if the operating system supports `SO_KEEPALIVE` and might still not solve the issue of firewalls as they might still ignore or drop these packets meaning it has no effect.
92
+
### What does the keepAlive setting do?
93
+
`keepAlive` is a socket setting available from Node.js that in theory
94
+
will keep a socket alive by sending periodic probes to MongoDB.
95
+
However, this only works if the operating system supports
96
+
`SO_KEEPALIVE`, and still might not work if a firewalls
97
+
ignores or drops the `keepAlive` packets.
62
98
63
99
### On misconfigured firewalls
64
-
Internal firewalls in between applications servers and MongoDB are in many cases misconfigured, being to aggressive in their culling of sockets connections. Many a problem have been diagnosed to a misconfiguration in a firewall between a DMC and internal MongoDB databases. If you are experiencing weird behavior it might be wise to investigate the settings on said firewall. Things to check for are.
100
+
Internal firewalls which exist between application servers and MongoDB
101
+
are often misconfigured, and are overly aggressive in their culling of
102
+
socket connections. If you experience unexpected network behavior, here
103
+
are some things to check:
65
104
66
-
1. The firewall should send a FIN packet when closing a socket allowing the driver to detect the socket as closed.
67
-
2. The firewall should allow keepAlive probes to allow for persistent connections.
105
+
1. The firewall should send a FIN packet when closing a socket,
106
+
allowing the driver to detect that the socket is closed.
107
+
2. The firewall should allow keepAlive probes.
68
108
69
109
# I'm getting ECONNRESET when calling MongoClient.connect
70
-
You might have decided to use a big connection pool with your node project.
110
+
This can occur if the connection pool is too large.
If this operation causes an `ECONNRESET` error, you may have run into
119
+
the file descriptor limit for your Node.js process.
78
120
79
-
When executing the operation you receive an error containg the `ECONNRESET` message. You have run into the file descriptor limit for your node.js process.
80
-
81
-
In most operating systems each socket connection is associated with a file descriptor. Many operating systems have a limit on how many such file descriptors can be used by a single process.
121
+
In most operating systems, each socket connection is associated with a
122
+
file descriptor. Many operating systems have a limit on how many such
123
+
file descriptors can be used by a single process.
82
124
83
-
For the example above let's assume the limit of file descriptors for each process is `1000`. Once the driver attempts to open it's `1001` socket the operating system returns an error as the process has exceeded the maximum file descriptors allowed for a single process.
84
-
85
-
The way to fix this issue is to increase the number of file descriptors for the Node.js process. On OSX and Linux you do this using the `ulimit`method.
125
+
The way to fix the descriptor limit issue is to increase the number of
126
+
file descriptors for the Node.js process. On Mac OS and Linux you do
127
+
this with the `ulimit`shell command.
86
128
87
129
```
88
130
ulimit -n 6000
89
131
```
90
132
91
-
The command above will set the maximum number of file descriptors for the process to `6000` descriptors allowing us to correctly connect with a pool size of `5000` sockets.
92
-
93
-
# How can I avoid a very slow operation delaying other operations ?
94
-
You have run into the `Slow Train` problem. It's tied to the fact that although the driver is Async, MongoDB is not. MongoDB as of 3.2 uses a single execution thread per socket. This means that it will only execute a single operation on a socket at any given point in time. Any operations sent to that socket will have to wait until the current operation is finished. This causes a slow train effect.
133
+
This sets the maximum number of file descriptors for the process to
134
+
6000, allowing Node.js to connect with a pool size of 5000 sockets.
95
135
96
-
```
97
-
Socket 1 <- [S, F, F, F]
98
-
Socket 2 <- [S, F, F, F]
99
-
...
100
-
Socket N <- [S, F, F, F]
101
-
```
136
+
# How can I prevent a slow operation from delaying other operations?
102
137
103
138
{{% note %}}
104
139
The driver is only affected by the slow train operations if the number of slow operations is larger than the max pool size.
105
140
{{% /note %}}
106
141
107
-
# Ensure you connection string is valid for Replica Set
142
+
While Node.js is asynchronous, MongoDB is not. Currently, MongoDB uses a single execution thread per socket. This means that it will only execute a single operation on a socket at any given point in time. Any other operations sent to that socket will have to wait until the current operation is finished. If you have a slow-running operation which holds up other operations,
143
+
the best solution is to create a separate connection pool for the slow operation, isolating it from other, faster
144
+
operations.
145
+
146
+
# Ensure your connection string is valid for Replica Set
108
147
109
148
The connection string passed to the driver **MUST** use the fully qualified host names for the servers as set in the replicaset config. Given the following configuration settings for your replicaset.
110
149
@@ -131,3 +170,4 @@ The connection string passed to the driver **MUST** use the fully qualified host
131
170
```
132
171
133
172
You must ensure `server1`, `server2` and `server3` are resolvable from the driver for the Replicaset discovery and failover to work correctly.
0 commit comments