Error: 23984 trailing bytes

[shivashahu07]

"socket.io": "^4.5.4"
"@socket.io/redis-adapter": "^8.0.0",
AWS elastic cache - redis engine version - 6.2.6

Above are the version in my application running on mutliple container and i'm getting this error when again and again and below are the exception are coming from nowhere and I don't know the exact root cause:

Error: 23984 trailing bytes at Object.decode (/app/node_modules/notepack.io/lib/decode.js:289:11) at RedisAdapter.onmessage (/app/node_modules/@socket.io/redis-adapter/dist/index.js:131:34) at /app/node_modules/@socket.io/redis-adapter/dist/index.js:84:22 at PubSub._PubSub_emitPubSubMessage (/app/node_modules/@redis/client/dist/lib/client/pub-sub.js:294:9) at PubSub.handleMessageReply (/app/node_modules/@redis/client/dist/lib/client/pub-sub.js:237:93) at Object.onReply (/app/node_modules/@redis/client/dist/lib/client/commands-queue.js:40:87) at RESP2Decoder.write (/app/node_modules/@redis/client/dist/lib/client/RESP2/decoder.js:119:26) at RedisCommandsQueue.onReplyChunk (/app/node_modules/@redis/client/dist/lib/client/commands-queue.js:154:72) at RedisSocket.<anonymous> (/app/node_modules/@redis/client/dist/lib/client/index.js:410:84) at RedisSocket.emit (node:events:513:28) at Socket.<anonymous> (/app/node_modules/@redis/client/dist/lib/client/socket.js:201:42)

After this error my application got socket connection timeout reached issue and This issue is coming when i'm hitting 500+ connection all together:

Error: timeout reached while waiting for fetchSockets response at Timeout._onTimeout (/app/node_modules/@socket.io/redis-adapter/dist/index.js:568:28) at listOnTimeout (node:internal/timers:564:17) at process.processTimers (node:internal/timers:507:7)

Can anyone help why this is happening again and again.

[darrachequesne]

Hi!

Error: xxx trailing bytes means the adapter has received an ill-defined packet from Redis PUB/SUB. Not sure how that could happen though.

timeout reached while waiting for fetchSockets() means the fetchSockets() operation has timed out because the other servers did not respond in the given delay. You might want to increase the requestsTimeout option:

const io = new Server({
  adapter: createAdapter(pubClient, subClient, {
    requestsTimeout: 10000
  })
});

Or simply retry upon timeout.

[shivashahu07]

@darrachequesne : I already did the requestTimeout to 10000 ms. but fetchSocket() is timedout since yesterdayand still giving the same error and my server cpu and memory utilization is 2-3%. Anything that redis are storing while pub/sub that are not cleared out within timeframe?

Whats your thought whether this is socket server issue or redis-adaptor issue?

[cody-evaluate]

@shivashahu07 its prbly caused by this

#367 (comment)

[shivashahu07]

@cody-evaluate : So how did you resolve this issue?

[cody-evaluate]

@shivashahu07 nope, happens intermittently but we will be moving to the sharded adapter soon.

[Peixer]

@cody-evaluate @shivashahu07 did you find a solution? I've just changed to sharded adapter, let's see if that solves the issue.

[Peixer]

just sharing my updates, I updated to sharded adapter but I was still getting some problems with memory leaking, so I removed it and put an AMQP approach.

[cherviakovtaskworld]

I can see it happens like this:

subClient redis error: SocketClosedUnexpectedlyError: Socket closed unexpectedly
subClient redis is reconnecting
subClient redis is connected
/app/node_modules/.pnpm/notepack.io@3.0.1/node_modules/notepack.io/lib/decode.js:289
    throw new Error((buffer.length - decoder.offset) + ' trailing bytes');

then server crashing on this unhandled error