Connection timeout 10000ms when upgraded mongoose from 7.4.5 to 8.7.1

[sukhminderx]

Prerequisites

• I have written a descriptive issue title

Mongoose version

8.7.1

Node.js version

20.x

MongoDB version

6.9

Operating system

None

Operating system version (i.e. 20.04, 11.3, 10)

No response

Issue

Last night, I upgraded my mongodb and mongoose from versions 5.8.1 to 6.9 and. 7.4.5 to 8.7.1 respectively.

I tested on my local. Everything worked fine. I promoted the changes to test env - AWS amplify - things were fine. Then, I moved the changes to production.

My db is hosted on mongo atlas serverless cluster.

On production, sometimes same queries will work fine but other times give timeout error with 10000ms. I mean same queries behaving differently.

I have been scratching my head because both test and prod env are exactly similar except that prod is connected to my domain name.
Please assist if someone has knowledge about this.

sharing my connection code and one query that fails/passes abruptly.

import { ServerApiVersion } from "mongodb";
import mongoose from "mongoose";

const uri = process.env.MONGODB_URI as string;


const connection: {
  isConnected?: boolean;
} = {};

const dbConnect = async () => {
  if (connection.isConnected) return;
  mongoose.set("strictQuery", false);
  const db = await mongoose.connect(uri);

  connection.isConnected = Boolean(db.connections[0].readyState);
};
export default dbConnect;

export async function GET(req: Request) {
  try {
    await apiHandler(req, "");
    const countries: any = await Countries.find({}).exec();
    return NextResponse.json(countries, { status: 200 });
  } catch (error: any) {
    console.log("wb saas: countries");
    console.log(
      typeof error !== "string"
        ? error.message
          ? error.message
          : (error || {}).toString()
        : error
    );
    return errorHandler(error);
  }
}

[garethlowriehive]

We're also experiencing a very similar issue.

Using MongoDB Server 8.0 in Atlas and locally in Docker.
Node: 20.11.1
Mongoose 8.7.1

Everything works intermittently, but lots of buffering error across lots of different find’s and aggregates in all of our microservices.

Buffering timed out after 10000ms

Posted about it here: https://www.mongodb.com/community/forums/t/buffering-timed-out-after-10000ms-8-7-0-mongoose-node-20-10-0/299932

[ksmithut]

We found the issue was introduced in 8.7. We were on 8.6, but only saw the issues when we upgraded to 8.7.

[slavik-pastushenko]

the same issue.
Mongo: 8.0
Node: 20.13.1
Mongoose 8.7.1

Buffering timed out after 10000ms

[vkarpov15]

The "Buffering timed out after 10000ms" is indicative of Mongoose not being connected to MongoDB. Here's a couple of suggestions you can try:

1. Reduce your serverSelectionTimeoutMS. This will likely just change the error message, but the new error message will provide a little bit more info on why you can't connect to MongoDB:

const db = await mongoose.connect(uri, { serverSelectionTimeoutMS: 5000 });

2. Fix your isConnected logic. isConnected is not tied to Mongoose's readyState, if Mongoose's readyState changes then isConnected is still tied to the initial value. Try the following instead.

mongoose.set("strictQuery", false);

const dbConnect = async () => {
  if (mongoose.connection.readyState === 1) {
    return;
  }
  
  await mongoose.connect(uri);
};
export default dbConnect;

You can read more on our Lambda docs

[sukhminderx]

My error got resolved somehow when I used these:

const options = {
serverApi: {
version: ServerApiVersion.v1,
strict: true,
deprecationErrors: true,
},
};

Please check.

[github-actions]

This issue is stale because it has been open 14 days with no activity. Remove stale label or comment or this will be closed in 5 days

[tabz24]

We are also facing the same issue, which only happens in the production environment. We have tried multiple versions after 8.7.1 and none of them seems to be working so far.

We are stuck at using 8.6.x as of now.

Another thing to point out is that we have about 15 services and all of them fail when we go to 8.7.1 or above and that too only in production.

[sukhminderx]

hi.. did you check with latest version ie 8.8.2?

[VanCoding]

I have the same issue. Downgrading to mongoose 8.6.x solved the issue. I tried everything, even created a new mongodb atlas cluster to make sure it's not the cluster, even though connecting to the db worked fine from other tools. There's definitely something wrong with mongoose or the mongodb package.

Worth mentioning: It only fails to connect with the mongodb+srv:// scheme.

@sukhminderx

hi.. did you check with latest version ie 8.8.2?

I did, no success.

[slavik-pastushenko]

any updates on it? I have the same issue as @VanCoding described above.

[atayahmet]

We are also experiencing this issue in version 8.0.1.

[sukhminderx]

My issue was resolved by this code:

import { ServerApiVersion } from "mongodb";
import mongoose from "mongoose";

const uri = process.env.MONGODB_URI as string;
const optionsOld = {
  useUnifiedTopology: true,
  useNewUrlParser: true,
  serverApi: ServerApiVersion.v1,
};

const options = {
  serverApi: {
    version: ServerApiVersion.v1,
    strict: true,
    deprecationErrors: true,
  },
};

const dbConnect = async () => {
  console.log("MONGO CONNECTION STATUS -- " + mongoose.connection.readyState);
  if (mongoose.connection.readyState === 1) {
    console.log("MONGO already connected");
    return;
  }
  try {
    const db = await mongoose.connect(uri, options);
    // mongoose.set("debug", true); // Logs every query

    // Listen for connection events
    mongoose.connection.on("connected", () => {
      console.log("MONGO connection established");
    });

    mongoose.connection.on("disconnected", () => {
      console.error("MONGO connection lost");
    });

    mongoose.connection.on("error", (error) => {
      console.error("MONGO connection error:", error);
    });
  } catch (error) {
    console.error("MONGO connection failed:", error);
    throw error;
  }
};
// Disconnect function
const dbDisconnect = async () => {
  if (mongoose.connection.readyState !== 0) {
    await mongoose.disconnect();
    console.log("MONGO disconnected");
  }
};

// Handle shutdown signals
process.on("SIGINT", async () => {
  await dbDisconnect();
  process.exit(0);
});

process.on("SIGTERM", async () => {
  await dbDisconnect();
  process.exit(0);
});

export default dbConnect;

Can anyone try this and confirm if this works for them?

[vkarpov15]

@sukhminderx what did you add, serverApi: ServerApiVersion.v1?

[halilbrhmtrn]

In case you encounter such connection errors where your connections are frequently disconnected (readyState = 0), consider using a Mongoose version below 8.7.0. Version 8.7.0 introduces a new variable to store the last heartbeat from the MongoDB server, which can lead to connection inconsistencies. For more details, refer to this comment:
#14812 (comment).

[maxuai]

Will there be any fix soon? With this now being public but not being able to update mongoose 8.6.4 to any newer version it creates some urgency from my perspective.

[halilbrhmtrn]

I haven't received any response from the maintainers, so I assume they don't believe this PR introduces a bug. For now, the best option, if you want to use the latest version with security updates, is to fork the repository, revert the changes introduced by this PR, republish the package, and use it until they acknowledge the issue with this PR for the serverless cold start bug. Hopefully, they either come up with a reliable solution or decide to revert the changes. @maxuai

[sukhminderx]

@sukhminderx what did you add, serverApi: ServerApiVersion.v1?

I replace my optionsOld with options,ie, the ones I got from Atlas connection steps. Did it work for anyone?