3.6 Release Notes

Mongoose 3.6 Release Notes

The primary focus of the 3.6 release was a rewrite and expansion of the query population feature. Population capabilities are now far more robust, flexible, and perform much faster than before. Also of significant note are additional new features in support of MongoDB 2.4. What follows are the details of all 58 new features, enhancements, and bug fixes.

Population

Added Model.populate(docs, opts, cb)

The core population functionality has been rewritten and exposed through Model.populate(docs, paths, cb). This refactor and exposure opens the door to populating plain js objects as well as mongoose documents, meaning population of documents returned from lean queries and mapReduce output is supported.

While not directly supported through the query.populate() syntax, recursive population of multi-level deep refs can be achieved by calling Model.populate ad infinitum to populate structures that span more than 2 collections. Check out this gist for an example.

Query.populate now accepts an options object

populate accepts up to five arguments making it a bit obscure what is each argument is when all are passed. We now support an options object which clarifies the arguments and allows applying several options to more than one path at a time.

// old (still supported)
Model
  .find()
  .populate('friends', 'name age', 'ModelName', { age: { $gte: 18 }}, { sort: { age: -1 }})
  .populate('author', 'name age', 'ModelName', { age: { $gte: 18 }}, { sort: { age: -1 }})

// new
Model.find().populate({
    path: 'friends author'        // either single path or multiple space delimited paths
  , select: 'name age'            // optional
  , model: 'ModelName'            // optional
  , match: { age: { $gte: 18 }}   // optional
  , options: { sort: { age: -1 }} // optional
})

Added document.populate(opts | path, [cb])

Documents themselves now support population.

Model.findById(id, function (err, doc) {
  doc
  .populate('owner')
  .populate({ path: 'friends', select: 'name' }, function (err, pop) {
    console.log(pop.owner.name)
    assert.equal(doc.id, pop.id) // same doc
  })
})

The document passed to the callback is the same as the document that executed populate().

Added document.populated(path)

Returns original _id(s) used in population call for that path. If the path was not populated, undefined is returned.

Model.findOne(function (err, doc) {
  console.log(doc.comments) // [ObjectId('xxx'), ObjectId('yyy')]
  doc.populate('comments', function (err, doc) {
    console.log(doc.comments) // [{ _id: ObjectId('xxx'), name: 'populated' }, { _id: ObjectId('yyy') }]
    console.log(doc.populated('comments')) // [ObjectId('xxx'), ObjectId('yyy')]
  })
})

Syntax support for population of multiple fields

Model.find().populate('owner comments friends').exec(callback)
document.populate('owner comments friends', callback)
Model.populate(docs, 'owner comments friends', callback)

Specifying identical options for all paths:

var opts = { path: 'owner comments friends', select: 'name' }

Model.find().populate(opts).exec(callback)
document.populate(opts, callback)
Model.populate(docs, opts, callback)

Separate options per path:

var opts = [
    { path: 'owner', select: 'name' }
  , { path: 'comments', select: 'title -body', match: { createdAt: { $gt: lastMonth }}}
  , { path: 'friends', options: { sort: { name:-1 }}}
]

Model.find().populate(opts).exec(callback)
document.populate(opts, callback)
Model.populate(docs, opts, callback)

Improved population support for arrays within arrays

Arrays within arrays are now able to be populated:

var schema = new Schema({
    name: String
  , em: [{ arr: [{ person: { type: Schema.ObjectId, ref: 'Person'}}] }]
});
A.findById(id).populate('em.arr.person').exec(function (err, doc) {
  console.log(doc.em[0].arr[0].person instanceof mongoose.Document) // true
})

Support adding documents to populated paths

Change from 3.5.x

Previously, setting a populated path to a document, or adding a document to a populated array would cause the document to be cast back to an _id. This is now fixed and the argument is cast to a document.

#570

Model.findOne().populate('owner comments').exec(function (err, doc) {
  console.log(doc.owner) // a populated document
  doc.owner = { name: 'a different document' }
  console.log(doc.owner) // { _id: 'xxx', name: 'a different document' }
  assert(doc.owner instanceof mongoose.Document) // true

  console.log(doc.comments) // an array of populated documents
  doc.comments.push({ body: 'a new comment' })
  var added = doc.comments[doc.comments.length - 1]
  console.log(added)        // { _id: 'xxx', body: 'a new comment' }
  assert(added instanceof mongoose.Document) // true
})

Support population in combination with lean() queries

Previously, we could not combine lean() with populate(). For example, this would fail to populate batch:

#1260

Model.find().lean().populate('batch').exec(fn)

Support population of non-schema ad-hoc paths

Populating paths that are not defined in the schema is now possible as long as the model name is passed. This is particularly helpful for populating mapReduce results and opens the door to do things with the aggregation framework in the future.

commit

Model.populate({ friend: 4815162342 }, { path: 'friend', model: 'ModelName' }, callback);

Population performance improvements

The refactor of populate brought with it opportunities for performance improvements. Here are some very non-scientific benchmarks run on my MacBook Air using node.js 0.8.21.

node populate.js 1
==================================
3.5.7:     8117  completed queries
3.6.0-rc1: 11226 completed queries

node populate.js 10
==================================
3.5.7:     1662
3.6.0-rc1: 4200

node populate.js 20
==================================
3.5.7:     898
3.6.0-rc1: 2376

node populate.js 50
==================================
3.5.7:     370
3.6.0-rc1: 1088

node populate.js 100
==================================
3.5.7:     188
3.6.0-rc1: 590

Prevent potentially destructive operations on populated arrays

When saving a document with an array populated using a query condition, skip and/or limit options, or exclusion of the _id property, we now produce an error if the modification to the array results in either a $set of the entire array or a $pop. Reason: the view mongoose has of the array has diverged from the array in the database and these operations would have unknown consequences on that data. Use doc.update() or Model.update() instead. See the commit for more information and examples.

Prevent updates to $elemMatch projected arrays

Similar to why potentially destructive array operations are disallowed, we now also disallow modifications made to arrays selected using the $elemMatch projection. The view mongoose has of the array has diverged from what is in the database and these operations would have unknown consequences as a result. Use doc.update() or Model.update() instead.

#1334
commit

Validation errors now include values

Example:

var schema = Schema({ age: { type: Number, min: 18 }})
var M = mongoose.model('Model', schema);
var doc = new M({ age: 4 });
doc.save(function (err) {
  console.log(err) // ValidationError: Validator "min" failed for path name with value `4`
  console.log(err.errors.age.value) // 4
})

commit

Refactor internal document properties and methods

To allow greater flexibility in designing your document schemas, mongoose internal properties are no longer littered everywhere and private methods have been renamed with a $ prefix (document property names cannot begin with $ in MongoDB) to avoid naming collisions. The exception is the private init method which remains to facilitate pre('init') hooks.

Support for non-strict `document.set()` and `Model.update()`

The schemas strict option can now be optionally overridden during document.set() and Model.update(). This is helpful when cleaning up old properties that are no longer in the schema.

commit

// set
document.set('path', value, { strict: false })

// update
Model.update(selector, doc, { strict: false })

Object literal schema support

mongoose.model() and connection.model() now support passing an object literal schema. Useful for quick scripts, etc.

mongoose.model('Name', { object: { literal: Boolean }})

Auto-generated ObjectIds support

Set the auto option to true to create an ObjectId at document construction time automatically.

#1285

var M = mongoose.model('Blog', { owner: { type: Schema.Types.ObjectId, auto: true }})
var m = new M;
console.log(m.owner) // ObjectId('513f5f7a2b4024250053bec8')

Improved sub-document schema types declaration

Passing a capitalized type as a string is now supported.

new Schema({ path: [{ type: 'String' }] });

commit

Support optional command buffering

When running with the drivers autoReconnect option disabled and connected to a single mongod (non-replica-set), mongoose buffers commands when the connection goes down until you manually reconnect. To disable mongoose buffering under these conditions, set this option to false. Note: by default, both autoReconnect and bufferCommands are true.

mongoose.connect(uri, { server: { autoReconnect: false }});
new Schema({ stuff: String }, { bufferCommands: false });

Improve Boolean casting

The strings 'true' and 'false' now cast to the Boolean true and false respectively.

var schema = Schema({ deleted: Boolean })
var M = mongoose.model('Media', schema)
var doc = new M({ deleted: 'false' })
assert(false === doc.deleted)

Support nulls in arrays of Buffer

For consistency between all array types, buffer arrays now accept null values too.

Schema({ buffers: [Buffer] })
new Doc({ buffers: [new Buffer(0), null, new Buffer('supported')] })

Support doc.array.remove(non-object value)

Passing the _id directly to documentArray#remove() now works, instead of needing to pass { _id: value }.

#1278

doc.array.remove('513f5f7a2b4024250053bec8') // now same as
doc.array.remove({ _id: '513f5f7a2b4024250053bec8' })

commit

Add `transform` option for QueryStreams

We may want to tranform a document before it is emitted on data to convert it to a string for example.

// JSON.stringify all documents before emitting
var stream = Thing.find().stream({ transform: JSON.stringify });
stream.pipe(writeStream);

Expose list of added model names

#1362

mongoose.model('Thing', aSchema);
console.log(mongoose.modelNames()) // ['Thing']

var conn = mongoose.createConnection();
conn.model('Thing2', aSchema);
conn.model('Thing3', anotherSchema);
console.log(connection.modelNames()) // ['Thing2', 'Thing3']

Expose mongoose constructors to all Mongoose instances

Previously, when creating more instances of Mongoose, the various mongoose constructors were not exposed the same way they are through the module itself. This has been fixed. Example:

var mongoose = require('mongoose');
console.log(typeof mongoose.Document) // function

var mongoose2 = new mongoose.Mongoose;

// old
console.log(typeof mongoose2.Document) // undefined
// mongoose 3.6
console.log(typeof mongoose2.Document) // function

commit

Hashed indexes support (MongoDB >= 2.4)

MongoDB 2.4 supports a new type of index called hashed indexes. We may enable it as follows:

Schema({ someProp: { type: String, index: 'hashed' }})
// or
schema.index({ something: 'hashed' })

commit

GeoJSON support (MongoDB >= 2.4)

MongoDB 2.4 supports GeoJSON. In support of GeoJSON, mongoose 3.6 has the following new features:

2dsphere indexes

new Schema({ loc: { type: [Number], index: '2dsphere'}})
// or
schema.index({ line: '2dsphere' })

$geoIntersects

Casting of GeoJSON coordinates.

var geojsonLine = { type: 'LineString', coordinates: [[180.0, 11.0], [180.0, '9.00']] }
Model.find({ line: { $geoIntersects: { $geometry: geojsonLine }}}, callback);

You might also notice the new $geometry operator is supported.

query.intersects.geometry()

Similar to the $geoIntersects example above, the mongoose Query builder now supports an intersects getter and geometry() method for a more fluid query declaration:

Model.where('line').intersects.geometry(geojsonLine).exec(cb);

query.within.geometry()

$within supports the $geometry operator as well.

var geojsonPoly = { type: 'Polygon', coordinates: [[[-5,-5], ['-5',5], [5,5], [5,-5],[-5,'-5']]] }

Model.find({ loc: { $within: { $geometry: geojsonPoly }}})
// or
Model.where('loc').within.geometry(geojsonPoly)

For more information about MongoDB GeoJSON, read the 2.4 pre-release notes.

commit

array `$push` with `$each`, `$slice`, and `$sort` (MongoDB >= 2.4)

Mongoose 3.6 supports these new MongoDB 2.4 array operators.

Model.update(matcher, { $push: { docs: { $each: [{ x: 1 }, { x: 23 }, { x: 5 }], $slice: -2, $sort: { x: 1 }}}})

Support $setOnInsert (MongoDB >= 2.4)

Mongoose 3.6 supports the MongoDB 2.4 $setOnInsert operator for upserts.

Support authSource

Support for the authSource driver option (driver version 1.2.14) is now supported.

mongoose.connect('mongodb://user:pass@host:27017/db?authSource=databaseName')
// or
mongoose.connect(uri, { auth: { authSource: 'databaseName' }})

Promises A+ 1.2 conformant

The promise module has now been refactored and published separately with support for version 1.2 of the Promises A+ spec.

Added `promise.then(onFulfill, onReject)`

then creates and returns a new promise which is "tied" to the original promise (read the spec). If onFulfill or onReject are passed, they are added as success/error callbacks to this promise after the nextTick. See the spec for the full rundown.

var promise = Model.findOne({ version: '3.6' }).exec();
promise.then(function (doc) {
  console.log('the version is %s', doc.version);
}, function (err) {
  console.error('awww shucks', err)
})

Added `promise.end()`

end signifies that this promise was the last in a chain of then() calls. This is useful when no onReject handler has been registered and a handler passed to then() throws, in which case the exception would be silently swallowed. Declaring an end() to your then() calls allows the error to go uncaught.

var p = Model.findOne({ version: '3.6' }).exec();
p.then(function(){ throw new Error('shucks') });

setTimeout(function () {
  p.fulfill();
}, 10);
// error was caught and swallowed by the promise returned from
// p.then(). we either have to always register handlers on
// the returned promises OR we can do the following...

// this time we use .end() which prevents catching thrown errors
var p = Model.findOne({ version: '3.6' }).exec();
var p2 = p.then(function(){ throw new Error('shucks') }).end(); // <--

setTimeout(function () {
  p.fulfill(); // throws "shucks"
}, 10);

For more information, read the mongoose promise docs, the mpromise docs and the Promises A+ docs.

Bug fixes

fixed; lean population #1382
fixed; empty object mixed defaults now return unique empty objects #1380 change from 3.5
fixed; populate w/ deselected _id using string syntax
fixed; attempted save of divergent populated arrays #1334 related
fixed; better error msg when attempting use of toObject as property name
fixed; setting populated paths #570
fixed; casting when added docs to populated arrays #570
fixed; prevent updating arrays selected with $elemMatch #1334
fixed; pull/set subdoc combination #1303
fixed; multiple background index creation #1365
fixed; manual reconnection to single mongod
fixed; Constructor / version exposure #1124
fixed; CastError race condition
fixed; no longer swallowing misuse of subdoc#invalidate()
fixed; utils.clone retains RegExp opts
fixed; population of non-schema properties
fixed; allow updating versionKey #1265
fixed; add EventEmitter props to reserved paths #1338
fixed; can now deselect populated doc _ids #1331
fixed; properly pass subtype to Binary in MongooseBuffer
fixed; casting to _id from document with non-ObjectId _id
fixed; schema type edge case { path: [{type: "String" }] }
fixed; typo in schemadate #1329 jplock

Updated

driver to version 1.2.14

Deprecated

deprecated; collection name pluralization

The collection name pluralization rules are confusing. We generally expect that model names are used for collection names without alteration. Though deprecated, the rules will stay in effect until 4.x.