Creating a new app project on mobile or web is an exciting time while you’re thinking of the architecture and data structures. It’s a chance to build something different and explore new ideas and techniques, but some parts of an application are mostly always the same - database querying. This post is an accumulation of a few years of “gottchas” and snippets to form a kind of cheat sheat for MongoDB’s ORM, Mongoose.
Updated 11th September 2018. Thanks to Peter Cooper for sending in a few typo corrections.
MongoDB vs Mongoose
MongoDB is a noSQL database which uses Object Document Mapping (ODM) to enable flexible data structures which don’t adhere to a strict Schema. This is useful for unpredictable data sets where the attributes of data can change any time.
Say for example you have a database of cars which stores “colour”, “engine size” and “top speed”. Later, you find that you need to add electric cars which require another attribute for “charge time”, and after that you introduce flying cars which require a “fuel type” attribute. MongoDB will be great for this situation because you can add any data to the structure.
Mongoose is an abstraction layer which sits on top of MongoDB and actually requires MongoDB as a dependency. While still being a noSQL database, Mongoose uses Object Relational Mapping (ORM) for it’s data modelling which is one of it’s defining features. This means the data conforms to a Schema which outlines the data attributes, types, default values etc which can be a huge help for most projects.
Aside from the fundemental data modelling mentioned above (which I believe are the biggest/most important differences) MongoDB is reported to be faster than Mongoose for larger scale projects, but Mongoose is thought to make development times generally faster.
One of the reasons for writing this post is that when I do use MongoDB, I will mostly always go with Mongoose for an ORM, but I find the documentation saturated and fragmented to find how to do the simplest tasks which almost all apps will do.
Here’s an example Schema which Mongoose will use to define the data. When something is saved, updated, deleted etc. it will need to conform to this data structure. It can also be changed over time if more data needs to be added.
The following structure outlines basicuser information, as well as an embedded array of objects. Remember - nosql databases are flat lists:
Creating a Schema looks for plural of collection name
One of the gotcha’s when defining and using a Schema is that the model will automatically look for the plural of the collection name. For example:
var Users = mongoose.model('User', usersSchema);
The above will look for the plural of collection (first argument) which in this case would be
Users. If you’ve already created the
Users collection and use that as the first argument, the call will not find the collection:
You can, however, explicitly define the collection name as a third parameter:
Saving created and updated time by using schema:
A common pattern is to save the current time/date of when something is created, and then when it’s updated thereafter. This can be done in the Schema instead of littering it throughout different queries:
Mongoose query structuring
There are different ways to write queries with Mongoose - so much so that it can be confusing. Here are some of the main ways at a basic level:
The callback route is the more traditional Node async pattern:
Callbacks are sometimes seen as the less favorable way of structuring a query because it can lead to multiple callback levels when there’s a need to run other queries based on the returned data:
If a query has the second parameter passed (the callback parameter as shown above) the query is executed and results passed to the callback (as shown in the above examples).
If a query doesn’t have the second parameter (callback parameter) a
Query instance is returned. This query instance can be used as a promise, and as such it returns
then() returned from a
Query object mentioned above is not part of a fully-fledged ES6 Promise, although it is a promise which conforms to the Promises/A+ specification. This means you can use the
Query promise by chaining
catch() for errors (which I think is nicer than error handling with the callback approach), and even use as part of other Promise functional such as
Promise.all(). Here’s an example of a simple promise based query structure:
Using this promise type can also be great for avoiding the callback hell mentioned above when updating data by using previously queried data:
then()’s can be chained to run as many separate queries as you need, and all errors will be caught by the last
catch() function which keeps the code squeaky clean.
As mentioned above, the
Query object which is returned from a query without a callback parameter can be used as a promise which is not the official ES6 Promise. However, the
exec() function can be used after any query function such as
find() to return a fully-fledged ES6 Promise:
I’ve used the same example as the previous one here to illustrate that there’s not much of a difference whether you use
exec() or not.
Another feature of the
Query object is that it can be used to chain query functions together, for example:
This is a different way of querying data as opposed to passing an object to the
find() function as shown here:
exec() function is used at the end of the query build chain to execute the query. This can be written another way to save the query to a variable and later use the
exec() function to execute the query:
The query building syntax can also be used alongside the promise functionality from the
Query object, which I prefer to use:
So that may leave the question, what’s the difference between
then()? As far as I can tell, the
exec()returns a full Promise and that’s it. The
then()function can execute queries later from the
Queryobject, as well as execute queries and be used in the query builder process.
Mongoose file structure and general usage
The Node ecosystem is an extremely unopinionated as it allows you to write code anywhere in the application - either all in the same file or splitting up into chunks and using
import syntax. When an app gets bigger it can be better to separate the concerns of the app into small, managable parts. Here’s how I like to do that with the database file structure:
As shown here, all database related files go in a folder called
util. Lines 27-29 in the screen shot also shows the only place the database is initiated in the main entry point file (server.js in my case). The boilerplate setup code for making the database connection is in the
mongo.js file inside
util, which has the following:
You’ll notice I’m using environment variables to work with the sensitive and instance specific code here (local or remote). I’ve written another post about that if you want to take a look!
The database setup and Schema definitions are all in the same folder and only 3 lines of code initiate the database connection when the server loads, but what about actually makign queries? That’s simple, just include the Schema export in any file which needs it. For example, if you have one file which handles all API requests, you may include both database Schemas:
Here are some popular queries which I end up using constantly, so it’s nice to have then in this quick reference quide. I’ve used the callback style of querying here rather than using the returned promise and chaining as it’s a little quicker to write and easy to understand if you’re new as many examples you see may be like this. I would use the promise syntax though when adding in to my code as it works a lot better when the app grows.
Creating an entry
Find all entries
Find entry where
Upate an entry where
Upate an entries embedded array
Thanks for reading. I’ll try and update this post every now and again to add to the list of popular queries. Even now, I find myself learning new ways to save data and implement new functionality. Tweet me if you have any questions or I’ve made any mistakes :).