Article summary
Recently, on my project, I was tasked with migrating data from a relational database to a document database. Specifically, the database we were migrating to was AWS DocumentDB. Along the way, I hit a few snags while developing against MongoDB locally and deploying to AWS DocumentDB; here is what I discovered.
Local Development Constraints
The first hurdle was writing integration tests that could run locally and in our code pipelines since there’s no official AWS DocumentDB Docker image. Not even LocalStack has a better option for local development other than MongoDB. Given how much Amazon touts the compatibility of AWS DocumentDB with MongoDB, we decided to proceed with the official MongoDB image for our testing.
MongoDB Compass
While manually testing, I used MongoDB Compass to view the the database collections. When I connected to the DocumentDB, I saw this warning:
This server or service appears to be an emulation of MongoDB rather than an official MongoDB product.
When continuing past this warning, I could view the collections and filter and sort documents as expected. The only issue I ran into was using Collations, but other issues may arise from using other features in Compass. This warning shows every time you connect to AWS DocumentDB, which is not a big deal. This was just my first clue that there might not be as much feature parity between the software as I had thought.
Collation
While developing queries for the new database, I used collation to index and query for values case-insensitively. When running the query against my local MongoDB, everything worked as expected. The issue came into play when this code was deployed to the development environment: Field 'collation' is currently not supported. After some research, I found out DocumentDB does not support collations. In fact, many cursor methods you would find in MongoDB are not supported in DocumentDB. We had a few options to get around this incompatibility:
- Use a regex query to match the property case-insensitive. The issue with this approach is that this query cannot use an index, so it must scan the whole collection to find a document, so we didn’t go with this option.
- Another option we considered was creating a new property normalized to have only lowercase characters to create, index, and query against for these case-insensitive queries.
- The last option was to standardize the values to lowercase, as my specific use case allowed for the values to be transformed to lowercase. This would enable using queries and indexing on the same property as before, as long as the query ensures any lookup values are also lowercase.
Index Names
Another challenge was the character limit for index names in AWS DocumentDB, which is less than that of MongoDB. AWS DocumentDB has a maximum character limit of a fully qualified index name of 127 characters. In contrast, the latest versions of MongoDB do not limit the length of the name. To resolve this, we had to make an exception to our index naming, which had been based on our property name. That way, the index name could fit within the character limit for DocumentDB.
Conclusion
When using AWS DocumentDB as a substitute for MongoDB, take note of the compatibility challenges that can arise. Next time you use another MongoDB feature, ensure that AWS DocumentDB has support, too. Another great way to mitigate any issues in production is, as always, testing in your development and QA environments before the feature makes its way into production.