Eliot's Ramblings

Innovate vs. Appropriate

One theme I kept harping on at MongoDB World a few weeks ago was knowing when to innovate around new ideas and when to just reuse what already works well for products that have been successful. This comes up continuously at MongoDB, because having a good understanding of it is a significant competitive advantage. I attribute a large extent of MongoDB’s success to our unbending adherence to this discipline.

When we started MongoDB, we had a clear goal - make data and databases easier for developers and operators, so that data and databases serve their users, not the other way around. To that end, there were two key things we wanted to change. First, we wanted to reduce the impedance mismatch between application code and data, which we addressed by building our database around collections of documents, rather than tables of rows. Second, we wanted to make distributed systems accessible and usable by most organizations, which we have done by making them first-class components and intuitive to use, instead of leaving them to higher layers of the stack to build.

Everything else we wanted to leave the same. We would ask ourselves, “Does this need to be different in a document based distributed database?” For example, indexes in MongoDB have the same semantics as they do in their relational counterparts, because they have worked quite well for decades, and their semantics match those used to query a document database. Query results composed of documents need the same filtering and/or sorting as those composed of rows, and indexes that cover an ordered set of fields (a, b, c) are redundant with indexes that cover a same-ordered subset of those fields (a, b). MongoDB provides a shell that mimics a relational shell, because in our world, the needs to explore databases and collections, do ad hoc queries, create indexes, get stats, and perform analytics is identical to those needs in a relational world. Even mongodump name came about because I had been using mysqldump for a decade, and making data easy to work with and distributed systems accessible would not be in any way furthered by changing that aspect of a database.

Today, we are constantly improving features and adding new ones to MongoDB. Every time we do, the question is: do we need to invent something new to make this feature fit into the document world or a distributed systems world? If the answer is yes, we innovate to try to make suitable for MongoDB. If not, we try to find the best solution out there and apply it to MongoDB (in a patent safe way of course).

Why is this so important? First off, it takes a lot more though to invent something than to copy something. Taking semantics or ideas from successful systems focuses design and architecture work where it is needed most. Focusing innovation also makes it easier for users to learn a new system. If everything is needlessly different, it will be more frustrating for your users, so there better be a good reason. And lastly, every innovation involves risk. You think you are improving something, but if you’re wrong, you’ve wasted time and have to do it all over again.

This is an important concept for all companies to master, both new and old. Like the adage “Is this a core competency” for helping decide if you should build or buy, all product teams adding features should be asking themselves “Does this need to be different in our domain?”

Comments