The Document Model

Motivation

• Need for greater scalability that relational databases can easily provide, when it comes to very high write throughput
• Free and open source software FTW
• Specialized queries that SQL may not support well
• SQL is pretty darn restrictive

The one-to-many problem 🤔

• SQL kinda craps out when it comes to one-to-many relationships
• You have one of the following three options:
  • Store the one-to-many relationship in a separate table
  • Use structured data types to represent the one-to-many relationship (e.g., JSON, XML, CSV, etc.)
  • Use a structured datatype and encode it into text

The one-to-many problem

SQL

The one-to-many problem

NoSQL

{
  "id": 1,
  "name": "Ethan",
  "jobs": [
    {
      "id": 1,
      "title": "Software Engineer at Hootsuite"
    },
    {
      "id": 2,
      "title": "Software Engineer at PDT"
    },
    {
      "id": 3,
      "title": "Floor mopper at The Miguel Company"
    }
  ],
  "location": "New York"
}

NoSQL trying to join the cool table

{
  "id": 1,
  "name": "Ethan",
  "jobs": [
    {
      "id": 1,
      "title": "Software Engineer at Hootsuite"
    },
    {
      "id": 2,
      "title": "Software Engineer at PDT"
    },
    {
      "id": 3,
      "title": "Floor mopper at The Miguel Company"
    }
  ],
  "location": 314141414
},
{
  "id": 2,
  "name": "Miguel",
  "jobs": [],
  "location": 314141414
}

{
    "location_id": 314141414,
    "location_name": "Goblins",
}

NoSQL joining the table

// Joins the people table with the locations table
// to find out where Ethan and Miguel live
db.people.find({
  $or: [
    { id: 1 },
    { id: 2 }
  ]
}, {
  _id: 0,
  name: 1,
  location: 1
}).forEach(function(person) {
  db.locations.findOne({
    location_id: person.location
  }, {
    _id: 0,
    location_name: 1
  }, function(location) {
    console.log(person.name + " lives in " + location.location_name);
  });
});

$ node index.js
Ethan lives in Goblins
Miguel lives in Goblins

NoSQL vs SQL - Fight! 🤜 🤛

• Pros for Document Database:
  • Schema flexibility
  • Better performance due to locality
  • Closer to the data structures the used by the application

NoSQL vs SQL - Fight! 🤜 🤛

• Pros for Document Database:
  • Schema flexibility
  • Better performance due to locality
  • Closer to the data structures the used by the application
• Pros for Relational Database:
  • Better support for joins
  • Many-to-many and many-to-one relationships are better supported

NoSQL vs SQL - Fight! 🤜 🤛

What data model leads to simpler application code?

• If your data has a document like structure (with one-to-many relationships), then most likely document database!
• If your data has a relational structure (with many-to-many relationships), then most likely relational database!

NoSQL vs SQL - Fight! 🤜 🤛

Schema Flexibility

• Document databases are schema-on-read:
  • The data assumes the structure of the schema, when it is read
  • The schema is not enforced
• Relational databases are schema-on-write:
  • The schema is enforced
  • The data is assumed to have the structure of the schema when it is written to the database

NoSQL vs SQL - Fight! 🤜 🤛

Schema Flexibility

How to change the schema

• SQL:

  --- Migrate the people table to have a first and last name
  --- Then move the split the old name column into first and last name columns
  ALTER TABLE people
  ADD COLUMN first_name VARCHAR(255)
  ADD COLUMN last_name VARCHAR(255);
  --- Split the name column into first and last name columns
  UPDATE people
  SET first_name = SUBSTRING(name, 1, LOCATE(' ', name) - 1),
  last_name = SUBSTRING(name, LOCATE(' ', name) + 1);
  --- Drop the old name column
  ALTER TABLE people
  DROP COLUMN name;

NoSQL vs SQL - Fight! 🤜 🤛

Schema Flexibility

How to change the schema

• NoSQL:

// If the user doesn't have a `first_name` or `last_name` field,
// then we can add them by splitting the `name` field
if (!person.first_name || !person.last_name) {
  person.first_name = person.name.split(' ')[0];
  person.last_name = person.name.split(' ')[1];
}
// Then we can assume the user has a `first_name` and `last_name` field...

NoSQL vs SQL - Fight! 🤜 🤛

Schema Flexibility

And what if not all documents have the same structure? 😱

NoSQL vs SQL - Fight! 🤜 🤛

Data locality

• In the Document model, all the data is stored in the same document, so there is an advantage of loading all of it up
• In the Relational model, the data could be split across multiple tables, requiring multiple index lookups
• That said, newer technologies have done wonders..

Declarative vs Imperative Query Languages

• Declarative languages hide the implementation details of the query, and describe only the data that it needs
• Imperative languages describe the steps needed to retrieve the data

Declarative vs Imperative Query Languages

SQL

--- We join the location table to get the people
--- who live in the Goblins
SELECT people.first_name
FROM people
JOIN locations
ON people.location = locations.location_id
WHERE locations.location_name = 'Goblins';

JavaScript

// we have to look at the people array and
// location array to get the people that live in Goblins
const result = [];
for (const person of people) {
  for (const location of locations) {
    if (person.location === location.location_id &&
     location.location_name === 'Goblins') {
      result.push(person.first_name);
    }
  }
}

Declarative vs Imperative Query Languages

MapReduce

• We'll talk about MapReduce later (psst Batch Processing!)
• It's not either of the two, but it's a nice abstraction.
• Consumers write two functions, a map function and a reduce function. The map function is called once for each document in the collection. The reduce function is called once for each key in the collection.

Declarative vs Imperative Query Languages

// MongoDB mapReduce example
// to emit the number of people in each location
const res = db.people.mapReduce(
  // The map function
  function() {
    emit(this.location, 1);
  },
  // The reduce function
  function(key, values) {
    return Array.sum(values);
  },
  {
    out: 'people_by_location'
  }
);
// output the results
res.find().forEach(function(doc) {
    const location = db.locations.findOne({
        location_id: doc.key
    });
  console.log(doc.value + " people live in " + location.location_name);
});

$ node index.js
2 people live in Goblins

ENTER GRAPH DATA MODEL 📈 📈 📈

• Graph data models are a way to represent relationships between data
• Graph data models are great for modeling very connected data
• In graph data models, relationships are represented by edges and nodes represent entities
• Graph data models are often used to represent social networks

ENTER GRAPH DATA MODEL 📈 📈 📈

// dgraph query to get the people and their locations (Looks like GraphQL!)
const query = `
query {
  people(func: has(location)) {
    name
    location {
      location_name
    }
  }
}
`;
const res = await dgraph.newTxn().query(query);
console.log(res);

ENTER GRAPH DATA MODEL 📈 📈 📈

$ node index.js
{
  "people": [
    {
      "name": "Ethan",
      "location": {
        "location_name": "Goblins"
      }
    },
    {
      "name": "Miguel",
      "location": {
        "location_name": "Goblins"
      }
    }
  ]
}

Summary

• Deciding on a data model is a big decision
• It's important to understand the data models you can choose from
• There are a few models that are great for different types of data
• There are many many models out there, the following are the most common nowadays
  • Relational data models
  • Document data models
  • Graph data models
• The query language matters too, and there are two main types:
  • Declarative query languages
  • Imperative query languages

PS I didn't cover those:

• Network data models
• The history of it all
• RDF data models
• SPARQL
• Cypher query languages
• Datalog and functional programming (almost)