Data Mapping

Data Mappers

Data Mappers map data from one schema to another. For flat objects, this allows a simple mapping of properties. For more complex and nested structures, an advanced specification is required.

A Data Mapper uses jsonpath expressions to map from one object format to another. A Data Mapper uses a mapping template to define these expressions. A mapping template can be likened to the “wiring” of a complicated Hi-Fi system or an old fashioned telephone exchange, each property in the target object needs to be specified/available and then the value is “wired-in” for it to recieve its input from a specific place.

JsonPath

JsonPath is a domain specific language which helps us easily locate deeply nested values from within a complex object.

Consider the following example of a book store who also sells software and also has a bicycle for sale:

const store = {
  books: [
    {
      id: 1,
      title: 'Clean Code',
      author: { name: 'Robert C. Martin' },
      price: 17.96,
    },
    {
      id: 2,
      title: 'Maintainable JavaScript',
      author: { name: 'Nicholas C. Zakas' },
      price: 10,
    },
    {
      id: 3,
      title: 'Agile Software Development',
      author: { name: 'Robert C. Martin' },
      editor: { name: 'Yussef Miller' },
      price: 20,
    }
  ],
  bicycle: {
    color: 'red',
    price: 19.95,
  },
  software: [{
    id: 1,
    title: 'jsonpath-mapper',
    author: { name: 'Neil Flatley' },
    cost: 0.0,
  }],
};

To create a normalised representation of the store we can apply the following mapping template to do the transformations and create the object we need for our application to use:

// This is defining the mapping template
const sampleMapping = {
  authors: 'books[*].author.name', // Fetch all 'author.name' values from the books array
  editors: 'books[*].editor.name', // Fetch all 'editor.name' values from the books array
  allNames: 'books[*]..name', // Fetch all '.name' values from any object member inside the books array
  bikeColour: 'bicycle.color',
}

// This is invoking the mapper function, applying our mapping template and returning our new object
const names = mapJson(store, sampleMapping);

console.log(names);

# Object({
#   authors: [
#       "Robert C. Martin",
#       "Nicholas C. Zakas",
#       "Robert C. Martin",
#   ],
#   editors: [
#       "Yussef Miller"
#   ],
#   allNames: [
#       "Robert C. Martin",
#       "Nicholas C. Zakas",
#       "Robert C. Martin",
#       "Yussef Miller"
#   ],
#   bikeColour: "red",
# })

Providing fallback mappings

We can define any jsonpath expression as an array of strings that will attempt to find a value from the provided paths, searching each path sequentially until a value is found at one of the locations.

const imageMapping = {
  imageUri: [
    'overviewImage.asset.sys.uri',
    'overviewImageUri',
    'thumbnailImage.asset.sys.uri',
    'thumbnailImageUri',
  ],
};

We would expect this mapping to return an object with a single property - imageUri, that has a single value - a uri, if a value exists under any of the paths defined in the array:

const testObject = mapJson(entry, imageMapping);

console.log(testObject);

# Object({
#   imageUri: "/path/to/image.png"
# })

Wire functions

Instead of using just Jsonpath to find our value, we can open a function on any property we define to ‘wire’ in our value.

The following example converts an object into an object array.

const input = {"text": "text to analyze"};

const mappingTemplate = `(root) => [{ text: root.text }]`;

const output = mapJson(input, mappingTemplate);

# [{"text": "text to analyze"}]

root represents the original object we are transforming.

Refining values

Instead of defining a property value as a jsonpath expression or a wire function, we can create an object under a given property that can contain a combination of special reserved functions to do a series of operations to find and process our final returned value.

Take the following exampe which is mapping a simple search result from a returned entry. Let’s say we need to truncate our description field to only show the first 1,000 characters, and we also need to append a different host address to our image uri…

const searchResult = {
    title: 'entryTitle',
    description: ['summary', 'leadIn', 'entryDescription'],
    thumbnail: 'image.asset.sys.uri',
    uri: 'sys.uri',
};

const searchResult = {
    title: 'entryTitle',
    description: {
        $path: ['summary', 'leadIn', 'entryDescription'],
        $formatting: description => description && description.substring(0, 1000),
    },
    thumbnail: {
        $path: 'image.asset.sys.uri',
        $formatting: uri => uri ? `https://images.mysite.com/${uri}` : null,
    },
    uri: 'sys.uri',
};

$path: string | Array

You would generally use this in combination with one or more of the other special functions here. To use this functionality in isolation without any other special functions, you would just define a jsonpath expression as a string against the property.

Adding a $path expression to the object will resolve any value(s) found at that location and scope any further processing directly to the found value(s)

$formatting: function

You would always use this in combination with a $path expression. To use this functionality in isolation without a $path we can just use a plain wire function instead.

Adding a $formatting function to the object will take any value(s) found from the $path expression and will allow us to process them with a function that is scoped to these value(s).

$return: function

This does the same job as the $formatting function and really comes into play when $path has returned an array and we need to further process/refine the final returned array, and not the values inside each array item.

$default: function

You can provide a function that returns a default value for the property if all of the above functions return a null or undefined value.

const searchResult = {
    title: 'entryTitle',
    description: ['summary', 'leadIn', 'entryDescription'],
    thumbnail: {
        $path: 'image.asset.sys.uri',
        $formatting: uri => uri ? `https://images.mysite.com/${uri}` : null,
        $default: () => 'https://images.mysite.com/placeholder.png',
    },
    uri: 'sys.uri',
};

$disable: function

You can provide a function which returns a boolean value of whether or not to disable the property from being mapped this iteration.

In the example below instead of returning description: false, we would prefer to just not include this property in the final mapped object. We can create a function that checks the found value and ensure it returns true if the found value is falsy, we will effectively be saying $disable: true.

const searchResult = {
    title: 'entryTitle',
    description: {
        $path: ['summary', 'leadIn', 'entryDescription'],
        $formatting: description => description && description.substring(0, 1000),
        $disable: description => !description
    },
    thumbnail: 'image.asset.sys.uri',
    uri: 'sys.uri',
};