Programming Series with Array and Object Data in Javascript - Handling Plain Data

Problem

In any programming language, whether it's back-end or front-end, data processing is very common. It can involve tasks such as preprocessing before displaying it on the user interface, or smoothing the data and returning it to the client through an API, for example.

Array and Object are two objects that we need to process the most. In this series, I will present to readers some experience in data processing with these two objects.

The powerful trio: map, filter, and reduce

It must be said that the three functions map, filter, and reduce are very helpful. If you still have a vague idea about them, you should take the time to learn about them. I will summarize a little about the usefulness of these three functions.

map function

The map function works with data types being an Array. It takes a function as input and returns an array with the length of the elements always equal to the original data. In essence, we use map when we want to iterate over the elements in the array to add/modify/delete... their data.

The callback of map has the element, index, and array parameters corresponding to the current element, position, and data of the array.

map(function callbackFn(element, index, array) { ... }, thisArg)

map is an implementation of a Functor, if you don't know what a functor is, you can read my article titled What is a Functor? Do I need to know about Functors?.

For example, an array of users with basic information is as follows:

const users = [
  {
    id: 1,  
    name: "Nguyễn Văn A",  
    age: 18,  
    status: "active",  
    city_code: "HN"
  },  
  {
    id: 2,  
    name: "Trần Thị B",  
    age: 20,  
    status: "active",  
    city_code: "HCM"
  },  
  {
    id: 3,  
    name: "Phạm Thị Xuân C",  
    age: 26,  
    status: "block",  
    city_code: "HN"
  },  
]

Increase the age of each person by 1:

const usersIncAge = users.map(item => {
  ...item,  
  age: item.age + 1;
});

Note: In this case, I deliberately create a new object and return it to avoid reference, so after running, there is an array usersIncAge that does not reference users at all.

I have seen people often write:

users.map(item => item.age += 1);

It is much shorter, but unintentionally modifies the data of users, which can lead to data discrepancies if you cannot manage the use of users in other places. So if possible, you should avoid this writing style, instead, create a new array. If you are interested in this issue, read the article Pure Function in Javascript. Why should we know it as soon as possible?.

filter function

filter is used to filter out elements that satisfy the condition in the array. The output of filter is always an array with a length less than or equal to the original data.

The callback function of filter is similar to map, it contains the parameters element, index, and array.

For example, I want to filter out a list of users with age >= 20:

users.filter(item => item.age >= 20);

Filter will use the result of the callback function returning true or false to filter the data, if true is taken, and if false is not. Since filter can only filter elements from the original array, the newly created data from filter will have a reference to the original data, so be careful.

reduce function

Unlike map and filter, the output of reduce is not guaranteed. It can be any data type depending on the purpose.

Reduce will iterate over each element in the array, perform a calculation function, and return a single result.

The callback function of reduce has 4 values accumulator, currentValue, index, array, which are the initial input parameter, the current element, the current element's position, and the initial array respectively.

reduce(function callbackFn(accumulator, currentValue, index, array) { ... }, initialValue)

The theory is verbose, but I will take a simple example first, below is an example using reduce to calculate the sum of the numbers in the array:

const arr = [1, 2, 3, 4];
const sum = arr.reduce((accumulator, currentValue) => accumulator + currentValue, 0);
console.log(sum); // 10

0 is the initial input value corresponding to accumulator, then after each iteration, it performs the operation accumulator + currentValue, the result is assigned back to accumulator. So accumulator is the accumulated value after each iteration and it is also the final result reduce will return.

A more complex example, I don't use filter to filter out a list of users with age >= 20, instead, I will use reduce:

const usersFilter = users.reduce((accumulator, currentValue) => {
  if (currentValue.age >= 20) {
    return [...accumulator, { ...currentValue }];
  } else return accumulator;
}, []);

// or even shorter
const usersFilter = users.reduce((accumulator, currentValue) => currentValue.age >= 20 ? [...accumulator, { ...currentValue }] : accumulator, []);

With this way, I can avoid referencing as when using filter, although it is a bit verbose.

Two levels when using callback function

In the examples, I wrote the callback function directly into the functions, which makes me write code faster but in return, you will have longer code snippets that can sometimes be difficult for the reader to understand.

I call the way of writing as mentioned above as level 1. So how would you write at level 2?

That's how you write with currying, if you don't know about curry, you can refer to the article titled What is a Curry Function? A delicious "curry" dish and how to enjoy it?.

For example, I solve the initial requirement of increasing the age of each user by 1:

const incAge = item => ({...item, age: item.age + 1});
const usersIncAge = users.map(incAge);

I create a function incAge that is responsible for receiving an object, increasing the value of the age property by 1, and then returning a completely new object. Then map accepts it as a callback to process the data. Looking at line 2, you can see the code focuses on what it is doing rather than how it is doing it...

Up to now, the data editing or filtering requirements I presented have been addressed, although it takes a little more time to write code. So let's move on to a more complex requirement: I want to group users by city_code and sort the users by descending age within each group?

I won't implement the code for this requirement because it may be quite long, instead, I propose the following solution: First, get all the city_code values and filter duplicate data (unique), then iterate through each city_code to find the matching users. Finally, iterate through each group to sort the order of users in each group.

That's the solution I thought of, if you have another way, please comment for everyone to know.

So is there a shorter way to solve this problem without needing much code? The answer is yes! Let's use lodash.

Using lodash to speed up programming

For those who don't know, lodash is a very popular data processing library, with over 50k stars on GitHub. This should give you an idea of how popular this library is.

Regarding lodash, it is a collection of utility functions for data processing on arrays, objects, even data types like strings and numbers. It has a lot of functions, you can see them on the Document page.

In lodash, there is a groupBy function for grouping and a sortBy function for sorting data. Applying it to solve my requirement of grouping by city_code:

const _ = require("lodash");
const usersGroupByCity = _.groupBy(users, "city_code");

The result will look like this:

{
  HN: [
    {
      id: 1,  
      name: 'Nguyễn Văn A',  
      age: 18,  
      status: 'active',  
      city_code: 'HN'
    },  
    {
      id: 3,  
      name: 'Phạm Thị Xuân C',  
      age: 26,  
      status: 'block',  
      city_code: 'HN'
    }
  ],  
  HCM: [
    {
      id: 2,  
      name: 'Trần Thị B',  
      age: 20,  
      status: 'active',  
      city_code: 'HCM'
    }
  ]
}

Don't forget we still have one more requirement to sort the data within each group by descending age.
In lodash, there is a function called orderBy that can help me sort the data in ascending or descending order:

const _ = require("lodash");
const usersGroupByCity = _.groupBy(users, "city_code");
const orderByAge = _.mapValues(usersGroupByCity, o => {
  return _.orderBy(o, "age", "desc");
});

mapValues is a function that allows modifying the value of each attribute in the object. Here, I use mapValues to sort the data in each group.

Lodash also supports the chain function, which is a chain of successive functions. The output of this function will be the input of the next function. It can be said that it is like the compose/pipe functions that I have written about in the article Implement more efficiently with compose & pipe functions in Javascript.

const _ = require("lodash");
const orderByAgeDesc = o => _.orderBy(o, "age", "desc");

const result = _(users).groupBy("city_code").mapValues(orderByAgeDesc).value();

Both ways address the problem, however, way 1 focuses on what it is doing while way 2 focuses more on how it is doing it.

Conclusion

Data processing is an integral part of programming, it involves tasks such as adding/modifying/deleting data to serve a specific purpose.

For Array data type, Javascript has the trio map, filter, reduce as powerful assistants in processing. However, there are many other data processing utility functions that can be mentioned such as lodash. Integrating lodash will help reduce the time to rewrite code while also increasing the time to understand the code.