JQ functions in Shell Scripting

JQ is a powerful command-line tool for manipulating JSON data in Linux. It provides a wide range of functions that can be used to perform complex operations on JSON data. In this tutorial, we’ll explore some of the most commonly used JQ functions:

The select() Function

The select() function in JQ allows us to filter JSON objects based on a specific criterion. The function takes a filter expression as its argument, which is evaluated against each element in the JSON object. If the filter expression returns a true value, the element is selected and included in the output. If the filter expression returns a false value, the element is excluded from the output.

For example, consider the following JSON object:

[
  {
    "name": "John",
    "age": 25,
    "city": "New York"
  },
  {
    "name": "Jane",
    "age": 30,
    "city": "London"
  },
  {
    "name": "Bob",
    "age": 35,
    "city": "Paris"
  }
]

To select only the objects where the city key is equal to "New York", we can use the following command:

jq 'select(.city == "New York")' file.json

This will output:

{
  "name": "John",
  "age": 25,
  "city": "New York"
}

Note that the select() function returns the entire object that matches the condition inside the parentheses.

The has() Function

The has() function in JQ allows us to check if a key exists in a JSON object. The function takes a key name as its argument and returns a true value if the key exists in the object and a false value if it does not.

For example, consider the following JSON object:

{
  "name": "John",
  "age": 25,
  "city": "New York"
}

To check if the object has a city key, we can use the following command:

jq 'has("city")' file.json

This will output:

true

Note that the has() function returns a boolean value indicating whether the key exists or not.

The empty() Function

The empty() function in JQ allows us to check if a value is empty or null. The function returns a true value if the value is empty or null and a false value if it is not.

For example, consider the following JSON object:

{
  "name": "John",
  "age": 25,
  "city": null
}

To check if the city value is empty or null, we can use the following command:

jq '.city | empty' file.json

This will output:

true

Note that we first use the dot notation to select the city value, and then apply the empty() function to it.

The map() Function

The map() function in JQ allows us to apply a transformation to each element in a JSON array. The function takes a filter expression as its argument, which is evaluated against each element in the array. The output of the expression is then used to replace the original element in the array.

For example, consider the following JSON array:

[
  {
    "name": "John",
    "age": 25
  },
  {
    "name": "Jane",
    "age": 30
  },
  {
    "name": "Bob",
    "age": 35
  }
]

To add a new key-value pair to each object in the array, we can use the following command:

jq 'map(. + {"city": "New York"})' file.json

This will output:

[
  {
    "name": "John",
    "age": 25,
    "city": "New York"
  },
  {
    "name": "Jane",
    "age": 30,
    "city": "New York"
  },
  {
    "name": "Bob",
    "age": 35,
    "city": "New York"
  }
]

Note that the map() function returns a new array with the transformed elements.

The reduce() Function

The reduce() function in JQ allows us to perform an aggregation operation on a JSON array. The function takes a filter expression and an initial value as its arguments. The filter expression is evaluated against each element in the array, and the result is used to update the accumulator value. The final value of the accumulator is returned as the output.

For example, consider the following JSON array:

[1, 2, 3, 4, 5]

To calculate the sum of all the elements in the array, we can use the following command:

jq 'reduce .[] as $item (0; . + $item)' file.json

This will output:

15

Note that the reduce() function returns a single value, which is the sum of all the elements in the array.

The sort() Function

The sort() function in JQ allows us to sort a JSON array based on a specific key or value. The function takes an optional filter expression as its argument, which is used to extract the value that is used for sorting. If no filter expression is provided, the default behavior is to sort the array based on the values of the elements.

For example, consider the following JSON array:

[
  {
    "name": "John",
    "age": 25
  },
  {
    "name": "Jane",
    "age": 30
  },
  {
    "name": "Bob",
    "age": 35
  }
]

To sort the array based on the name key, we can use the following command:

jq 'sort_by(.name)' file.json

This will output:

[
  {
    "name": "Bob",
    "age": 35
  },
  {
    "name": "Jane",
    "age": 30
  },
  {
    "name": "John",
    "age": 25
  }
]

The max() and min() Functions

The max() and min() functions in JQ allow us to find the maximum and minimum values in a JSON array. Both functions take an optional filter expression as their argument, which is used to extract the value that is used for comparison.

For example, consider the following JSON array:

[3, 7, 2, 5, 1, 9]

To find the maximum value in the array, we can use the following command:

jq 'max' file.json

This will output:

9

To find the minimum value in the array, we can use the following command:

jq 'min' file.json

This will output:

1

The length Function

The length function in JQ allows us to find the number of elements in a JSON array. It takes no arguments and returns an integer.

For example, consider the following JSON array:

[1, 2, 3, 4, 5]

To find the number of elements in the array, we can use the following command:

jq 'length' file.json

This will output:

5

The add() Function

The add() function in JQ allows us to concatenate multiple JSON arrays. It takes an array of arrays as its argument and returns a single array containing all the elements from the input arrays.

For example, consider the following two JSON arrays:

[1, 2, 3]
[4, 5, 6]

To concatenate these two arrays, we can use the following command:

python
jq 'add' file.json

This will output:

[1, 2, 3, 4, 5, 6]

Note that the add() function returns a new array containing all the elements from the input arrays.

Also, see the example code shell-scripting-examples in our GitHub repository. See complete examples in our GitHub repositories.

Follow us on social media
Follow Author