Back to: Linux
Linux shell scripting is a powerful tool that allows users to automate tasks and process data in a simple and efficient way. One of the most useful tools in shell scripting is JQ, a command-line JSON processor that allows users to manipulate and extract data from JSON files.
In this tutorial, we will explore how to use JQ in Linux shell scripting and some of its most useful features.
Installing JQ
Before we can start using JQ, we need to install it on our system. JQ is available for Linux, macOS, and Windows, and can be installed using various package managers.
On Ubuntu or Debian-based systems, you can install JQ using the following command:
sudo apt-get install jq
On CentOS or Fedora-based systems, you can install JQ using the following command:
sudo yum install jq
If you are using a different distribution or operating system, refer to the JQ installation documentation for instructions on how to install it.
Basic JQ Usage
Once JQ is installed, we can start using it to process JSON files. JQ provides a wide range of features, but we will start with some basic usage.
Consider the following JSON file named data.json:
{
"name": "John Doe",
"age": 30,
"address": {
"city": "New York",
"state": "NY"
},
"friends": [
{
"name": "Alice",
"age": 28
},
{
"name": "Bob",
"age": 32
}
]
}
To extract the value of a specific field from this JSON file, we can use the following command:
jq '.name' data.json
This command will output the value of the name field:
"John Doe"
We can also extract nested fields by specifying the field hierarchy. For example, to extract the value of the state field from the address field, we can use the following command:
jq '.address.state' data.json
This command will output the value of the state field:
"NY"
JQ also allows us to filter JSON objects based on specific conditions. For example, to extract only the friends whose age is greater than 30, we can use the following command:
jq '.friends[] | select(.age > 30)' data.json
This command will output the following JSON object:
{
"name": "Bob",
"age": 32
}
Advanced JQ Usage
JQ provides a wide range of features for manipulating and processing JSON data. Here are some of the most useful advanced features.
1. Selecting Specific Fields
JQ provides a simple way to select specific fields from a JSON file using the map function. Consider the following JSON file named data.json:
[
{
"name": "John Doe",
"age": 30,
"address": {
"city": "New York",
"state": "NY"
},
"friends": [
{
"name": "Alice",
"age": 28
},
{
"name": "Bob",
"age": 32
}
]
},
{
"name": "Jane Smith",
"age": 35,
"address": {
"city": "Los Angeles",
"state": "CA"
},
"friends": [
{
"name": "Chris",
"age": 36
},
{
"name": "David",
"age": 34
}
]
}
]
To select specific fields from this JSON file, we can use the map function as follows:
jq 'map({name: .name, age: .age})' data.json
This command will output the following JSON object:
[
{
"name": "John Doe",
"age": 30
},
{
"name": "Jane Smith",
"age": 35
}
]
2. Sorting JSON Data
JQ also allows us to sort JSON data using the sort_by function. Consider the following command:
jq 'sort_by(.age)[]' data.json
This command will sort the JSON data by the age field and output the following JSON objects in ascending order of age:
{
"name": "Alice",
"age": 28
}
{
"name": "John Doe",
"age": 30
}
{
"name": "David",
"age": 34
}
{
"name": "Bob",
"age": 32
}
{
"name": "Jane Smith",
"age": 35
}
{
"name": "Chris",
"age": 36
}
Working with Arrays
Arrays are a common data structure in JSON, and JQ provides several functions for working with arrays. For example, to select the first element of an array, we can use the .[0] syntax. To select the last element, we can use the .[length-1] syntax. Consider the following JSON data:
{
"users": [
{
"name": "John",
"age": 25
},
{
"name": "Mary",
"age": 30
},
{
"name": "David",
"age": 28
}
]
}
To select the first user, we can use the following command:
jq '.users[0]' data.json
This will output the following JSON object:
{
"name": "John",
"age": 25
}
To select the last user, we can use the following command:
jq '.users[.users | length-1]' data.json
This will output the following JSON object:
{
"name": "David",
"age": 28
}
Advanced Filtering
JQ provides powerful filtering capabilities for selecting and filtering JSON data based on complex conditions. For example, consider the following JSON data:
{
"users": [
{
"name": "John",
"age": 25,
"gender": "male"
},
{
"name": "Mary",
"age": 30,
"gender": "female"
},
{
"name": "David",
"age": 28,
"gender": "male"
}
]
}
To select all the male users, we can use the following command:
jq '.users[] | select(.gender == "male")' data.json
This will output the following JSON objects:
{
"name": "John",
"age": 25,
"gender": "male"
}
{
"name": "David",
"age": 28,
"gender": "male"
}
To select all the female users who are older than 25, we can use the following command:
jq '.users[] | select(.gender == "female" and .age > 25)' data.json
This will output the following JSON object:
{
"name": "Mary",
"age": 30,
"gender": "female"
}
Also, see the example code shell-scripting-examples in our GitHub repository. See complete examples in our GitHub repositories.
Follow us on social media
Follow Author