How to Format JSON in Bash

How to Format JSON in Bash

Formatting JSON data is a crucial step in many development workflows, especially when working with APIs, data exchange, or debugging. In Bash, formatting JSON can be achieved using the jq command-line JSON processor. In this guide, we will explore how to format JSON in Bash using jq, covering the basics, edge cases, common mistakes, and performance tips.

Installation

Before we begin, make sure you have jq installed on your system. You can install it using your package manager:

sudo apt-get install jq  # Ubuntu/Debian
brew install jq          # macOS (Homebrew)

Quick Example

Here's a minimal example that formats a JSON string:

json_string='{"name":"John","age":30,"city":"New York"}'
formatted_json=$(jq '.' <<< "$json_string")
echo "$formatted_json"

This code uses the jq command to parse the JSON string and pretty-print it with indentation.

Step-by-Step Breakdown

Let's break down the code:

• json_string: a variable containing the JSON string to be formatted.
• jq '.': the jq command with the . filter, which simply outputs the input JSON.
• <<< "$json_string": a here-string that feeds the JSON string to jq as input.
• formatted_json=$(...): captures the output of jq in a variable.
• echo "$formatted_json": prints the formatted JSON string.

Handling Edge Cases

Empty/Null Input

If the input JSON string is empty or null, jq will output an error message. To handle this, you can add a simple check:

if [ -n "$json_string" ]; then
  formatted_json=$(jq '.' <<< "$json_string")
  echo "$formatted_json"
else
  echo "Error: Empty input"
fi

Invalid Input

If the input JSON string is invalid (e.g., malformed syntax), jq will output an error message. You can use the --exit-status option to exit with a non-zero status code:

if jq --exit-status '.' <<< "$json_string" > /dev/null; then
  formatted_json=$(jq '.' <<< "$json_string")
  echo "$formatted_json"
else
  echo "Error: Invalid input"
fi

Large Input

When working with large JSON files, you may encounter performance issues. To improve performance, you can use the jq option --stream to process the input in chunks:

jq --stream '.' large_json_file.json

Unicode/Special Characters

By default, jq outputs JSON strings in UTF-8 encoding. If you need to handle special characters or non-ASCII characters, you can use the --ascii-output option:

jq --ascii-output '.' input_json.json

Common Mistakes

1. Forgetting to quote the JSON string

Wrong code:

formatted_json=$(jq '.' $json_string)

Corrected code:

formatted_json=$(jq '.' <<< "$json_string")

2. Not handling empty/null input

Wrong code:

formatted_json=$(jq '.' <<< "$json_string")

Corrected code:

if [ -n "$json_string" ]; then
  formatted_json=$(jq '.' <<< "$json_string")
  echo "$formatted_json"
else
  echo "Error: Empty input"
fi

3. Not checking for invalid input

Wrong code:

formatted_json=$(jq '.' <<< "$json_string")

Corrected code:

if jq --exit-status '.' <<< "$json_string" > /dev/null; then
  formatted_json=$(jq '.' <<< "$json_string")
  echo "$formatted_json"
else
  echo "Error: Invalid input"
fi

Performance Tips

1. Use --stream for large inputs

When working with large JSON files, use the --stream option to process the input in chunks.

2. Use --ascii-output for special characters

If you need to handle special characters or non-ASCII characters, use the --ascii-output option.

3. Avoid unnecessary piping

Avoid piping the output of jq to another command unless necessary, as this can introduce additional overhead.

FAQ

Q: What is the difference between jq and jsonlint?

A: jq is a JSON processor that can format, parse, and transform JSON data, while jsonlint is a JSON validator that checks the syntax and structure of JSON data.

Q: Can I use jq with JSON files?

A: Yes, you can use jq with JSON files by specifying the file path as an argument.

Q: How can I pretty-print JSON data with jq?

A: Use the . filter with jq to pretty-print JSON data.

Q: Can I use jq with non-ASCII characters?

A: Yes, jq supports Unicode characters and can handle non-ASCII characters.

Q: What is the purpose of the --exit-status option?

A: The --exit-status option tells jq to exit with a non-zero status code if the input JSON is invalid.