Parsing JSON in Rust easily explained

2021-10-20
Rust JSON

Parsing JSON in a proper way requires representing the JSON data in a structured format so that the language can find out which fields have to be parsed and which type they are. The Rust way of representing the structured data are so called structs . We could make appropriate structs manually, but thanks to the great online converter, transform.tools , we can generate the structs very easily. In this article we will parse data from songsterr.com , a web site with guitar tabs. Let’s make the request to their API. Just click here and look at the data in your browser. Scarry? Not really, just copy all data and paste here at the left hand side box. Rust structs should appear on the right side almost immediately.

use serde_derive::Deserialize;
use serde_derive::Serialize;

pub type Root = Vec<Root2>;

#[derive(Default, Debug, Clone, PartialEq, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct Root2 {
    pub id: i64,
    #[serde(rename = "type")]
    pub type_field: String,
    pub title: String,
    pub artist: Artist,
    pub chords_present: bool,
    pub tab_types: Vec<String>,
}

#[derive(Default, Debug, Clone, PartialEq, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct Artist {
    pub id: i64,
    #[serde(rename = "type")]
    pub type_field: String,
    pub name_without_the_prefix: String,
    pub use_the_prefix: bool,
    pub name: String,
}

What can we see from here? First, the root of the JSON body is an array or vector in Rust. This array contains many objects of type Root2 which is actually a song and it contains an Artist object underneath. Let’s rename the objects in the following way:

use serde_derive::Deserialize;
use serde_derive::Serialize;

pub type Response = Vec<Song>;

#[derive(Default, Debug, Clone, PartialEq, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct Song {
    pub id: i64,
    #[serde(rename = "type")]
    pub type_field: String,
    pub title: String,
    pub artist: Artist,
    pub chords_present: bool,
    pub tab_types: Vec<String>,
}

#[derive(Default, Debug, Clone, PartialEq, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct Artist {
    pub id: i64,
    #[serde(rename = "type")]
    pub type_field: String,
    pub name_without_the_prefix: String,
    pub use_the_prefix: bool,
    pub name: String,
}

It already looks better. The idea behind this is that we now make the same query in Rust, fetch the date, parse it using these structs and display for example song titles, artist names and tabs types for each of them. Let’s start from scratch. To create a project in Rust, just type cargo new tabs and open this directory in your favorite IDE. Open src/main.rs and copy previous code block except the first two (use) lines. You can paste the content bellow the main function which was generated by cargo. By the way, cargo is a powerful dependencies management tool for go and it should be installed all together with rust if you follow these instructions . For the next step we will need cargo-edit plugin to be able to easily add dependencies to our project. Just type cargo install cargo-edit and then type cargo add serde and cargo add serde_json. serde is famous Rust library for serializing and deserializing data and serde_json is it’s partner library to do this for JSON format. These two commands will just add these two dependencies in Cargo.toml file. Open the file and modify the line which imports serde to look like following:

serde = { version = "1.0.130", features = ["derive"] }

Of course, you can keep the version previously defined by cargo if it’s differ from the version at the time of writing this article. Now type cargo run and if everything compiles correctly, you should see Hello, world! which was also generated by cargo. At this point it is important that you do not get any compiler errors. If so, we can continue. If not, just copy the following block over your main.rs contents and the next block over your Cargo.toml.

use serde::{Deserialize,Serialize};

fn main() {
    println!("Hello, world!");
}

pub type Response = Vec<Song>;

#[derive(Default, Debug, Clone, PartialEq, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct Song {
    pub id: i64,
    #[serde(rename = "type")]
    pub type_field: String,
    pub title: String,
    pub artist: Artist,
    pub chords_present: bool,
    pub tab_types: Vec<String>,
}

#[derive(Default, Debug, Clone, PartialEq, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct Artist {
    pub id: i64,
    #[serde(rename = "type")]
    pub type_field: String,
    pub name_without_the_prefix: String,
    pub use_the_prefix: bool,
    pub name: String,
}
[package]
name = "tabs"
version = "0.1.0"
edition = "2018"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
serde = { version = "1.0.130", features = ["derive"] }
serde_json = "1.0.68"

In order to parse JSON, we have to fetch JSON contents first and we will do this using reqwest , another famous Rust library. Type cargo add reqwest and modify the new entry in Cargo.toml to look like this:

reqwest = { version = "0.11.6", features = ["blocking", "json"] }

Again, if you are reading this when a new version is present, just keep the previously set version and add the features part.

In order that we can use reqwest and make real HTTP request to fetch the data, we have to add new use statement at the beginning of main.rs:

use reqwest::blocking::get;

Now we can fetch the data just using one line of code. Delete the content of the main function and put the following:

  let res = get("https://www.songsterr.com/a/ra/songs.json?pattern=Beatles").unwrap();
  let songs = res.json::<Response>().unwrap();
  for song in songs {
      println!("Song: {} Artist: {} Tab types: {:?}", song.title, song.artist.name, song.tab_types);
  }

Run your program by cargo run and voila, you should see lot of data on your screen. But let’s explain now what is going on here. First we make a get request to fetch the data and resolve the result by unwrap() method. Take care that this is not proper error handling and that you should handle errors much better in a real program. But for the simplicity, I did it this way. So, unwrap() will panic if something goes wrong (i.e. remote API not available).

In the next line we parse JSON using generic json<T> method by telling that we want to use our Response struct to collect the data. Again, we unwrap to get the result (assign the songs variable) and panic in case that parsing fails.

Finally, we loop over songs vector (array) and display some data that we need. {} is a placeholder in Rust to display basic types like strings and numbers and {:?} is a placeholder do display (actually debug) the more complex data like vectors, structs, enums, etc. Take care that we used println!() and not println(). When you see an exclamation mark at the end of some function, this means that you are actually using a macro. And macro means that the real code for this will be generated by compiler at the compilation time. But this is quite complex topic for this article, we will discuss it at some later point.

If you have any trouble running this, be sure that the contents of main.rs and Cargo.toml are the following:

use serde::{Deserialize,Serialize};
use reqwest::blocking::get;

fn main() {
  let res = get("https://www.songsterr.com/a/ra/songs.json?pattern=Beatles").unwrap();
  let songs = res.json::<Response>().unwrap();
  for song in songs {
      println!("Song: {} Artist: {} Tab types: {:?}", song.title, song.artist.name, song.tab_types);
  }
}

pub type Response = Vec<Song>;

#[derive(Default, Debug, Clone, PartialEq, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct Song {
    pub id: i64,
    #[serde(rename = "type")]
    pub type_field: String,
    pub title: String,
    pub artist: Artist,
    pub chords_present: bool,
    pub tab_types: Vec<String>,
}

#[derive(Default, Debug, Clone, PartialEq, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct Artist {
    pub id: i64,
    #[serde(rename = "type")]
    pub type_field: String,
    pub name_without_the_prefix: String,
    pub use_the_prefix: bool,
    pub name: String,
}
[package]
name = "tabs"
version = "0.1.0"
edition = "2018"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
reqwest = { version = "0.11.6", features = ["blocking", "json"] }
serde = { version = "1.0.130", features = ["derive"] }
serde_json = "1.0.68"

And that’s all, happy rusting. 🙂

Parsing custom datetime format in Rust using serde

Parsing standard datetime format is no hassle, but what to do when you have custom datetime format?
Rust

A first impression of Rust from the perspective of a Go developer

Rust is very powerful, but let's see how it compares to Go
Go Rust Kubernetes controller secret replicator