39

I want to deserialize the chemical elements JSON file from Bowserinator on github using Serde. For this I created a structure with all the needed fields and derived the needed macros:

#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct Element {
    name: String,
    appearance: String,
    atomic_mass: f64,
    boil: f64, 
    category: String,
    #[serde(default)]
    color: String,
    density: f64,
    discovered_by: String,
    melt: f64, 
    #[serde(default)]
    molar_heat: f64,
    named_by: String,
    number: String,
    period: u32,
    phase: String,
    source: String,
    spectral_img: String,
    summary: String,
    symbol: String,
    xpos: u32,
    ypos: u32,
}

This works fine until it gets to fields which contain a "null" value. E.g. for the field "color": null, in Helium.

The error message I get is { code: Message("invalid type: unit value, expected a string"), line: 8, column: 17 } for this field.

I experimented with the #[serde(default)] Macro. But this only works when the field is missing in the JSON file, not when there is a null value.

I like to do the deserialization with the standard macros avoiding to program a Visitor Trait. Is there a trick I miss?

Improve this question
4
  • 1
    It is strongly recommended that you read The Rust Programming Language, which covers the concept of Option and Result, which are very pervasive in Rust. Commented May 26, 2017 at 16:32
  • 1
    I already did this, but a hint would be helpful how to handle this case, as it seems I need to think a little bit different than I expected. As I said above my assumption is that I need to implement the Visitor Trait and I wanted to avoid that. As I said below: I also wanted to avoid to parse all the read Structures a second time and hoped that Serde has some kind of magic to help. Commented May 26, 2017 at 17:18
  • Your question would be clearer if you provided a minimal reproducible example. As-is, you've provided code and the input, but not what output you want. As you can see, the ambiguity you've presented has resulted in two wildly different answers. Commented May 27, 2017 at 0:31
  • ok, thanks, I will do this the next time. Commented May 30, 2017 at 11:00

4 Answers 4

Reset to default
56

A deserialization error occurs because the struct definition is incompatible with the incoming objects: the color field can also be null, as well as a string, yet giving this field the type String forces your program to always expect a string. This is the default behaviour, which makes sense. Be reminded that String (or other containers such as Box) are not "nullable" in Rust. As for a null value not triggering the default value instead, that is just how Serde works: if the object field wasn't there, it would work because you have added the default field attribute. On the other hand, a field "color" with the value null is not equivalent to no field at all.

One way to solve this is to adjust our application's specification to accept null | string, as specified by @user25064's answer:

#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct Element {
    color: Option<String>,
}

Playground with minimal example

Another way is to write our own deserialization routine for the field, which will accept null and turn it to something else of type String. This can be done with the attribute #[serde(deserialize_with=...)].

#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct Element {
    #[serde(deserialize_with="parse_color")]
    color: String,
}

fn parse_color<'de, D>(d: D) -> Result<String, D::Error> where D: Deserializer<'de> {
    Deserialize::deserialize(d)
        .map(|x: Option<_>| {
            x.unwrap_or("black".to_string())
        })
}

Playground

See also:

Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

Thank you, especially for the explanation. I think I will go with the second way, so I can avoid a translation class (from a structure with Option<T> to the structure I like to work with)
I really wish Serde would handle this in a better way. null is not a valid value in Rust but is a valid value in a JSON, thus Serde shall just implement the basic JSON standards. The solutions at the moment are all verbose and sub-optimals. Either using Option<...> for every field (and then unwrap_or_else for every field, huh...), or append a #[serde(deserialize_with="...")] to every field, which seems actually maybe better
8

Any field that can be null should be an Option type so that you can handle the null case. Something like this?

#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct Element {
    ...
    color: Option<String>,
    ...
}
Improve this answer

1 Comment

I hoped there is some trick to automate this conversion. I wanted to avoid to parse the Element struct a second time after it came back from the Serde parser and repair all the null values myself.
7

Based on code from here, when one needs default values to be deserialized if null is present.

// Omitting other derives, for brevity 
#[derive(Deserialize)]
struct Foo {
   #[serde(deserialize_with = "deserialize_null_default")]
   value: String, 
}

fn deserialize_null_default<'de, D, T>(deserializer: D) -> Result<T, D::Error>
where
    T: Default + Deserialize<'de>,
    D: Deserializer<'de>,
{
    let opt = Option::deserialize(deserializer)?;
    Ok(opt.unwrap_or_default())
}

playground link with full example. This also works for Vec and HashMap.

Improve this answer

Comments

1

https://crates.io/crates/serde_with has a helper that can deserialize null into a default value https://docs.rs/serde_with/3.4.0/serde_with/struct.DefaultOnNull.html

#[serde_as]
#[derive(Serialize, Deserialize)]
pub struct Element {
    #[serde_as(as = "DefaultOnNull")]
    color: Option<String>
}
Improve this answer

1 Comment

This should be the accepted answer. The idea that it "obviously" shouldn't accept null as being default, despite the #[serde(default)], is beyond me.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.