Basic JData Examples

1. Constants and scalars
2. Strings
3. Structures
4. 1-D row vectors
5. 1-D column vectors
6. 2-D matrices
7. 2-D arrays in the annotated format
8. 3-D and higher dimensional arrays
9. Array data with compression

The goal of this section is to provide you a set of easy-to-understand examples on how to use JData constructs to represent real-world data. Please note that

  1. JData constructs use the "name":value pair sequences as the fundamental building blocks of the data representation, which is 100% compatible with the JSON and UBJSON (for UBJSON, an extended N-D array container can be optionally used, see item#5 below) specifications in syntax
  2. In the provided text-JData/JSON examples, all white spaces (newlines, tabs, spaces etc) are optional, and have no effect to the data according to the JSON specification.
  3. In the provided binary-JData samples (last column), all white spaces and square-brackets enclosing each data element are shown only for formatting purposes, and MUST BE REMOVED when storing the data to file
  4. The light-green-shaded data snippets indicate the use of JData-defined annotations, but the outcome data files are 100% compatible with existing JSON and UBJSON parsers
  5. The light-blue-shaded data snippets indicate the use of JData-extended optimized array header, and such data must be parsed using JData-compliant UBJSON parsers, but may produce an error if your parser does not support this extended syntax

Please also checkout the Advanced Examples next.

1. Constants and scalars

Numerical values are directly supported by either JSON or UBJSON specifications. A numerical value is typically unchanged when converting to the JData annotation. When storing as files, they are directly stored in the JSON/UBJSON numerical value forms. For example

Native data text-JData/JSON form binary-JData(UBJSON)
a=3
=>
{"a":3}
[{] [U][1][a][U][3] [}]

Similar for a floating point number

Native data text-JData/JSON form binary-JData(UBJSON) †
a=3.14159
=>
{"a":3.14159}
[{] [U][1][a][D][3.15169] [}]
† In the case of the binary-JData, the type marker of the floating point number, here [D] must match the native data's type.

There are a few special constants, namely "NaN", "Infinity" and "-Infinity", they are encoded as special string keywords when stored in the JSON/text-JData formats, but stay unchanged when stored in the binary JData format

Native data text-JData/JSON form binary-JData(UBJSON) †
a=nan
=>

{"a":"_NaN_"}

[{] [U][1][a][D][nan] [}]
a=inf
=>

{"a":"_Inf_"}

[{] [U][1][a][D][inf] [}]
a=-inf
=>

{"a":"-_Inf_"}

[{] [U][1][D][-inf] [}]

Three special constants are directly supported by the JSON/UBJSON specifications, these constants are logical "True", logical "False" and "Null", they are stored as

Native data text-JData/JSON form binary-JData(UBJSON)
a=true
=>
{"a":true}
[{] [U][1][a][T] [}]
a=false
=>
{"a":false}
[{] [U][1][a][F] [}]
a=null
=>
{"a":null}
[{] [U][1][a][Z] [}]

2. Strings

Strings are natively supported in both JSON and UBJSON, therefore, when converting to the JData annotations, they are mostly unchanged. For JSON strings, 9 special characters can be escaped to represent control sequences (\",\\,\/,\b,\f,\n,\r,\t,\uXXXX). For example

Native data text-JData/JSON form binary-JData(UBJSON)
a="JData will prevail"
=>
{"a":"JData will prevail"}
[{] [U][1][a][S][U][18][JData will prevail] [}]
a="Tabs\tNewlines\n"
=>
{"a":"Tabs\tNewlines\n"}
[{] [U][1][a][S][U][14][Tabs\tNewlines\n] [}]
a="中文UTF-8格式"
=>
{"a":"中文UTF-8格式"}
[{] [U][1][a][S][U][17][中文UTF-8格式] [}]

We want to mention that in JData, "non-strict" JSON string mode is allowed: that means one can store ASCII values 0-31 as part of the string without escaping.

3. Structures

Hierarchical structures are often needed when representing metadata or simple lists with named-members. Because "structure" data-type can be directly mapped to the "object" construct in JSON and UBJSON, therefore, they do not need to be converted when using the JData annotation.

Native data text-JData/JSON form binary-JData(UBJSON)

a=struct(
'i1',1,
'f1',2.0,
's1':"string"
)

=>

{
"a":{
"i1":1,
"f1":2.0,
"s1":"string"
}}

[{]
[U][1][a]
[{]
[U][2][i1][U][1]
[U][2][f1][D][2.0]
[U][2][s1][S][6][string]
[}]
[}]

4. 1-D row vectors

Simple 1-dimensional vectors are supported in both JSON and UBJSON using the "array" construct. For example

Native data text-JData/JSON form binary-JData(UBJSON)
a=[1,2,3]
=>
{"a":[1,2,3]}

[{]
[U][1][a]
[[]
[U][1][U][2][U][3]
[]]
[}]

when array element have the same type, binary JData/UBJSON
can use the "optimized" array header to save space. For the array
above, one can write in the right form, where [$][U] indicates array
element type is "unsigned byte" [U] and [#][U][3] indicates that
there are 3 elements

[{]
[U][1][a]
[[] [$][U] [#][U][3]
[1][2][3]
[}]

a=[2.0,"str",nan]
=>

{"a":[2.0,"str","_NaN_"]}

[{]
[U][1][a]
[[]
[D][2.0]
[S][U][3][str]
[D][nan]
[]]
[}]

5. 1-D column vectors

To distinguish column from row vectors, we use embedded array constructs in JSON/UBJSON to represent a column vector

Native data text-JData/JSON form binary-JData(UBJSON)
a=[[1],[2],[3]]
=>
{"a":[[1],[2],[3]]}

[{]
[U][1][a]
[[]
[[] [U][1] []]
[[] [U][2] []]
[[] [U][3] []]
[]]
[}]

a=[[2.0],["str"],[nan]]
=>

{"a":[[2.0],["str"],["_NaN_"]]}

[{]
[U][1][a]
[[]
[[] [D][2.0] []]
[[] [S][U][3][str] []]
[[] [D][nan] []]
[]]
[}]

6. 2-D matrices

Simple 1-dimensional vectors are supported in both JSON and UBJSON using the "array" construct. For example

Native data text-JData/JSON form binary-JData(UBJSON)

a=[
1,2,3
4,5,6
]

=>

{
"a":[
[1,2,3],
[4,5,6]
]
}

[{]
[U][1][a]
[[]
[[] [U][1][U][2][U][3] []]
[[] [U][4][U][5][U][6] []]
[]]
[}]

similar to the 1-D raow vector example above, we can
use the type [$] and count [#] markers to simplify
this array in the binary form

[{]
[U][1][a]
[[]
[[] [$][U] [#][U][3] [1][2][3]
[[] [$][U] [#][U][3] [4][5][6]
[]]
[}]

to simplify this further, in the JData Specification,
we further extended UBJSON array count marker [#] to
accept a 1-D array count-type, representing the dimension
vector of an N-D array, in this case [2,3] for a 2x3 matrix

[{]
[U][1][a]
[[] [$][U] [#][[] [$][U][#][2] [2][3]
[1][2][3][4][5][6]
[}]

7. 2-D arrays in the annotated format

In JData specification, we introduced a light-weight data annotation approach to allow one to specify additional information, such as data type, data size and compression, in the stored data record. This is achieved using a "structure-like" data container (a structure is supported in almost all programming language) with JData-specified human-readable subfield keywords. This construct is also easily serialized using many of the existing JSON/UBJSON libraries.

For example, the above 2-D array can be alternatively stored using the annotated format to allow fine-grained data storage

Native data text-JData/JSON form binary-JData(UBJSON)

a=[
1,2,3
4,5,6
]

=>

{
"a":{
"_ArrayType_":"uint8",
"_ArraySize_":[2,3],
"_ArrayData_":[1,2,3,4,5,6]
}
}

[{]
[U][1][a]
[{]
[U][11][_ArrayType_] [S][U][5][uint8]
[U][11][_ArraySize_] [[] [U][2][U][3] []]
[U][11][_ArrayData_] [[] [$][U][#][6] [1][2][3][4][5][6]
[}]
[}]

Please note that the binary JData format already has built-in support to strongly-typed binary data, therefore, it is not very beneficial, despite it is permitted, to use the above annotated format compared to the packed-array format with optimized header shown in the above section.

8. 3-D and higher dimensional arrays

One can use either the direct-format or annotated-format for storing higher dimensional arrays, as natively supported by both JSON/UBJSON, but the benefit of using the annotated format for text-based JData, and the packed-array optimized format for binary-JData becomes more advantageous due to faster processing speed.

Native datatext-JData/JSON formbinary-JData(UBJSON)
a=[
      [
          [1,9,6,0],
          [2,9,3,1],
          [8,0,9,6]
      ],
      [
          [6,4,2,7],
          [8,5,1,2],
          [3,3,2,6]
      ]
  ]
=>
{
  "a":[
        [
          [1,9,6,0],
          [2,9,3,1],
          [8,0,9,6]
        ],
        [
          [6,4,2,7],
          [8,5,1,2],
          [3,3,2,6]
        ]
    ]
}
[{]
 [U][1][a]
 [[]
   [[]
     [[] [U][1][U][9][U][6][u][0] []]
     [[] [U][2][U][9][U][3][u][1] []]
     [[] [U][8][U][0][U][9][u][6] []]
   []]
   [[]
     [[] [U][6][U][4][U][2][u][7] []]
     [[] [U][8][U][5][U][1][u][2] []]
     [[] [U][3][U][3][U][2][u][6] []]
   []]
 []]
[}]
More efficient alternative formats using JData annotations
{
  "a":{
        "_ArrayType_":"uint8",
        "_ArraySize_":[2,3,4],
        "_ArrayType_":[1,9,6,0,2,9,3,1,8,0,9,6,6,4,2,
                       7,8,5,1,2,3,3,2,6]
  }
}
[{]
 [U][1][a]
 [[] [$][U] [#][[] [$][U][#][3] [2][3][4]
    [1][9][6][0][2][9][3][1][8][0][9][6][6][4][2]
    [7][8][5][1][2][3][3][2][6]
[}]

9. Array data with compression

JData annotations allows easy extensions. One of the immediate benefits is to introduce data compression to save space. To permit data compression, several additional keywords can be used, including "_ArrayZipType" - the compression method used, "_ArrayZipSize_" - the dimension vector of the "preprocessed" data stored in the "_ArrayData_" construct before compression, and "_ArrayZipData_" - the compressed data byte-stream. For example

Native data text-JData/JSON form binary-JData(UBJSON)

a=[
1,2,3
4,5,6
]

=>

{
"a":{
"_ArrayType_":"uint8",
"_ArraySize_":[2,3],
"_ArrayZipType_":"zlib"
"_ArrayZipSize_":[1,6]
"_ArrayZipData_":"eJxjZGJmYWUDAAA+ABY="
}
}

[{]
[U][1][a]
[{]
[U][11][_ArrayType_] [S][U][5][uint8]
[U][11][_ArraySize_] [[] [U][2][U][3] []]
[U][14][_ArrayZipType_] [S][U][4][zlib]
[U][14][_ArrayZipSize_] [[] [U][1][U][6] []]
[U][14][_ArrayZipData_] [[] [$][U][#][14] [...compressed byte stream...]
[}]
[}]

Powered by Habitat