Showing revision 13.0

Basic JData Examples

1. Constants and scalars
2. Strings
3. Structures
4. 1-D row vectors
5. 1-D column vectors
6. 2-D matrices
7. 2-D arrays in the annotated format
8. 3-D and higher dimensional arrays

1. Constants and scalars

Numerical values are directly supported by either JSON or UBJSON specifications. A numerical value is typically unchanged when converting to the JData annotation. When storing as files, they are directly stored in the JSON/UBJSON numerical value forms. For example

Native data text-JData/JSON form binary-JData(UBJSON)
a=3
=>
{"a":3}
[{] [U][1][a][U][3] [}]

Similar for a floating point number

Native data text-JData/JSON form binary-JData(UBJSON) †
a=3.14159
=>
{"a":3.14159}
[{] [U][1][a][D][3.15169] [}]
† In the case of the binary-JData, the type marker of the floating point number, here [D] must match the native data's type.

There are a few special constants, namely "NaN", "Infinity" and "-Infinity", they are encoded as special string keywords when stored in the JSON/text-JData formats, but stay unchanged when stored in the binary JData format

Native data text-JData/JSON form binary-JData(UBJSON) †
a=nan
=>
{"a":"_NaN_"}
[{] [U][1][a][D][nan] [}]
a=inf
=>
{"a":"_Inf_"}
[{] [U][1][a][D][inf] [}]
a=-inf
=>
{"a":"-_Inf_"}
[{] [U][1][D][-inf] [}]

Three special constants are directly supported by the JSON/UBJSON specifications, these constants are logical "True", logical "False" and "Null", they are stored as

Native data text-JData/JSON form binary-JData(UBJSON)
a=true
=>
{"a":true}
[{] [U][1][a][T] [}]
a=false
=>
{"a":false}
[{] [U][1][a][F] [}]
a=null
=>
{"a":null}
[{] [U][1][a][N] [}]

2. Strings

Strings are natively supported in both JSON and UBJSON, therefore, when converting to the JData annotations, they are mostly unchanged. For JSON strings, 9 special characters can be escaped to represent control sequences (\",\\,\/,\b,\f,\n,\r,\t,\uXXXX). For example

Native data text-JData/JSON form binary-JData(UBJSON)
a="JData will prevail"
=>
{"a":"JData will prevail"}
[{] [U][1][a][S][U][18][JData will prevail] [}]
a="Tabs\tNewlines\n"
=>
{"a":"Tabs\tNewlines\n"}
[{] [U][1][a][S][U][14][Tabs\tNewlines\n] [}]
a="中文UTF-8格式"
=>
{"a":"中文UTF-8格式"}
[{] [U][1][a][S][U][17][中文UTF-8格式] [}]

We want to mention that in JData, "non-strict" JSON string mode is allowed: that means one can store ASCII values 0-31 as part of the string without escaping.

3. Structures

Hierarchical structures are often needed when representing metadata or simple lists with named-members. Because "structure" data-type can be directly mapped to the "object" construct in JSON and UBJSON, therefore, they do not need to be converted when using the JData annotation.

Native data text-JData/JSON form binary-JData(UBJSON)

a=struct(
'i1',1,
'f1',2.0,
's1':"string"
)

=>

{
"a":{
"i1":1,
"f1":2.0,
"s1":"string"
}}

[{]
[U][1][a]
[{]
[U][2][i1][U][1]
[U][2][f1][D][2.0]
[U][2][s1][S][6][string]
[}]
[}]

4. 1-D row vectors

Simple 1-dimensional vectors are supported in both JSON and UBJSON using the "array" construct. For example

Native data text-JData/JSON form binary-JData(UBJSON)
a=[1,2,3]
=>
{"a":[1,2,3]}

[{]
[U][1][a]
[[]
[U][1][U][2][U][3]
[]]
[}]

when array element have the same type, binary JData/UBJSON
can use the "optimized" array header to save space. For the array
above, one can write in the right form, where [$][U] indicates array
element type is "unsigned byte" [U] and [#][U][3] indicates that
there are 3 elements

[{]
[U][1][a]
[[] [$][U] [#][U][3]
[1][2][3]
[}]

a=[2.0,"str",nan]
=>
{"a":[2.0,"str","_NaN_"]}

[{]
[U][1][a]
[[]
[D][2.0]
[S][U][3][str]
[S][U][5][_NaN_]
[]]
[}]

5. 1-D column vectors

To distinguish column from row vectors, we use embedded array constructs in JSON/UBJSON to represent a column vector

Native data text-JData/JSON form binary-JData(UBJSON)
a=[[1],[2],[3]]
=>
{"a":[[1],[2],[3]]}

[{]
[U][1][a]
[[]
[[] [U][1] []]
[[] [U][2] []]
[[] [U][3] []]
[]]
[}]

a=[[2.0],["str"],[nan]]
=>
{"a":[[2.0],["str"],["_NaN_"]]}

[{]
[U][1][a]
[[]
[[] [D][2.0] []]
[[] [S][U][3][str] []]
[[] [S][U][5][_NaN_] []]
[]]
[}]

6. 2-D matrices

Simple 1-dimensional vectors are supported in both JSON and UBJSON using the "array" construct. For example

Native data text-JData/JSON form binary-JData(UBJSON)

a=[
1,2,3
4,5,6
]

=>

{
"a":[
[1,2,3],
[4,5,6]
]
}

[{]
[U][1][a]
[[]
[[] [U][1][U][2][U][3] []]
[[] [U][4][U][5][U][6] []]
[]]
[}]

similar to the 1-D raow vector example above, we can
use the type [$] and count [#] markers to simplify
this array in the binary form

[{]
[U][1][a]
[[]
[[] [$][U] [#][U][3] [1][2][3]
[[] [$][U] [#][U][3] [4][5][6]
[]]
[}]

to simplify this further, in the JData Specification,
we further extended UBJSON array count marker [#] to
accept a 1-D array count-type, representing the dimension
vector of an N-D array, in this case [2,3] for a 2x3 matrix

[{]
[U][1][a]
[[] [$][U] [#][[] [$][U][#][2] [2][3]
[1][2][3][4][5][6]
[}]

7. 2-D arrays in the annotated format

In JData specification, we introduced a light-weight data annotation approach to allow one to specify additional information, such as data type, data size and compression, in the stored data record. This is achieved using a "structure-like" data container (a structure is supported in almost all programming language) with JData-specified human-readable subfield keywords. This construct is also easily serialized using many of the existing JSON/UBJSON libraries.

For example, the above 2-D array can be alternatively stored using the annotated format to allow fine-grained data storage

Native data text-JData/JSON form binary-JData(UBJSON)

a=[
1,2,3
4,5,6
]

=>

{
"a":{
"_ArrayType_":"uint8",
"_ArraySize_":[2,3],
"_ArrayData_":[1,2,3,4,5,6]
}
}

[{]
[U][1][a]
[{]
[U][11][_ArrayType_] [S][U][5][uint8]
[U][11][_ArraySize_] [[] [U][2][U][3] []]
[U][11][_ArrayData_] [[] [$][U][#][6] [1][2][3][4][5][6]
[}]
[}]

Please note that the binary JData format already has built-in support to strongly-typed binary data, therefore, it is not very beneficial, despite it is permitted, to use the above annotated format compared to the packed-array format with optimized header shown in the above section.

8. 3-D and higher dimensional arrays

One can use either the direct-format or annotated-format for storing higher dimensional arrays, as natively supported by both JSON/UBJSON, but the benefit of using the annotated format for text-based JData, and the packed-array optimized format for binary-JData becomes more advantageous due to faster processing speed.

Native datatext-JData/JSON formbinary-JData(UBJSON)
a=[
      [
          [1,9,6,0],
          [2,9,3,1],
          [8,0,9,6]
      ],
      [
          [6,4,2,7],
          [8,5,1,2],
          [3,3,2,6]
      ]
  ]
=>
{
  "a":[
        [
          [1,9,6,0],
          [2,9,3,1],
          [8,0,9,6]
        ],
        [
          [6,4,2,7],
          [8,5,1,2],
          [3,3,2,6]
        ]
    ]
}
[{]
 [U][1][a]
 [[]
   [[]
     [[] [U][1][U][9][U][6][u][0] []]
     [[] [U][2][U][9][U][3][u][1] []]
     [[] [U][8][U][0][U][9][u][6] []]
   []]
   [[]
     [[] [U][6][U][4][U][2][u][7] []]
     [[] [U][8][U][5][U][1][u][2] []]
     [[] [U][3][U][3][U][2][u][6] []]
   []]
 []]
[}]
More efficient alternative formats using JData annotations
{
  "a":{
        "_ArrayType_":"uint8",
        "_ArraySize_":[2,3,4],
        "_ArrayType_":[1,9,6,0,2,9,3,1,8,0,9,6,6,4,2,7,8,5,1,2,3,3,2,6]
  }
}
[{]
 [U][1][a]
 [[] [$][U] [#][[] [$][U][#][3] [2][3][4]
    [1][9][6][0][2][9][3][1][8][0][9][6]
    [6][4][2][7][8][5][1][2][3][3][2][6]
[}]
Powered by Habitat