Showing revision 2

Advanced JData Examples

: 1. Complex-number and complex-valued arrays
: 2. Sparse arrays
: 3. Complex-valued sparse arrays
: 4. Special matrices

The goal of this section is to show how one can use the lightweight and portable JData annotations to conveniently represent advanced data structures, including complex-valued arrays, sparse arrays, tables, trees, linked lists and graphs. The notation used in this section is similar to those used in the basic examples.

1. Complex-number and complex-valued arrays

Complex numbers are supported in many scientific programming languages, such as MATLAB, python and R, but there is no standardized format to store complex-valued data and share among these programming languages. Using the portable JData annotation system, we can conveniently represent complex-valued data and make sharing such data possible.

A complex-valued data record must be stored using the "annotated array format" as shown in the basic examples. This is achieved via the presence of "_ArrayIsComplex_" keyword and the serialization of the complex values in the "_ArrayData_" constructs in the order of [[serialized real-part values], [serialized imag-part values]]

Native data		text-JData/JSON form	binary-JData(UBJSON)
a=10.0+6.0j	=>	{ "a":{ "_ArrayType_":"double", "_ArraySize_":[1,1], "_ArrayIsComplex_":true, "_ArrayData_":[[10.0],[6.0]] } }	[{] [U][1][a] [{] [U][11][_ArrayType_] [S][U][6][double] [U][11][_ArraySize_] [[] [U][2][U][3] []] [U][16][_ArrayIsComplex_] [T] [U][11][_ArrayData_] [[] [[][D][10.0][]] [[][D][6.0][]] []] [}] [}]
a=[ 1+2j,3+4j 5+6j,7+8j ]	=>	{ "a":{ "_ArrayType_":"uint8", "_ArraySize_":[2,2], "_ArrayIsComplex_":true "_ArrayData_":[[1,3,5,7],[2,4,6,8]] } }	[{] [U][1][a] [{] [U][11][_ArrayType_] [S][U][5][uint8] [U][11][_ArraySize_] [[] [U][2][U][2] []] [U][16][_ArrayIsComplex_] [T] [U][11][_ArrayData_] [[] [[] [$][U][#][4] [1][3][5][7] [[] [$][U][#][4] [2][4][6][8] []] [}] [}]

2. Sparse arrays

Sparse array is an important data type widely used in computational models. A small number of scientific programming languages provide built-in support to sparse arrays, such as MATLAB; others support sparse arrays via 3rd party libraries, such as scipy.sparse for Python, Blas for C/C++, Lapack for FORTRAN etc.

Similar to the above case, a sparse array must be stored using the "annotated array format" as shown in the basic examples. This is achieved via the presence of "_ArrayIsSparse_" keyword and the serialization of the non-zero sparse data elements and indices in the "_ArrayData_" constructs in the order of [[first index i1], [second index i2], ... [last index iN], [serialized non-zero values]]

Native data		text-JData/JSON form	binary-JData(UBJSON)
a=sparse(5,4); a(1,1)=2.0; a(2,3)=9.0; a(4,2)=7.0;	=>	{ "a":{ "_ArrayType_":"double", "_ArraySize_":[5,4], "_ArrayIsSparse_":true, "_ArrayData_":[[1,2,4],[1,3,2],[2.0,9.0,7.0]] } }	[{] [U][1][a] [{] [U][11][_ArrayType_] [S][U][6][double] [U][11][_ArraySize_] [[] [U][2][U][3] []] [U][16][_ArrayIsSparse_] [T] [U][11][_ArrayData_] [[] [[][$][U][#][3] [1][2][4] [[][$][U][#][3] [1][3][2] [[][$][D][#][3] [2.0][9.0][7.0] []] [}] [}]

3. Complex-valued sparse arrays

Combining the "_ArrayIsComplex_" and "_ArrayIsSparse_" tags, we can easily support complex-valued sparse arrays using JData annotations. The "_ArrayData_" stores the information regarding the non-zero values in the order of [[first index i1], [second index i2], ... [last index iN], [serialized real-part of non-zero values], [serialized imag-part of non-zero values]]

Native data		text-JData/JSON form	binary-JData(UBJSON)
a=sparse(5,4); a(1,1)=2.0+1.2j; a(2,3)=9.0-4.7j; a(4,2)=7.0+1.0j;	=>	{ "a":{ "_ArrayType_":"double", "_ArraySize_":[5,4], "_ArrayIsSparse_":true, "_ArrayData_":[[1,2,4],[1,3,2], [2.0,9.0,7.0],[1.2,-4.7,1.0]] } }	[{] [U][1][a] [{] [U][11][_ArrayType_] [S][U][6][double] [U][11][_ArraySize_] [[] [U][2][U][3] []] [U][16][_ArrayIsSparse_] [T] [U][11][_ArrayData_] [[] [[][$][U][#][3] [1][2][4] [[][$][U][#][3] [1][3][2] [[][$][D][#][3] [2.0][9.0][7.0] [[][$][D][#][3] [1.2][-4.7][1.0] []] [}] [}]

4. Special matrices

The light-weight data annotation mechanism provided by JData specification can further enable efficient storage of special matrices, such as diagonal, triangular, Toeplitz matrices etc. This is enabled via the "_ArrayShape_" keyword. For example

Native data		text-JData/JSON form	binary-JData(UBJSON)
a=diag([1.0,1.0,2.1,2.2,5.3])	=>	{ "a":{ "_ArrayType_":"double", "_ArraySize_":[5,5], "_ArrayShape_":"diag", "_ArrayData_":[1.0,1.0,2.1,2.2,5.3] } }	[{] [U][1][a] [{] [U][11][_ArrayType_] [S][U][6][double] [U][11][_ArraySize_] [[] [U][5][U][5] []] [U][12][_ArrayShape_] [S][U][4][diag] [U][11][_ArrayData_] [[] [$][U][D][#][5] [1.0][1.0][2.1][2.2][5.3] [}] [}]