Welcome Guest! Log in
Stambia versions 2.x, 3.x, S17, S18, S19 and S20 are reaching End of Support January, 15th, 2024. Please consider upgrading to the supported Semarchy xDI versions. See Global Policy Support and the Semarchy Documentation.

The Stambia User Community is moving to Semarchy! All the applicable resources have already been moved or are currently being moved to their new location. Read more…


Parquet files can be reversed in Metadata like any other technology.

This article is a quick guide explaining how to work with Parquet Files.

Parquet Component must have been installed before following this guide, refer to this article for the download and installation.

 

Metadata creation and reverse

Reversing a Parquet file can be performed through the following steps.

First, create a new Metadata and choose Parquet Schemas.

new metadata

 

Create a new "Schema" node in it:

 

new schema

 

Then, specify the file path to the Parquet File to reverse:

 

reverse path

 

Finally, perform the reverse by doing a "right click > Reverse" on the schema node:

 

reverse schema

 

That's it, the Parquet file is now reversed:

 

reversed metadata

 

Additional Information about Parquet Metadata

Parquet Metadata has the following structure.

Note that you can reverse multiple Parquet schemas by creating multiple "schema" nodes inside the Metadata.

 

Parquet root node

The "Parquet" root node is just a container to store many Parquet schemas in one place.

You can define a user-friendly name to represent this Metadata, and you must define a Parquet Module to use.

 

parquet metadata node root

 

Parquet Schema Node

"Schema" nodes are created under the "Parquet" root node.

A schema node is the entry point to reverse and use Parquet schemas in the Metadata.

When you want to reverse the schema from an existing file, you must fill the file path to the Parquet file to reverse.

 

parquet metadata node schema

 

Note that you can also define all the attributes manually, if needed.

 

Parquet Fields nodes

Under a schema node, the fields will be defined.

They represent all the fields of the Parquet file structure.

parquet metadata node field

 

Field logical Type

On a field, you can find the reference to the related logical types, regarding predefined list.

 

Parquet Logical

 

Notes:
  • Each "time" type is separated, such as nano/micro precision or UTC reference.
  • The same behaviour can also be noticed for signed or unsigned integers.

 

Field Physical Type

On a field, you can find the reference to the related physical type, regarding predefined list

Parquet Primitive


Note that the "Group" primitive type is specific Stambia one.

It is automatically set when doing a reverse.

It is used to indicate this node contains a sub-node.


Parquet Group

 

 

Articles

Suggest a new Article!