Copyright	(c) 2013-2021 Brendan Hay
License	Mozilla Public License, v. 2.0.
Maintainer	Brendan Hay <brendan.g.hay+amazonka@gmail.com>
Stability	auto-generated
Portability	non-portable (GHC extensions)
Safe Haskell	None

Amazonka.Firehose.Types.ParquetSerDe

Description

Documentation

data ParquetSerDe Source #

A serializer to use for converting data to the Parquet format before storing it in Amazon S3. For more information, see Apache Parquet.

See: newParquetSerDe smart constructor.

Constructors

ParquetSerDe'

Fields

writerVersion :: Maybe ParquetWriterVersion
Indicates the version of row format to output. The possible values are V1 and V2. The default is V1.
compression :: Maybe ParquetCompression
The compression code to use over data blocks. The possible values are UNCOMPRESSED, SNAPPY, and GZIP, with the default being SNAPPY. Use SNAPPY for higher decompression speed. Use GZIP if the compression ratio is more important than speed.
maxPaddingBytes :: Maybe Natural
The maximum amount of padding to apply. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is 0.
enableDictionaryCompression :: Maybe Bool
Indicates whether to enable dictionary compression.
pageSizeBytes :: Maybe Natural
The Parquet page size. Column chunks are divided into pages. A page is conceptually an indivisible unit (in terms of compression and encoding). The minimum value is 64 KiB and the default is 1 MiB.
blockSizeBytes :: Maybe Natural
The Hadoop Distributed File System (HDFS) block size. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is 256 MiB and the minimum is 64 MiB. Kinesis Data Firehose uses this value for padding calculations.

Instances

Instances details

Eq ParquetSerDe Source #
Instance details Defined in Amazonka.Firehose.Types.ParquetSerDe Methods (==) :: ParquetSerDe -> ParquetSerDe -> Bool # (/=) :: ParquetSerDe -> ParquetSerDe -> Bool #
Read ParquetSerDe Source #
Instance details Defined in Amazonka.Firehose.Types.ParquetSerDe Methods readsPrec :: Int -> ReadS ParquetSerDe # readList :: ReadS [ParquetSerDe] # readPrec :: ReadPrec ParquetSerDe # readListPrec :: ReadPrec [ParquetSerDe] #
Show ParquetSerDe Source #
Instance details Defined in Amazonka.Firehose.Types.ParquetSerDe Methods showsPrec :: Int -> ParquetSerDe -> ShowS # show :: ParquetSerDe -> String # showList :: [ParquetSerDe] -> ShowS #
Generic ParquetSerDe Source #
Instance details Defined in Amazonka.Firehose.Types.ParquetSerDe Associated Types type Rep ParquetSerDe :: Type -> Type # Methods from :: ParquetSerDe -> Rep ParquetSerDe x # to :: Rep ParquetSerDe x -> ParquetSerDe #
NFData ParquetSerDe Source #
Instance details Defined in Amazonka.Firehose.Types.ParquetSerDe Methods rnf :: ParquetSerDe -> () #
Hashable ParquetSerDe Source #
Instance details Defined in Amazonka.Firehose.Types.ParquetSerDe Methods hashWithSalt :: Int -> ParquetSerDe -> Int # hash :: ParquetSerDe -> Int #
ToJSON ParquetSerDe Source #
Instance details Defined in Amazonka.Firehose.Types.ParquetSerDe Methods toJSON :: ParquetSerDe -> Value # toEncoding :: ParquetSerDe -> Encoding # toJSONList :: [ParquetSerDe] -> Value # toEncodingList :: [ParquetSerDe] -> Encoding #
FromJSON ParquetSerDe Source #
Instance details Defined in Amazonka.Firehose.Types.ParquetSerDe Methods parseJSON :: Value -> Parser ParquetSerDe # parseJSONList :: Value -> Parser [ParquetSerDe] #
type Rep ParquetSerDe Source #
Instance details Defined in Amazonka.Firehose.Types.ParquetSerDe type Rep ParquetSerDe = D1 ('MetaData "ParquetSerDe" "Amazonka.Firehose.Types.ParquetSerDe" "libZSservicesZSamazonka-kinesis-firehoseZSamazonka-kinesis-firehose" 'False) (C1 ('MetaCons "ParquetSerDe'" 'PrefixI 'True) ((S1 ('MetaSel ('Just "writerVersion") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (Maybe ParquetWriterVersion)) :: (S1 ('MetaSel ('Just "compression") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (Maybe ParquetCompression)) :: S1 ('MetaSel ('Just "maxPaddingBytes") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (Maybe Natural)))) :: (S1 ('MetaSel ('Just "enableDictionaryCompression") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (Maybe Bool)) :: (S1 ('MetaSel ('Just "pageSizeBytes") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (Maybe Natural)) :*: S1 ('MetaSel ('Just "blockSizeBytes") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (Maybe Natural))))))

newParquetSerDe :: ParquetSerDe Source #

Create a value of ParquetSerDe with all optional fields omitted.

Use generic-lens or optics to modify other optional fields.

The following record fields are available, with the corresponding lenses provided for backwards compatibility:

$sel:writerVersion:ParquetSerDe', parquetSerDe_writerVersion - Indicates the version of row format to output. The possible values are V1 and V2. The default is V1.

$sel:compression:ParquetSerDe', parquetSerDe_compression - The compression code to use over data blocks. The possible values are UNCOMPRESSED, SNAPPY, and GZIP, with the default being SNAPPY. Use SNAPPY for higher decompression speed. Use GZIP if the compression ratio is more important than speed.

$sel:maxPaddingBytes:ParquetSerDe', parquetSerDe_maxPaddingBytes - The maximum amount of padding to apply. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is 0.

$sel:enableDictionaryCompression:ParquetSerDe', parquetSerDe_enableDictionaryCompression - Indicates whether to enable dictionary compression.

$sel:pageSizeBytes:ParquetSerDe', parquetSerDe_pageSizeBytes - The Parquet page size. Column chunks are divided into pages. A page is conceptually an indivisible unit (in terms of compression and encoding). The minimum value is 64 KiB and the default is 1 MiB.

$sel:blockSizeBytes:ParquetSerDe', parquetSerDe_blockSizeBytes - The Hadoop Distributed File System (HDFS) block size. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is 256 MiB and the minimum is 64 MiB. Kinesis Data Firehose uses this value for padding calculations.

parquetSerDe_writerVersion :: Lens' ParquetSerDe (Maybe ParquetWriterVersion) Source #

Indicates the version of row format to output. The possible values are V1 and V2. The default is V1.

parquetSerDe_compression :: Lens' ParquetSerDe (Maybe ParquetCompression) Source #

The compression code to use over data blocks. The possible values are UNCOMPRESSED, SNAPPY, and GZIP, with the default being SNAPPY. Use SNAPPY for higher decompression speed. Use GZIP if the compression ratio is more important than speed.

parquetSerDe_maxPaddingBytes :: Lens' ParquetSerDe (Maybe Natural) Source #

The maximum amount of padding to apply. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is 0.

parquetSerDe_enableDictionaryCompression :: Lens' ParquetSerDe (Maybe Bool) Source #

Indicates whether to enable dictionary compression.

parquetSerDe_pageSizeBytes :: Lens' ParquetSerDe (Maybe Natural) Source #

The Parquet page size. Column chunks are divided into pages. A page is conceptually an indivisible unit (in terms of compression and encoding). The minimum value is 64 KiB and the default is 1 MiB.

parquetSerDe_blockSizeBytes :: Lens' ParquetSerDe (Maybe Natural) Source #

The Hadoop Distributed File System (HDFS) block size. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is 256 MiB and the minimum is 64 MiB. Kinesis Data Firehose uses this value for padding calculations.