Copyright	(c) 2013-2021 Brendan Hay
License	Mozilla Public License, v. 2.0.
Maintainer	Brendan Hay <brendan.g.hay+amazonka@gmail.com>
Stability	auto-generated
Portability	non-portable (GHC extensions)
Safe Haskell	None

Amazonka.Glue.CreateMLTransform

Contents

Creating a Request
Request Lenses
Destructuring the Response
Response Lenses

Description

Creates an Glue machine learning transform. This operation creates the transform and all the necessary parameters to train it.

Call this operation as the first step in the process of using a machine learning transform (such as the FindMatches transform) for deduplicating data. You can provide an optional Description, in addition to the parameters that you want to use for your algorithm.

You must also specify certain parameters for the tasks that Glue runs on your behalf as part of learning from your data and creating a high-quality machine learning transform. These parameters include Role, and optionally, AllocatedCapacity, Timeout, and MaxRetries. For more information, see Jobs.

Synopsis

data CreateMLTransform = CreateMLTransform' {
- numberOfWorkers :: Maybe Int
- workerType :: Maybe WorkerType
- glueVersion :: Maybe Text
- maxRetries :: Maybe Int
- maxCapacity :: Maybe Double
- timeout :: Maybe Natural
- transformEncryption :: Maybe TransformEncryption
- description :: Maybe Text
- tags :: Maybe (HashMap Text Text)
- name :: Text
- inputRecordTables :: [GlueTable]
- parameters :: TransformParameters
- role' :: Text
}
newCreateMLTransform :: Text -> TransformParameters -> Text -> CreateMLTransform
createMLTransform_numberOfWorkers :: Lens' CreateMLTransform (Maybe Int)
createMLTransform_workerType :: Lens' CreateMLTransform (Maybe WorkerType)
createMLTransform_glueVersion :: Lens' CreateMLTransform (Maybe Text)
createMLTransform_maxRetries :: Lens' CreateMLTransform (Maybe Int)
createMLTransform_maxCapacity :: Lens' CreateMLTransform (Maybe Double)
createMLTransform_timeout :: Lens' CreateMLTransform (Maybe Natural)
createMLTransform_transformEncryption :: Lens' CreateMLTransform (Maybe TransformEncryption)
createMLTransform_description :: Lens' CreateMLTransform (Maybe Text)
createMLTransform_tags :: Lens' CreateMLTransform (Maybe (HashMap Text Text))
createMLTransform_name :: Lens' CreateMLTransform Text
createMLTransform_inputRecordTables :: Lens' CreateMLTransform [GlueTable]
createMLTransform_parameters :: Lens' CreateMLTransform TransformParameters
createMLTransform_role :: Lens' CreateMLTransform Text
data CreateMLTransformResponse = CreateMLTransformResponse' {
- transformId :: Maybe Text
- httpStatus :: Int
}
newCreateMLTransformResponse :: Int -> CreateMLTransformResponse
createMLTransformResponse_transformId :: Lens' CreateMLTransformResponse (Maybe Text)
createMLTransformResponse_httpStatus :: Lens' CreateMLTransformResponse Int

Creating a Request

data CreateMLTransform Source #

See: newCreateMLTransform smart constructor.

Constructors

CreateMLTransform'

Fields

numberOfWorkers :: Maybe Int
The number of workers of a defined workerType that are allocated when this task runs.
If WorkerType is set, then NumberOfWorkers is required (and vice versa).
workerType :: Maybe WorkerType
The type of predefined worker that is allocated when this task runs. Accepts a value of Standard, G.1X, or G.2X.
- For the Standard worker type, each worker provides 4 vCPU, 16 GB of memory and a 50GB disk, and 2 executors per worker.
- For the G.1X worker type, each worker provides 4 vCPU, 16 GB of memory and a 64GB disk, and 1 executor per worker.
- For the G.2X worker type, each worker provides 8 vCPU, 32 GB of memory and a 128GB disk, and 1 executor per worker.
MaxCapacity is a mutually exclusive option with NumberOfWorkers and WorkerType.
- If either NumberOfWorkers or WorkerType is set, then MaxCapacity cannot be set.
- If MaxCapacity is set then neither NumberOfWorkers or WorkerType can be set.
- If WorkerType is set, then NumberOfWorkers is required (and vice versa).
- MaxCapacity and NumberOfWorkers must both be at least 1.
glueVersion :: Maybe Text
This value determines which version of Glue this machine learning transform is compatible with. Glue 1.0 is recommended for most customers. If the value is not set, the Glue compatibility defaults to Glue 0.9. For more information, see Glue Versions in the developer guide.
maxRetries :: Maybe Int
The maximum number of times to retry a task for this transform after a task run fails.
maxCapacity :: Maybe Double
The number of Glue data processing units (DPUs) that are allocated to task runs for this transform. You can allocate from 2 to 100 DPUs; the default is 10. A DPU is a relative measure of processing power that consists of 4 vCPUs of compute capacity and 16 GB of memory. For more information, see the Glue pricing page.
MaxCapacity is a mutually exclusive option with NumberOfWorkers and WorkerType.
- If either NumberOfWorkers or WorkerType is set, then MaxCapacity cannot be set.
- If MaxCapacity is set then neither NumberOfWorkers or WorkerType can be set.
- If WorkerType is set, then NumberOfWorkers is required (and vice versa).
- MaxCapacity and NumberOfWorkers must both be at least 1.
When the WorkerType field is set to a value other than Standard, the MaxCapacity field is set automatically and becomes read-only.
When the WorkerType field is set to a value other than Standard, the MaxCapacity field is set automatically and becomes read-only.
timeout :: Maybe Natural
The timeout of the task run for this transform in minutes. This is the maximum time that a task run for this transform can consume resources before it is terminated and enters TIMEOUT status. The default is 2,880 minutes (48 hours).
transformEncryption :: Maybe TransformEncryption
The encryption-at-rest settings of the transform that apply to accessing user data. Machine learning transforms can access user data encrypted in Amazon S3 using KMS.
description :: Maybe Text
A description of the machine learning transform that is being defined. The default is an empty string.
tags :: Maybe (HashMap Text Text)
The tags to use with this machine learning transform. You may use tags to limit access to the machine learning transform. For more information about tags in Glue, see Amazon Web Services Tags in Glue in the developer guide.
name :: Text
The unique name that you give the transform when you create it.
inputRecordTables :: [GlueTable]
A list of Glue table definitions used by the transform.
parameters :: TransformParameters
The algorithmic parameters that are specific to the transform type used. Conditionally dependent on the transform type.
role' :: Text
The name or Amazon Resource Name (ARN) of the IAM role with the required permissions. The required permissions include both Glue service role permissions to Glue resources, and Amazon S3 permissions required by the transform.
- This role needs Glue service role permissions to allow access to resources in Glue. See Attach a Policy to IAM Users That Access Glue.
- This role needs permission to your Amazon Simple Storage Service (Amazon S3) sources, targets, temporary directory, scripts, and any libraries used by the task run for this transform.

Instances

Instances details

Eq CreateMLTransform Source #
Instance details Defined in Amazonka.Glue.CreateMLTransform Methods (==) :: CreateMLTransform -> CreateMLTransform -> Bool # (/=) :: CreateMLTransform -> CreateMLTransform -> Bool #
Read CreateMLTransform Source #
Instance details Defined in Amazonka.Glue.CreateMLTransform Methods readsPrec :: Int -> ReadS CreateMLTransform # readList :: ReadS [CreateMLTransform] # readPrec :: ReadPrec CreateMLTransform # readListPrec :: ReadPrec [CreateMLTransform] #
Show CreateMLTransform Source #
Instance details Defined in Amazonka.Glue.CreateMLTransform Methods showsPrec :: Int -> CreateMLTransform -> ShowS # show :: CreateMLTransform -> String # showList :: [CreateMLTransform] -> ShowS #
Generic CreateMLTransform Source #
Instance details Defined in Amazonka.Glue.CreateMLTransform Associated Types type Rep CreateMLTransform :: Type -> Type # Methods from :: CreateMLTransform -> Rep CreateMLTransform x # to :: Rep CreateMLTransform x -> CreateMLTransform #
NFData CreateMLTransform Source #
Instance details Defined in Amazonka.Glue.CreateMLTransform Methods rnf :: CreateMLTransform -> () #
Hashable CreateMLTransform Source #
Instance details Defined in Amazonka.Glue.CreateMLTransform Methods hashWithSalt :: Int -> CreateMLTransform -> Int # hash :: CreateMLTransform -> Int #
ToJSON CreateMLTransform Source #
Instance details Defined in Amazonka.Glue.CreateMLTransform Methods toJSON :: CreateMLTransform -> Value # toEncoding :: CreateMLTransform -> Encoding # toJSONList :: [CreateMLTransform] -> Value # toEncodingList :: [CreateMLTransform] -> Encoding #
AWSRequest CreateMLTransform Source #
Instance details Defined in Amazonka.Glue.CreateMLTransform Associated Types type AWSResponse CreateMLTransform # Methods request :: CreateMLTransform -> Request CreateMLTransform # response :: MonadResource m => Logger -> Service -> Proxy CreateMLTransform -> ClientResponse ClientBody -> m (Either Error (ClientResponse (AWSResponse CreateMLTransform))) #
ToHeaders CreateMLTransform Source #
Instance details Defined in Amazonka.Glue.CreateMLTransform Methods toHeaders :: CreateMLTransform -> [Header] #
ToPath CreateMLTransform Source #
Instance details Defined in Amazonka.Glue.CreateMLTransform Methods toPath :: CreateMLTransform -> ByteString #
ToQuery CreateMLTransform Source #
Instance details Defined in Amazonka.Glue.CreateMLTransform Methods toQuery :: CreateMLTransform -> QueryString #
type Rep CreateMLTransform Source #
Instance details Defined in Amazonka.Glue.CreateMLTransform type Rep CreateMLTransform = D1 ('MetaData "CreateMLTransform" "Amazonka.Glue.CreateMLTransform" "libZSservicesZSamazonka-glueZSamazonka-glue" 'False) (C1 ('MetaCons "CreateMLTransform'" 'PrefixI 'True) (((S1 ('MetaSel ('Just "numberOfWorkers") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (Maybe Int)) :: (S1 ('MetaSel ('Just "workerType") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (Maybe WorkerType)) :: S1 ('MetaSel ('Just "glueVersion") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (Maybe Text)))) :: (S1 ('MetaSel ('Just "maxRetries") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (Maybe Int)) :: (S1 ('MetaSel ('Just "maxCapacity") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (Maybe Double)) :: S1 ('MetaSel ('Just "timeout") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (Maybe Natural))))) :: ((S1 ('MetaSel ('Just "transformEncryption") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (Maybe TransformEncryption)) :: (S1 ('MetaSel ('Just "description") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (Maybe Text)) :: S1 ('MetaSel ('Just "tags") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (Maybe (HashMap Text Text))))) :: ((S1 ('MetaSel ('Just "name") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 Text) :: S1 ('MetaSel ('Just "inputRecordTables") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 [GlueTable])) :: (S1 ('MetaSel ('Just "parameters") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 TransformParameters) :: S1 ('MetaSel ('Just "role'") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 Text))))))
type AWSResponse CreateMLTransform Source #
Instance details Defined in Amazonka.Glue.CreateMLTransform type AWSResponse CreateMLTransform = CreateMLTransformResponse

newCreateMLTransform Source #

Arguments

:: Text	`$sel:name:CreateMLTransform'`
-> TransformParameters	`$sel:parameters:CreateMLTransform'`
-> Text	`$sel:role':CreateMLTransform'`
-> CreateMLTransform

Create a value of CreateMLTransform with all optional fields omitted.

Use generic-lens or optics to modify other optional fields.

The following record fields are available, with the corresponding lenses provided for backwards compatibility:

$sel:numberOfWorkers:CreateMLTransform', createMLTransform_numberOfWorkers - The number of workers of a defined workerType that are allocated when this task runs.

If WorkerType is set, then NumberOfWorkers is required (and vice versa).

$sel:workerType:CreateMLTransform', createMLTransform_workerType - The type of predefined worker that is allocated when this task runs. Accepts a value of Standard, G.1X, or G.2X.

For the Standard worker type, each worker provides 4 vCPU, 16 GB of memory and a 50GB disk, and 2 executors per worker.
For the G.1X worker type, each worker provides 4 vCPU, 16 GB of memory and a 64GB disk, and 1 executor per worker.
For the G.2X worker type, each worker provides 8 vCPU, 32 GB of memory and a 128GB disk, and 1 executor per worker.

MaxCapacity is a mutually exclusive option with NumberOfWorkers and WorkerType.

If either NumberOfWorkers or WorkerType is set, then MaxCapacity cannot be set.
If MaxCapacity is set then neither NumberOfWorkers or WorkerType can be set.
If WorkerType is set, then NumberOfWorkers is required (and vice versa).
MaxCapacity and NumberOfWorkers must both be at least 1.

$sel:glueVersion:CreateMLTransform', createMLTransform_glueVersion - This value determines which version of Glue this machine learning transform is compatible with. Glue 1.0 is recommended for most customers. If the value is not set, the Glue compatibility defaults to Glue 0.9. For more information, see Glue Versions in the developer guide.

$sel:maxRetries:CreateMLTransform', createMLTransform_maxRetries - The maximum number of times to retry a task for this transform after a task run fails.

$sel:maxCapacity:CreateMLTransform', createMLTransform_maxCapacity - The number of Glue data processing units (DPUs) that are allocated to task runs for this transform. You can allocate from 2 to 100 DPUs; the default is 10. A DPU is a relative measure of processing power that consists of 4 vCPUs of compute capacity and 16 GB of memory. For more information, see the Glue pricing page.

MaxCapacity is a mutually exclusive option with NumberOfWorkers and WorkerType.

If either NumberOfWorkers or WorkerType is set, then MaxCapacity cannot be set.
If MaxCapacity is set then neither NumberOfWorkers or WorkerType can be set.
If WorkerType is set, then NumberOfWorkers is required (and vice versa).
MaxCapacity and NumberOfWorkers must both be at least 1.

When the WorkerType field is set to a value other than Standard, the MaxCapacity field is set automatically and becomes read-only.

$sel:timeout:CreateMLTransform', createMLTransform_timeout - The timeout of the task run for this transform in minutes. This is the maximum time that a task run for this transform can consume resources before it is terminated and enters TIMEOUT status. The default is 2,880 minutes (48 hours).

$sel:transformEncryption:CreateMLTransform', createMLTransform_transformEncryption - The encryption-at-rest settings of the transform that apply to accessing user data. Machine learning transforms can access user data encrypted in Amazon S3 using KMS.

$sel:description:CreateMLTransform', createMLTransform_description - A description of the machine learning transform that is being defined. The default is an empty string.

$sel:tags:CreateMLTransform', createMLTransform_tags - The tags to use with this machine learning transform. You may use tags to limit access to the machine learning transform. For more information about tags in Glue, see Amazon Web Services Tags in Glue in the developer guide.

$sel:name:CreateMLTransform', createMLTransform_name - The unique name that you give the transform when you create it.

$sel:inputRecordTables:CreateMLTransform', createMLTransform_inputRecordTables - A list of Glue table definitions used by the transform.

$sel:parameters:CreateMLTransform', createMLTransform_parameters - The algorithmic parameters that are specific to the transform type used. Conditionally dependent on the transform type.

$sel:role':CreateMLTransform', createMLTransform_role - The name or Amazon Resource Name (ARN) of the IAM role with the required permissions. The required permissions include both Glue service role permissions to Glue resources, and Amazon S3 permissions required by the transform.

This role needs Glue service role permissions to allow access to resources in Glue. See Attach a Policy to IAM Users That Access Glue.
This role needs permission to your Amazon Simple Storage Service (Amazon S3) sources, targets, temporary directory, scripts, and any libraries used by the task run for this transform.

Request Lenses

createMLTransform_numberOfWorkers :: Lens' CreateMLTransform (Maybe Int) Source #

The number of workers of a defined workerType that are allocated when this task runs.

If WorkerType is set, then NumberOfWorkers is required (and vice versa).

createMLTransform_workerType :: Lens' CreateMLTransform (Maybe WorkerType) Source #

The type of predefined worker that is allocated when this task runs. Accepts a value of Standard, G.1X, or G.2X.

For the Standard worker type, each worker provides 4 vCPU, 16 GB of memory and a 50GB disk, and 2 executors per worker.
For the G.1X worker type, each worker provides 4 vCPU, 16 GB of memory and a 64GB disk, and 1 executor per worker.
For the G.2X worker type, each worker provides 8 vCPU, 32 GB of memory and a 128GB disk, and 1 executor per worker.

MaxCapacity is a mutually exclusive option with NumberOfWorkers and WorkerType.

If either NumberOfWorkers or WorkerType is set, then MaxCapacity cannot be set.
If MaxCapacity is set then neither NumberOfWorkers or WorkerType can be set.
If WorkerType is set, then NumberOfWorkers is required (and vice versa).
MaxCapacity and NumberOfWorkers must both be at least 1.

createMLTransform_glueVersion :: Lens' CreateMLTransform (Maybe Text) Source #

This value determines which version of Glue this machine learning transform is compatible with. Glue 1.0 is recommended for most customers. If the value is not set, the Glue compatibility defaults to Glue 0.9. For more information, see Glue Versions in the developer guide.

createMLTransform_maxRetries :: Lens' CreateMLTransform (Maybe Int) Source #

The maximum number of times to retry a task for this transform after a task run fails.

createMLTransform_maxCapacity :: Lens' CreateMLTransform (Maybe Double) Source #

The number of Glue data processing units (DPUs) that are allocated to task runs for this transform. You can allocate from 2 to 100 DPUs; the default is 10. A DPU is a relative measure of processing power that consists of 4 vCPUs of compute capacity and 16 GB of memory. For more information, see the Glue pricing page.

MaxCapacity is a mutually exclusive option with NumberOfWorkers and WorkerType.

If either NumberOfWorkers or WorkerType is set, then MaxCapacity cannot be set.
If MaxCapacity is set then neither NumberOfWorkers or WorkerType can be set.
If WorkerType is set, then NumberOfWorkers is required (and vice versa).
MaxCapacity and NumberOfWorkers must both be at least 1.

When the WorkerType field is set to a value other than Standard, the MaxCapacity field is set automatically and becomes read-only.

createMLTransform_timeout :: Lens' CreateMLTransform (Maybe Natural) Source #

The timeout of the task run for this transform in minutes. This is the maximum time that a task run for this transform can consume resources before it is terminated and enters TIMEOUT status. The default is 2,880 minutes (48 hours).

createMLTransform_transformEncryption :: Lens' CreateMLTransform (Maybe TransformEncryption) Source #

The encryption-at-rest settings of the transform that apply to accessing user data. Machine learning transforms can access user data encrypted in Amazon S3 using KMS.

createMLTransform_description :: Lens' CreateMLTransform (Maybe Text) Source #

A description of the machine learning transform that is being defined. The default is an empty string.

createMLTransform_tags :: Lens' CreateMLTransform (Maybe (HashMap Text Text)) Source #

The tags to use with this machine learning transform. You may use tags to limit access to the machine learning transform. For more information about tags in Glue, see Amazon Web Services Tags in Glue in the developer guide.

createMLTransform_name :: Lens' CreateMLTransform Text Source #

The unique name that you give the transform when you create it.

createMLTransform_inputRecordTables :: Lens' CreateMLTransform [GlueTable] Source #

A list of Glue table definitions used by the transform.

createMLTransform_parameters :: Lens' CreateMLTransform TransformParameters Source #

The algorithmic parameters that are specific to the transform type used. Conditionally dependent on the transform type.

createMLTransform_role :: Lens' CreateMLTransform Text Source #

The name or Amazon Resource Name (ARN) of the IAM role with the required permissions. The required permissions include both Glue service role permissions to Glue resources, and Amazon S3 permissions required by the transform.

This role needs Glue service role permissions to allow access to resources in Glue. See Attach a Policy to IAM Users That Access Glue.
This role needs permission to your Amazon Simple Storage Service (Amazon S3) sources, targets, temporary directory, scripts, and any libraries used by the task run for this transform.

Destructuring the Response

data CreateMLTransformResponse Source #

See: newCreateMLTransformResponse smart constructor.

Constructors

CreateMLTransformResponse'
Fields transformId :: Maybe Text A unique identifier that is generated for the transform. httpStatus :: Int The response's http status code.

Instances

Instances details

Eq CreateMLTransformResponse Source #
Instance details Defined in Amazonka.Glue.CreateMLTransform Methods (==) :: CreateMLTransformResponse -> CreateMLTransformResponse -> Bool # (/=) :: CreateMLTransformResponse -> CreateMLTransformResponse -> Bool #
Read CreateMLTransformResponse Source #
Instance details Defined in Amazonka.Glue.CreateMLTransform Methods readsPrec :: Int -> ReadS CreateMLTransformResponse # readList :: ReadS [CreateMLTransformResponse] # readPrec :: ReadPrec CreateMLTransformResponse # readListPrec :: ReadPrec [CreateMLTransformResponse] #
Show CreateMLTransformResponse Source #
Instance details Defined in Amazonka.Glue.CreateMLTransform Methods showsPrec :: Int -> CreateMLTransformResponse -> ShowS # show :: CreateMLTransformResponse -> String # showList :: [CreateMLTransformResponse] -> ShowS #
Generic CreateMLTransformResponse Source #
Instance details Defined in Amazonka.Glue.CreateMLTransform Associated Types type Rep CreateMLTransformResponse :: Type -> Type # Methods from :: CreateMLTransformResponse -> Rep CreateMLTransformResponse x # to :: Rep CreateMLTransformResponse x -> CreateMLTransformResponse #
NFData CreateMLTransformResponse Source #
Instance details Defined in Amazonka.Glue.CreateMLTransform Methods rnf :: CreateMLTransformResponse -> () #
type Rep CreateMLTransformResponse Source #
Instance details Defined in Amazonka.Glue.CreateMLTransform type Rep CreateMLTransformResponse = D1 ('MetaData "CreateMLTransformResponse" "Amazonka.Glue.CreateMLTransform" "libZSservicesZSamazonka-glueZSamazonka-glue" 'False) (C1 ('MetaCons "CreateMLTransformResponse'" 'PrefixI 'True) (S1 ('MetaSel ('Just "transformId") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (Maybe Text)) :*: S1 ('MetaSel ('Just "httpStatus") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 Int)))

newCreateMLTransformResponse Source #

Arguments

:: Int	`$sel:httpStatus:CreateMLTransformResponse'`
-> CreateMLTransformResponse

Create a value of CreateMLTransformResponse with all optional fields omitted.

Use generic-lens or optics to modify other optional fields.

The following record fields are available, with the corresponding lenses provided for backwards compatibility:

$sel:transformId:CreateMLTransformResponse', createMLTransformResponse_transformId - A unique identifier that is generated for the transform.

$sel:httpStatus:CreateMLTransformResponse', createMLTransformResponse_httpStatus - The response's http status code.

Response Lenses

createMLTransformResponse_transformId :: Lens' CreateMLTransformResponse (Maybe Text) Source #

A unique identifier that is generated for the transform.

createMLTransformResponse_httpStatus :: Lens' CreateMLTransformResponse Int Source #

The response's http status code.