API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design...

153
Cloud Data Migration API Reference Issue 10 Date 2018-08-03 HUAWEI TECHNOLOGIES CO., LTD.

Transcript of API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design...

Page 1: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Cloud Data Migration

API Reference

Issue 10

Date 2018-08-03

HUAWEI TECHNOLOGIES CO., LTD.

Page 2: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Copyright © Huawei Technologies Co., Ltd. 2018. All rights reserved.No part of this document may be reproduced or transmitted in any form or by any means without prior writtenconsent of Huawei Technologies Co., Ltd. Trademarks and Permissions

and other Huawei trademarks are trademarks of Huawei Technologies Co., Ltd.All other trademarks and trade names mentioned in this document are the property of their respectiveholders. NoticeThe purchased products, services and features are stipulated by the contract made between Huawei and thecustomer. All or part of the products, services and features described in this document may not be within thepurchase scope or the usage scope. Unless otherwise specified in the contract, all statements, information,and recommendations in this document are provided "AS IS" without warranties, guarantees orrepresentations of any kind, either express or implied.

The information in this document is subject to change without notice. Every effort has been made in thepreparation of this document to ensure accuracy of the contents, but all statements, information, andrecommendations in this document do not constitute a warranty of any kind, express or implied.

Huawei Technologies Co., Ltd.Address: Huawei Industrial Base

Bantian, LonggangShenzhen 518129People's Republic of China

Website: http://e.huawei.com

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

i

Page 3: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Contents

1 Before You Start............................................................................................................................. 1

2 API Overview................................................................................................................................. 2

3 API Use............................................................................................................................................ 43.1 Overview of RESTful APIs............................................................................................................................................43.2 Obtaining Request Authentication Information............................................................................................................. 83.2.1 Token Authentication...................................................................................................................................................83.3 Obtaining a Project ID....................................................................................................................................................93.4 API Use Case................................................................................................................................................................10

4 API Description........................................................................................................................... 174.1 Cluster Management.....................................................................................................................................................174.1.1 Creating a Cluster...................................................................................................................................................... 174.1.2 Querying the Cluster List.......................................................................................................................................... 214.1.3 Querying Cluster Details........................................................................................................................................... 234.1.4 Restarting a Cluster................................................................................................................................................... 284.1.5 Deleting a Cluster...................................................................................................................................................... 304.1.6 Stopping a Cluster..................................................................................................................................................... 314.1.7 Starting a Cluster....................................................................................................................................................... 324.2 Link Management.........................................................................................................................................................334.2.1 Creating a Link.......................................................................................................................................................... 344.2.2 Link Parameter Description.......................................................................................................................................394.2.2.1 Link to a Relational Database.................................................................................................................................394.2.2.2 Link to OBS............................................................................................................................................................424.2.2.3 Link to OSS on Alibaba Cloud...............................................................................................................................434.2.2.4 Link to Qiniu Cloud Object Storage (KODO)........................................................................................................444.2.2.5 Link to HDFS......................................................................................................................................................... 454.2.2.6 Link to HBase.........................................................................................................................................................484.2.2.7 Link to CloudTable................................................................................................................................................. 504.2.2.8 Link to Hive............................................................................................................................................................514.2.2.9 Link to an FTP or SFTP Server.............................................................................................................................. 524.2.2.10 Link to MongoDB/DDS....................................................................................................................................... 534.2.2.11 Link to Redis/DCS................................................................................................................................................544.2.2.12 Link to NAS..........................................................................................................................................................55

Cloud Data MigrationAPI Reference Contents

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

ii

Page 4: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

4.2.2.13 Link to Kafka........................................................................................................................................................564.2.2.14 Link to DIS........................................................................................................................................................... 574.2.2.15 Link to Elasticsearch............................................................................................................................................ 584.2.2.16 Link to DLI...........................................................................................................................................................594.2.3 Querying a Link.........................................................................................................................................................604.2.4 Modifying a Link.......................................................................................................................................................634.2.5 Deleting a Link.......................................................................................................................................................... 664.3 Job Management...........................................................................................................................................................674.3.1 Creating a Job............................................................................................................................................................ 674.3.2 Source Job Parameters...............................................................................................................................................714.3.2.1 From a Relational Database....................................................................................................................................714.3.2.2 From OBS/OSS...................................................................................................................................................... 734.3.2.3 From HDFS............................................................................................................................................................ 784.3.2.4 From Hive...............................................................................................................................................................814.3.2.5 From HBase/CloudTable........................................................................................................................................ 824.3.2.6 From FTP/SFTP/NAS............................................................................................................................................ 834.3.2.7 From HTTP/HTTPS............................................................................................................................................... 884.3.2.8 From MongoDB/DDS............................................................................................................................................ 884.3.2.9 From Redis/DCS.....................................................................................................................................................894.3.2.10 From DIS.............................................................................................................................................................. 914.3.2.11 From Kafka........................................................................................................................................................... 924.3.2.12 From Elasticsearch/Cloud Search Service............................................................................................................934.3.3 Destination Job Parameters....................................................................................................................................... 944.3.3.1 To a Relational Database........................................................................................................................................ 944.3.3.2 To OBS................................................................................................................................................................... 964.3.3.3 To HDFS...............................................................................................................................................................1014.3.3.4 To Hive................................................................................................................................................................. 1024.3.3.5 To HBase/CloudTable...........................................................................................................................................1034.3.3.6 To FTP/SFTP/NAS...............................................................................................................................................1054.3.3.7 To DDS................................................................................................................................................................. 1074.3.3.8 To DCS................................................................................................................................................................. 1084.3.3.9 To Elasticsearch/Cloud Search Service................................................................................................................ 1104.3.3.10 To DLI.................................................................................................................................................................1114.3.4 Job Parameter Description....................................................................................................................................... 1124.3.5 Querying a Job......................................................................................................................................................... 1164.3.6 Modifying a Job.......................................................................................................................................................1204.3.7 Starting a Job........................................................................................................................................................... 1224.3.8 Stopping a Job......................................................................................................................................................... 1244.3.9 Querying Job Status.................................................................................................................................................1264.3.10 Querying Job Execution History........................................................................................................................... 1304.3.11 Deleting a Job........................................................................................................................................................ 131

5 Common Parameters................................................................................................................. 133

Cloud Data MigrationAPI Reference Contents

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

iii

Page 5: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

5.1 Error Code.................................................................................................................................................................. 1335.2 Status Code................................................................................................................................................................. 141

A Change History......................................................................................................................... 145

Cloud Data MigrationAPI Reference Contents

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

iv

Page 6: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

1 Before You Start

This document provides detailed information about application programming interfaces(APIs) of Cloud Data Migration (CDM), including API description, syntax, parameters, andexamples. You can switch to your desired content according to Table 1-1.

Table 1-1 Document overview

If You Want to Know Link

API constituents and list API Overview

Representational State Transfer (REST)message bodies, calling methods, andexamples

API Use

API description API Description

Common parameters Common Parameters

Cloud Data MigrationAPI Reference 1 Before You Start

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

1

Page 7: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

2 API Overview

CDM provides Huawei-developed APIs. You can perform the following operations withCDM APIs.

Type API Description

ClusterManagementAPIs

Creating a Cluster Creates a cluster.

Querying the Cluster List Queries and displays the cluster list.

Querying Cluster Details Queries cluster details.

Restarting a Cluster Restarts a cluster.

Deleting a Cluster Deletes a cluster.

Stopping a Cluster Stops a specified cluster.

Starting a Cluster Starts a specified cluster.

LinkManagementAPIs

Creating a Link Connects to a data source.

Querying a Link Queries the link list.

Modifying a Link Modifies the link configuration.

Deleting a Link Deletes a link.

Job ManagementAPIs

Creating a Job Creates a table/file migration job.

Querying a Job Queries and displays the job list.

Modifying a Job Modifies the job configuration.

Starting a Job Starts a data migration job.

Stopping a Job Stops a running job.

Querying Job Status Queries and displays the runningstatus of a job.

Querying Job ExecutionHistory

Queries and displays the historicalstatus of job execution.

Cloud Data MigrationAPI Reference 2 API Overview

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

2

Page 8: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Type API Description

Deleting a Job Deletes a job.

Cloud Data MigrationAPI Reference 2 API Overview

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

3

Page 9: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

3 API Use

3.1 Overview of RESTful APIsHUAWEI CLOUD APIs comply with RESTful API design specifications. RepresentationalState Transfer (REST) allocates Uniform Resource Identifiers (URIs) to dispersed resourcesso that the resources can be located. Applications on clients use unified resource locators(URLs) to obtain the resources.

The URL is in the following format: https://Endpoint/uri. uri indicates the resource path.That is, the API access path.

HUAWEI CLOUD APIs use HTTP as the transfer protocol. Requests/Responses aretransferred by using JSON packets, with media type represented by application/json.

A RESTful API request or response consists of the following five parts:

l Request URIl Request Headerl Request Bodyl Response Headerl Response Body

Sending a Request

You can send a request based on the constructed request body using any of the following threemethods:

l cURLcURL is a command-line tool used to perform URL operations and transfer information.It can serve as an HTTP client to send HTTP requests to the server end and receiveresponse messages. cURL is suitable for use in API tuning scenarios. For moreinformation about cURL, visit https://curl.haxx.se/.

l CodeYou can call APIs through code to assemble, send, and process requests.

l REST clients

Cloud Data MigrationAPI Reference 3 API Use

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

4

Page 10: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Both Mozilla and Google provide a graphical browser plug-in, that is, REST client, tosend and process requests.

– For Mozilla Firefox, see Firefox REST Client.

– For Google Chrome, see Postman Interceptor.

Request URI

Although the request URI is included in the request header, most languages or frameworksrequire that it be transferred separately from the request message. Therefore, the request URIis listed independently.

A request URI consists of the following:

{URI-scheme} :// {Endpoint} / {resource-path} ? {query-string}

Table 3-1 Parameters in a URI

Parameter Description

URI-scheme Protocol used to transfer requests. CDM APIs use https.

Endpoint Domain name or IP address of the server bearing the REST serviceendpoint. Obtain this value from Regions and Endpoints.

resource-path API access path for performing a specified operation. Obtain thevalue from the URI of the API, for example, v3/auth/tokens.

Query string (Optional) For example, API version or resource selection criteria.

Request Header

A request header consists of the following two parts:

l HTTP methods, which are also called operations or actions, specify the type ofoperations that you are requesting. Table 3-2 describes methods supported by CDMRESTful APIs.

Table 3-2 HTTP methods

Method Description

GET Requests the server to return a specific resource.

PUT Requests the server to update a specific resource.

POST Requests the server to add resources or perform special operations.

DELETE Requests the server to delete a specific resource, for example, todelete an object.

HEAD Allows the client to query the server for the headers of a givenresource without actually downloading the resource itself.

Cloud Data MigrationAPI Reference 3 API Use

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

5

Page 11: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Method Description

PATCH Requests the server to apply partial modifications to a resource.If the resource does not exist, a resource can be created using thePATCH method.

l (Optional) Additional fields in the request header required by a specific URI and HTTPmethod.

For details about common request headers in token authentication, see Table 3-3. Fordetails about the request authentication information, see Obtaining RequestAuthentication Information.

Table 3-3 Common request headers in token authentication

Parameter Description Mandatory Example Value

x-sdk-date Time when therequest is sent. Thetime is inYYYYMMDD'T'HHMMSS'Z' format.The value is thecurrent GreenwichMean Time (GMT)time of the system.

No 20150907T101459Z

Authorization AuthenticationinformationThe value can beobtained from therequest signingresult.

No SDK-HMAC-SHA256Credential=ZIRRKMTWPTQFQI1WKNKB/20150907//ec2/sdk_request,SignedHeaders=content-type;host;x-sdk-date,Signature=55741b610f3c9fa3ae40b5a8021ebf7ebc2a28a603fc62d25cb3bfe6608e1994

Host Information aboutthe requested server.The value can beobtained from theURL of the serviceAPI inhostname[:port]format. If the portnumber is notspecified, the defaultport is used. Thedefault port numberfor https is 443.

No code.test.comorcode.test.com:443

Cloud Data MigrationAPI Reference 3 API Use

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

6

Page 12: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Parameter Description Mandatory Example Value

Content-type Request body MIMEtype

Yes application/json

Content-Length

Length of therequest body. Theunit is byte.

This parameteris mandatoryfor POST andPUT requestsbut must beleft blank forGET requests.

3495

X-Project-Id Project ID. Thisparameter is used toobtain the token forthe project.This parameter ismandatory for arequest from a DeCor multi-project user.

No e9993fc787d94b6c886cbaa340f9c0f4

X-Auth-Token User token Yes -

X-Language Request language Yes zh_cn

NOTE

For details about other parameters in the header, see the HTTP protocol documentation.

(Optional) Request BodyA request body is generally sent in a structured format (for example, JSON or XML),corresponding to Content-type in the request header, and is used to transfer content otherthan the request header.

If the request body contains Chinese characters, convert the Chinese characters into theUTF-8 encoding format.

Response HeaderA response header consists of the following two parts:

l An HTTP status code, from 2xx success code to 4xx or 5xx error code, or the status codedefined by the service. See Status Code.

l Additional fields in the response header required by a specified response, for example,the Content-type response header. For details about common response headers, seeTable 3-4.

Cloud Data MigrationAPI Reference 3 API Use

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

7

Page 13: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Table 3-4 Common response headers

Parameter Description Example Value

Date Standard HTTP header, whichspecifies the date and time at whichthe message was sent. The format isdefined by RFC 822.

Mon, 12 Nov 200715:55:01 GMT

Server Standard HTTP header, whichincludes the software informationthat the server uses to process therequest.

Apache

Content-Length Standard HTTP header, whichspecifies the size of the entity body,in decimal number of bytes, sent tothe recipient.

-

Content-Type Standard HTTP header, whichspecifies the media type of theentity body sent to the recipient.

application/json

(Optional) Response Body

The response body is generally returned in a structured format (for example, JSON or XML),corresponding to Content-type in the response header, and is used to transfer content otherthan the response header.

3.2 Obtaining Request Authentication InformationCDM supports token authentication in API calling. You can use tokens for authentication.

3.2.1 Token Authentication

Scenarios

To use token authentication, obtain the user token, and add X-Auth-Token to the requestheader of the service API when making an API call.

This section describes how to make an API call using token authentication.

Procedure1. Send POST https://Endpoint of IAM/v3/auth/tokens to obtain the endpoint of IAM and

the region name in the message body.

See Regions and Endpoints.

If the service region name is ALL, replace Endpoint of IAM with the endpoint of CNNorth-Beijing1.

The following code provides a sample request.

Cloud Data MigrationAPI Reference 3 API Use

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

8

Page 14: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

NOTE

Replace the items in italic in the following example with actual ones. For details, see the Identityand Access Management API Reference.

{ "auth": { "identity": { "methods": [ "password" ], "password": { "user": { "name": "Username", "password": "Password", "domain": { "name": "Domain name" } } } }, "scope": { "project": { "id": "0215ef11e49d4743be23dd97a1561e91" //This ID is used as an example. } } }}

2. Obtain the token by performing the operations in section "Obtaining a User Token" inthe Identity and Access Management API Reference. The token value is the X-Subject-Token value in the response header.

3. Make a call to a service API, add X-Auth-Token to the message header, and set thevalue of X-Auth-Token to the token obtained in 2.

3.3 Obtaining a Project IDA project ID is required for some URLs when an API is called. It can be project_id ortenant_id because project_id has the same meaning as tenant_id in this document. Beforecalling an API, you need to obtain a project ID on the console. The steps are as follows:

1. Log in to the management console.2. Hover the cursor on the username and select My Credential from the drop-down list.

On the page that is displayed, view the project ID in the project list.

Figure 3-1 Viewing the project ID

Cloud Data MigrationAPI Reference 3 API Use

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

9

Page 15: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

3.4 API Use CaseThis section describes how to use CDM APIs with an example of creating a table/filemigration job by making API calls.

The process of creating a CDM job to migrate data from the local MySQL database to DataWarehouse Service (DWS) on the cloud is as follows:

1. Call the API in Token Authentication to obtain the user token, which will be put intothe request header for authentication in a subsequent request.

2. Create a cluster and obtain the cluster ID.

NOTE

If you have created a cluster, skip the steps for creating a cluster and directly use the createdcluster ID.

a. Obtain the project ID.b. Call the API in Creating a Cluster to create a cluster.c. Call the API in Querying Cluster Details to obtain the cluster ID.

3. Call the API in Creating a Link to create the MySQL link and DWS link.4. Call the API in Creating a Job to create a job of migrating data from MySQL to DWS.5. Call the API in Starting a Job to execute the job.

Prerequisitesl You have created a VPC, subnet, and elastic IP address (EIP), and obtained the IDs of

the VPC, subnet, and security group. For details, see the Virtual Private Cloud APIReference.

l You have obtained the endpoints of IAM and CDM. For details, see Regions andEndpoints.

l You can access the local MySQL database using a public IP address, and have obtainedthe IP address, port number, database name, username, and password of the MySQLdatabase. The account has the read, write, and delete permissions on the MySQLdatabase.

l You have purchased a DWS instance and obtained the IP address, port number, databasename, username, and password for accessing DWS. The account has the read, write, anddelete permissions on the database installed on DWS.

Creating a JobThe following values are examples (replace them based on the actual situation).

l IAM endpoint: iam_endpointl CDM endpoint: cdm_endpointl Project ID: 1551c7f6c808414d8e9f3c514a170f2el VPC ID (the same as that of DWS): 6b47302a-bf79-4b20-bf7a-80987408e196l Subnet ID (the same as that of DWS): 63bdc3cb-a4e7-486f-82ee-d9bf208c8f8cl Security group ID (the same as that of DWS): 005af77a-cce5-45ac-99c7-2ea50ea8addf

Perform the following operations to create a job.

Cloud Data MigrationAPI Reference 3 API Use

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

10

Page 16: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

1. Before calling other APIs, obtain the token and set it as an environment variable.curl -H "Content-Type:application/json" https://{iam_endpoint}/v3/auth/tokens -X POST -d '{ "auth": { "identity": { "methods": [ "password" ], "password": { "user": { "name": "testname", "domain": { "name": "testname" }, "password": "Password" } } }, "scope": { "project": { "name": "cn-north-1" } } }}' -v -k

The value of X-Subject-Token in the response header is the token.X-Subject-Token:MIIDkgYJKoZIhvcNAQcCoIIDgzCCA38CAQExDTALBglghkgBZQMEAgEwgXXXXX...

Run the following command to set the token as an environment variable for future use:export X-Auth-Token = MIIDkgYJKoZIhvcNAQcCoIIDgzCCA38CAQExDTALBglghkgBZQMEAgEwgXXXXX...

2. Call the API in Creating a Cluster to create a cluster. The following values areexamples:

– Cluster name: cdm-ab82

– Cluster flavor: cdm.medium

– The VPC, subnet, and security group are the same as those of DWS, and the EIP isautomatically bound.

curl -X POST -H 'Content-Type:application/json;charset=utf-8' -H "X-Auth-Token:$Token" -d '{ "cluster": { "name": "cdm-ab82", "vpcId": "6b47302a-bf79-4b20-bf7a-80987408e196", "instances": [{ "specCode": "cdm.medium", "nics": [{ "net-id": "63bdc3cb-a4e7-486f-82ee-d9bf208c8f8c", "securityGroupId": "005af77a-cce5-45ac-99c7-2ea50ea8addf" }], "availability_zone": "cn-north-1b", "type": "cdm", "publicIP": { "publicBindType": "auto_assign" } }] }}' https://{cdm_endpoint}/cdm/v1.0/1551c7f6c808414d8e9f3c514a170f2e/clusters -v -k

If status code 200 is returned, the cluster is successfully created.

3. Call the API in Querying the Cluster List to query cluster information, obtain thecluster ID, and set the cluster ID to a global variable.

Cloud Data MigrationAPI Reference 3 API Use

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

11

Page 17: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

curl -X GET -H 'Content-Type:application/json;charset=utf-8' -H "X-Auth-Token:$Token" https://{cdm_endpoint}/cdm/v1.0/1551c7f6c808414d8e9f3c514a170f2e/clusters -k –vThe response is as follows:{ "clusters": [{ "version": "1.0.0", "updated": "2017-09-05T08:38:25", "name": "cdm-ab82", "created": "2017-09-05T08:38:25", "id": "bae65496-643e-47ca-84af-948672de7eeb", "status": "200", "isFrozen": "0", "statusDetail": "Normal", "actionProgress": { }, "config_status": "In-Sync" }]}If the value of status is 200, the cluster is successfully created. The cluster ID isbae65496-643e-47ca-84af-948672de7eeb.Run the following command to set the cluster ID to a global variable for future use:export ID = bae65496-643e-47ca-84af-948672de7eeb

4. Call the API in Creating a Link to create the MySQL link mysql_link. The followingvalues are examples:– Public IP address: 1xx.120.85.24– Port number: 3306– Database name: DB_name– Login username: username– Password: DB_passwordcurl -X POST -H "Content-Type:application/json" -H "X-Auth-Token:$Token" -d '{ "links": [{ "enabled": true, "update-user": null, "name": "mysql_link", "link-config-values": { "configs": [ { "name": "linkConfig", "inputs": [ { "name": "linkConfig.databaseType", "value": "MYSQL" }, { "name": "linkConfig.host", "value": "1xx.120.85.24" }, { "name": "linkConfig.port", "value": "3306" }, { "name": "linkConfig.database", "value": "DB_name" }, { "name": "linkConfig.username", "value": "username" }, { "name": "linkConfig.password",

Cloud Data MigrationAPI Reference 3 API Use

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

12

Page 18: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

"value": "DB_password" }, { "name": "linkConfig.fetchSize", "value": "100000" }, { "name": "linkConfig.usingNative", "value": "true" } ] } ] }, "connector-name": "generic-jdbc-connector", "creation-date": 1496654788622, "update-date": 1496654788622, "creation-user": null }]}' https://{cdm_endpoint}/cdm/v1.0/1551c7f6c808414d8e9f3c514a170f2e/clusters/bae65496-643e-47ca-84af-948672de7eeb/cdm/link -k -vIf status code 200 is returned, the link is successfully created.

5. Call the API in Creating a Link to create the DWS link dws_link. The following valuesare examples:– IP address of the database: 10.120.85.24– Port number: 3306– Database name: DWS– Login username: user_dws– Password: dws_passwordcurl -X POST -H "Content-Type:application/json" -H "X-Auth-Token:$Token" -d '{ "links": [{ "enabled": true, "update-user": null, "name": "dws_link", "link-config-values": { "configs": [ { "name": "linkConfig", "inputs": [ { "name": "linkConfig.databaseType", "value": "DWS" }, { "name": "linkConfig.host", "value": "10.120.85.24" }, { "name": "linkConfig.port", "value": "3306" }, { "name": "linkConfig.database", "value": "DWS" }, { "name": "linkConfig.username", "value": "user_dws" }, { "name": "linkConfig.password", "value": "dws_password" },

Cloud Data MigrationAPI Reference 3 API Use

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

13

Page 19: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

{ "name": "linkConfig.fetchSize", "value": "100000" }, { "name": "linkConfig.usingNative", "value": "true" } ] } ] }, "connector-name": "generic-jdbc-connector", "creation-date": 1496654789622, "update-date": 1496654789622, "creation-user": null }]}' https://{cdm_endpoint}/cdm/v1.0/1551c7f6c808414d8e9f3c514a170f2e/clusters/bae65496-643e-47ca-84af-948672de7eeb/cdm/link -k -vIf status code 200 is returned, the link is successfully created.

6. After the two links are created, call the API in Creating a Job to create a migration job.The following is a sample job:– The job name is mysql2dws.– The name of the MySQL database from which data is exported is default, and the

name of the exported table is mysql_tbl. The job is split into multiple tasks by idand the tasks are executed concurrently.

– The name of the database on DWS to which the data is imported is public, and thetable name is cdm_all_type. Do not clear the data in the table before import.

– If no table in the local MySQL database exists in the database on DWS, CDMautomatically creates the table on DWS.

– The field list loaded to DWS is id&gid&name.– When the job extracts data, a number of three Extractor tasks are concurrently

executed.curl -X POST -H "Content-Type:application/json" -H "X-Cluster-ID:$ID" -H "X-Auth-Token:$Token" -d '{ "jobs": [{ "job_type": "NORMAL_JOB", "name": "mysql2dws", "from-link-name": "mysql_link", "from-connector-name": "generic-jdbc-connector", "to-link-name": "dws_link", "to-connector-name": "generic-jdbc-connector", "from-config-values": { "configs": [{ "name": "fromJobConfig", "inputs": [{ "name": "fromJobConfig.schemaName", "value": "default" }, { "name": "fromJobConfig.tableName", "value": "mysql_tbl" }, { "name": "fromJobConfig.partitionColumn", "value": "id" }] }] },"to-config-values": { "configs": [

Cloud Data MigrationAPI Reference 3 API Use

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

14

Page 20: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

{ "inputs": [ { "name": "toJobConfig.schemaName", "value": "public" }, { "name": "toJobConfig.tablePreparation", "value": "CREATE_WHEN_NOT_EXIST" }, { "name": "toJobConfig.tableName", "value": "cdm_all_type" }, { "name": "toJobConfig.columnList", "value": "id&gid&name" }, { "name": "toJobConfig.shouldClearTable", "value": "false" } ], "name": "toJobConfig" } ] }, "driver-config-values": { "configs": [{ "name": "throttlingConfig", "inputs": [{ "name": "throttlingConfig.numExtractors", "value": "3" }] }] } }]}' https://{cdm_endpoint}/cdm/v1.0/1551c7f6c808414d8e9f3c514a170f2e/clusters/bae65496-643e-47ca-84af-948672de7eeb/cdm/job -k -vIf status code 200 is returned, the job is successfully created.

7. Call the API in Starting a Job to execute the job.curl -X GET -H 'Content-Type:application/json;charset=utf-8' -H "X-Cluster-ID:$ID" -H "X-Auth-Token:$Token" https://{cdm_endpoint}/cdm/v1.0/1551c7f6c808414d8e9f3c514a170f2e/clusters/bae65496-643e-47ca-84af-948672de7eeb/cdm/job/mysql2dws/start -k -vThe response is as follows:{ "submissions": [{ "progress": 1, "job-name": "mysql2dws", "status": "BOOTING", "creation-date": 1496905778725, "creation-user": "cdm" }]}

8. Call the API in Querying Job Status to query the job status.curl -X GET -H 'Content-Type:application/json;charset=utf-8' -H "X-Cluster-ID:$ID" -H "X-Auth-Token:$Token" https://{cdm_endpoint}/cdm/v1.0/1551c7f6c808414d8e9f3c514a170f2e/clusters/6ec9a0a4-76be-4262-8697-e7af1fac7920/cdm/job/mysql2dws/status -k -vIf the job status is successfully queried, the response is as follows:{ "submissions": [{ "progress": 0, "job-name": "mysql2dws", "status": "SUCCEEDED",

Cloud Data MigrationAPI Reference 3 API Use

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

15

Page 21: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

"creation-date": 1496906582997, "creation-user": "cdm", "isStopingIncrement": "", "last-update-date": 1496906580005, "is-execute-auto": false, "last-udpate-user": "cdm", "isDeleteJob": false, "isIncrementing": false, "external-id": "job_local1127970451_0009", "counters": { "org.apache.sqoop.submission.counter.SqoopCounters": { "BYTES_WRITTEN": -1, "TOTAL_FILES": -1, "BYTES_READ": -1, "FILES_WRITTEN": -1, "TOTAL_SIZE": -1, "FILES_READ": -1, "ROWS_WRITTEN": 80, "ROWS_READ": 80 } } }]}

NOTE

l BYTES_WRITTEN: number of written bytes

l BYTES_READ: number of read bytes

l TOTAL_FILES: total number of files

l FILES_WRITTEN: number of written files

l FILES_READ: number of read files

l ROWS_WRITTEN: number of rows that are successfully written

l ROWS_READ: number of rows that are successfully read

Cloud Data MigrationAPI Reference 3 API Use

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

16

Page 22: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

4 API Description

4.1 Cluster Management

4.1.1 Creating a Cluster

FunctionThis API is used to create a CDM cluster. Creating a CDM cluster is an asynchronousoperation. You can query the creation progress and result by periodically calling the API inQuerying Cluster Details.

URIl URI format

POST /cdm/v1.0/{project_id}/clustersl Parameter description

Parameter Mandatory Type Description

project_id Yes String Project ID. For detailsabout how to obtain theproject ID, see Obtaininga Project ID.

Requestl Sample request

POST /cdm/v1.0/1551c7f6c808414d8e9f3c514a170f2e/clusters{ "cluster": { "name": "cdm-ab82", "vpcId": "6b47302a-bf79-4b20-bf7a-80987408e196", "instances": [{ "specCode": "cdm.medium", "nics": [{ "net-id": "63bdc3cb-a4e7-486f-82ee-d9bf208c8f8c", "securityGroupId": "005af77a-cce5-45ac-99c7-2ea50ea8addf"

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

17

Page 23: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

}], "availability_zone": "cn-north-1b", "type": "cdm", "publicIP": { "publicBindType": "auto_assign" } }] }}

l Parameter description

Parameter Mandatory Type Description

cluster Yes List Cluster object. Fordetails, see Descriptionof the clusterparameter.

l Description of the cluster parameter

Parameter Mandatory Type Description

name Yes String Cluster name

vpcId Yes String VPC ID, which is usedfor configuring the clusternetwork

instances Yes List Node list. For details, seeDescription of theinstances parameter.

l Description of the instances parameter

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

18

Page 24: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Parameter Mandatory Type Description

specCode Yes String Instance flavor. The optionsare as follows:l cdm.small: 2 vCPUs with

4 GB memory, applicableto Proof of Concept(PoC) verification anddevelopment tests

l cdm.medium: 4 vCPUswith 8 GB memory,applicable to migration ofa single database tablewith fewer than 10million pieces of data

l cdm.large: 8 vCPUs with16 GB memory,applicable to migration ofa single database tablewith 10 million pieces ofdata or more

l cdm.xlarge: 16 vCPUswith 32 GB memory,applicable to TB-leveldata migration requiring10GE high-speedbandwidth

nics Yes List NIC list. A maximum of twoNICs are supported. Fordetails, see Description ofthe nics parameter.

availability_zone Yes String Currently, CDM supports thefollowing AZs:l CN North-Beijng1:

– cn-north-1a– cn-north-1b

l CN East-Shanghai2:– cn-east-2a– cn-east-2b– cn-east-2c

l CN South-Guangzhou:– cn-south-1a– cn-south-2b– cn-south-1c

type Yes String Node type. Currently, onlycdm is available.

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

19

Page 25: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Parameter Mandatory Type Description

publicIP No Dictionary EIP. This parameter is notrequired when no EIP isbound. For details, seeDescription of the publicIPparameter.

l Description of the nics parameter

Parameter Mandatory Type Description

net-id Yes String Subnet ID

securityGroupId Yes String Security group ID

l Description of the publicIP parameter

Parameter Mandatory Type Description

publicBindType Yes String Currently, this parameter hasonly one value.auto_assign indicates that theelastic IP address isdynamically allocated.

Responsel Sample response

{ "id": "7d85f602-a948-4a30-afd4-e84f47471c15", "name": "cdm-ab82"}

l Parameter description

Parameter Mandatory Type Description

id Yes String Cluster ID

name Yes Sring Cluster name

Return Valuel Normal

200l Abnormal

Return Value Description

400 Bad Request Request error.

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

20

Page 26: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Return Value Description

401 Unauthorized Authentication failed.

403 Forbidden No operation permission.

404 Not Found The requested resource is not found.

500 Internal Server Error Internal error. For details about the returnederror code, see Error Code.

503 Service Unavailable Service unavailable.

4.1.2 Querying the Cluster List

Function

This API is used to query the CDM cluster list.

URIl URI format

GET /cdm/v1.0/{project_id}/clustersl Parameter description

Parameter Mandatory Type Description

project_id Yes String Project ID

Request

Sample requestGET /cdm/v1.0/1551c7f6c808414d8e9f3c514a170f2e/clusters

Responsel Sample response

{ "clusters": [{ "version": "1.0.0", "updated": "2017-09-05T08:38:25", "name": "cdm-c018", "created": "2017-09-05T08:38:25", "id": "bae65496-643e-47ca-84af-948672de7eeb", "status": "200", "isFrozen": "0", "statusDetail": "Normal", "actionProgress": { }, "config_status": "In-Sync" }]}

l Parameter description

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

21

Page 27: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Parameter Mandatory Type Description

clusters Yes List Cluster list. For details,see Description of theclusters parameter.

l Description of the clusters parameter

Parameter Mandatory Type Description

version Yes String Cluster version

updated Yes String Time when a cluster isupdated. The format isYYYY-MM-DDThh:mm:ssZ(ISO 8601).

name Yes String Cluster name

created Yes String Time when a cluster iscreated. The format isYYYY-MM-DDThh:mm:ssZ(ISO 8601).

id Yes String Cluster ID

status Yes String Cluster status. The optionsare as follows:l 100: Creatingl 200: Runningl 300: Failedl 303: Creation failedl 400: Deletedl 800: Frozenl 900: Stoppedl 910: Stoppingl 920: Starting

isFrozen Yes String Whether a cluster is frozen.The options are as follows:l 0: Nol 1: Yes

statusDetail No String Cluster status description

actionProgress No Dictionary Cluster operation progress.For details, see Descriptionof the actionProgressparameter.

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

22

Page 28: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Parameter Mandatory Type Description

config_status No String Cluster configurationstatus. The options are asfollows:l In-Sync: Configuration

synchronizedl Applying: Being

configuredl Sync-Failure:

Configuration failed

l Description of the actionProgress parameter

Parameter Mandatory Type Description

CREATING No String Cluster creation progress. Forexample, 29%.

Return Valuel Normal

200l Abnormal

Return Value Description

400 Bad Request Request error.

401 Unauthorized Authentication failed.

403 Forbidden No operation permission.

404 Not Found The requested resource is not found.

500 Internal Server Error Internal error. For details about the returnederror code, see Error Code.

503 Service Unavailable Service unavailable.

4.1.3 Querying Cluster Details

Function

This API is used to query CDM cluster details.

URIl URI format

GET /cdm/v1.0/{project_id}/clusters/{cluster_id}

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

23

Page 29: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

l Parameter description

Parameter Mandatory Type Description

project_id Yes String Project ID

cluster_id Yes String Cluster ID

RequestSample requestGET /cdm/v1.0/1551c7f6c808414d8e9f3c514a170f2e/clusters/bae65496-643e-47ca-84af-948672de7eeb

Responsel Sample response

{ "version": "1.0.0", "instances": [{ "flavor": { "id": "fb8fe666-6734-4b11-bc6c-43d11db3c745" }, "volume": { "type": "LOCAL_DISK", "size": 100 }, "status": "200", "actions": ["REBOOTING"], "type": "cdm", "id": "635dce67-3df8-4756-b4c7-90e45e687367", "name": "cdm-c018", "isFrozen": "0", "config_status": "In-Sync" }], "updated": "2017-09-05T08:38:25", "name": "cdm-c018", "created": "2017-09-05T08:38:25", "id": "bae65496-643e-47ca-84af-948672de7eeb", "status": "200", "actions": ["REBOOTING"], "isFrozen": "0", "statusDetail": "Normal", "actionProgress": { }, "config_status": "In-Sync"}

l Parameter description

Parameter Mandatory Type Description

version Yes String Cluster version

instances Yes List Cluster node information.For details, see Descriptionof the instances parameter.

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

24

Page 30: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Parameter Mandatory Type Description

updated Yes String Time when a cluster isupdated. The format isYYYY-MM-DDThh:mm:ssZ(ISO 8601).

name Yes String Cluster name

created Yes String Time when a cluster iscreated. The format is YYYY-MM-DDThh:mm:ssZ (ISO8601).

id Yes String Cluster ID

status Yes String Cluster status. The optionsare as follows:l 100: Creatingl 200: Runningl 300: Failedl 303: Creation failedl 400: Deletedl 800: Frozenl 900: Stoppedl 910: Stoppingl 920: Starting

actions No String Cluster operation status list.The options are as follows:l REBOOTING:

Restartingl RESTORING:

Restoringl REBOOT_FAILURE:

Restart failed

isFrozen Yes String Whether a cluster is frozen.The options are as follows:l 0: Nol 1: Yes

statusDetail No String Cluster status description

actionProgress No List Cluster operation progress.For details, see Descriptionof the actionProgressparameter.

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

25

Page 31: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Parameter Mandatory Type Description

config_status No String Cluster configuration status.The options are as follows:l In-Sync: Configuration

synchronizedl Applying: Being

configuredl Sync-Failure:

Configuration failed

l Description of the instances parameter

Parameter Mandatory Type Description

flavor Yes Dictionary VM flavor of a node. Fordetails, see Description ofthe flavor parameter.

volume Yes Dictionary Disk information of a node.For details, see Descriptionof the volume parameter.

status Yes String Node status. The options areas follows:l 100: Creatingl 200: Normall 300: Failedl 303: Creation failedl 400: Deletedl 800: Frozen

actions No String list Node operation status list.The options are as follows:l REBOOTING: Restartingl RESTORING: Restoringl REBOOT_FAILURE:

Restart failed

type Yes String Node type. Currently, onlycdm is available.

id Yes String VM ID

name Yes Strng VM name

isFrozen Yes String Whether the node is frozen.The options are as follows:l 0: Nol 1: Yes

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

26

Page 32: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Parameter Mandatory Type Description

config_status No String Node configuration status.The options are as follows:l In-Sync: Configuration

synchronizedl Applying: Being

configuredl Sync-Failure:

Configuration failed

l Description of the flavor parameter

Parameter Mandatory Type Description

id Yes String VM flavor ID

l Description of the volume parameter

Parameter Mandatory Type Description

type Yes String Type of disks on the node. Onlylocal disks are supported.

size Yes Integer Size of the disk on the node(GB)

l Description of the actionProgress parameter

Parameter Mandatory Type Description

CREATING No String Cluster creation progress. Forexample, 29%.

Return Valuel Normal

200

l Abnormal

Return Value Description

400 Bad Request Request error.

401 Unauthorized Authentication failed.

403 Forbidden No operation permission.

404 Not Found The requested resource is not found.

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

27

Page 33: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Return Value Description

500 Internal Server Error Internal error. For details about the returnederror code, see Error Code.

503 Service Unavailable Service unavailable.

4.1.4 Restarting a Cluster

FunctionThis API is used to restart a CDM cluster. Restarting a CDM cluster is an asynchronousoperation. You can query the restart result by periodically calling the API in QueryingCluster Details.

URIl URI format

POST /cdm/v1.0/{project_id}/clusters/{cluster_id}/actionl Parameter description

Parameter Mandatory Type Description

project_id Yes String Project ID

cluster_id Yes String Cluster ID

Requestl Sample request

POST /cdm/v1.0/1551c7f6c808414d8e9f3c514a170f2e/clusters/bae65496-643e-47ca-84af-948672de7eeb/action{ "restart": { "type": "cdm" }}

l Parameter description

Parameter Mandatory Type Description

restart Yes Dictionary Cluster restart. For detailsabout how to define clusterrestart, see Description ofthe restart parameter.

l Description of the restart parameter

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

28

Page 34: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Parameter Mandatory Type Description

type Yes String Type of the cluster to berestarted. The parameter can beset to cdm only.

restartMode No Enumeration Restart mode. The options are asfollows:l IMMEDIATELY:

Immediate restartl GRACEFULL: Graceful

restartl FORCELY: Forcible restartThe default value isIMMEDIATELY.

restartLevel No Enumeration Restart level. The options are asfollows:l SERVICE: Service restartl VM: VM restartThe default value is SERVICE.

restartDelayTime

No Integer Restart delay, in seconds

Responsel Sample response

{ "jobId": "ff8080815e59d92d015e5b27ccb0004d"}

l Parameter description

Parameter Mandatory Type Description

jobId No String Job ID

Return Valuel Normal

200l Abnormal

Return Value Description

400 Bad Request Request error.

401 Unauthorized Authentication failed.

403 Forbidden No operation permission.

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

29

Page 35: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Return Value Description

404 Not Found The requested resource is not found.

500 Internal Server Error Internal error. For details about the returnederror code, see Error Code.

503 Service Unavailable Service unavailable.

4.1.5 Deleting a Cluster

Function

This API is used to delete a specified CDM cluster. Deleting a CDM cluster is anasynchronous operation. You can query the deletion result by periodically calling the API inQuerying Cluster Details.

URIl URI format

DELETE /cdm/v1.0/{project_id}/clusters/{cluster_id}l Parameter description

Parameter Mandatory Type Description

project_id Yes String Project ID

cluster_id Yes String Cluster ID

Request

Sample requestDELETE /cdm/v1.0/1551c7f6c808414d8e9f3c514a170f2e/clusters/6ec9a0a4-76be-4262-8697-e7af1fac7920

Responsel Sample response

{ "jobId": "ff8080815e55125a015e552eddba001a"}

l Parameter description

Parameter Mandatory Type Description

jobId No String Job ID

Return Valuel Normal

200

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

30

Page 36: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

l Abnormal

Return Value Description

400 Bad Request Request error.

401 Unauthorized Authentication failed.

403 Forbidden No operation permission.

404 Not Found The requested resource is not found.

500 Internal Server Error Internal error. For details about the returnederror code, see Error Code.

503 Service Unavailable Service unavailable.

4.1.6 Stopping a Cluster

Function

This API is used to stop a specified CDM cluster. Stopping a CDM cluster is an asynchronousoperation. You can query the stop result by periodically calling the API in Querying ClusterDetails. A stopped cluster cannot execute migrations jobs or access the Job Managementpage. You are charged lower fees for a stopped cluster than a running cluster. When a CDMcluster is no longer used, you can call this API to save costs.

URIl URI format

POST /cdm/v1.0/{project_id}/clusters/{cluster_id}/actionl Parameter description

Parameter Mandatory Type Description

project_id Yes String Project ID

cluster_id Yes String Cluster ID

Requestl Sample request

POST /cdm/v1.0/1551c7f6c808414d8e9f3c514a170f2e/clusters/bae65496-643e-47ca-84af-948672de7eeb/action{ "stop": { }}

l Parameter description

Parameter Mandatory Type Description

stop Yes Dictionary Stopping a cluster

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

31

Page 37: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Responsel Sample response

{ "jobId": "ff8080815e59d92d015e5b27ccb0004d"}

l Parameter description

Parameter Mandatory Type Description

jobId No String Job ID

Return Valuel Normal

200l Abnormal

Return Value Description

400 Bad Request Request error.

401 Unauthorized Authentication failed.

403 Forbidden No operation permission.

404 Not Found The requested resource is not found.

500 Internal Server Error Internal error. For details about the returnederror code, see Error Code.

503 Service Unavailable Service unavailable.

4.1.7 Starting a Cluster

Function

This API is used to start a specified CDM cluster. Starting a CDM cluster is an asynchronousoperation. You can query the start result by periodically calling the API in Querying ClusterDetails. When a cluster is stopped, you can call this API to start the cluster.

URIl URI format

POST /cdm/v1.0/{project_id}/clusters/{cluster_id}/actionl Parameter description

Parameter Mandatory Type Description

project_id Yes String Project ID

cluster_id Yes String Cluster ID

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

32

Page 38: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Requestl Sample request

POST /cdm/v1.0/1551c7f6c808414d8e9f3c514a170f2e/clusters/bae65496-643e-47ca-84af-948672de7eeb/action{ "start": { }}

l Parameter description

Parameter Mandatory Type Description

start Yes Dictionary Starting a cluster

Responsel Sample response

{ "jobId": "ff8080815e59d92d015e5b27ccb0004d"}

l Parameter description

Parameter Mandatory Type Description

jobId No String Job ID

Return Valuel Normal

200

l Abnormal

Return Value Description

400 Bad Request Request error.

401 Unauthorized Authentication failed.

403 Forbidden No operation permission.

404 Not Found The requested resource is not found.

500 Internal Server Error Internal error. For details about the returnederror code, see Error Code.

503 Service Unavailable Service unavailable.

4.2 Link Management

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

33

Page 39: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

4.2.1 Creating a Link

Function

When you create links for connectors, required parameters of links vary with connectors. Forexample, to create a link for the JDBC connector, you must enter the IP address, port number,username, and password of the database server.

URIl URI format

POST /cdm/v1.0/{project_id}/clusters/{cluster_id}/cdm/link?validate={validate}

l Parameter description

Parameter Mandatory Type Description

project_id Yes String Project ID

cluster_id Yes String Cluster ID

Requestl Sample request

POST /cdm/v1.0/1551c7f6c808414d8e9f3c514a170f2e/clusters/6ec9a0a4-76be-4262-8697-e7af1fac7920/cdm/link

{ "links": [{ "enabled": true, "update-user": null, "name": "mysql_link", "link-config-values": { "configs": [ { "name": "linkConfig", "inputs": [ { "name": "linkConfig.databaseType", "value": "MYSQL" }, { "name": "linkConfig.host", "value": "10.120.85.24" }, { "name": "linkConfig.port", "value": "3306" }, { "name": "linkConfig.database", "value": "DB_name" }, { "name": "linkConfig.username", "value": "username" }, { "name": "linkConfig.password", "value": "DB_password" },

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

34

Page 40: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

{ "name": "linkConfig.fetchSize", "value": "100000" }, { "name": "linkConfig.usingNative", "value": "true" } ] } ] }, "connector-name": "generic-jdbc-connector", "creation-date": 1496654788622, "update-date": 1496654788622, "creation-user": null }]}

l Parameter description

Parameter Mandatory Type Description

links Yes List Link list. For details, seeDescription of the linksparameter.

l Description of the links parameter

Parameter Mandatory Type Description

enabled No Boolean Whether to activate a link.The default value is true.

update-user No String User who updated a link

name Yes String Link name

link-config-values Yes Dictionary Link configuration. Fordetails, see Description ofthe link-config-valuesparameter.

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

35

Page 41: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Parameter Mandatory Type Description

connector-name Yes String Connector name. Theoptions are as follows:l generic-jdbc-connector:

relational databaseconnector

l obs-connector:HUAWEI CLOUD OBSconnector or AlibabaCloud OSS connector

l hdfs-connector: HDFSconnector

l hive-connector: Hiveconnector

l hbase-connector: HBaseconnector

l ftp-connector: FTPconnector

l sftp-connector: SFTPconnector

l mangodb-connector:MangoDB connector

l redis-connector: Redisconnector

l nas-connector: NASconnector

l elasticsearch-connector:Elasticsearch connector

l kafka-connector: Kafkaconnector

l dis-connector: DataIngestion Service (DIS)connector

l dli-connector: Data LakeInsight (DLI) connector

l http-connector: HTTP/HTTPS connector

l thirdparty-obs-connector: third-partyobject storage connector

creation-date No Timestamp Time when a link is created

update-date No Timestamp Time when a link is updated

creation-user No String User who created a link

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

36

Page 42: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Parameter Mandatory Type Description

validate No Boolean When the parameter is set totrue, the API only validateswhether the parameters arecorrectly configured, but notcreate any link.

l Description of the link-config-values parameter

Parameter Mandatory Type Description

configs Yes List Data structure of linkconfiguration items. Fordetails, see Description of theconfigs parameter.

l Description of the configs parameter

Parameter Mandatory Type Description

name Yes String Configuration item name

inputs Yes List Input parameter list. Eachelement in the list is inname,value format. Fordetails, see Description ofthe inputs parameter.For details about theparameter names, see LinkParameter Description.

l Description of the inputs parameter

Parameter Mandatory Type Description

name Yes String Parameter name

value Yes String Parameter value

Responsel Successful sample response

{ "name": "rdb_link", "validation-result": [{ }]}

l Failed sample response{ "validation-result": [{ "linkConfig": [{

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

37

Page 43: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

"message": "Can't connect to the database with given credentials: The authentication type 12 is not supported. Check that you have configured the pg_hba.conf file to include the client's IP address or subnet, and that it is using an authentication scheme supported by the driver.", "status": "ERROR" }] }]}

l Parameter description

Parameter Mandatory Type Description

name Yes String Link name

validation-result Yes List Validation structurel If a link fails to be

created, the failurecause is returned. Fordetails, see Descriptionof the validation-resultparameter.

l If a link is successfullycreated, a blank list isreturned.

l Description of the validation-result parameter

Parameter Mandatory Type Description

linkConfig No List Validation result of linkcreation or update. Fordetails, see Description ofthe linkConfigparameter.

l Description of the linkConfig parameter

Parameter Mandatory Type Description

message No String Error message

status No Enumeration l ERRORl WARNING

Return Valuel Normal

200l Abnormal

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

38

Page 44: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Return Value Description

400 Bad Request Request error.

401 Unauthorized Authentication failed.

403 Forbidden No operation permission.

404 Not Found The requested resource is not found.

500 Internal Server Error Internal error. For details about the returnederror code, see Error Code.

503 Service Unavailable Service unavailable.

4.2.2 Link Parameter Description

4.2.2.1 Link to a Relational Database

Description

By creating a JDBC link, you can extract data from or load data to the following relationaldatabases:l Data Warehouse Servicel RDS (MySQL)l RDS (PostgreSQL)l RDS (SQL Server)l DDMl MySQLl PostgreSQLl Microsoft SQL Serverl Oraclel IBM Db2l FusionInsight LibrAl Derecho (GaussDB)

Sample Link{ "links": [ { "link-config-values": { "configs": [ { "inputs": [ { "name": "linkConfig.databaseType", "value": "MYSQL" }, { "name": "linkConfig.host",

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

39

Page 45: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

"value": "10.120.205.30" }, { "name": "linkConfig.port", "value": "3306" }, { "name": "linkConfig.database", "value": "DB_name" }, { "name": "linkConfig.username", "value": "username" }, { "name": "linkConfig.password", "value": "Add password here" }, { "name": "linkConfig.fetchSize", "value": "100000" }, { "name": "linkConfig.usingNative", "value": "false" }, { "name": "linkConfig.useSSL", "value": "false" } ], "name": "linkConfig" } ] }, "name": "mysql_link", "connector-name": "generic-jdbc-connector" } ]}

Link ParametersParameter Mandatory Type Description

linkConfig.databaseType

Yes Enumeration Database type. The options areas follows:l ORACLEl MYSQLl SQLSERVERl DB2l POSTGRESQLl GAUSSDBl DWS

linkConfig.host No String IP address of the database server

linkConfig.port No String Port number of the databaseserver

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

40

Page 46: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Parameter Mandatory Type Description

linkConfig.database

No String Database name

linkConfig.username

Yes String Username

linkConfig.password

Yes String Password

linkConfig.fetchSize

No String Number of data rows obtainedeach time

linkConfig.usingNative

No Boolean Whether to use the local APIacceleration function of thedatabaseWhen creating a MySQL link,you can use the LOAD DATAfunction of MySQL to acceleratedata import and improve theperformance of importing data tothe MySQL database.

linkConfig.useSSL

No Boolean Whether to use encryptedtransmission. You can enableSSL encrypted transmission forRelational Database Service(RDS).

linkConfig.jdbcProperties

No Map Link attribute, which specifiesthe JDBC connector attributes ofthe data source. For details abouthow to configure the linkattributes, see the JDBCconnector description of thecorresponding database.

dialect.identifierEnclose

No String Reference identifier, which is thedelimiter between the referencedtable names or column names.For details, see the productdocumentation of thecorresponding database.

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

41

Page 47: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Parameter Mandatory Type Description

linkConfig.importMode

No Enumeration Data import mode. Thisparameter is available only whenthe database type is DWS.l GDS: In GDS mode, port

25000 is enabled on CDM,and multiple DWSDataNodes extract data fromCDM.

l COPY: In COPY mode,CDM copies data to DWS byusing the JDBC API ofDWS.

4.2.2.2 Link to OBS

DescriptionBy creating an OBS link, you can extract files from or load files to HUAWEI CLOUD OBS.Files in CSV, JSON, and binary formats are supported.

Sample Link{ "links": [ { "link-config-values": { "configs": [ { "inputs": [ { "name": "linkConfig.storageType", "value": "OBS" }, { "name": "linkConfig.server", "value": "10.121.16.183" }, { "name": "linkConfig.port", "value": "5443" }, { "name":"linkConfig.accessKey", "value": "RSO6TTEZMJ6TTFBBAACE" }, { "name":"linkConfig.securityKey", "value":"Add password here" } ], "name": "linkConfig" } ] }, "name": "obs_link", "connector-name": "obs-connector" }

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

42

Page 48: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

]}

Link ParametersParameter Mandatory Type Description

linkConfig.storageType

Yes String Storage class of an object

linkConfig.server Yes String IP address of the OBS server

linkConfig.port Yes String Port number

linkConfig.accessKey

Yes String AK

linkConfig.securityKey

Yes String SK

4.2.2.3 Link to OSS on Alibaba Cloud

DescriptionBy creating an OSS link, you can extract files from Object Storage Service (OSS) on AlibabaCloud. Files in CSV, JSON, and binary formats are supported.

Sample Link{ "links": [ { "link-config-values": { "configs": [ { "inputs": [ { "name": "linkConfig.storageType", "value": "OSS" }, { "name": "linkConfig.ossEndpoint", "value": "oss-cn-qingdao.aliyuncs.com" }, { "name": "linkConfig.ossAuthType", "value": "ACCESS_KEY" }, { "name": "linkConfig.accessKey", "value": "LTAI1mR805IE0cVr" }, { "name": "linkConfig.securityKey", "value": "Add password here" } ], "name": "linkConfig" } ] },

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

43

Page 49: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

"name": "op_oss_link", "connector-name": "obs-connector" } ]}

Link ParametersParameter Mandatory Type Description

linkConfig.storageType

Yes String Storage class of an object

linkConfig.ossEndpoint

Yes String Endpoint of OSS

linkConfig.ossAuthType

Yes Boolean Identity authentication methodl ACCESS_KEY: Use the

access key to access OSS.l STS: Use the temporary key

and security token to accessOSS.

linkConfig.accessKey

Yes String AK

linkConfig.securityKey

Yes String SK

linkConfig.securityToken

No String Temporary security tokenrequired when a temporaryaccess key is used to log in toOSS

4.2.2.4 Link to Qiniu Cloud Object Storage (KODO)

DescriptionBy creating a KODO link, you can extract files from Qiniu Cloud Object Storage (KODO).Files in CSV, JSON, and binary formats are supported.

Sample Link{ "links": [ { "link-config-values": { "configs": [ { "inputs": [ { "name": "linkConfig.storageType", "value": "KODO" }, { "name": "linkConfig.zone", "value": "z1"

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

44

Page 50: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

}, { "name": "linkConfig.accessKey", "value": "xib1R_oeN4KBYTA-yNoV0Kd3AbUwi1yuvzbPF4LE" }, { "name": "linkConfig.securityKey", "value": "Add password here" }, { "name": "linkConfig.useCustomDomain", "value": "false" } ], "name": "linkConfig" } ] }, "name": "kodo_link", "connector-name": "thirdparty-obs-connector" } ]}

Link ParametersParameter Mandatory Type Description

linkConfig.storageType

Yes String Storage class of an object

linkConfig.zone Yes String Storage area of KODO

linkConfig.accessKey

Yes String AK

linkConfig.securityKey

Yes String SK

linkConfig.useCustomDomain

No Boolean Whether to use the user-defineddomain name to downloadobjects

4.2.2.5 Link to HDFS

DescriptionBy creating an HDFS link, you can extract files from or load files to MRS, FusionInsight HD,or Apache Hadoop. Files in CSV, Parquet, and binary formats are supported.

Sample Link{ "links": [ { "link-config-values": { "configs": [ { "inputs": [ {

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

45

Page 51: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

"name": "linkConfig.hadoopType", "value": "FusionInsight HD" }, { "name": "linkConfig.host", "value": "10.120.205.143" }, { "name": "linkConfig.casPort", "value": "20009" }, { "name": "linkConfig.port", "value": "28443" }, { "name": "linkConfig.authType", "value": "KERBEROS" }, { "name": "linkConfig.user", "value": "admin" }, { "name": "linkConfig.password", "value": "Add password here" }, { "name": "linkConfig.runMode", "value": "STANDALONE" } ], "name": "linkConfig" } ] }, "name": "hdfslink", "connector-name": "hdfs-connector" } ]}

Link ParametersParameter Mandatory Type Description

linkConfig.hadoopType

Yes Enumeration Hadoop type. The options are asfollows:l MRS: link to HDFS of MRSl FusionInsight HD: link to

HDFS of FusionInsight HDl Apache Hadoop: link to HDFS

of Apache Hadoop

linkConfig.uri No String NameNode URI required for thelink to Apache Hadoop. The formatis ip:port.

linkConfig.host No String IP address of Manager required forthe link to MRS or FusionInsightHD

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

46

Page 52: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Parameter Mandatory Type Description

linkConfig.port No String Port number of Manager requiredfor the link to FusionInsight HD

linkConfig.casPort

No String Port number of CAS Server thatconnects to FusionInsight HDrequired for the link toFusionInsight HD

linkConfig.user No String Username for logging in toManager. If SIMPLEauthentication is used forconnecting to MRS, you do notneed to enter the username andpassword for logging in toManager.

linkConfig.password

No String Password for logging in toManager. If SIMPLEauthentication is used forconnecting to MRS, you do notneed to enter the username andpassword for logging in toManager.

linkConfig.authType

No Enumeration Authentication method. Theoptions are as follows:l Simple: for non-security model Kerberos: for security mode

linkConfig.principal

No String Account principal required forKerberos authentication. You cancontact the administrator to obtainthe account.

linkConfig.keytab

No FileContent Local absolute path of the keytabfile required for Kerberosauthentication. You can contact theadministrator to obtain the file.

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

47

Page 53: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Parameter Mandatory Type Description

linkConfig.runMode

No Enumeration Running mode of the HDFS link.Options are as follows:l Embedded: The link instance

runs with CDM. This mode hasbetter performance.

l Standalone: The link instanceruns in an independent process.If CDM needs to connect tomultiple Hadoop data sources(MRS, Hadoop, or CloudTable)with both Kerberos and Simpleauthentication methods,Standalone prevails.

If Standalone is selected, CDMcan migrate data between HDFSsof multiple MRS clusters.

4.2.2.6 Link to HBase

DescriptionBy creating an HBase link, you can extract data from or load data to HBase of MRS,FusionInsight HD, and Apache Hadoop.

Sample Link{ "links": [ { "link-config-values": { "configs": [ { "inputs": [ { "name": "linkConfig.hbaseType", "value": "MRS" }, { "name": "linkConfig.host", "value": "192.168.2.23" }, { "name": "linkConfig.authType", "value": "SIMPLE" }, { "name": "linkConfig.runMode", "value": "STANDALONE" } ], "name": "linkConfig" } ] }, "name": "mrshbase",

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

48

Page 54: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

"connector-name": "hbase-connector" } ]}

Link ParametersParameter Mandatory Type Description

linkConfig.hbaseType

Yes Enumeration HBase type. The options are asfollows:l CloudTable: link to CloudTable

Service (CloudTable)l MRS: link to HBase of MRSl FusionInsight HD: link to HBase

of FusionInsight HDl Apache Hadoop: link to HBase

of Apache Hadoop

linkConfig.uri No String NameNode URI required for the linkto Apache Hadoop. The format isip:port.

llinkConfig.host

No String IP address of Manager required forthe link to MRS or FusionInsight HD

linkConfig.port No String Port number of Manager required forthe link to FusionInsight HD

linkConfig.casPort

No String Port number of CAS Server thatconnects to FusionInsight HDrequired for the link to FusionInsightHD

linkConfig.user No String Username for logging in to Manager.If SIMPLE authentication is used forconnecting to MRS, you do not needto enter the username and passwordfor logging in to Manager.

linkConfig.password

No String Password for logging in to Manager.If SIMPLE authentication is used forconnecting to MRS, you do not needto enter the username and passwordfor logging in to Manager.

linkConfig.authType

No Enumeration Authentication method. The optionsare as follows:l Simple: for non-security model Kerberos: for security mode

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

49

Page 55: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Parameter Mandatory Type Description

linkConfig.principal

No String Account principal required forKerberos authentication. You cancontact the administrator to obtain theaccount.

linkConfig.keytab

No FileContent Local absolute path of the keytab filerequired for Kerberos authentication.You can contact the administrator toobtain the file.

linkConfig.serviceType

No String Service type

linkConfig.runMode

No Enumeration Running mode of the HBase link.Options are as follows:l Embedded: The link instance

runs with CDM. This mode hasbetter performance.

l Standalone: The link instanceruns in an independent process. IfCDM needs to connect to multipleHadoop data sources (MRS,Hadoop, or CloudTable) with bothKerberos and Simpleauthentication methods,Standalone prevails.

4.2.2.7 Link to CloudTable

DescriptionBy creating a CloudTable link, you can extract data from or load data to CloudTable.

Sample Link{ "links": [ { "link-config-values": { "configs": [ { "inputs": [ { "name": "linkConfig.hbaseType", "value": "CloudTable" }, { "name": "linkConfig.zookeeperQuorum", "value": "cloudtable-cdm-zk1.cloudtable.com:2181,cloudtable-cdm-zk2.cloudtable.com:2181" } ], "name": "linkConfig" }

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

50

Page 56: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

] }, "name": "hbase_link", "connector-name": "hbase-connector" } ]}

Link ParametersParameter Mandatory Type Description

linkConfig.hbaseType

Yes Enumeration HBase type. The options are asfollows:l CloudTable: link to CloudTablel MRS: link to MRSl FusionInsight HD: link to

FusionInsight HDl Apache Hadoop: link to Apache

Hadoop

linkConfig.zookeeperQuorum

Yes String ZooKeeper link of CloudTable. Thisparameter is mandatory for theCloudTable link.

4.2.2.8 Link to Hive

DescriptionBy creating a Hive link, you can extract data from or load data to Hive of MRS.

Sample Link{ "links": [ { "link-config-values": { "configs": [ { "inputs": [ { "name": "linkConfig.host", "value": "10.120.205.230" }, { "name": "linkConfig.authType", "value": "KERBEROS" }, { "name": "linkConfig.user", "value": "cdm" }, { "name": "linkConfig.password", "value": "Add password here" } ], "name": "linkConfig"

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

51

Page 57: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

} ] }, "name": "hive_link", "connector-name": "hive-connector" } ]}

Link ParametersParameter Mandatory Type Description

llinkConfig.host

No String IP address of MRS Manager

linkConfig.authType

No Enumeration Authentication method of MRS. Theoptions are as follows:l SIMPLE: for non-security model KERBEROS: for security mode

linkConfig.principal

No String Account principal required forKerberos authentication. You cancontact the administrator to obtain theaccount.

linkConfig.keytab

No FileContent Local absolute path of the keytab filerequired for Kerberos authentication.You can contact the administrator toobtain the file.

4.2.2.9 Link to an FTP or SFTP Server

DescriptionBy creating an FTP or SFTP link, you are able to extract files from or load files to the FTP orSFTP server. Files in CSV, JSON, and binary format are supported.

Sample Link{ "links": [ { "link-config-values": { "configs": [ { "inputs": [ { "name": "linkConfig.server", "value": "10.120.85.167" }, { "name": "linkConfig.port", "value": "22" }, { "name": "linkConfig.username", "value": "username"

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

52

Page 58: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

}, { "name": "linkConfig.password", "value": "Add password here" } ], "name": "linkConfig" } ] }, "name": "sftp_link", "connector-name": "sftp-connector" } ]}

Link ParametersParameters for creating the FTP or SFTP link are the same.

Parameter Mandatory Type Description

linkConfig.server

Yes String IP address of the FTP or SFTP server

linkConfig.port Yes String Port number of the FTP or SFTPserver

linkConfig.username

Yes String Username for logging in to the FTPor SFTP server

linkConfig.password

Yes String Password for logging in to the FTP orSFTP server

4.2.2.10 Link to MongoDB/DDS

DescriptionBy creating a MongoDB link, you can extract data from or load data to the MongoDB serveror Document Database Service (DDS).

Sample Link{ "links": [ { "link-config-values": { "configs": [ { "inputs": [ { "name": "linkConfig.serverList", "value": "10.120.84.149:27017" }, { "name": "linkConfig.database", "value": "DB_name" }, { "name": "linkConfig.userName",

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

53

Page 59: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

"value": "username" }, { "name": "linkConfig.password", "value": "Add password here" } ], "name": "linkConfig" } ] }, "name": "mongo_link", "connector-name": "mongodb-connector" } ]}

Link ParametersParameter Mandatory Type Description

linkConfig.serverList

Yes String Server IP address list inhost1:port1;host2:port2 format

linkConfig.database

Yes String Name of the MongoDB database

linkConfig.userName

Yes String Username for logging in to theMongoDB server

linkConfig.password

Yes String Password for logging in to theMongoDB server

4.2.2.11 Link to Redis/DCS

DescriptionBy creating a Redis link, you can extract data from or load data to the Redis server. Bycreating a DCS link, you can load data to Data Cache Service (DCS), but not extract data forDCS. The data can be stored in string or hash format.

Sample Link{ "links": [ { "link-config-values": { "configs": [ { "inputs": [ { "name": "linkConfig.deploymentMode", "value": "Cluster" }, { "name": "linkConfig.serverlist", "value": "10.120.84.149:7300" }, { "name": "linkConfig.password",

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

54

Page 60: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

"value": "Add password here" }, { "name": "linkConfig.dbIndex", "value": "0" } ], "name": "linkConfig" } ] }, "name": "redis_link", "connector-name": "redis-connector" } ]}

Link ParametersParameter Mandatory Type Description

linkConfig.deploymentMode

Yes Enumeration Redis deployment mode. Theoptions are as follows:l Single: single-node deploymentl Cluster: cluster deployment

linkConfig.serverlist

Yes String Server IP address list inhost1:port1;host2:port2 format

linkConfig.password

Yes String Password for logging in to the Redisserver

linkConfig.dbIndex

Yes String Redis database index

4.2.2.12 Link to NAS

DescriptionBy creating a NAS link, you are able to extract files from or load files to the NAS server.Currently, the SMB and CIFS protocols are supported. Files in CSV, JSON, and binaryformats are supported.

Sample Link{ "links": [ { "link-config-values": { "configs": [ { "inputs": [ { "name": "linkConfig.protocol", "value": "CIFS" }, { "name": "linkConfig.server", "value": "\\\\10.121.16.20\\data"

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

55

Page 61: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

}, { "name": "linkConfig.username", "value": "username" }, { "name": "linkConfig.password", "value": "Add password here" } ], "name": "linkConfig" } ] }, "name": "nas_link", "connector-name": "nas-connector" } ]}

Link ParametersParameter Mandatory Type Description

linkConfig.protocol Yes String NAS file protocol. Currently,only the SMB and CIFSprotocols are supported.

linkConfig.server Yes String Shared path of the NASserver, in which \\\\ isconverted into \\

linkConfig.username Yes String Username for logging in tothe NAS server

linkConfig.password Yes String Password for logging in to theNAS server

4.2.2.13 Link to Kafka

DescriptionBy creating a Kafka link, you are able to access open source Kafka and migrate data fromKafka to other data sources as required. Currently, only data export from Kafka is supported.

Sample Link{ "links": [ { "link-config-values": { "configs": [ { "inputs": [ { "name": "linkConfig.brokerList", "value": "127.0.0.1:9092" } ], "name": "linkConfig"

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

56

Page 62: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

} ] }, "name": "kafka_link", "connector-name": "kafka-connector" } ]}

Link ParametersParameter Mandatory Type Description

linkConfig.brokerList

Yes String Kafka broker list inhost1:port1,host2:port2 format

4.2.2.14 Link to DIS

DescriptionBy creating a DIS link, you can access DIS on HUAWEI CLOUD and migrate data from DISto other data sources as required.

Sample Link{ "links": [ { "link-config-values": { "configs": [ { "inputs": [ { "name": "linkConfig.region", "value": "cn-north-1" }, { "name": "linkConfig.endpoint", "value": "https://dis.cn-north-1.myhuaweiclouds.com:20004" }, { "name": "linkConfig.ak", "value": "RSO6TTEZMJ6TTFBBAACE" }, "name": "linkConfig.sk", "value": "Add password here" }, { "name": "linkConfig.projectId", "value": "11d4d5af17c84660bc90b6631327d7c7" } ], "name": "linkConfig" } ] }, "name": "dis_link", "connector-name": "dis-connector" } ]}

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

57

Page 63: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Link Parameters

Parameter Mandatory Type Description

linkConfig.region Yes String Region where DIS resides

linkConfig.endpoint Yes String Domain name or IP addressof the DIS server

linkConfig.ak Yes String AK of the DIS server

linkConfig.sk Yes String SK of the DIS server

linkConfig.projectId Yes String Project ID

4.2.2.15 Link to Elasticsearch

Description

By creating an Elasticsearch link, you can extract data from or load data to the Elasticsearchserver or Cloud Search Service.

Sample Link{ "links": [ { "link-config-values": { "configs": [ { "inputs": [ { "name": "linkConfig.host", "value": "10.66.65.91" }, { "name": "linkConfig.port", "value": "9200" }, { "name": "linkConfig.user", "value": "sqoop" }, { "name": "linkConfig.password", "value": "Add password here" }, { "name": "linkConfig.linkType", "value": "ElasticSearch" } ], "name": "linkConfig" } ] }, "name": "es_link", "connector-name": "elasticsearch-connector" } ]}

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

58

Page 64: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Link ParametersParameter Mandatory Type Description

linkConfig.host Yes String Domain name or IP address of theElasticsearch server

linkConfig.port Yes String Port number of the Elasticsearchserver

linkConfig.user No String For Elasticsearch that supportsusername and passwordauthentication, configure theusername and password whencreating a link.

linkConfig.password

No String Password for logging in to theElasticsearch server

linkConfig.linkType

Yes String Link type, which is used todistinguish the Elasticsearch linkfrom the Cloud Search Service link

4.2.2.16 Link to DLI

Description

By creating a DLI link, you can import data to DLI. Currently, you cannot export data fromDLI using CDM.

Sample Link{ "links": [ { "link-config-values": { "configs": [ { "inputs": [ { "name": "linkConfig.ak", "value": "GRC2WR0IDC6NGROYLWU2" }, { "name": "linkConfig.sk", "value": "Add password here" }, { "name": "linkConfig.region", "value": "cn-north-1" }, { "name": "linkConfig.projectId", "value": "c48475ce8e174a7a9f775706a3d5ebe2" } ], "name": "linkConfig" } ]

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

59

Page 65: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

}, "name": "dli", "connector-name": "dli-connector" } ]}

Link Parameters

Parameter Mandatory Type Description

linkConfig.ak Yes String AK for logging in to the DLIdatabase

linkConfig.sk Yes String SK for logging in to the DLIdatabase

linkConfig.region

Yes String Region where DLI resides

linkConfig.projectId

Yes String Project ID of the DLI service

4.2.3 Querying a Link

Function

This API is used to query the link list.

URIl URI format

GET /cdm/v1.0/{project_id}/clusters/{cluster_id}/cdm/link/{link_name}l Parameter description

Parameter Mandatory Type Description

project_id Yes String Project ID

cluster_id Yes String Cluster ID

link_name Yes String Link name. When theparameter is set to all, all linksare queried.

Request

Sample requestGET /cdm/v1.0/1551c7f6c808414d8e9f3c514a170f2e/clusters/6ec9a0a4-76be-4262-8697-e7af1fac7920/cdm/link/sftplink

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

60

Page 66: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Responsel Sample response

{ "links": [ { "link-config-values": { "configs": [ { "name": "linkConfig", "inputs": [ { "size": 128, "name": "linkConfig.server", "type": "STRING", "mandatory": true, "value": "10.120.85.163" }, { "defaultValue": "22", "name": "linkConfig.port", "type": "INTEGER", "mandatory": true, "value": "22" }, { "size": 32, "name": "linkConfig.username", "type": "STRING", "mandatory": true, "value": "root" }, { "size": 32, "name": "linkConfig.password", "sensitive": true, "type": "STRING", "mandatory": true } ] } ] }, "creation-user": "cdm", "name": "sftp_link", "creation-date": 1516674482640, "connector-name": "sftp-connector", "update-date": 1516674476022, "enabled": true, "update-user": "cdm" } ]}

l Parameter description

Parameter Mandatory Type Description

links Yes List Link list. For details,see Description ofthe links parameter.

l Description of the links parameter

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

61

Page 67: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Parameter Mandatory Type Description

link-config-values Yes Dictionary Link configuration.For details, seeDescription of thelink-config-valuesparameter.

creation-user Yes String User who created alink

name Yes String Link name

creation-date Yes Timestamp Time when a link iscreated

connector-name Yes String Connector name

update-date Yes Timestamp Time when a link isupdated

enabled No Boolean Whether to activate alink

update-user No String User who updated alink

l Description of the link-config-values parameter

Parameter Mandatory Type Description

configs Yes List Data structure of linkconfiguration items. Fordetails, see Description of theconfigs parameter.

l Description of the configs parameter

Parameter Mandatory Type Description

name Yes String Configuration item name

inputs Yes List Parameter list. For details, seeDescription of the inputsparameter.

l Description of the inputs parameter

Parameter Mandatory Type Description

size No String Maximum length of theparameter

name Yes String Parameter name

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

62

Page 68: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Parameter Mandatory Type Description

type No String Parameter type

defaultValue No String Default value of theparameter

dependent No Boolean Whether the parameter isaffected by other components

value No String Parameter value

mandatory No Boolean Whether the parameter ismandatory

editable No String Whether a parameter can beedited

sensitive No Boolean Whether the parameter is apassword

Return Valuel Normal

200l Abnormal

Return Value Description

400 Bad Request Request error.

401 Unauthorized Authentication failed.

403 Forbidden No operation permission.

404 Not Found The requested resource is not found.

500 Internal Server Error Internal error. For details about the returnederror code, see Error Code.

503 Service Unavailable Service unavailable.

4.2.4 Modifying a Link

Function

This API is used to modify a link that you have created.

URIl URI format

PUT /cdm/v1.0/{project_id}/clusters/{cluster_id}/cdm/link/{link_name}l Parameter description

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

63

Page 69: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Parameter Mandatory Type Description

project_id Yes String Project ID

cluster_id Yes String Cluster ID

link_name Yes String Link name

l Parameter description of a message body

The parameters for modifying a link are the same as those for creating a link. For details,see Request in Creating a Link.

Request

Sample request

PUT /cdm/v1.0/1551c7f6c808414d8e9f3c514a170f2e/clusters/6ec9a0a4-76be-4262-8697-e7af1fac7920/cdm/link/rdb_link{ "links": [{ "enabled": true, "update-user": null, "name": "mysql_link", "link-config-values": { "configs": [ { "name": "linkConfig", "inputs": [ { "name": "linkConfig.databaseType", "value": "MYSQL" }, { "name": "linkConfig.host", "value": "10.120.85.24" }, { "name": "linkConfig.port", "value": "3306" }, { "name": "linkConfig.database", "value": "DB_name" }, { "name": "linkConfig.username", "value": "username" }, { "name": "linkConfig.password", "value": "Add password here" }, { "name": "linkConfig.fetchSize", "value": "100000" }, { "name": "linkConfig.usingNative", "value": "true" } ] } ] },

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

64

Page 70: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

"connector-name": "generic-jdbc-connector", "creation-date": 1496654788622, "update-date": 1496654788622, "creation-user": null }]}

Responsel Successful sample response

{ "name": "mysql_link", "validation-result": [{ }]}

l Failed sample response{ "validation-result": [{ "linkConfig": [{ "message": "Can't connect to the database with given credentials: The authentication type 12 is not supported. Check that you have configured the pg_hba.conf file to include the client's IP address or subnet, and that it is using an authentication scheme supported by the driver.", "status": "ERROR" }] }]}

l Parameter description

Parameter Mandatory Type Description

name Yes String Link name

validation-result Yes List Validation structurel If a link fails to be

modified, the failurecause is returned. Fordetails, see Descriptionof the validation-resultparameter.

l If a link is successfullymodified, a blank list isreturned.

l Description of the validation-result parameter

Parameter Mandatory Type Description

linkConfig No List Validation result of linkcreation or update. Fordetails, see Description ofthe linkConfigparameter.

l Description of the linkConfig parameter

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

65

Page 71: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Parameter Mandatory Type Description

message No String Error message

status No Enumeration l ERRORl WARNING

Return Valuel Normal

200l Abnormal

Return Value Description

400 Bad Request Request error.

401 Unauthorized Authentication failed.

403 Forbidden No operation permission.

404 Not Found The requested resource is not found.

500 Internal Server Error Internal error. For details about the returnederror code, see Error Code.

503 Service Unavailable Service unavailable.

4.2.5 Deleting a Link

FunctionThis API is used to delete a specified link. However, you cannot delete the links used by jobs.

URIl URI format

DELETE /cdm/v1.0/{project_id}/clusters/{cluster_id}/cdm/link/{link_name}

l Parameter description

Parameter Mandatory Type Description

project_id Yes String Project ID

cluster_id Yes String Cluster ID

link_name Yes String Link name

RequestSample request

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

66

Page 72: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

DELETE /cdm/v1.0/1551c7f6c808414d8e9f3c514a170f2e/clusters/6ec9a0a4-76be-4262-8697-e7af1fac7920/cdm/link/jdbclink

Responsel Parameter description of a successful response

None

l If a failure occurs, the HTTP response code is 500. A sample response is as follows:{ "errCode": "Cdm.0021", "externalMessage": "Given link name is in use"}

l Parameter description of a failed response

Parameter Mandatory Type Description

errCode Yes String Error code

externalMessage Yes String Error message

Return Valuel Normal

200

l Abnormal

Return Value Description

400 Bad Request Request error.

401 Unauthorized Authentication failed.

403 Forbidden No operation permission.

404 Not Found The requested resource is not found.

500 Internal Server Error Internal error. For details about the returnederror code, see Error Code.

503 Service Unavailable Service unavailable.

4.3 Job Management

4.3.1 Creating a Job

Function

This API is used to create a job. You must specify the source and destination links and jobparameters in the message body.

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

67

Page 73: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

URIl URI format

POST /cdm/v1.0/{project_id}/clusters/{cluster_id}/cdm/jobl Parameter description

Table 4-1 URI parameter description

Parameter Mandatory Type Description

project_id Yes String Project ID

cluster_id Yes String Cluster ID

Requestl Sample request

POST /cdm/v1.0/1551c7f6c808414d8e9f3c514a170f2e/clusters/6ec9a0a4-76be-4262-8697-e7af1fac7920/cdm/job

{ "jobs": [{ "job_type": "NORMAL_JOB", "name": "mysql2hive", "from-link-name": "mysqllink", "from-connector-name": "generic-jdbc-connector", "to-link-name": "hivelink", "to-connector-name": "hive-connector", "from-config-values": { "configs": [{ "name": "fromJobConfig", "inputs": [{ "name": "fromJobConfig.schemaName", "value": "default" }, { "name": "fromJobConfig.tableName", "value": "mysql_tbl" }, { "name": "fromJobConfig.partitionColumn", "value": "id" }] }] }, "to-config-values": { "configs": [{ "name": "toJobConfig", "inputs": [{ "name": "toJobConfig.hive", "value": "hive" }, { "name": "toJobConfig.database", "value": "default" }, { "name": "toJobConfig.table", "value": "test_tbl" }] }] }, "driver-config-values": { "configs": [{

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

68

Page 74: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

"name": "throttlingConfig", "inputs": [{ "name": "throttlingConfig.numExtractors", "value": "1" }] }] } }]}

l Parameter description

Parameter Mandatory Type Description

jobs Yes List Job list. For details, seeDescription of the jobsparameter.

l Description of the jobs parameter

Parameter Mandatory Type Description

job_type Yes String Job typel NORMAL_JOB: table/

file migrationl BATCH_JOB: entire

DB migration

name Yes String Job name

from-link-name Yes String Source link name

from-connector-name Yes String Name of the connector ofthe source link

to-link-name Yes String Destination link name

to-connector-name Yes String Name of the connector ofthe destination link

from-config-values Yes Dictionary Source link configuration.For details, see Descriptionof from-config-values.

to-config-values Yes Dictionary Destination linkconfiguration. For details,see Description of to-config-values.

driver-config-values Yes Dictionary Job parameterconfiguration. For details,see Description of driver-config-values.

l Description of from-config-values, to-config-values, and driver-config-values

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

69

Page 75: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Parameter Mandatory Type Description

configs Yes List The data structures of sourcelink parameters, destinationlink parameters, and jobparameters are the same.However, the inputsparameter varies. For details,see Description of theconfigs parameter.

l Description of the configs parameter

Parameter Mandatory Type Description

name Yes String Configuration item name

inputs Yes List Input parameter list. Eachelement in the list is inname,value format. Fordetails, see Description ofthe inputs parameter.l In from-config-values,

different types of sourcelinks have different inputsparameter lists. Fordetails, see Source JobParameters.

l In to-config-values,different types ofdestination links havedifferent inputs parameterlists. For details, seeDestination JobParameters.

l For details about theinputs parameter indriver-config-values, seeJob ParameterDescription.

l Description of the inputs parameter

Parameter Mandatory Type Description

name Yes String Parameter name

value Yes String Parameter value

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

70

Page 76: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Responsel Sample response

{ "name": "mysql2hive"}

l Parameter description

Parameter Mandatory Type Description

name Yes String Job name

Return Valuel Normal

200l Abnormal

Return Value Description

400 Bad Request Request error.

401 Unauthorized Authentication failed.

403 Forbidden No operation permission.

404 Not Found The requested resource is not found.

500 Internal Server Error Internal error. For details about the returnederror code, see Error Code.

503 Service Unavailable Service unavailable.

4.3.2 Source Job ParametersCDM allows you to export data from multiple types of data sources. Each data source has itsown job parameters. You can view specific input parameters based on the correspondingsection of the data source type.

Set the input parameters of the source link as described in Description of the inputsparameter of from-config-values in Creating a Job.

4.3.2.1 From a Relational Database

Sample JSON File"from-config-values": { "configs": [{ "inputs": [{ "name": "fromJobConfig.schemaName", "value": "default" }, { "name": "fromJobConfig.tableName", "value": "cdm_tbl" },

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

71

Page 77: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

{ "name": "fromJobConfig.partitionColumn", "value": "id" }], "name": "fromJobConfig" }] }

Parameter DescriptionParameter Mandatory Type Description

fromJobConfig.schemaName

Yes String Database mode or tablespace. Forexample, public.

fromJobConfig.tableName

Yes String Table name. For example,TBL_EXAMPLE.

fromJobConfig.whereClause

No String WHERE clauses to be extracted. Ifno WHERE clause is specified, theentire table will be extracted. Forexample, age > 18 and age <= 60.

fromJobConfig.columnList

No String List of fields to be extracted. Use& to separate field names. Forexample, id&gid&name.

fromJobConfig.partitionColumn

No String Partition field to be extracted, bywhich a job is split in multiple sub-jobs executed concurrently. Forexample, id.

fromJobConfig.regainSymbol

No String After Regain Symbol is set to aspecified field, CDM queries thetable imported from the destinationdatabase every time the job isstarted. If the table does not containthe specified field, CDM performsfull migration. If the destinationtable contains the specified fieldand the field has a value, CDMperforms incremental migration tomigrate only the data whose valueis greater than the value of thisfield.

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

72

Page 78: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Parameter Mandatory Type Description

fromJobConfig.incrMigration

No Boolean Whether to migrate incrementaldata in MySQL Binlog mode.Currently, this mode can be used inthe table/file migration from theMySQL database to the DWSdatabase only.After this function is enabled, datain the source table and destinationtable can be synchronized in realtime. One MySQL link supportsonly one incremental migration job,and one source table supports onlyone incremental migration job.

4.3.2.2 From OBS/OSS

Sample JSON File"from-config-values": { "configs": [ { "inputs": [ { "name": "fromJobConfig.bucketName", "value": "cdm" }, { "name": "fromJobConfig.inputDirectory", "value": "/data/" }, { "name": "fromJobConfig.inputFormat", "value": "JSON_FILE" }, { "name": "fromJobConfig.jsonReferenceNode", "value": "data.list" }, { "name": "fromJobConfig.fieldSeparator", "value": "," }, { "name": "fromJobConfig.encodeType", "value": "UTF-8" }, { "name": "fromJobConfig.splitType", "value": "FILE" }, { "name": "fromJobConfig.fromFileOpType", "value": "DELETE" }, { "name": "fromJobConfig.markerFile", "value": "ok.txt" }, {

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

73

Page 79: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

"name": "fromJobConfig.filterType", "value": "WILDCARD" }, { "name": "fromJobConfig.pathFilter", "value": "*input" }, { "name": "fromJobConfig.fileFilter", "value": "*.csv" } ], "name": "fromJobConfig" } ] }

Parameter Descriptionl OBS/OSS job parameter description

Parameter Mandatory Type Description

fromJobConfig.bucketName

Yes String OBS/OSS bucket name. Forexample, cdm.

fromJobConfig.inputDirectory

Yes String Data extraction path. A single filepath can contain a maximum of 50files, which are separated byvertical bars (|). For example,FROM/example.csv|FROM/b.txt.

fromJobConfig.inputFormat

Yes Enumeration File format required for datatransmission. Currently, thefollowing file formats aresupported:l CSV_FILE: CSV formatl JSON_FILE: JSON formatl BINARY_FILE: binary formatl CARBONDATA: CarbonData

formatWhen the parameter is set toBINARY_FILE, the connector ofthe destination link must be HDFS,OBS, SFTP, FTP, or NAS.

fromJobConfig.lineSeparator

No String Line feed character in the file. Forexample, \n. By default, \n, \r, or\r\n is automatically identified.

fromJobConfig.columnList

No String Numbers of columns to beextracted. Use & to separatecolumn numbers in ascendingorder. For example, 1&3&5.

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

74

Page 80: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Parameter Mandatory Type Description

fromJobConfig.lineSeparator

No String Line feed character. Thisparameter is valid only when thefile format is CSV_FILE. Thedefault value is \r\n.

fromJobConfig.regexSeparator

No Boolean Whether to use the regularexpression to separate fields. Thisparameter is valid only when thefile format is CSV_FILE.

fromJobConfig.regex

No String Regular expression. Thisparameter is valid only when theregular expression is used toseparate fields.

fromJobConfig.fieldSeparator

No String Field delimiter. This parameter isvalid only when the file format isCSV_FILE. The default value is ,.

fromJobConfig.quoteChar

No Boolean Whether to use the encirclingsymbol. If this parameter is set totrue, the field delimiters in theencircling symbol are regarded asa part of the string value.Currently, the default encirclingsymbol of CDM is doublequotation mark (").

fromJobConfig.firstRowAsHeader

No Boolean Whether to regard the first line asthe heading line. This parameter isvalid only when the file format isCSV_FILE. When this parameteris set to true, CDM reads the firstline as the heading line whenextracting data, which means theheading line of the CSV file isremoved in data transmission.

fromJobConfig.fromCompression

No Enumeration Compression format. Thisparameter is valid only when thefile format is CSV_FILE orJSON. The options are as follows:l None: Files in all formats are

transferred.l gzip: Files in gzip format are

transferred.l Zip: Files in Zip format are

transferred.

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

75

Page 81: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Parameter Mandatory Type Description

fromJobConfig.jsonReferenceNode

No String Reference node. This parameter isvalid when the file format isJSON_FILE. Resolve data on theJSON node. If the datacorresponding to the node is aJSON array, the system extractsdata from the array in the samemode. Nested JSON nodes areseparated by periods (.). Forexample, data.list.

fromJobConfig.encodeType

No String Encoding type. For example,UTF_8 or GBK.

fromJobConfig.splitType

No Enumeration Job split mode. The options are asfollows:l FILE: Files are split by file

quantity.l SIZE: Files are split by file

size.

fromJobConfig.fromFileOpType

No Enumeration Source file processing mode. Aftera job is completed, operations onthe source file can be performed.The source file can be renamed ordeleted.

fromJobConfig.useMarkerFile

No Boolean Whether to start a job by a markerfile. A job is started only when amarker file for starting the jobexists in the source path.Otherwise, the job will besuspended for a period of timespecified by Suspension Period.

fromJobConfig.markerFile

No String Name of the identifier file forstarting a job. After an identifierfile is specified, the task isexecuted only when the file existsin the source path. If the file is notspecified, this function is disabledby default. For example, ok.txt.

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

76

Page 82: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Parameter Mandatory Type Description

fromJobConfig.waitTime

No String Period of waiting for a marker file.If you set Start Job by MarkerFile to Yes but no marker fileexists in the source path, the jobfails upon suspension timeout.If you set this parameter to 0 andno marker file exists in the sourcepath, the job will fail immediately.Unit: second

fromJobConfig.filterType

No Enumeration Whether to filter files usingwildcard characters or regularexpressions. When you select aregular expression, the Javaregular expression is used.l WILDCARD: Files are filtered

by wildcard character.l REGEX: Files are filtered by

Java regular expression.

fromJobConfig.pathFilter

No String Path filter, which is used to filterdirectories in the input path. Forexample, *input.

fromJobConfig.fileFilter

No String File filter, which is used to filterfiles in the input path. Forexample, *.csv.

fromJobConfig.formats

No Datastructure

Time format. This parameter ismandatory only when the fileformat is CSV_FILE and the timefield exists in the file. For details,see Description of thefromJobConfig.formatsparameter.

l Description of the fromJobConfig.formats parameter

Parameter Mandatory Type Description

name Yes String Column number. For example, 1.

value Yes String Time format. For example, yyyy-MM-dd.

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

77

Page 83: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

4.3.2.3 From HDFS

Sample JSON File"from-config-values": { "configs": [ { "inputs": [ { "name": "fromJobConfig.inputDirectory", "value": "/user/cdm" }, { "name": "fromJobConfig.inputFormat", "value": "PARQUET_FILE" }, { "name": "fromJobConfig.fieldSeparator", "value": "," }, { "name": "fromJobConfig.encodeType", "value": "UTF-8" }, { "name": "fromJobConfig.fromCompression", "value": "NONE" }, { "name": "fromJobConfig.splitType", "value": "FILE" }, { "name": "fromJobConfig.fromFileOpType", "value": "RENAME" }, { "name": "fromJobConfig.markerFile", "value": "ok.txt" }, { "name": "fromJobConfig.filterType", "value": "REGEX" }, { "name": "fromJobConfig.pathFilter", "value": "*input" }, { "name": "fromJobConfig.fileFilter", "value": "*.csv" } ], "name": "fromJobConfig" } ] }

Parameter Descriptionl HDFS job parameter description

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

78

Page 84: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Parameter Mandatory Type Description

fromJobConfig.inputDirectory

Yes String Path for storing data to beextracted. For example, /data_dir.

fromJobConfig.inputFormat

Yes Enumeration

File format required for datatransmission. Currently, thefollowing file formats aresupported:l CSV_FILE: CSV formatl PARQUET_FILE: Parquet

formatl BINARY_FILE: binary

formatWhen the parameter is set toBINARY_FILE, the connectorof the destination link must beHDFS, OBS, SFTP, FTP, orNAS.

fromJobConfig.columnList

No String Numbers of columns to beextracted. Use & to separatecolumn numbers in ascendingorder. For example, 1&3&5.

fromJobConfig.lineSeparator

No String Line feed character. Thisparameter is valid only when thefile format is CSV_FILE. Thedefault value is \r\n.

fromJobConfig.fieldSeparator

No String Field delimiter. This parameter isvalid only when the file format isCSV_FILE. The default valueis ,.

fromJobConfig.encodeType

No String Encoding type. For example,UTF_8 or GBK.

fromJobConfig.firstRowAsHeader

No Boolean Whether to regard the first lineas the heading line. Thisparameter is valid only when thefile format is CSV_FILE. Whenthis parameter is set to true,CDM reads the first line as theheading line when extractingdata, which means the headingline of the CSV file is removedin data transmission.

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

79

Page 85: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Parameter Mandatory Type Description

fromJobConfig.fromCompression

No Enumeration

Compression format. Only thesource files in specifiedcompression format aretransferred. NONE indicatesfiles in all formats aretransferred.

fromJobConfig.splitType

No Enumeration

Job split mode. The options areas follows:l FILE: Files are split by file

quantity.l SIZE: Files are split by file

size.

fromJobConfig.fromFileOpType

No Enumeration

Source file processing mode.After a job is completed,operations on the source file canbe performed. The source filecan be renamed or deleted.

fromJobConfig.markerFile

No String Name of the identifier file forstarting a job. After an identifierfile is specified, the task isexecuted only when the fileexists in the source path. If thefile is not specified, this functionis disabled by default. Forexample, ok.txt.

fromJobConfig.filterType

No Enumeration

Whether to filter files usingwildcard characters or regularexpressions. When you select aregular expression, the Javaregular expression is used.l WILDCARD: Files are

filtered by wildcardcharacter.

l REGEX: Files are filtered byJava regular expression.

fromJobConfig.pathFilter

No String Path filter, which is used to filterdirectories in the input path. Forexample, *input.

fromJobConfig.fileFilter

No String File filter, which is used to filterfiles in the input path. Forexample, *.csv.

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

80

Page 86: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Parameter Mandatory Type Description

fromJobConfig.formats

No Datastructure

Time format. This parameter ismandatory only wheninputFormat is set toCSV_FILE and the time fieldexists in the file. For details, seeDescription of thefromJobConfig.formatsparameter.

l Description of the fromJobConfig.formats parameter

Parameter Mandatory Type Description

name Yes String Column number. For example, 1.

value Yes String Time format. For example, yyyy-MM-dd.

4.3.2.4 From Hive

Sample JSON File"from-config-values": { "configs": [ { "inputs": [ { "name": "fromJobConfig.hive", "value": "hive" }, { "name": "fromJobConfig.database", "value": "sqoop" }, { "name": "fromJobConfig.table", "value": "invites" }, { "name": "fromJobConfig.columnList", "value": "foo&bar&ds" } ], "name": "fromJobConfig" } ] }

Parameter Description

Parameter Mandatory Type Description

fromJobConfig.hive

No String Data source to be extracted. Forexample, hive.

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

81

Page 87: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Parameter Mandatory Type Description

fromJobConfig.database

No String Database from which data isextracted. For example, default.

fromJobConfig.table

Yes String Name of the table from which data isextracted. For example,TBL_EXAMPLE.

fromJobConfig.columnList

No String Numbers of columns to be extracted.Use & to separate column numbersin ascending order. For example,1&3&5.

4.3.2.5 From HBase/CloudTable

Sample JSON File"from-config-values": { "configs": [ { "inputs": [ { "name": "fromJobConfig.table", "value": "sqooptest" } ], "name": "fromJobConfig" } ] }

Parameter Descriptionl HBase job parameter description

Parameter Mandatory Type Description

fromJobConfig.table

Yes String Name of the table from which data isextracted. For example,TBL_EXAMPLE.

fromJobConfig.columnFamilies

No String Column family to which the data tobe extracted belongs

fromJobConfig.columns

No String Columns to be extracted. Use & toseparate column numbers and : toseparate column families andcolumns. For example,cf1:c1&cf2:c2.

fromJobConfig.isSplit

No Boolean Whether to split the rowkey. Forexample, true.

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

82

Page 88: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Parameter Mandatory Type Description

fromJobConfig.delimiter

No String Delimiter used for splitting rowkeys.If this parameter is not set, row keyswill not be split. For example,vertical bars (|).

fromJobConfig.startTime

No String Minimum timestamp of the timerange (the time point is included).The format is yyyy-MM-ddhh:mm:ss.Only the data created at this point intime and later is extracted.

fromJobConfig.endTime

No String Maximum timestamp of the timerange (the time point is notincluded). The format is yyyy-MM-dd hh:mm:ss.Only the data created before thispoint in time is extracted.

fromJobConfig.formats

No Datastructure

Time format. This parameter ismandatory when the field is a timetype field.

l Description of the fromJobConfig.formats parameter

Parameter Mandatory Type Description

name Yes String Column number. For example, 1.

value Yes String Time format. For example, yyyy-MM-dd.

4.3.2.6 From FTP/SFTP/NAS

Sample JSON File "from-config-values": { "configs": [ { "inputs": [ { "name": "fromJobConfig.inputDirectory", "value": "/from/bigdata" }, { "name": "fromJobConfig.inputFormat", "value": "CSV_FILE" }, { "name": "fromJobConfig.columnList", "value": "1&2&3&4&5&6&7&8&9&10&11&12" }, { "name": "fromJobConfig.fieldSeparator",

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

83

Page 89: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

"value": "," }, { "name": "fromJobConfig.encodeType", "value": "UTF-8" }, { "name": "fromJobConfig.splitType", "value": "SIZE" }, { "name": "fromJobConfig.filterType", "value": "WILDCARD" } ], "name": "fromJobConfig" } ] }

Parameter Description

Source link job parameters of FTP, SFTP, and NAS are the same.

l FTP, SFTP, or NAS job parameter description

Parameter Mandatory Type Description

fromJobConfig.inputDirectory

Yes String Data extraction path. A singlefile path can contain a maximumof 50 files, which are separatedby vertical bars (|). For example,FROM/example.csv|FROM/b.txt.

fromJobConfig.inputFormat

Yes Enumeration File format required for datatransmission. Currently, thefollowing file formats aresupported:l CSV_FILE: CSV formatl JSON_FILE: JSON formatl BINARY_FILE: binary

formatWhen the parameter is set toBINARY_FILE, the connectorof the destination link must beHDFS, OBS, SFTP, FTP, orNAS.

fromJobConfig.lineSeparator

No String Line feed character in the file.For example, \n. By default, \n,\r, or \r\n is automaticallyidentified.

fromJobConfig.columnList

No String Numbers of columns to beextracted. Use & to separatecolumn numbers in ascendingorder. For example, 1&3&5.

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

84

Page 90: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Parameter Mandatory Type Description

fromJobConfig.lineSeparator

No String Line feed character. Thisparameter is valid only when thefile format is CSV_FILE. Thedefault value is \r\n.

fromJobConfig.fieldSeparator

No String Field delimiter. This parameter isvalid only when the file format isCSV_FILE. The default valueis ,.

fromJobConfig.quoteChar

No Boolean Whether to use the encirclingsymbol. If this parameter is setto true, the field delimiters inthe encircling symbol areregarded as a part of the stringvalue. Currently, the defaultencircling symbol of CDM isdouble quotation mark (").

fromJobConfig.regexSeparator

No Boolean Whether to use the regularexpression to separate fields.This parameter is valid onlywhen the file format isCSV_FILE.

fromJobConfig.regex

No String Regular expression. Thisparameter is valid only when theregular expression is used toseparate fields.

fromJobConfig.firstRowAsHeader

No Boolean Whether to regard the first lineas the heading line. Thisparameter is valid only when thefile format is CSV_FILE. Whenthis parameter is set to true,CDM reads the first line as theheading line when extractingdata, which means the headingline of the CSV file is removedin data transmission.

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

85

Page 91: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Parameter Mandatory Type Description

fromJobConfig.fromCompression

No Enumeration Compression format. Thisparameter is valid only when thefile format is CSV_FILE orJSON. The options are asfollows:l None: Files in all formats are

transferred.l gzip: Files in gzip format are

transferred.l Zip: Files in Zip format are

transferred.

fromJobConfig.jsonReferenceNode

No String Reference node. This parameteris valid when the file format isJSON_FILE. Resolve data onthe JSON node. If the datacorresponding to the node is aJSON array, the system extractsdata from the array in the samemode. Nested JSON nodes areseparated by periods (.). Forexample, data.list.

fromJobConfig.encodeType

No String Encoding type. For example,UTF_8 or GBK.

fromJobConfig.splitType

No Enumeration Job split mode. The options areas follows:l FILE: Files are split by file

quantity.l SIZE: Files are split by file

size.

fromJobConfig.fromFileOpType

No Enumeration Source file processing mode.After a job is completed,operations on the source file canbe performed. The source filecan be renamed or deleted.

fromJobConfig.useMarkerFile

No Boolean Whether to start a job by amarker file. A job is started onlywhen a marker file for startingthe job exists in the source path.Otherwise, the job will besuspended for a period of timespecified by Suspension Period.

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

86

Page 92: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Parameter Mandatory Type Description

fromJobConfig.markerFile

No String Name of the identifier file forstarting a job. After an identifierfile is specified, the task isexecuted only when the fileexists in the source path. If thefile is not specified, this functionis disabled by default. Forexample, ok.txt.

fromJobConfig.waitTime

No String Period of waiting for a markerfile. If you set Start Job byMarker File to Yes but nomarker file exists in the sourcepath, the job fails uponsuspension timeout.If you set this parameter to 0 andno marker file exists in thesource path, the job will failimmediately.Unit: second

fromJobConfig.filterType

No Enumeration Whether to filter files usingwildcard characters or regularexpressions. When you select aregular expression, the Javaregular expression is used.l WILDCARD: Files are

filtered by wildcardcharacter.

l REGEX: Files are filtered byJava regular expression.

fromJobConfig.pathFilter

No String Path filter, which is used to filterdirectories in the input path. Forexample, *input.

fromJobConfig.fileFilter

No String File filter, which is used to filterfiles in the input path. Forexample, *.csv.

fromJobConfig.formats

No Datastructure

Time format. This parameter ismandatory only when the fileformat is CSV_FILE and thetime field exists in the file. Fordetails, see Description of thefromJobConfig.formatsparameter.

l Description of the fromJobConfig.formats parameter

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

87

Page 93: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Parameter Mandatory Type Description

name Yes String Column number. For example, 1.

value Yes String Time format. For example, yyyy-MM-dd.

4.3.2.7 From HTTP/HTTPS

Sample JSON File"from-config-values": { "configs": [ { "inputs": [ { "name": "fromJobConfig.inputDirectory", "value": "http://esdk.huawei.com/ilink/esdk/download/HW_456706" }, { "name": "fromJobConfig.inputFormat", "value": "BINARY_FILE" } ], "name": "fromJobConfig" } ] }

Parameter DescriptionParameter Mandatory Type Description

fromJobConfig.inputDirectory

Yes String URL of the file to be extractedThese connectors are used to readfiles with an HTTP/HTTPS URL,such as reading public files on thethird-party object storage systemand web disks.

fromJobConfig.inputFormat

Yes Enumeration File format required for datatransmission. Currently, only thebinary format is supported:

4.3.2.8 From MongoDB/DDS

Sample JSON File"from-config-values": { "configs": [ { "inputs": [ { "name": "fromJobConfig.database", "value": "cdm" },

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

88

Page 94: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

{ "name": "fromJobConfig.collectionName", "value": "person" }, { "name": "fromJobConfig.columnList", "value": "id&name&gender&age&score" } ], "name": "fromJobConfig" } ] }

Parameter Description

Parameter Mandatory Type Description

fromJobConfig.database

Yes String Name of the MongoDB/DDSdatabase

fromJobConfig.collectionName

Yes String Name of the MongoDB/DDScollection

fromJobConfig.columnList

No String List of fields to be extracted. Use& to separate field names. Forexample, id&gid&name.

4.3.2.9 From Redis/DCS

Sample JSON File"from-config-values": { "configs": [ { "inputs": [ { "name": "fromJobConfig.isBatchMigration", "value": "false" }, { "name": "fromJobConfig.keyPrefix", "value": "cdm_string" }, { "name": "fromJobConfig.keySeparator", "value": ":" }, { "name": "fromJobConfig.valueStoreType", "value": "STRING" }, { "name": "fromJobConfig.valueSeparator", "value": "," }, { "name": "fromJobConfig.columnList", "value": "1" } ], "name": "fromJobConfig"

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

89

Page 95: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

} ] }

Parameter Descriptionl Redis job parameter description

Parameter Mandatory Type Description

fromJobConfig.isBatchMigration

No Boolean Whether to migrate all data in thedatabase

fromJobConfig.keyPrefix

Yes String Key prefix, which is the name ofthe corresponding associationtable.Mapping between Redis and theassociation table: Name of theassociation table + delimiter is aRedis key, and a row of data inthe association table is a Redisvalue.

fromJobConfig.keySeparator

Yes String Key delimiter, which separatesthe association table and primarykey

fromJobConfig.valueStoreType

Yes String Storage mode of rows of data inthe association table on Redis.The options are string and hash.l string: indicates that a row of

data is stored as a characterstring using delimiters toseparate columns. This modereduces storage spaceoccupation.

l hash: indicates that a row ofdata is stored in columnname:column value format inthe hash table.

fromJobConfig.valueSeparator

No String Value delimiter. The defaultvalue is \tab. This parameter isvalid when valueStoreType isset to string.

fromJobConfig.columnList

No String List of fields to be extracted. Use& to separate field names. Forexample, id&gid&name.

fromJobConfig.formats

No Datastructure

Time format. For details, seeDescription of thefromJobConfig.formatsparameter.

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

90

Page 96: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

l Description of the fromJobConfig.formats parameter

Parameter Mandatory Type Description

name Yes String Column number. For example, 1.

value Yes String Time format. For example, yyyy-MM-dd.

4.3.2.10 From DIS

Sample JSON File"from-config-values": { "configs": [ { "inputs": [ { "name": "fromJobConfig.streamName", "value": "test" }, { "name": "fromJobConfig.disConsumerStrategy", "value": "FROM_LAST_STOP" }, { "name": "fromJobConfig.isPermanency", "value": "true" }, { "name": "fromJobConfig.maxPollRecords", "value": "100" }, { "name": "fromJobConfig.shardId", "value": "0" }, { "name": "fromJobConfig.separator", "value": "," } ], "name": "fromJobConfig" } ] }

Parameter DescriptionParameter Mandatory Type Description

fromJobConfig.streamName

Yes String DIS stream name

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

91

Page 97: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Parameter Mandatory Type Description

fromJobConfig.disConsumerStrategy

Yes Enumeration Used to set the initial offset whendata is pulled from DIS. Theoptions are as follows:l LATEST: maximum offset,

that is, the latest datal FROM_LAST_STOP: data

after the last stopl EARLIEST: minimum offset,

that is, the earliest data

fromJobConfig.isPermanency

Yes Boolean Whether to run permanently

fromJobConfig.maxPollRecords

No String Maximum number of requests thatcan be sent to DIS each time

fromJobConfig.shardId

Yes String ID of the DIS partition. You canenter multiple partition IDs, whichare separated by commas (,).

fromJobConfig.separator

No String Field delimiter

4.3.2.11 From Kafka

Sample JSON File"from-config-values": { "configs": [ { "inputs": [ { "name": "fromJobConfig.topicsList", "value": "test1,test2" }, { "name": "fromJobConfig.kafkaConsumerStrategy", "value": "EARLIEST" }, { "name": "fromJobConfig.isPermanency", "value": "true" } ], "name": "fromJobConfig" } ] }

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

92

Page 98: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Parameter DescriptionParameter Mandatory Type Description

fromJobConfig.topicsList

Yes String List of Kafka topics. Separatemultiple topics by commas (,).

fromJobConfig.kafkaConsumerStrategy

Yes Enumeration Used to set the initial offset whendata is pulled from Kafka. Theoptions are as follows:l LATEST: maximum offset,

that is, the latest datal EARLIEST: minimum offset,

that is, the earliest data

fromJobConfig.isPermanency

Yes Boolean Whether to run permanently

fromJobConfig.groupId

No String Group of the Kafka offset

fromJobConfig.maxPollRecords

No String Maximum number of requests thatcan be sent to Kafka each time

fromJobConfig.maxPollInterval

No String Maximum interval between eachpoll

fromJobConfig.separator

No String Field delimiter

4.3.2.12 From Elasticsearch/Cloud Search Service

Sample JSON File"from-config-values": { "configs": [ { "inputs": [ { "name": "fromJobConfig.index", "value": "dg" }, { "name": "fromJobConfig.type", "value": "es" }, { "name": "fromJobConfig.columnList", "value": "a1:numeric&s1:string" }, { "name": "fromJobConfig.splitNestedField", "value": "true" }, { "name": "fromJobConfig.queryString", "value": "last_name:Smith" } ],

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

93

Page 99: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

"name": "fromJobConfig" } ] }

Parameter DescriptionParameter Mandatory Type Description

fromJobConfig.index

Yes String Index of the extracted data, whichis similar to the database name inthe relational database

fromJobConfig.type

Yes String Type of the extracted data, which issimilar to the table name in therelational database

fromJobConfig.columnList

No String List of fields to be extracted. Use &to separate field names. Forexample, id&gid&name.

fromJobConfig.splitNestedField

No Boolean Whether to split the JSON contentof the nested field. For example, a:{ b:{ c:1, d:{ e:2, f:3 } } } can besplit into a.b.c, a.b.d.e, and a.b.d.f.

fromJobConfig.queryString

No String Whether to use the Elasticsearchquery string to filter the sourcedata. CDM migrates only the datathat meets the filter criteria.

4.3.3 Destination Job ParametersCDM allows you to import data to multiple types of data sources. Each data source has itsown job parameters. You can view specific input parameters based on the correspondingsection of the data source type.

Set the input parameters of the source link as described in Description of the inputsparameter of to-config-values in Creating a Job.

4.3.3.1 To a Relational Database

Sample JSON File"to-config-values": { "configs": [ { "inputs": [ { "name": "toJobConfig.schemaName", "value": "public" }, { "name": "toJobConfig.tablePreparation", "value": "DO_NOTHING" }, {

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

94

Page 100: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

"name": "toJobConfig.tableName", "value": "cdm_all_type" }, { "name": "toJobConfig.columnList", "value": "Ttiny&Tsmall&Tmediumint&Tint" }, { "name": "toJobConfig.shouldClearTable", "value": "false" } ], "name": "toJobConfig" } ] }

Parameter DescriptionParameter Mandatory Type Description

toJobConfig.schemaName

No String Database mode or tablespace. Forexample, public.

toJobConfig.tablePreparation

Yes Enumeration This parameter is available onlywhen both the source anddestination databases are relationaldatabases. The options for datawrite to tables are as follows:l DO_NOTHING: Do not create

the table automatically.l CREATE_WHEN_NOT_EXI

ST: If the destination databasedoes not contain the tablespecified by tableName, CDMautomatically creates the table.

l DROP_AND_CREATE:Delete the table specified bytableName, and then create thetable again.

toJobConfig.tableName

Yes String Name of the table to which data iswritten. For example,TBL_EXAMPLE.

toJobConfig.columnList

No String List of fields to be loaded. Use &to separate field names. Forexample, id&gid&name.

toJobConfig.shouldClearTable

No Boolean If this parameter is set to true, datain the target table is cleared beforedata is written into the table.

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

95

Page 101: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Parameter Mandatory Type Description

orientation No Enumeration Storage mode. This parameter isenabled only for the DWSdatabase. When the DWS databasetable needs to be automaticallycreated, the optional data storagemodes of the table are as follows:l ROW: Data in the table is

stored in rows.l COLUMN: Data in the table is

stored in columns.

isCompress No Boolean Whether to perform compression.This parameter is enabled only forthe DWS database. When theDWS database table needs to beautomatically created, you canspecify whether to store the data inthe table after compression.

oJobConfig.useStageTable

No Boolean Whether to import data to thephase table first. If this parameteris set to true, the data is importedto the phase table before it isimported to the destination table.After the data is successfullyimported to the phase table, it isthen imported from the phase tableto the destination table. In thisway, the data that is successfullyimported to the destination tableremains in case the data importfails.

extendCharLength

No Boolean Whether to extend the length ofthe character string field. If thisparameter is set to true, the lengthof the character string field in thedestination table is three times thelength of the corresponding field inthe source table when thedestination table needs to beautomatically created.

4.3.3.2 To OBS

Sample JSON File"to-config-values": { "configs": [ {

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

96

Page 102: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

"inputs": [ { "name": "toJobConfig.bucketName", "value": "cdm01" }, { "name": "toJobConfig.kmsEncryption", "value": "false" }, { "name": "toJobConfig.outputDirectory", "value": "/oss2obs/" }, { "name": "toJobConfig.outputFormat", "value": "BINARY_FILE" }, { "name": "toJobConfig.fieldSeparator", "value": "," }, { "name": "toJobConfig.writeToTempFile", "value": "false" }, { "name": "toJobConfig.validateMD5", "value": "true" }, { "name": "toJobConfig.recordMD5Result", "value": "true" }, { "name": "toJobConfig.recordMD5Link", "value": "obslink" }, { "name": "toJobConfig.recordMD5Bucket", "value": "cdm05" }, { "name": "toJobConfig.recordMD5Directory", "value": "/md5/" }, { "name": "toJobConfig.encodeType", "value": "UTF-8" }, { "name": "toJobConfig.duplicateFileOpType", "value": "REPLACE" } ], "name": "toJobConfig" } ] }

Parameter DescriptionParameter Mandatory Type Description

toJobConfig.bucketName

Yes String OBS bucket name. For example,cdm.

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

97

Page 103: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Parameter Mandatory Type Description

toJobConfig.outputDirectory

Yes String Path to which data is written. Forexample, /data_dir.

toJobConfig.outputFormat

Yes Enumeration File format required for datawrites (except the binaryformat). Currently, the followingfile formats are supported:l CSV_FILE: Write data in

CSV format.l BINARY_FILE: Files are

directly transferred withoutresolving the content. CDMwrites the file withoutchanging the file format.

l CARBONDATA: Write datain CarbonData format.

When the parameter is set toBINARY_FILE, the connectorof the source link must beHDFS, OBS, SFTP, or FTP.

toJobConfig.fieldSeparator

No String Column delimiter. Thisparameter is valid only when thefile format is CSV_FILE. Thedefault value is ,.

toJobConfig.lineSeparator

No String Line feed character. Thisparameter is valid only when thefile format is CSV_FILE. Thedefault value is \r\n.

toJobConfig.writeFileSize

No String Whether to fragment multiplefiles by size so that the files areexported in proper size. The unitis MB. This parameter is validwhen the migration source is adatabase.

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

98

Page 104: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Parameter Mandatory Type Description

toJobConfig.duplicateFileOpType

No Enumeration Method for processing duplicatefiles. If the name and size of afile are the same as those ofanother file, the file is regardedas a duplicate file. Duplicatefiles can be processed in thefollowing ways:l REPLACE: Replace

duplicate files.l SKIP: Skip duplicate files.l ABANDON: Stop the job

when any duplicate file isfound.

toJobConfig.kmsEncryption

No Boolean Whether to encrypt the uploadeddata using Key ManagementService (KMS). If KMSencryption is enabled, MD5verification for data cannot beperformed.

toJobConfig.kmsID

No String Key used for encryption duringdata upload. You must create akey in KMS before data upload.

toJobConfig.validateMD5

No Boolean Whether to verify the MD5value. MD5 verification cannotbe used together with KMSencryption. MD5 values can beverified only when files aretransferred in BINARY_FILEformat.Calculate the MD5 value of thesource file and verify it with theMD5 value returned by OBS. Ifan MD5 file exists on the sourceend, directly read the MD5 fileand verify the MD5 file with theMD5 value returned by OBS.

toJobConfig.recordMD5Result

No Boolean Whether to record theverification result when the MD5value is verified

toJobConfig.recordMD5Link

No String OBS link where the bucket towhich the MD5 verificationresult is written resides

toJobConfig.recordMD5Bucket

No String OBS bucket to which the MD5verification result is written

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

99

Page 105: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Parameter Mandatory Type Description

toJobConfig.recordMD5Directory

No String Directory to which the MD5verification result is written

toJobConfig.encodeType

No String Encoding type. For example,UTF_8 or GBK.

toJobConfig.markerFile

No String Whether to generate a markerfile with a custom name in thedestination directory after a jobis executed successfully. If youdo not specify a file name, thisfunction is disabled by default.

toJobConfig.copyContentType

No Boolean This parameter is displayed onlywhen File Format is Binary andboth the migration source anddestination are object storage.If you set this parameter to Yes,the Content-Type attribute of thesource file is copied duringobject file migration. Thisfunction is mainly used for staticwebsite migrationThe Content-Type attributecannot be written to Archivebuckets. Therefore, if you setthis parameter to Yes, themigration destination must be anon-Archive bucket.

toJobConfig.quoteChar

No Boolean This parameter is displayed onlywhen File Format is CSV. It isused when database tables aremigrated to file systems.If you set this parameter to Yesand a field in the source datatable contains a field delimiter orline separator, CDM uses doublequotation marks (") as the quotecharacter to quote the fieldcontent as a whole to prevent afield delimiter from dividing afield into two fields, or a lineseparator from dividing a fieldinto different lines. For example,if the hello,world field in thedatabase is quoted, it will beexported to the CSV file as awhole.

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

100

Page 106: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

4.3.3.3 To HDFS

Sample JSON File"to-config-values": { "configs": [ { "inputs": [ { "name": "toJobConfig.outputDirectory", "value": "/data" }, { "name": "toJobConfig.outputFormat", "value": "CSV_FILE" }, { "name": "toJobConfig.fieldSeparator", "value": "," }, { "name": "toJobConfig.duplicateFileOpType", "value": "REPLACE" }, { "name": "toJobConfig.compression", "value": "NONE" }, { "name": "toJobConfig.appendMode", "value": "true" } ], "name": "toJobConfig" } ] }

Parameter Description

Parameter Mandatory Type Description

toJobConfig.outputDirectory

Yes String Path to which data is written. Forexample, /data_dir.

toJobConfig.outputFormat

Yes Enumeration File format required for data writes(except the binary format).Currently, the following fileformats are supported:l CSV_FILE: Write data in CSV

format.l BINARY_FILE: Files are

directly transferred withoutresolving the content. CDMwrites the file without changingthe file format.

When the parameter is set toBINARY_FILE, the connector ofthe source link must be HDFS,OBS, SFTP, or FTP.

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

101

Page 107: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Parameter Mandatory Type Description

toJobConfig.lineSeparator

No String Line feed character. This parameteris valid only when the file format isCSV_FILE. The default value is \r\n.

toJobConfig.fieldSeparator

No String Column delimiter. This parameter isvalid only when the file format isCSV_FILE. The default value is ,.

toJobConfig.duplicateFileOpType

No Enumeration Method for processing duplicatefiles. If the name and size of a fileare the same as those of anotherfile, the file is regarded as aduplicate file. Duplicate files can beprocessed in the following ways:l REPLACE: Replace duplicate

files.l SKIP: Skip duplicate files.l ABANDON: Stop the job when

any duplicate file is found.

toJobConfig.compression

No Enumeration After the file is written, select thecompression format of the file. Thefollowing compression formats aresupported:l None: Do not compress the file.l DEFLATE: Compress the file

in DEFLATE format.l gzip: Compress the file in gzip

format.l bzip2: Compress the file in

bzip2 format.l LZ4: Compress the file in LZ4

format.l Snappy: Compress the file in

Snappy format.

toJobConfig.appendMode

Yes Boolean Whether to write data when one ormultiple files exist in the loadingpath. The default value is false.

4.3.3.4 To Hive

Sample JSON File"to-config-values": { "configs": [{ "inputs": [{

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

102

Page 108: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

"name": "toJobConfig.hive", "value": "hive" }, { "name": "toJobConfig.database", "value": "default" }, { "name": "toJobConfig.table", "value": "cdm_tbl" }], "name": "toJobConfig" }]}

Parameter Description

Parameter Mandatory Type Description

toJobConfig.hive

No String Data source to which data iswritten

toJobConfig.database

No String Name of the database to whichdata is written. For example,default.

toJobConfig.table

Yes String Name of the table to which datais written

4.3.3.5 To HBase/CloudTable

Sample JSON File "to-config-values": [ { "inputs": [ { "name": "toJobConfig.table", "value": "sqoop_bulkload" }, { "name": "toJobConfig.storageType", "value": "BULKLOAD" }, { "name": "toJobConfig.columns", "value": "cf1:a&cf2:b&cf2:c" }, { "name": "toJobConfig.rowKeyColumn", "value": "cf1:a&cf2:b" }, { "name": "toJobConfig.isOverride", "value": "true" }, { "name": "toJobConfig.algorithm", "value": "GZ" } ], "name": "toJobConfig"

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

103

Page 109: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

} ]

Parameter DescriptionParameter Mandatory Type Description

toJobConfig.table Yes String Name of the table to which data iswritten. For example,TBL_EXAMPLE.

toJobConfig.storageType

Yes Enumeration Mode for writing data to an HBasetable. The options are as follows:l BULKLOAD: The

BULKLOAD mode isrecommended to improve theloading performance.

l PUTLIST: The PUTLISTmode is recommended onlywhen the data volume is small.

toJobConfig.columns

No String Columns to be extracted. Use & toseparate column numbers and : toseparate column families andcolumns. For example,cf1:c1&cf2:c2.

toJobConfig.rowKeyColumn

Yes String Columns serve as rowkeys. Use &to separate column numbers and :to separate column families andcolumns. For example,cf1:c1&cf2:c2.

toJobConfig.isOverride

No Boolean Whether to clear data when data isimported in BULKLOAD mode.For example, true.

toJobConfig.delimiter

No String Delimiter used for separatingcolumns when multiple columnsare used as rowkeys. For example,vertical bars (|).

toJobConfig.isRowkeyRedundancy

No Boolean Whether to write rowkey data tothe HBase column at the sametime

toJobConfig.algorithm

No Enumeration Compression algorithm used whena new HBase table is created. TheSnappy and GZ algorithms aresupported. The default value isNone.

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

104

Page 110: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Parameter Mandatory Type Description

toJobConfig.writeToWAL

No Boolean Whether to enable Write AheadLog (WAL) of HBase. The optionsare as follows:l Yes: If the HBase server breaks

down after the function isenabled, you can replay theoperations that have not beenperformed in WAL.

l No: If you set this parameter toNo, the write performance isimproved. However, if theHBase server breaks down,data may be lost.

toJobConfig.transType

No Boolean l true: Data of the Short, Int,Long, Float, Double, andDecimal columns in the sourcedatabase is converted intoByte[] arrays (binary) andwritten into HBase. Other typesof data are written as characterstrings. If several types of datamentioned above are combinedas rowkeys, they will be writtenas character strings.This function saves storagespace. In specific scenarios, therowkey distribution is evener.

l false: All types of data in thesource database are written intoHBase as character strings.

4.3.3.6 To FTP/SFTP/NAS

Sample JSON File"to-config-values": { "configs": [ { "inputs": [ { "name": "toJobConfig.outputDirectory", "value": "/opt/data" }, { "name": "toJobConfig.outputFormat", "value": "CSV_FILE" }, { "name": "toJobConfig.fieldSeparator", "value": "," }, {

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

105

Page 111: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

"name": "toJobConfig.duplicateFileOpType", "value": "REPLACE" } ], "name": "toJobConfig" } ] }

Parameter DescriptionParameter Mandatory Type Description

toJobConfig.outputDirectory

Yes String Path to which data is written. Forexample, /data_dir.

toJobConfig.outputFormat

Yes Enumeration File format required for data writes(except the binary format).Currently, the following fileformats are supported:l CSV_FILE: Write data in CSV

format.l BINARY_FILE: Files are

directly transferred withoutresolving the content. CDMwrites the file without changingthe file format.

When the parameter is set toBINARY_FILE, the connector ofthe source link must be HDFS,OBS, SFTP, or FTP.

toJobConfig.duplicateFileOpType

No Enumeration Method for processing duplicatefiles. If the name and size of a fileare the same as those of anotherfile, the file is regarded as aduplicate file. Duplicate files canbe processed in the followingways:l REPLACE: Replace duplicate

files.l SKIP: Skip duplicate files.l ABANDON: Stop the job

when any duplicate file isfound.

toJobConfig.lineSeparator

No String Line feed character. Thisparameter is valid only when thefile format is CSV_FILE. Thedefault value is \r\n.

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

106

Page 112: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Parameter Mandatory Type Description

toJobConfig.fieldSeparator

No String Column delimiter. This parameteris valid only when the file formatis CSV_FILE. The default valueis ,.

toJobConfig.encodeType

No String Encoding type. For example,UTF_8 or GBK.

toJobConfig.writeToTempFile

No Boolean This parameter is displayed onlywhen the migration source is a filesystem (OBS/FTP/SFTP/NAS/HDFS), the migration destinationis FTP/SFP/NAS, and File Formatis Binary.The binary file is written to a .tmpfile first. After the migration issuccessful, run the rename ormove command at the migrationdestination to restore the file.

toJobConfig.recordMD5Result

No Boolean This parameter is displayed onlywhen the migration source is a filesystem (OBS/FTP/SFTP/NAS/HDFS), the migration destinationis FTP/SFP/NAS, and File Formatis Binary.An MD5 hash value is generatedfor each transferred file, and thevalue is recorded in a new .md5file. You can specify the directorywhere the MD5 value is generated.

toJobConfig.recordMD5Directory

No String Directory for storing MD5 values

toJobConfig.markerFile

No String Whether to generate a marker filewith a custom name in thedestination directory after a job isexecuted successfully. If you donot specify a file name, thisfunction is disabled by default.

4.3.3.7 To DDS

Sample JSON File"to-config-values": { "configs": [ { "inputs": [ {

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

107

Page 113: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

"name": "toJobConfig.database", "value": "demo" }, { "name": "toJobConfig.collectionName", "value": "cdmbase" }, { "name": "toJobConfig.columnList", "value": "_char&_varchar" }, { "name": "toJobConfig.isBatchMigration", "value": "false" } ], "name": "toJobConfig" } ] }

Parameter DescriptionParameter Mandatory Type Description

toJobConfig.database

Yes String Name of the MongoDB/DDSdatabase

toJobConfig.collectionName

Yes String Name of the MongoDB/DDScollection

toJobConfig.columnList

No String List of fields to be extracted. Use& to separate field names. Forexample, id&gid&name.

toJobConfig.isBatchMigration

No Boolean Whether to migrate all data in thedatabase

4.3.3.8 To DCS

Sample JSON File "to-config-values": { "configs": [ { "inputs": [ { "name": "toJobConfig.isBatchMigration", "value": "false" }, { "name": "toJobConfig.shouldClearDatabase", "value": "false" }, { "name": "toJobConfig.keyPrefix", "value": "cdm_string" }, { "name": "toJobConfig.keySeparator", "value": ":" },

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

108

Page 114: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

{ "name": "toJobConfig.primaryKeyList", "value": "1" }, { "name": "toJobConfig.valueStoreType", "value": "STRING" }, { "name": "toJobConfig.valueSeparator", "value": "," }, { "name": "toJobConfig.columnList", "value": "1&2&3&4&5&6&7&8&9&10&11&12" } ], "name": "toJobConfig" } ] }

Parameter Descriptionl Parameter description

Parameter Mandatory Type Description

toJobConfig.isBatchMigration

No Boolean Whether to migrate all data in thedatabase

toJobConfig.shouldClearDatabase

No Boolean Whether to clear data beforeimport

toJobConfig.keyPrefix

Yes String Key prefix, which is similar tothe table name of a relationaldatabaseMapping between Redis and theassociation table: Name of theassociation table + delimiter is aRedis key, and a row of data inthe association table is a Redisvalue.

toJobConfig.keySeparator

Yes String Key delimiter, which separatesthe association table and primarykey

toJobConfig.primaryKeyList

Yes String List of primary keys. Use & toseparate field names. Forexample, id&gid.

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

109

Page 115: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Parameter Mandatory Type Description

toJobConfig.valueStoreType

Yes Enumeration

Storage mode of rows of data inthe association table on Redis.The options are string and hash.l STRING: indicates that a row

of data is stored as a characterstring, in which columns inthe row are separated byvalueSeparator.

l Hash: indicates that a row ofdata is stored in columnname:column value format inthe hash table.

toJobConfig.valueSeparator

No String Value delimiter. The default valueis \tab. This parameter is validwhen valueStoreType is set tostring.

toJobConfig.columnList

No String List of fields to be written. Use &to separate field names. Forexample, id&gid&name.

toJobConfig.formats

No Datastructure

Time format. For details, seeDescription of thetoJobConfig.formatsparameter.

l Description of the toJobConfig.formats parameter

Parameter Mandatory Type Description

name Yes String Column number. For example, 1.

value Yes String Time format. For example, yyyy-MM-dd.

4.3.3.9 To Elasticsearch/Cloud Search Service

Sample JSON File"to-config-values": { "configs": [ { "inputs": [ { "name": "toJobConfig.index", "value": "cdm" }, { "name": "toJobConfig.type", "value": "type1" }, {

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

110

Page 116: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

"name": "toJobConfig.shouldClearType", "value": "false" }, { "name": "toJobConfig.pipeLine", "value": "es_03" } ], "name": "toJobConfig" } ] }

Parameter DescriptionParameter Mandatory Type Description

toJobConfig.index Yes String Index of the written data, which issimilar to the database name in therelational database

toJobConfig.type Yes String Type of the written data, which issimilar to the table name in therelational database

toJobConfig.shouldClearType

No Boolean Whether to clear data before import

toJobConfig.primaryKey

No String Primary key or unique index

toJobConfig.columnList

No String List of fields to be written. Use &to separate field names. Forexample, id&gid&name.

toJobConfig.pipeLine

No String This parameter is available onlyafter a pipeline ID is created inKibana. It is used to convert thedata format using the datatransformation pipeline of CloudSearch Service or Elasticsearchafter data is transferred to CloudSearch Service or Elasticsearch.

4.3.3.10 To DLI

Sample JSON File"to-config-values": { "configs": [ { "inputs": [ { "name": "toJobConfig.queue", "value": "cdm" }, { "name": "toJobConfig.database",

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

111

Page 117: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

"value": "sqoop" }, { "name": "toJobConfig.table", "value": "t_test1" }, { "name": "toJobConfig.columnList", "value": "string_&int_&date_&double_&boolean_&short_&timestamp_&long_&smallint_&bigint_" }, { "name": "toJobConfig.shouldClearTable", "value": "false" } ], "name": "toJobConfig" } ] }

Parameter DescriptionParameter Mandatory Type Description

toJobConfig.queue

Yes String Resource queue to which data iswritten

toJobConfig.database

Yes String DLI database to which data iswritten

toJobConfig.table Yes String Name of the table to which data iswritten

toJobConfig.columnList

No String List of fields to be loaded. Use & toseparate field names. For example,id&gid&name.

toJobConfig.shouldClearTable

No Boolean Whether to clear data in theresource queue before data import

4.3.4 Job Parameter Description

Sample JSON File"driver-config-values": { "configs": [ { "inputs": [ { "name": "throttlingConfig.numExtractors", "value": "3" }, { "name": "throttlingConfig.numLoaders", "value": "1" }, { "name": "throttlingConfig.recordDirtyData", "value": "true" },

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

112

Page 118: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

{ "name": "throttlingConfig.writeToLink", "value": "obs_link_auto" }, { "name": "throttlingConfig.obsBucket", "value": "dirtydata" }, { "name": "throttlingConfig.dirtyDataDirectory", "value": "/user/sqoop/dirtydir" }, { "name": "throttlingConfig.maxErrorRecords", "value": "0" } ], "name": "throttlingConfig" }, { "inputs": [], "name": "jarConfig" }, { "inputs": [ { "name": "schedulerConfig.isSchedulerJob", "value": "true" }, { "name": "schedulerConfig.cycleType", "value": "minute" }, { "name": "schedulerConfig.cycle", "value": "30" }, { "name": "schedulerConfig.startDate", "value": "2018-01-22 14:57:23" } ], "name": "schedulerConfig" }, { "inputs": [], "name": "transformConfig" }, { "inputs": [ { "name": "schedulerConfig.isSchedulerJob", "value": "false" }, { "name": "schedulerConfig.disposableType", "value": "NONE" } ], "name": "schedulerConfig" }, { "inputs": [ { "name": "retryJobConfig.retryJobType", "value": "RETRY_TRIPLE" } ], "name": "retryJobConfig"

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

113

Page 119: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

} ] }

Parameter DescriptionParameter Mandatory Type Description

throttlingConfig.numExtractors

No Integer Maximum number of extractionjobs. For example, 20.

throttlingConfig.numLoaders

No Integer This parameter is available onlywhen HBase or Hive serves as thedestination data source.Maximum number of loading jobs.For example, 5.

throttlingConfig.recordDirtyData

No Boolean Whether to write dirty data. Forexample, true.

throttlingConfig.writeToLink

No String Link to which dirty data is written.Currently, dirty data can be writtenonly to OBS or HDFS. Forexample, obslink.

throttlingConfig.obsBucket

No String Name of the OBS bucket to whichdirty data is written. This parameteris valid only when dirty data iswritten to OBS. For example,dirtyData.

throttlingConfig.dirtyDataDirectory

No String Directory to which dirty data iswrittenl To write dirty data to HDFS, set

this parameter to the specifiedHDFS directory.

l To write dirty data to OBS, setthis parameter to the directory inthe OBS bucket. For example, /data/dirtydata/.

throttlingConfig.maxErrorRecords

No String Maximum number of error recordsin a single shard. When the numberof error records of a map exceedsthe upper limit, the taskautomatically ends. The importeddata will not be rolled back.

schedulerConfig.isSchedulerJob

No Boolean Whether to enable a scheduled task.For example, true.

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

114

Page 120: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Parameter Mandatory Type Description

schedulerConfig.cycleType

No String Cycle type of a scheduled task. Theoptions are as follows:l minute: minutel hour: hourl day: dayl week: weekl month: month

schedulerConfig.cycle

No Integer Cycle of a scheduled task. IfcycleType is set to minute andcycle is set to 10, the scheduledtask is executed every 10 minutes.

schedulerConfig.runAt

No String Time when a scheduled task istriggered in a cycle. This parameteris valid only when cycleType is setto hour, week, or month.l If cycleType is set to month,

cycle is set to 1, and runAt isset to 15, the scheduled task isexecuted on the 15th day ofeach month. You can set runAtto multiple values and separatethe values with commas (,).For example, if runAt is set to1,2,3,4,5, the scheduled task isexecuted on the first day, secondday, third day, fourth day, andfifth day of each month.

l If cycleType is set to week andrunAt is set tomon,tue,wed,thu,fri, thescheduled task is executed onMonday to Friday.

l If cycleType is set to hour andrunAt is set to 27,57, thescheduled task is executed at the27th and 57th minute in thecycle.

schedulerConfig.startDate

No String Start time of a scheduled task. Forexample, 2018-01-24 19:56:19.

schedulerConfig.stopDate

No String End time of a scheduled task. Forexample, 2018-01-27 23:59:00.If you do not set the end time, thescheduled task is always executedand will never stop.

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

115

Page 121: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Parameter Mandatory Type Description

schedulerConfig.disposableType

No Enumeration Whether to delete a job after the jobis executed. The options are asfollows:l NONE: A job will not be

deleted after it is executed.l DELETE_AFTER_SUCCEE

D: A job will be deleted onlyafter it is successfully executed.It is applicable to massive one-time jobs.

l DELETE: A job will be deletedafter it is executed, regardless ofthe execution result.

retryJobConfig.retryJobType

No Enumeration Whether to automatically retry if ajob fails to be executed. Theoptions are as follows:l NONE: Do not retry.l RETRY_TRIPLE: Retry three

times.

4.3.5 Querying a Job

FunctionThis API is used to query all or a specified job.

URIl URI format

GET /cdm/v1.0/{project_id}/clusters/{cluster_id}/cdm/job/{job_name}?filter={filter}&page_no={page_no}&page_size={page_size}

l Parameter description

Parameter Mandatory Type Description

project_id Yes String Project ID

cluster_id Yes String Cluster ID

job_name Yes String Job name. When this parameter isset to all, all jobs are returned.

filter No String This parameter is used to enablefuzzy filtration when job_name isset to all.

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

116

Page 122: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Parameter Mandatory Type Description

page_no No Integer Page number. Jobs on the specifiedpage will be returned.

page_size No Integer Number of jobs on each page

RequestSample requestGET /cdm/v1.0/1551c7f6c808414d8e9f3c514a170f2e/clusters/6ec9a0a4-76be-4262-8697-e7af1fac7920/cdm/job/all

Responsel Sample response

{ "total": 1, "jobs": [{ "name": "oracle2hive", "type": "NORMAL_JOB", "creation-user": "cdm", "creation-date": 1514880878504, "update-date": 1514880878083, "update-user": "cdm", "status": "NEW", "from-link-name": "oraclelink", "from-connector-name": "generic-jdbc-connector", "to-link-name": "hivelink", "to-connector-name": "hive-connector", "from-config-values": { "configs": [{ "inputs": [{ "name": "fromJobConfig.schemaName", "value": "SQOOP" }, { "name": "fromJobConfig.tableName", "value": "SQ_INPUT" }, { "name": "fromJobConfig.columnList", "value": "SQI_ID&SQI_NAME" }], "name": "fromJobConfig", "type": "JOB" }] }, "to-config-values": { "configs": [{ "inputs": [{ "name": "toJobConfig.hive", "value": "hive" }, { "name": "toJobConfig.database", "value": "default" }, { "name": "toJobConfig.table", "value": "sq_input" }, { "name": "toJobConfig.columnList",

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

117

Page 123: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

"value": "sqi_id&sqi_name" }], "name": "toJobConfig", "type": "JOB" }], }, "driver-config-values": { "configs": [{ "inputs": [{ "name": "throttlingConfig.numExtractors", "value": "1" }] }] } }, }], "page_no": 1, "page_size": 20}

l Response parameter description

Parameter Mandatory Type Description

total No Integer Number of jobs

jobs Yes List Job list. For details, seeDescription of the jobsparameter.

page_no No Integer Page number. Jobs on thespecified page will be returned.

page_size No Integer Number of jobs on each page

simple No Boolean When this parameter is set totrue, compact information willbe returned. In other words,only parameter names andvalues will be returned.Attributes such as size, type,and id will not be returned.

l Description of the jobs parameter

Parameter Mandatory Type Description

name Yes String Job name

type Yes String Job type. The options are asfollows:l NORMAL_JOB: table/file

migrationl BATCH_JOB: entire DB

migration

creation-user Yes String User who created a job

creation-date Yes Timestamp Time when a job is created

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

118

Page 124: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Parameter Mandatory Type Description

update-date Yes Timestamp Time when a job is updated

update-user No String User who updated a job

status No String Status of a job. The options areas follows:l BOOTING: The job is

starting.l RUNNING: The job is

running.l SUCCEEDED: The job is

executed successfully.l FAILED: The job failed.l NEW: The job has not been

executed.

from-link-name

Yes String Source link name

from-connector-name

Yes String Name of the connector of thesource link

to-link-name Yes String Destination link name

to-connector-name

Yes String Name of the connector of thedestination link

from-config-values

Yes Dictionary Source link configuration. Fordetails, see Description offrom-config-values.

to-config-values

Yes Dictionary Destination link configuration.For details, see Description ofto-config-values.

driver-config-values

Yes Dictionary Job parameter configuration.For details, see Description ofdriver-config-values.

Return Valuel Normal

200l Abnormal

Return Value Description

400 Bad Request Request error.

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

119

Page 125: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Return Value Description

401 Unauthorized Authentication failed.

403 Forbidden No operation permission.

404 Not Found The requested resource is not found.

500 Internal Server Error Internal error. For details about the returnederror code, see Error Code.

503 Service Unavailable Service unavailable.

4.3.6 Modifying a Job

Function

This API is used to modify a job.

URIl URI format

PUT /cdm/v1.0/{project_id}/clusters/{cluster_id}/cdm/job/{job_id}l Parameter description

Parameter Mandatory Type Description

project_id Yes String Project ID

cluster_id Yes String Cluster ID

job_id Yes String Job ID. The job name is used asthe unique job ID.

l Parameter description of a message body

The message body parameters for modifying a job are the same as those for creating ajob. For details, see Description of the jobs parameter in Creating a Job.

Request

Sample request

PUT /cdm/v1.0/1551c7f6c808414d8e9f3c514a170f2e/cluster/6ec9a0a4-76be-4262-8697-e7af1fac7920/cdm/job/mysql2hive

{ "jobs": [{ "name": "mysql2hive", "from-link-name": "mysqllink", "from-connector-name": "generic-jdbc-connector", "to-link-name": "hivelink", "to-connector-name": "hive-connector", "from-config-values": { "configs": [{ "inputs": [{ "name": "fromJobConfig.schemaName",

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

120

Page 126: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

"value": "default" }, { "name": "fromJobConfig.tableName", "value": "test_tbl" }, { "name": "fromJobConfig.partitionColumn", "value": "id" }], "name": "fromJobConfig", "type": "JOB" }] }, "to-config-values": { "configs": [{ "inputs": [{ "name": "toJobConfig.hive", "value": "hive" }, { "name": "toJobConfig.database", "value": "default" }, { "name": "toJobConfig.table", "value": "test_tbl" }], "name": "toJobConfig", "type": "JOB" }] }, "driver-config-values": { "configs": [{ "inputs": [{ "name": "throttlingConfig.numExtractors", "value": "1" }], "name": "throttlingConfig", "type": "JOB" }] } }]}

Responsel Sample response

{ "name": "mysql2hive", "validation-result": [{},{},{}]}

l Parameter Description

Parameter Mandatory Type Description

name Yes String Job name

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

121

Page 127: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Parameter Mandatory Type Description

validation-result Yes List Validation resultl If a job fails to be

modified, the failurecause is returned.

l If a job is successfullymodified, a blank list isreturned.

Return Valuel Normal

200l Abnormal

Return Value Description

400 Bad Request Request error.

401 Unauthorized Authentication failed.

403 Forbidden No operation permission.

404 Not Found The requested resource is not found.

500 Internal Server Error Internal error. For details about the returnederror code, see Error Code.

503 Service Unavailable Service unavailable.

4.3.7 Starting a Job

FunctionThis API is used to start a job.

URIl URI format

PUT /cdm/v1.0/{project_id}/clusters/{cluster_id}/cdm/job/{job_name}/start

l Parameter description

Parameter Mandatory Type Description

project_id Yes String Project ID

cluster_id Yes String Cluster ID

job_name Yes String Job name

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

122

Page 128: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Request

Sample requestPUT /cdm/v1.0/1551c7f6c808414d8e9f3c514a170f2e/clusters/6ec9a0a4-76be-4262-8697-e7af1fac7920/cdm/job/jdbc2hive/start

Responsel Sample response

{ "submissions": [{ "progress": 1, "job-name": "jdbc2hive", "status": "BOOTING", "creation-date": 1496905778725, "creation-user": "cdm" }]}

l Parameter description

Parameter Mandatory Type Description

submissions Yes List Job running information.For details, seeDescription of thesubmissions parameter.

l Description of the submissions parameter

Parameter Mandatory Type Description

progress Yes Integer Job progress. If a job fails, thevalue is -1. Otherwise, thevalue is 0 to 100.

job-name Yes String Job name

status Yes String Job status. The options are asfollows:l BOOTING: The job is

starting.l FAILURE_ON_SUBMIT

: The job fails to besubmitted.

l RUNNING: The job isrunning.

l SUCCEEDED: The job isexecuted successfully.

l FAILED: The job failed.l UNKNOWN: The job

status is unknown.l NEVER_EXECUTED:

The job has not beenexecuted.

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

123

Page 129: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Parameter Mandatory Type Description

creation-date Yes Timestamp Time when a job is created

creation-user Yes String User who created a job

Return Valuel Normal

200l Abnormal

Return Value Description

400 Bad Request Request error.

401 Unauthorized Authentication failed.

403 Forbidden No operation permission.

404 Not Found The requested resource is not found.

500 Internal Server Error Internal error. For details about the returnederror code, see Error Code.

503 Service Unavailable Service unavailable.

4.3.8 Stopping a Job

Function

This API is used to stop a running job.

URIl URI format

PUT /cdm/v1.0/{project_id}/clusters/{cluster_id}/cdm/job/{job_name}/stop

l Parameter description

Parameter Mandatory Type Description

project_id Yes String Project ID

cluster_id Yes String Cluster ID

job_name Yes String Job name

Request

Sample request

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

124

Page 130: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

PUT /cdm/v1.0/1551c7f6c808414d8e9f3c514a170f2e/clusters/6ec9a0a4-76be-4262-8697-e7af1fac7920/cdm/job/jdbc2hive/stop

Responsel Sample response

{ "submissions": [{ "progress": -1, "job-name": "jdbc2hive", "status": "FAILED", "creation-date": 1496905778725, "creation-user": "cdm" }]}

l Parameter description

Parameter Mandatory Type Description

submissions Yes List Job running information.For details, seeDescription of thesubmissions parameter.

l Description of the submissions parameter

Parameter Mandatory Type Description

progress Yes Integer Job progress. If a job fails,the value is -1. Otherwise, thevalue is 0 to 100.

job-name Yes String Job name

status Yes String Job status. The options are asfollows:l BOOTING: The job is

starting.l FAILURE_ON_SUBMI

T: The job fails to besubmitted.

l RUNNING: The job isrunning.

l SUCCEEDED: The jobis executed successfully.

l FAILED: The job failed.l UNKNOWN: The job

status is unknown.l NEVER_EXECUTED:

The job has not beenexecuted.

creation-date Yes Timestamp Time when a job is created

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

125

Page 131: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Parameter Mandatory Type Description

creation-user Yes String User who created a job

Return Valuel Normal

200l Abnormal

Return Value Description

400 Bad Request Request error.

401 Unauthorized Authentication failed.

403 Forbidden No operation permission.

404 Not Found The requested resource is not found.

500 Internal Server Error Internal error. For details about the returnederror code, see Error Code.

503 Service Unavailable Service unavailable.

4.3.9 Querying Job Status

Function

This API is used to query job running status.

URIl URI format

GET /cdm/v1.0/{project_id}/clusters/{cluster_id}/cdm/job/{job_name}/status

l Parameter description

Parameter Mandatory Type Description

project_id Yes String Project ID

cluster_id Yes String Cluster ID

job_name Yes String Job name

Request

Sample requestGET /cdm/v1.0/1551c7f6c808414d8e9f3c514a170f2e/clusters/6ec9a0a4-76be-4262-8697-e7af1fac7920/cdm/job/jdbc2hive/status

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

126

Page 132: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Responsel Sample job running response

{ "submissions": [{ "progress": 1, "job-name": "jdbc2hive", "status": "BOOTING", "creation-date": 1496905778725, "creation-user": "cdm" }]}

l Sample response of a successful job{ "submissions": [{ "progress": 0, "job-name": "jdbc2hive", "status": "SUCCEEDED", "creation-date": 1496906582997, "creation-user": "cdm", "isStopingIncrement": "", "last-update-date": 1496916580005, "is-execute-auto": false, "last-udpate-user": "cdm", "isDeleteJob": false, "isIncrementing": false, "external-id": "job_local1127970451_0009", "counters": { "org.apache.sqoop.submission.counter.SqoopCounters": { "BYTES_WRITTEN": -1, "TOTAL_FILES": -1, "BYTES_READ": -1, "FILES_WRITTEN": -1, "TOTAL_SIZE": -1, "FILES_READ": -1, "ROWS_WRITTEN": 80, "ROWS_READ": 80 } } }]}

l Parameter description

Parameter Mandatory Type Description

submissions Yes List Job runninginformation. Fordetails, seeDescription of thesubmissionsparameter.

l Description of the submissions parameter

Parameter Mandatory Type Description

job-name Yes String Job name

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

127

Page 133: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Parameter Mandatory Type Description

status Yes String Job status. The options are asfollows:l BOOTING: The job is

starting.l RUNNING: The job is

running.l SUCCEEDED: The job

is executed successfully.l FAILED: The job failed.l NEW: The job has not

been executed.

creation-date Yes Timestamp Time when a job is created

creation-user Yes String User who created a job

isStopingIncrement

Yes Boolean Whether to stop incrementalmigration

last-update-date Yes Timestamp Time when a job is updated

is-execute-auto Yes Boolean Whether to execute a job atthe scheduled time

last-udpate-user Yes String Last user who updates the jobstatus

isDeleteJob Yes Boolean Whether to delete a job afterthe job is executed

isIncrementing Yes Boolean Whether a job is anincremental migration job

external-id Yes String Job ID

counters No Dictionary Job running result statistics.This parameter is availableonly when status isSUCCEEDED. For details,see Description of thecounters parameter.

l Description of the counters parameter

Parameter Mandatory Type Description

org.apache.sqoop.submission.counter.SqoopCounters

Yes List Job running resultstatistics. For details, seeDescription oforg.apache.sqoop.submission.counter.SqoopCounters.

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

128

Page 134: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

l Description of org.apache.sqoop.submission.counter.SqoopCounters

Parameter Mandatory Type Description

ROWS_WRITTEN No Integer Number of rows that arewritten

ROWS_READ No Integer Number of rows that areread

BYTES_WRITTEN No Integer Number of bytes that arewritten

BYTES_READ No Integer Number of bytes that areread

TOTAL_SIZE No Integer Total number of bytes

FILES_WRITTEN No Integer Number of files that arewritten

FILES_READ No Integer Number of files that areread

FILES_SKIPPED No Integer Number of files that areskipped

TOTAL_FILES No Integer Total number of files

Return Valuel Normal

200l Abnormal

Return Value Description

400 Bad Request Request error.

401 Unauthorized Authentication failed.

403 Forbidden No operation permission.

404 Not Found The requested resource is not found.

500 Internal Server Error Internal error. For details about the returnederror code, see Error Code.

503 Service Unavailable Service unavailable.

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

129

Page 135: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

4.3.10 Querying Job Execution History

FunctionThis API is used to query the historical job execution status.

URIl URI format

GET /cdm/v1.0/{project_id}/clusters/{cluster_id}/cdm/submissions?jname={job_name}

l Parameter description

Parameter Mandatory Type Description

project_id Yes String Project ID

cluster_id Yes String Cluster ID

job_name Yes String Job name

RequestSample requestGET /cdm/v1.0/1551c7f6c808414d8e9f3c514a170f2e/clusters/6ec9a0a4-76be-4262-8697-e7af1fac7920/cdm/submissions?jname=jdbc2hive

Responsel Sample job running response

{ "submissions": [{ "progress": 1, "job-name": "jdbc2hive", "status": "BOOTING", "creation-date": 1496905778725, "creation-user": "cdm" }]}

l Sample response of a successful job{ "submissions": [{ "progress": 0, "job-name": "jdbc2hive", "status": "SUCCEEDED", "creation-date": 1496906582997, "creation-user": "cdm", "isStopingIncrement": "", "last-update-date": 1496916580005, "is-execute-auto": false, "last-udpate-user": "cdm", "isDeleteJob": false, "isIncrementing": false, "external-id": "job_local1127970451_0009", "counters": { "org.apache.sqoop.submission.counter.SqoopCounters": { "BYTES_WRITTEN": -1, "TOTAL_FILES": -1, "BYTES_READ": -1, "FILES_WRITTEN": -1,

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

130

Page 136: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

"TOTAL_SIZE": -1, "FILES_READ": -1, "ROWS_WRITTEN": 80, "ROWS_READ": 80 } } }]}

l Parameter description

Parameter Mandatory Type Description

submissions Yes List Job running information.For details, seeDescription of thesubmissions parameter.

Return Valuel Normal

200l Abnormal

Return Value Description

400 Bad Request Request error.

401 Unauthorized Authentication failed.

403 Forbidden No operation permission.

404 Not Found The requested resource is not found.

500 Internal Server Error Internal error. For details about the returnederror code, see Error Code.

503 Service Unavailable Service unavailable.

4.3.11 Deleting a Job

Function

This API is used to delete a specified job.

URIl URI format

DELETE /cdm/v1.0/{project_id}/clusters/{cluster_id}/cdm/job/{job_name}

l Parameter description

Parameter Mandatory Type Description

project_id Yes String Project ID

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

131

Page 137: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Parameter Mandatory Type Description

cluster_id Yes String Cluster ID

job_name Yes String Job name

RequestSample requestDELETE /cdm/v1.0/1551c7f6c808414d8e9f3c514a170f2e/clusters/6ec9a0a4-76be-4262-8697-e7af1fac7920/cdm/job/jdbc2hive

Responsel Successful sample response

{}l Failed sample response

{ "errCode": "Cdm.0100", "externalMessage": "Job[jdbc2hive] doesn't exist."}

l Parameter description of a failed response

Parameter Mandatory Type Description

errCode Yes String Error code

externalMessage Yes String Error message

Return Valuel Normal

200l Abnormal

Return Value Description

400 Bad Request Request error.

401 Unauthorized Authentication failed.

403 Forbidden No operation permission.

404 Not Found The requested resource is not found.

500 Internal Server Error Internal error. For details about the returnederror code, see Error Code.

503 Service Unavailable Service unavailable.

Cloud Data MigrationAPI Reference 4 API Description

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

132

Page 138: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

5 Common Parameters

5.1 Error CodeNo data is returned if an API fails to be called. You can locate the cause of an error accordingto the error code of each API. When the API calling fails, HTTP status code 4xx or 5xx isreturned. The returned message body contains the specific error code and error information. Ifyou fail to locate the cause of an error, contact HUAWEI CLOUD service support and providethe error code so that we can help you solve the problem as soon as possible.

l Sample exception response{ "errCode": "Cdm.0100", "externalMessage": "Job[jdbc2hive] doesn't exist."}

l Parameter description

Parameter Mandatory Type Description

errCode No String Error code

externalMessage No String Error message

l Error code

In the following error message, %s is a variable. In the actual situation, the parametername, table name, job name, and link name are replaced with the actual values.

Error Code Status Code Error Message

Cdm.0000 500 System error.

Cdm.0009 500 %s is not an integer or is beyond the valuerange [0, 2147483647].

Cdm.0010 500 The integer must be within the range of[%s].

Cdm.0011 500 Validator has malformed argument.

Cloud Data MigrationAPI Reference 5 Common Parameters

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

133

Page 139: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Error Code Status Code Error Message

Cdm.0012 500 JDBC driver class is not found.

Cdm.0014 500 Invalid parameter.

Cdm.0015 500 An error occurred during file parse.

Cdm.0016 500 The file to be uploaded cannot be empty.

Cdm.0017 500 MRS Kerberos authentication failed.

Cdm.0018 500 The content of jobs or links is invalid.

Cdm.0019 500 Log %s does not exist.

Cdm.0020 500 The string must contain the followingsubstring: %s

Cdm.0021 500 Failed to connect to the server: %s .

Cdm.0024 500 [%s] must be within the range of [%s].

Cdm.0051 500 Invalid submission engine: %s

Cdm.0052 500 Job %s is running.

Cdm.0053 500 Job %s is not running.

Cdm.0054 500 Job %s does not exist.

Cdm.0055 500 Unsupported job type.

Cdm.0056 500 Failed to submit the job. Cause: %s

Cdm.0057 500 Invalid job execution engine: %s

Cdm.0058 500 Invalid combination of submission andexecution engines.

Cdm.0059 500 Job %s has been disabled. Failed to submitthe job.

Cdm.0060 500 Link %s for this job has been disabled.Failed to submit the job.

Cdm.0061 500 Connector %s does not support thespecified direction. Failed to submit thejob.

Cdm.0062 500 The binary file is applicable only to theSFTP, FTP, HDFS, or OBS connector.

Cdm.0063 500 An error occurred during table creation.Cause: %s

Cdm.0064 500 The data format is incorrect.

Cdm.0065 500 Failed to start the scheduler. Cause: %s

Cloud Data MigrationAPI Reference 5 Common Parameters

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

134

Page 140: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Error Code Status Code Error Message

Cdm.0066 500 Failed to obtain the sample value. Cause:%s

Cdm.0067 500 Failed to obtain the schema. Cause: %s

Cdm.0100 500 Job [%s] does not exist.

Cdm.0101 500 Link [%s] does not exist.

Cdm.0102 500 Connector [%s] does not exist.

Cdm.0104 500 The job name already exists.

Cdm.0010 500 The integer must be within the range of[%s].

Cdm.0201 500 Failed to obtain the instance.

Cdm.0202 500 The job status is unknown.

Cdm.0204 500 No MRS link is created.

Cdm.0230 500 Failed to load the specified class: %s

Cdm.0231 500 Failed to initialize the specified class: %s

Cdm.0232 500 Failed to write data. Cause: %s

Cdm.0233 500 An exception occurred during dataextraction. Cause: %s

Cdm.0234 500 An exception occurred during data loading.Cause: %s

Cdm.0235 500 All data has been consumed. Cause: %s

Cdm.0236 500 Invalid partitions have been retrieved fromPartitioner.

Cdm.0237 500 Failed to find the JAR file of the connector.

Cdm.0238 500 %s cannot be empty.

Cdm.0239 500 Failed to obtain HDFS. Cause: %s

Cdm.0240 500 Failed to obtain the status of the %s file.

Cdm.0241 500 Failed to obtain the type of the %s file.

Cdm.0242 500 An exception occurs during file check: %s

Cdm.0243 500 Failed to rename %s to %s.

Cdm.0244 500 Failed to create the %s file.

Cdm.0245 500 Failed to delete the %s file.

Cdm.0246 500 Failed to create the %s directory.

Cloud Data MigrationAPI Reference 5 Common Parameters

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

135

Page 141: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Error Code Status Code Error Message

Cdm.0247 500 HBase operation failure. Cause: %s

Cdm.0248 500 Failed to clear data in %s. Cause: %s

Cdm.0249 500 The file name %s is invalid.

Cdm.0250 500 Failed to perform operations in the path: %s

Cdm.0251 500 Failed to load data to HBase. Cause: %s

Cdm.0307 500 Failed to release the link. Cause: %s

Cdm.0315 500 The Link name %s already exists.

Cdm.0316 500 The link that does not exist cannot beupdated.

Cdm.0317 500 Link %s is invalid.

Cdm.0318 500 The job already exists and cannot becreated repeatedly.

Cdm.0319 500 The job that does not exist cannot beupdated.

Cdm.0320 500 Job %s is invalid.

Cdm.0321 500 Link %s has been used.

Cdm.0322 500 Job %s has been used.

Cdm.0323 500 The submission already exists and cannotbe created repeatedly.

Cdm.0327 500 Invalid link or job: %s

Cdm.0411 500 An error occurred when connecting to thefile server.

Cdm.0412 500 An error occurred when disconnecting fromthe file server.

Cdm.0413 500 An error occurred in data transfer to the fileserver.

Cdm.0415 500 An error occurred when downloading filesfrom the file server.

Cdm.0416 500 An error occurred during data extraction.

Cdm.0420 500 The source file or source directory does notexist.

Cdm.0423 500 Duplicate files exist in the destination path.

Cdm.0500 500 The source directory or the [%s] file doesnot exist.

Cloud Data MigrationAPI Reference 5 Common Parameters

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

136

Page 142: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Error Code Status Code Error Message

Cdm.0501 500 Invalid URI [%s].

Cdm.0518 500 Failed to connect to HDFS. Cause: %s

Cdm.0600 500 Failed to connect to the FTP server.

Cdm.0700 500 Failed to connect to the SFTP server.

Cdm.0800 500 Failed to connect to the OBS server.

Cdm.0801 500 OBS bucket [%s] does not exist.

Cdm.0900 500 Table [%s] does not exist.

Cdm.0901 500 Failed to connect to the database server.Cause: %s

Cdm.0902 500 Failed to execute the SQL statement.Cause: %s

Cdm.0903 500 Failed to obtain metadata. Cause: %s

Cdm.0904 500 An error occurred while retrieving datafrom the result. Cause: %s

Cdm.0905 500 No partition column is found.

Cdm.0906 500 No boundary is found in the partitioncolumn.

Cdm.0911 500 The table name or SQL must be specified.

Cdm.0912 500 The table name and SQL cannot bespecified at the same time.

Cdm.0913 500 Schema and SQL cannot be specified at thesame time.

Cdm.0914 500 Partition column is mandatory for query-based import.

Cdm.0915 500 The SQL-based import mode andcolumnList cannot be used at the sametime.

Cdm.0916 500 Last value is mandatory for incrementalread.

Cdm.0917 500 Last value cannot be obtained without fieldcheck.

Cdm.0918 500 If no transfer table is specified,shouldClearStageTable cannot bespecified.

Cdm.0921 500 Type %s is not supported.

Cloud Data MigrationAPI Reference 5 Common Parameters

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

137

Page 143: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Error Code Status Code Error Message

Cdm.0925 500 The partition column contains unsupportedvalues.

Cdm.0926 500 Failed to obtain the schema. Cause: %s

Cdm.0927 500 The transfer table is not empty.

Cdm.0928 500 An error occurred when data is migratedfrom the transfer table to the destinationtable.

Cdm.0931 500 Schema column size [%s] does not matchthe result set column size [%s].

Cdm.0932 500 Failed to obtain the maximum value of thefield.

Cdm.0934 500 Multiple tables of the same name exist indifferent schemas or catalogs.

Cdm.0935 500 No primary key. Specify the partitioncolumn.

Cdm.0936 500 The number of dirty data records reachesthe upper limit.

Cdm.0940 500 Failed to match the exact table name.

Cdm.0941 500 Failed to connect to the server. Cause: %s

Cdm.0950 500 Failed to connect to the database with theexisting authentication information.

Cdm.0960 500 serverlist must be specified.

Cdm.0961 500 Invalid serverlist format.

Cdm.0962 500 The host IP address must be specified.

Cdm.0963 500 The host port must be specified.

Cdm.0964 500 The database must be specified.

Cdm.1000 500 Hive table [%s] does not exist.

Cdm.1010 500 Invalid URI [%s]. The URI must be eithernull or a valid URI.

Cdm.1011 500 Failed to connect to Hive. Cause: %s

Cdm.1100 500 Table [%s] does not exist.

Cdm.1101 500 Failed to obtain the link. Cause: %s

Cdm.1102 500 Failed to create the table. Cause: %s

Cdm.1103 500 No rowkey is found.

Cloud Data MigrationAPI Reference 5 Common Parameters

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

138

Page 144: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Error Code Status Code Error Message

Cdm.1104 500 Failed to open the table. Cause: %s

Cdm.1105 500 Failed to initialize the job. Cause: %s

Cdm.1111 500 The table name is mandatory.

Cdm.1112 500 The import mode is mandatory.

Cdm.1113 500 Whether to clear data before import has notbeen specified.

Cdm.1114 500 The rowkey is empty. Set it in fieldmapping.

Cdm.1115 500 Columns is empty. Set it in field mapping.

Cdm.1116 500 Duplicate column names. Set it in fieldmapping.

Cdm.1117 500 An error occurred when checking whetherthe table exists. Cause: %s

Cdm.1118 500 Table %s does not contain the %s columnfamily.

Cdm.1119 500 The number of column families is %s andthe number of columns is %s.

Cdm.1120 500 The table contains data. Clear the table dataor set the configuration item to specifywhether to clear the table data before theimport.

Cdm.1121 500 Failed to close the link. Cause: %s

Cdm.1400 500 Failed to connect to the NAS server.

Cdm.1401 500 No permission to access the NAS server.

Cdm.1201 500 Failed to connect to the Redis server.Cause: %s

Cdm.1202 500 Failed to connect to the Redis cluster insingle-node mode.

Cdm.1203 500 Failed to extract data from the Redis server.Cause: %s

Cdm.1205 500 The prefix of the Redis value cannot beblank.

Cdm.1206 500 The storage type of the Redis value must bestring or hash.

Cdm.1207 500 When the value of the storage type isstring, valueSeparator must be specified.

Cloud Data MigrationAPI Reference 5 Common Parameters

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

139

Page 145: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Error Code Status Code Error Message

Cdm.1208 500 columnList of Redis must be specified.

Cdm.1209 500 keySeparator of Redis key cannot beempty.

Cdm.1210 500 primaryKeyList of Redis must bespecified.

Cdm.1211 500 primaryKeyList of Redis must exist incolumnList.

Cdm.1212 500 databaseType of Redis must be Originalor DCS.

Cdm.1213 500 serverlist of Redis must be specified.

Cdm.1301 500 Failed to connect to the MongoDB server.Cause: %s

Cdm.1302 500 Failed to extract data from the MongoDBserver. Cause: %s

Cdm.1304 500 The collection of MongoDB servers mustbe specified.

Cdm.1305 500 serverlist of MongoDB must be specified.

Cdm.1306 500 The database name of the MongoDBservice must be specified.

Cdm.1307 500 serverlist of MongoDB must be specified.

Cdm.1400 500 Failed to connect to the NAS server.

Cdm.1401 500 No permission to access the NAS server.

Cdm.1501 500 Failed to connect to the Elasticsearchserver. Cause: %s

Cdm.1502 500 Failed to write data to the Elasticsearchserver. Cause: %s

Cdm.1503 500 Failed to close the Elasticsearch link.Cause: %s

Cdm.1504 500 An error occurred when obtaining theElasticsearch index. Cause: %s

Cdm.1505 500 An error occurred when obtaining theElasticsearch type. Cause: %s

Cdm.1506 500 An error occurred when obtaining theElasticsearch field. Cause: %s

Cdm.1507 500 An error occurred when obtaining theElasticsearch sample data. Cause: %s

Cloud Data MigrationAPI Reference 5 Common Parameters

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

140

Page 146: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Error Code Status Code Error Message

Cdm.1508 500 The host name or IP address of theElasticsearch server must be specified.

Cdm.1509 500 The port of the Elasticsearch server must bespecified.

Cdm.1510 500 The Elasticsearch index must be specified.

Cdm.1511 500 The Elasticsearch type must be specified.

Cdm.1512 500 columnList of Elasticsearch must bespecified.

Cdm.1513 500 columnList must contain the field typedefinition.

Cdm.1514 500 columnList must contain primaryKey.

Cdm.1515 500 An error occurred when resolving the JSONcharacter string. Cause: %s

Cdm.1516 500 The column name %s is invalid.

Cdm.1517 500 An error occurred when obtaining thenumber of documents.

Cdm.1518 500 The partition fails to be created.

Cdm.1519 500 An error occurred during data extraction.

Cdm.1520 500 Failed to obtain the type. Cause: %s

Cdm.1601 500 Failed to connect to the server.

Cdm.1603 500 Failed to obtain the sample value of the %stopic.

Cdm.1604 500 No data exists in topic %s.

Cdm.1605 500 Invalid brokerList.

5.2 Status CodeTable 5-1 describes the status code.

Cloud Data MigrationAPI Reference 5 Common Parameters

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

141

Page 147: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Table 5-1 Status Code

StatusCode

Code Description

100 Continue The client continues sending the request.This interim response is used to inform the client thatthe initial part of the request has been received andhas not yet been rejected by the server.

101 Switching Protocols Switching protocols. The target protocol must bemore advanced than the source protocol.For example, the current HTTP protocol is switchedto a later version.

201 Created The request for creating a resource has been fulfilled.

202 Accepted The request has been accepted, but the processing hasnot been completed.

203 Non-AuthoritativeInformation

The server successfully processed the request, but isreturning information that may be from anothersource.

204 NoContent The server has successfully processed the request, buthas not returned any content.The status code is returned in response to an HTTPOPTIONS request.

205 Reset Content The server has fulfilled the request, but the requesteris required to reset the content.

206 Partial Content The server has successfully processed a part of theGET request.

300 Multiple Choices There are multiple options for the location of therequested resource. The response contains a list ofresource characteristics and addresses from which theuser or user agent (such as a browser) can choose themost appropriate one.

301 Moved Permanently The requested resource has been assigned a newpermanent URI, and the new URI is contained in theresponse.

302 Found The requested resource resides temporarily under adifferent URI.

303 See Other Retrieve a location.The response to the request can be found under adifferent URI and should be retrieved using a GET orPOST method.

304 Not Modified The requested resource has not been modified. Whenthe server returns this status code, it does not returnany resources.

Cloud Data MigrationAPI Reference 5 Common Parameters

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

142

Page 148: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

StatusCode

Code Description

305 Use Proxy The requested resource must be accessed through aproxy.

306 Unused The HTTP status code is no longer used.

400 BadRequest Invalid request.The client should not repeat the request withoutmodifications.

401 Unauthorized The status code is returned after the client providesthe authentication information, indicating that theauthentication information is incorrect or invalid.

402 Payment Required This status code is reserved for future use.

403 Forbidden The server understood the request, but is refusing tofulfill it.The client should not repeat the request withoutmodifications.

404 NotFound The requested resource cannot be found.The client should not repeat the request withoutmodifications.

405 MethodNotAllowed The method specified in the request is not supportedfor the requested resource.The client should not repeat the request withoutmodifications.

406 Not Acceptable The server cannot fulfill the request according to thecontent characteristics of the request.

407 Proxy AuthenticationRequired

This status code is similar to 401, but indicates thatthe client must first authenticate itself with the proxy.

408 Request Time-out The request timed out.The client may repeat the request withoutmodifications at any later time.

409 Conflict The request could not be completed due to a conflictwith the current state of the resource.This status code indicates that the resource that theclient attempts to create already exits, or the requestfails to be processed because of the update of theconflict request.

410 Gone The requested resource is no longer available.The status code indicates that the requested resourcehas been deleted permanently.

Cloud Data MigrationAPI Reference 5 Common Parameters

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

143

Page 149: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

StatusCode

Code Description

411 Length Required The server refuses to process the request without adefined Content-Length.

412 Precondition Failed The server does not meet one of the preconditionsthat the requester puts on the request.

413 Request Entity TooLarge

The request is larger than that a server is able toprocess. The server may close the connection toprevent the client from continuing the request. If theserver cannot process the request temporarily, theresponse will contain a Retry-After header field.

414 Request-URI TooLarge

The URI provided was too long for the server toprocess.

415 Unsupported MediaType

The server is unable to process the media format inthe request.

416 Requested range notsatisfiable

The requested range is invalid.

417 Expectation Failed The server fails to meet the requirements of theExpect request-header field.

422 UnprocessableEntity The request is well-formed but is unable to beprocessed due to semantic errors.

429 TooManyRequests The client has sent more requests than its rate limit isallowed within a given amount of time, or the serverhas received more requests than it is able to processwithin a given amount of time. In this case, it isadvisable for the client to re-initiate requests after thetime specified in the Retry-After header of theresponse expires.

500 InternalServerError The server is able to receive the request but it couldnot understand the request.

501 Not Implemented The server does not support the requested function.

502 Bad Gateway The server is acting as a gateway or proxy andreceives an invalid request from a remote server.

503 ServiceUnavailable The requested service is invalid.The client should not repeat the request withoutmodifications.

504 ServerTimeout The request cannot be fulfilled within a given time.This status code is returned to the client only whenthe Timeout parameter is specified in the request.

505 HTTP Version notsupported

The server does not support the HTTP protocolversion used in the request.

Cloud Data MigrationAPI Reference 5 Common Parameters

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

144

Page 150: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

A Change History

Release Date What's New

2018-08-03 This is the tenth official release.l Added the cdm.xlarge flavor to Creating a Cluster.l Updated the Start Job by Marker File and Use Quote

Character functions for file migration jobs in the followingsections:– From OBS/OSS– From FTP/SFTP/NAS– To OBS– To FTP/SFTP/NAS

l Added incremental migration in MySQL Binlog mode toFrom a Relational Database.

l Added the data type match function of HBase to To HBase/CloudTable.

2018-07-05 This is the ninth official release.l Added Link to Qiniu Cloud Object Storage (KODO).l Added the linkConfig.runMode parameter to the HDFS and

HBase links. The following sections are updated:– Link to HDFS– Link to HBase

l Updated From Elasticsearch/Cloud Search Service. WhenElasticsearch or Cloud Search Service is the migration source,data can be filtered using filter criteria.

l Updated To OBS. When exported to OBS, the data isfragmented by size.

Cloud Data MigrationAPI Reference A Change History

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

145

Page 151: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Release Date What's New

2018-06-02 This is the eighth official release.l Added From HTTP/HTTPS.l Added the CarbonData format of OBS to the following

sections:– From OBS/OSS– To OBS

l Added the retry function after configuration failure to JobParameter Description.

l Added the pipeline and authentication configurations of theElasticsearch data source to the following sections:– Link to Elasticsearch– To Elasticsearch/Cloud Search Service

2018-05-04 This is the seventh official release.l Added the following sections:

– Stopping a Cluster– Starting a Cluster– Link to CloudTable– To DDS

l Modified the following sections:– Creating a Link– Link to a Relational Database– Link to OBS– Link to OSS on Alibaba Cloud– Link to HDFS– Link to HBase– Source Job Parameters– Destination Job Parameters

l Changed Elasticsearch Service of HUAWEI CLOUD to CloudSearch Service.

l Changed Unlimited Query Service (UQuery) of HUAWEICLOUD to Data Lake Insight (DLI).

Cloud Data MigrationAPI Reference A Change History

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

146

Page 152: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Release Date What's New

2018-04-09 This is the sixth official release.l Added the following sections:

– Link to OSS on Alibaba Cloud– Link to DLI– To DLI

l Modified the following sections:– From OBS/OSS– From HBase/CloudTable– To OBS

2018-01-31 This is the fifth official release.l Deleted the type parameter in the configs field.l Updated the description of several link parameters and job

parameters.l Added a sample JSON file of each connector.l Added a sample JSON file of each type of data source.l Consolidated sections "Cluster Management", "Link

Management", and "Job Management" in section APIDescription.

l Added the GaussDB connector in section Link to aRelational Database.

l Added the following sections:– Before You Start– API Overview– API Use Case– Common Parameters

l Deleted the following sections:– Common Message Headers– AK/SK Authentication– Common Return Values– Common Exception Response

Cloud Data MigrationAPI Reference A Change History

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

147

Page 153: API Reference · 3.1 Overview of RESTful APIs HUAWEI CLOUD APIs comply with RESTful API design specifications. Representational State Transfer (REST) allocates Uniform Resource Identifiers

Release Date What's New

2018-01-11 This is the fourth official release.l Added the following sections:

– Link to HDFS– Link to CloudTable– Link to Kafka

l Deleted the description of the datastore parameter andmodified the description of the instances parameter in sectionCreating a Cluster.

l Deleted the description of the datastore parameter and addedthe description of the version parameter in section Queryingthe Cluster List.

l Deleted the description of the datastore parameter and addedthe description of the version parameter in section QueryingCluster Details.

l Modified the value of thefromJobConfig.disConsumerStrategy parameter in sectionFrom DIS.

l Modified the content in section Creating a Job.l Deleted the following sections:

– Querying Versions– Querying Flavors– Create MRS Data Source– Querying MRS Data Source– Deleting MRS Data Source– Public Configuration Data Structures

2017-11-30 This is the third official release.l Added the NAS, DIS, and Elasticsearch connectors.l Added the NAS and DIS data sources to the job parameter

description of source links.l Added the NAS and Elasticsearch data sources to the job

parameter description of destination links.

2017-10-31 This is the second official release.l Replaced projectId with project_id in this document.l Replaced datastoreId with datastore_id in this document.l Replaced clusterId with cluster_id in this document.l Added and modified the URI format and parameters in several

sections.l Added section "MongoDB Link".l Added section "Common Exception Response".

2017-9-30 This is the first official release.

Cloud Data MigrationAPI Reference A Change History

Issue 10 (2018-08-03) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

148