Skip to content

Commit

Permalink
Support catalog in MaxCompute Source
Browse files Browse the repository at this point in the history
  • Loading branch information
ruanwenjun committed Aug 11, 2023
1 parent 843d203 commit 930102e
Show file tree
Hide file tree
Showing 16 changed files with 631 additions and 328 deletions.
15 changes: 15 additions & 0 deletions docs/en/connector-v2/source/Maxcompute.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ Used to read data from Maxcompute.
| partition_spec | string | no | - |
| split_row | int | no | 10000 |
| common-options | string | no | |
| schema | config | no | |

### accessId [string]

Expand Down Expand Up @@ -59,6 +60,12 @@ Used to read data from Maxcompute.

Source plugin common parameters, please refer to [Source Common Options](common-options.md) for details.

### schema [config]

#### fields [Config]

The schema information of upstream data.

## Examples

```hocon
Expand All @@ -71,6 +78,13 @@ source {
table_name="<your table name>"
#partition_spec="<your partition spec>"
#split_row = 10000
schema {
fields {
name = string
age = int
gender = string
}
}
}
}
```
Expand All @@ -80,4 +94,5 @@ source {
### next version

- [Feature] Add Maxcompute Source Connector([3640](https://github.com/apache/seatunnel/pull/3640))
- [Feature] Support Schema in MaxCompute Source([3640](https://github.com/apache/seatunnel/pull/5283))

8 changes: 4 additions & 4 deletions docs/en/connector-v2/source/MyHours.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,13 +35,13 @@ Used to read data from My Hours.
In order to use the My Hours connector, the following dependencies are required.
They can be downloaded via install-plugin.sh or from the Maven central repository.

| Datasource | Supported Versions | Dependency |
|------------|--------------------|------------------------------------------------------------------------------------------------|
| My Hours | universal | [Download](https://mvnrepository.com/artifact/org.apache.seatunnel/seatunnel-connectors-v2) |
| Datasource | Supported Versions | Dependency |
|------------|--------------------|---------------------------------------------------------------------------------------------|
| My Hours | universal | [Download](https://mvnrepository.com/artifact/org.apache.seatunnel/seatunnel-connectors-v2) |

## Source Options

| Name | Type | Required | Default | Description |
| Name | Type | Required | Default | Description |
|-----------------------------|---------|----------|---------|--------------------------------------------------------------------------------------------------------------------------------------|
| url | String | Yes | - | Http request url. |
| email | String | Yes | - | My hours login email address. |
Expand Down

This file was deleted.

This file was deleted.

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package com.aliyun.odps.type;

import com.aliyun.odps.OdpsType;

import java.util.List;

/**
* This class is used to create TypeInfo objects, since TypeInfo constructors are only visitable
* under {@link com.aliyun.odps.type}.
*/
public class TypeInfoFactory {

public static DecimalTypeInfo newDecimalTypeInfo(int precision, int scale) {
return new DecimalTypeInfo(precision, scale);
}

public static SimpleArrayTypeInfo newSimpleArrayTypeInfo(TypeInfo valueTypeInfo) {
return new SimpleArrayTypeInfo(valueTypeInfo);
}

public static SimpleMapTypeInfo newSimpleMapTypeInfo(
TypeInfo keyTypeInfo, TypeInfo valueTypeInfo) {
return new SimpleMapTypeInfo(keyTypeInfo, valueTypeInfo);
}

public static SimplePrimitiveTypeInfo newSimplePrimitiveTypeInfo(OdpsType odpsType) {
return new SimplePrimitiveTypeInfo(odpsType);
}

public static SimpleStructTypeInfo newSimpleStructTypeInfo(
List<String> names, List<TypeInfo> typeInfos) {
return new SimpleStructTypeInfo(names, typeInfos);
}
}
Loading

0 comments on commit 930102e

Please sign in to comment.