Skip to content

Releases: apache/seatunnel

2.3.5 Release

21 Apr 03:05
@liugddx liugddx
2.3.5
fd78af9
Compare
Choose a tag to compare
2.3.5 Release Latest
Latest

2.3.5 Release Note

Bug fix

Core

  • [fix] fix get seatunnel row size npe issue (#6681)
  • [Hotfix] Fix DEFAULT TABLE problem (#6352)
  • [Chore] Fix file spell errors (#6606)
  • [BugFix][Spark-translation] map type cast error (#6552)
  • [Hotfix] Fix spark example (#6486)
  • [Hotfix] Fix compile error (#6463)

Transformer

  • [Fix][SQLTransform] fix the scale loss for the sql transform (#6553)
  • [Bug] Fix minus constant error in SQLTransform (#6533)

Connectors

  • [Fix][Kafka-Sink] fix kafka sink factory option rule (#6657)
  • [Hotfix] fix http source can not read yyyy-MM-dd HH:mm:ss format bug & Improve DateTime Utils (#6601)
  • [Bug] Fix OrcWriteStrategy/ParquetWriteStrategy doesn't login with kerberos (#6472)
  • [Fix][Doc] Fix FTP sink config key username to user (#6627)
  • [E2E] Fix AmazondynamodbIT unstable (#6640)
  • [Fix][Connector-V2] Fix add hive partition error when partition already existed (#6577)
  • [Fix][Connector-V2] Fixed doris/starrocks create table sql parse error (#6580)
  • [Fix][Connector-V2] Fix doris sink can not be closed when stream load not read any data (#6570)
  • [Fix][Connector-V2] Fix connector support SPI but without no args constructor (#6551)
  • [Fix][Connector-V2] Fix doris source select fields loss primary key information (#6339)
  • [Fix][FakeSource] fix random from template not include the latest value issue (#6438)
  • [Fix][Connector-V2] Fix mongodb cdc start up mode option values not right (#6338)
  • [BugFix][Connector-file-sftp] Fix SFTPInputStream.close does not correctly trigger the closing of the file stream (#6323) (#6329)
  • [Fix] Fix doris stream load failed not reported error (#6315)
  • [fix][connector-rocketmq]Fix a NPE problem when checkpoint.interval is set too small(#6624)
  • [Bugfix][TDengine] Fix the issue of losing the driver due to multiple calls to the submit job REST API #6581 (#6596)
  • [Fix][StarRocks] Fix NPE when upstream catalogtable table path only have table name part (#6540)

Formats

  • [Bug] [formats] Fix fail to parse line when content contains the file delimiter (#6589)

Zeta(ST-Engine)

  • [Hotfix] fix http source can not read yyyy-MM-dd HH:mm:ss format bug & Improve DateTime Utils (#6601)
  • [Fix][Zeta] Fix the thread stuck problem caused by savepoint checking mechanism (#6568)
  • [Fix][Zeta] improve the local mode hazelcast connection (#6521)
  • [Fix][Zeta] Fix thread classloader be set to null when use cache mode (#6509)
  • [Bug] [zeta] Fix null pointer exception when submitting jobs (#6492)
  • [bugfix] [Zeta] Fix the problem of class loader not releasing when using REST API to submit jobs
  • [BUG][Zeta]job name Display error #6470
  • [Hotfix][Zeta] Fix job deadlock when schema change (#6389)

E2E

  • [E2E] Enable StarRocksCDCSinkIT (#6626)

Improve

  • [Doc][Improve] Add Support Chinese for seatunnel-engine (#6656)
  • [Doc][Improve]Add Support Chinese for start-v2/locally/quick-start-flink.md and start-v2/locally/quick-start-spark.md (#6412)
  • [Improve] add icon for idea (#6394)
  • [Improve] Add deprecated annotation for ReadonlyConfig::toConfig (#6353)

Core

  • [Improve][RestAPI] always return jobId when call getJobInfoById API (#6422)
  • [Improve][RestAPI] return finished job info when job is finished (#6576)
  • [Improve] Improve MultiTableSinkWriter prepare commit performance (#6495)
  • [Improve] Add SaveMode log of process detail (#6375)
  • [Improve][API] Unify type system api(data & type) (#5872)

Formats

  • [Improve] Improve read with parquet type convert error (#6683)

Connector-V2

  • [Improve][Connector-V2]Support multi-table sink feature for redis (#6314)
  • [Improve][Connector-V2] oracle cdc e2e optimization (#6232)
  • [Improve][Connector-V2]Support multi-table sink feature for httpsink (#6316)
  • [Improve][Connector-V2] Support INFINI Easysearch (#5933)
  • [Improve][Connector-V2] Support hadoop ha and kerberos for paimon sink (#6585)
  • [Improve][CDC-Connector]Fix CDC option rule. (#6454)
  • [Improve][CDC] Optimize memory allocation for snapshot split reading (#6281)
  • [Improve][Connector-V2] Support TableSourceFactory on StarRocks (#6498)
  • [Improve][Jdbc] Using varchar2 datatype store string in oracle (#6392)
  • [Improve] StarRocksSourceReader use the existing client (#6480)
  • [Improve][JDBC] Optimized code style for getting jdbc field types (#6583)
  • [Improve][Connector-V2] Add ElasticSearch type converter (#6546)
  • [Improve][Connector-V2] Support read orc with schema config to cast type (#6531)
  • [Improve][Jdbc] Support custom case-sensitive config for dameng (#6510)
  • [Improve][Jdbc] Increase tyepe converter when auto creating tables (#6617)
  • [Improve][CDC] Optimize split state memory allocation in increment phase (#6554)
  • [Improve][CDC] Improve read performance when record not contains schema field (#6571)
  • [Improve][Jdbc] Add quote identifier for sql (#6669)
  • [Improve] Add disable 2pc in SelectDB cloud sink (#6266)
  • [Doc][Improve] Add Support Kerberos Auth For Kafka Connector #6653

CI

  • [CI] Fix error repository name in ci config files (#4795)

Zeta(ST-Engine)

  • [Improve][Zeta] Add classloader cache mode to fix metaspace leak (#6355)
  • [Improve][Test] Fix test unstable on ResourceManger and EventReport module (#6620)
  • [Improve][Test] Run all test when code merged into dev branch (#6609)
  • [Improve][Test] Make classloader cache testing more stable (#6597)
  • [Improve][Zeta][storage] update hdfs configuration, support more parameters (#6547)
  • [Improve][Zeta]Optimize the logic of RestHttpGetCommandProcessor#getSeaTunnelServer() (#6666)

Transformer

  • [Improve][Transform] Sql transform support inner strucy query (#6484)
  • [Improve][Transform] Remove Fallback during parsing Transform process (#6644)
  • [Improve][Transform] Remove can't find field exception (#6691)

Feature

Core

  • [Feature][Tool] Add connector check script for issue 6199 (#6635)
  • [Feature][Core] Support listening for message delayed events in cdc source (#6634)
  • [Feature][Core] Support event listener for job (#6419)

Connector-V2

  • [Feature][connector-v2] add xugudb connector (#6561)
  • [Feature][Connector-V2] Support multi-table sink feature for paimon #5652 (#6449)
  • [Feature][Connectors-V2][File]support assign encoding for file source/sink (#6489)
  • [Feature][Connector]update pgsql-cdc publication for add table (#6309)
  • [Feature][Paimon] Support specify paimon table write properties, partition keys and primary keys (#6535)
  • [Feature][Feature] Support nanosecond in Doris DateTimeV2 type (#6358)
  • [Feature][Feature] Support nanosecond in SelectDB DateTimeV2 type (#6332)
  • [Feature][Feature] Supports iceberg sink #6198 (#6265)

Zeta(ST-Engine)

  • [Zeta] Support config job retry times in job config (#6690)

Docs

  • [Docs] fix kafka format typo (#6633)
  • [Fix][Doc] Fixed links in some documents (#6673)
  • [Fix][Doc] Fix some spell errors (#6628)
  • [Fix][Doc] Fixed typography error in starrocks sink document (#6579)
  • [Hotfix][Doc][Chinese] Fix invalid link about configure logging related parameters (#6442)
  • [Fix][Doc]Seatunnel Engine/checkpoint-storage.md doc error(#6369)
Assets 2

2.3.4 Release

20 Feb 03:01
@EricJoy2048 EricJoy2048
2.3.4
b96bf8a
Compare
Choose a tag to compare
2.3.4 Release

2.3.4 Release Note

Bug Fix

Core

  • [Core] [API] Fixed generic class loss for lists (#4421)
  • [Starter] Fix the problem of "," being divided in [] (#5401)
  • [Core] [API] Fix ReadonlyConfig lose key error (#5565)
  • [Core] [API] Fix get bytes from LinkHashMap (#5622)
  • [Core] [API] Fix log error when multi-table sink close (#5683)
  • [Core] [API] Fix MultiTableSink return committer but sink do not support (#5710)
  • [Core] [API] Fix the error msg when parse schema with unsupported type (#5790)
  • [Core] [API] Fix flaky test OptionUtilTest.test (#5894)
  • [Core] [API] Fix SaveModeHandler not be closed (#5843)
  • [Core] [API] Fix MultiTableSinkWriter thread index always 1 (#5832)
  • [Core] [API] Fix SeaTunnelRow::getBytesSize not support map interface (#5990)
  • [Core] [Common] Fix FileUtils::createNewFile not create new file (#5943)
  • [Core] [API] Fix Debezium format cannot parse date/time/timestamp (#5887)
  • [Starter] When inside double quotes,',' are treated as normal characters instead of delimiters (#6042
  • [Core] [Common] Replace CommonErrorCodeDeprecated.JSON_OPERATION_FAILED (#5978)
  • [Core] [API] Fix Object.class option value can not return normal value (#6247)

Transformer-V2

  • [All] Fix PrimaryKey in transform (#5704)
  • [All] Fix cast to timestamp, date, time bug (#5812)

Formats

  • [Text] Allow the entry in the map to be null and allow the key in the entry to be null (#5277)

Connector-V2

  • [Connector-V2] [Clickhouse] Fix clickhouse old version compatibility (#5326)
  • [Connector-V2] [Clickhouse] Fix http header cover (#5446)
  • [Connector-V2] [StarRocks] Fix starrocks template sql parser (#5332)
  • [Connector-V2] [Hive] Fix the bug that hive-site.xml can not be injected in HiveConf (#5261)
  • [Connector-V2] [Clickhouse] Fix clickhouse sink flush bug (#5448)
  • [Connector-V2] [Hive] Fix An error occurred reading an empty directory (#5427)
  • [Connector-V2] [Hive] Fix An error occurred reading an empty directory (#5427)" (#5487)
  • [Connector-V2] [Oss jindo] Fix the problem of jindo driver download failure. (#5511)
  • [Connector-V2] [Oss jindo] Remove useless code (#5540)
  • [Connector-V2] [File] Fix WriteStrategy parallel writing thread unsafe issue (#5546)
  • [Connector-V2] [CDC] Fix the cdc bug about NPE when the original table deletes a field (#5579)
  • [Connector-V2] [Jdbc] oracle catalog create table repeat and oracle pg null point (#5517)
  • [Connector-V2] [CDC] Fix thread-unsafe collection container in cdc enumerator (#5614)
  • [Connector-V2] [Mongodb] Fixed unsupported exception caused by bsonNull (#5659)
  • [Connector-V2] [File] Fix file sink isPartitionFieldWriteInFile occurred exception when no columns are given (#5508)
  • [Connector-V2] [Doris] Fix RestService report NullPointerException (#5319)
  • [Connector-V2] [MaxCompute] Fix MaxCompute use not exist SCHEMA option (#5708)
  • [Connector-V2] [Doris] Using try-with-resources to simplify the code. (#4995)
  • [Connector-V2] [Clickhouse] Fixed an out-of-order BUG with output data fields of clickhouse-sink (#5346)
  • [Connector-V2] [Jdbc] support postgresql xml type (#5724)
  • [Connector-V2] [Jdbc] Nullable Column source have null data could be unexpected results. (#5560)
  • [Connector-V2] [Iceberg] Iceberg source lost data with parallelism option (#5732)
  • [Connector-V2] [Jdbc] Fix PG will not create index when using auto create table #5721
  • [Connector-V2] [Jdbc] Fix database identifier (#5756)
  • [Connector-V2] [CDC] Fix MultiTableSink restore failed when add new table (#5746)
  • [Connector-V2] [Postgres CDC] Fix Postgres create table test case failed (#5778)
  • [Connector-V2] [CDC] Clean unused code (#5785)
  • [Connector-V2] [CDC] Fix state recovery error when switching a single table to multiple tables (#5784)
  • [Connector-V2] [ElasticSearch] Fixed conversion exception of elasticsearch array format (#5825)
  • [Connector-V2] [Jdbc] Fix read from Oracle Date type value lose time (#5814)
  • [Connector-V2] [Pulsar] Fix: update IDENTIFIER = Pulsar for pulsar-datasource on project:seatunnel-web (#5852)
  • [Connector-V2] [Jdbc] Fix Hive-Jdbc use krb5 overwrite kerberosKeytabPath (#5891)
  • [Connector-V2] [InfluxDB] Resolve invalid SQL in initColumnsIndex method caused by direct QUERY_LIMIT appendage with 'tz' function. (#4829)
  • [Connector-V2] [Jdbc] Fix cdc updates were not filtering same primary key (#5923)
  • [Connector-V2] [File] Parquet reader parsing array type exception. (#4457)
  • [Connector-V2] [Http] Fix bug http config no schema option and improve e2e test add case (#5939)
  • [Connector-V2] [Doris] Fix DorisCatalog not implement name method (#5988)
  • [Connector-V2] [TDengine] Fix the degree of multiple parallelism affects driver loading (#6020)
  • [Connector-V2] [Jdbc] Fix jdbc setFetchSize error (#6005)
  • [Connector-V2] [CDC] Fix CDC job cannot consume incremental data After restore run (#625) (#6094)
  • [Connector-V2] [File] Fix the Issue of Abnormal Data Reading from Excel Files (#5932)
  • [Connector-V2] [CDC] Fix NPE caused by adding a table for restore job (#6145)
  • [Connector-V2] [Jdbc] Fix dameng catalog query table sql (#6141)
  • [Connector-V2] [Jdbc] update pgsql catalog for save mode (#6080)
  • [Connector-V2] [Jdbc] Fix Spliter Error in Case of Extensive Duplicate Data (#6026)
  • [Connector-V2] [CDC] Fix added columns cannot be parsed after job restore (#6118)
  • [Connector-V2] [CDC] Fix negative values in CDCRecordEmitDelay metric (#6259)
  • [Connector-V2] [Oracle CDC] Fix invalid split key when no primary key (#6251)

Zeta(ST-Engine)

  • [Zeta] Fix NotifyTaskRestoreOperation npe (#5362)
  • [Zeta] Fix Zeta will close task twice error (#5422)
  • [Zeta] Disable CheckpointTimeOutTest (#5438)
  • [Zeta] Fix CDC task restore throw NPE (#5507)
  • [Zeta] Multiple sink actions of the same type have the same name (#5499)
  • [Zeta] Checkpoint exception status messages exclude state data (#5547)
  • [Zeta] Fix memory leak issues related to checkpoints (#5539)
  • [Zeta] Fix the checkpoint be blocked with long time (#5695)
  • [Zeta] Fix the problem of unstable job status (#5450)
  • [Zeta] Fix submit job api (#5702)
  • [Zeta] Set default DeployMode to DeployMode.CLIENT (#5783)
  • [Zeta] rest api submit a job with chinese name return Garbled code name (#5870)
  • [Zeta] Fix CheckpointCoordinator report NPE when ack not existed pending checkpoint (#5909)
  • [Zeta] Fix submit job has the same job name error. (#6041)
  • [Zeta] Fixed the problem that the return list is empty due to no status parameters (#6040)
  • [Zeta] Fix zeta scheduler bug (#6050)
  • [Zeta] Fix job can not restore when last checkpoint failed (#6193)
  • [Zeta] [Rest-API] Submit or stop job from an inactive master node (#6217)

E2E

  • [E2E] [Common] Update test container version of seatunnel engine (#5323)
  • [E2E] [Jdbc] Fix not remove docker image after test finish on jdbc suite (#5586)
  • [E2E] [ClickHouse] Enhance ClickHouse E2E testing to trigger multiple checkpoints (#5476)
  • [E2E] Fix not remove docker image after test finish on jdbc suite (#5586)
  • [E2E] Fix ConnectorPackageServiceContainer miss implement getSavePointCommand/getRestoreCommand (#5780)
  • [E2E] Fix build failed cause by JdbcHiveIT and SparkSinkTest (#5798)
  • [E2E] Fix submit job case error (#6059)
  • [E2E] Fixed action related err (#6264)
  • [E2E] Lock mysql container version to 8.0 (#6263)

CI

  • [CI] Fix jindo oss connector name (#5385)
  • [Build] Fix error msg when fork repository not up to date. (#5497)
  • [CI] Fix file change not be check when CI run in fork repository (#5515)
  • [CI] remove jindo dependencies (#5528)
  • [CI] Fix phoenix ci error (#5530)
  • [Build] Update build version to 2.3.4-SNAPSHOT (#5619)
  • [Build] Ensure install-plugin.sh compatibility with sh on Debian #5630 (#5631)
  • [CI] [Chore] Remove useless sonar check script (#5665
  • [Chore] Remove DISCLAIMER file (#5673)
  • [CI] Fix CI unstable problem (#5896)
  • [Build] Fix empty line in config/plugin_config causes BUILD FAILURE (#5921)
  • [CI] Fix CI not run Kudu/AmazonSQS IT when not change api (#5955)
  • [CI] Split doris e2e into separate modules (#5999)
  • [CI] Fix Dead Links checker failure (#6016)
  • [CI] Fix e2e error (#6018)
  • [Build] Updated pom.xml (#6113)
  • [Build] Solve the problem of example running failure (#6173)
  • [Build] Fix build error (#6196)
  • [CI] Fix engine client not close  (#6241)

Examples

  • [Examples] modify the transform URL link (#5298)

Improve

  • [Improve][CheckStyle] Remove useless 'SuppressWarnings' annotation of checkstyle. (#5260)
  • [Improve][CheckStyle] Adjust the phase of spotless plugin for release plugin. (#5607)

Core

  • [Core] [API] Remove CatalogTable field in CatalogTableUtil (#5521)
  • [Core] [API] Move get schema logic from Config to ReadonlyConfig (#5534)
  • [Starter] Throw IllegalArgumentException when find multiple connector jar for one pluginIdentifier (#5551)
  • [Core] [API] Refactor CatalogTable and add SeaTunnelSource::getProducedCatalogTables (#5562)
  • [Core] [API] Support config column/primaryKey/constraintKey in schema (#5564)
  • [Core] [API] Remove useless ReadonlyConfig flatten feature (#5612)
  • [Core] [Flink & Spark] Refactor Spark/Flink execution processor (#5595)
  • [Core] [API] Mark SeaTunnelPluginLifeCycle as deprecated (#5625
  • [Core] [API] Support config tableIdentifier for schema (#5628)
  • [Core] [Pom] Add junit4 to the root pom (#5611)
  • [Core] [API] Remove catalog tag for config file (#5645)
  • [Core] [API] Remove useless transform code come from setTypeInfo (#5647)
  • [Core] [API] Make sure CatalogTable options and partitionKeys are mutable (#5681)
  • [Core] [API] Add default implement for SeaTunnelSource::getProducedType (#5670)
  • [Core] [API] Add default implement for SeaTunnelSink::setTypeInfo (#5682)
  • [Core] [API] Add warning for use fall back keys (#5753)
  • [Core] [API] Adjust the sleep mode of flink and spark e...
Read more
Assets 2

2.3.3 Release

22 Aug 08:21
@ic4y ic4y
2.3.3
6977e9f
Compare
Choose a tag to compare
2.3.3 Release

2.3.3 Release Note

Bug fix

Core

  • [Translation] [Spark] Fix SeaTunnelRowConvertor fail to convert when schema contains row type (#5170)

Connector

  • [Connector] [Jdbc] Fix the problem of JdbcOutputFormat database connection leak (#4802)
  • [Connector-V2] [AmazonDynamoDB] Fix the problem that all table data cannot be obtained (#5146)
  • [Connector-V2] [CDC-base] Fix cdc connection does not close (#4922)
  • [Connector-V2] [Clickhouse] Fix clickhouse source connector read Nullable() type is not null,example:Nullable(Float64) while value is null the result is 0.0 (#5080)
  • [Connector-V2] [ClickhouseFile] Fix ClickhouseFile write file failed when field value is null (#4937)
  • [Connector-V2] [Doris] Add streamload Content-type for doris URLdecode error (#4880)
  • [Connector-V2] [E2E] Fix maven scope (#4901)
  • [Connector-V2] [File-Base] Fix Hadoop Kerberos authentication related issues (#5171)
  • [Connector-V2] [Flink-runtime] Flink register table Environment The running mode is set tojob.mode (#4826)
  • [Connector-V2] [HbaseSink] Fix default timestamp (#4958)
  • [Connector-V2] [Hive] Support user-defined hive-site.xml (#4965)
  • [Connector-V2] [Jdbc] Support postgresql json type (#5194)
  • [Connector-V2] [Jdbc] Fix XA Transaction bug (#5020)
  • [Connector-V2] [Jdbc] Modify error comment in JdbcSourceConfigFactory (#5088)
  • [Connector-V2] [Jdbc-Oracle] Fix the Oracle number type mapping problem (#5209)
  • [Connector-V2] [Jdbc-Postgres] PostgreSQL versions below 9.5 are compatible use cdc sync problem (#5120)
  • [Connector-V2] [JindoOssFile] Fix plugin-mapping.properties (#5215)
  • [Connector-V2] [LocalFileSink] Fix LocalFile Sink file_format_type (#5118)
  • [Connector-V2] [Mongodb] Fix document error content and remove redundant code (#4982)
  • [Connector-V2] [Mongodb] Compatible with historical parameters (#4997)
  • [Connector-V2] [MongoDB-CDC] Incremental data kind error in snapshot phase (#5184)
  • [Connector-V2] [MongoDB-CDC] Solve startup resume token is negative (#5143)
  • [Connector-V2] [MongoDB-CDC] Refine data format to adapt to universal logic (#5162)
  • [Connector-V2] [MongoDB-CDC] Fix array index anomalies caused by #5057 (#5195)
  • [Connector-V2] [MySql-CDC] Fix jdbc connection leak for mysql (#5037)
  • [Connector-V2] [MySql-CDC] Fix mysql bit column to java byte (#4817)
  • [Connector-V2] [MySql-CDC] Fix listener not released when BinlogClient reuse (#5011)
  • [Connector-V2] [MySql-CDC] ServerId from int to long (#5033) (#5035)
  • [Connector-V2] [Paimon] Bump paimon-bundle version to 0.4.0-incubating (#5219)
  • [Connector-V2] [Redis] Fix redis nodes format error (#4981)
  • [Connector-V2] [StarRocks] Fix StarRocksJsonSerializer will transform array/map/row to string (#5281)
  • [Connector-V2] [StarRocks] Fix code style (#4966)

Zeta(ST-Engine)

  • [Zeta] Resolved the issue causing checkpoints to halt on tolerable-failure=0 (#5263)
  • [Zeta] Fix the issue of two identical IDs appearing when executing seatunnel.sh -l as the job resumes (#5191)
  • [Zeta] Fix SinkFlowLifeCycle without init lastCommitInfo (#5152)
  • [Zeta] Fix MultipleTableJobConfigParser ignore env option (#5067)
  • [Zeta] Fix checkpoint error report without msg (#5137)
  • [Zeta] [REST-API] Fix class not found exception (#5063)
  • [Zeta] [REST-API] Fix browser long type intercept (#5267)
  • [Zeta] Fix task state memory leak (#5139)
  • [Zeta] Fix when execute seatunnel.sh, console may print error (#4984)
  • [Zeta] Fixed the issue of duplicated metrics caused by job fault tolerance or restore (#5214)
  • [Zeta] Avoid Redundant Job Submissions by Checking Job Status (#5229)
  • [Zeta] Fix the serialization issue of GetMetricsOperation during multi-node operation (#5206)
  • [Zeta] Fix after the savepoint job is restored, the checkpoint file cannot be generated #4985 (#5051)
  • [Zeta] Fixed multi-table job data loss and latency issues (#149) (#5031)
  • [Zeta] Fix the HdfsStorage can not delete checkpoint file #5046 (#5054)
  • [Zeta] Fix if the initialization task throws an error and the task can not return to master (#4888)

TEST

  • [Test] Split updated modules integration test for part 5 (#5208)
  • [Test] Add code style check when docs changed (#5183)
  • [Test] Turn on the testSavepoint test (#5199)
  • [Test] Fix the CI Job name error (#5032)
  • [Test] [E2E] Fix testGetErrorInfo case error (#5282)
  • [Test] [Kafka] Kafka e2e error (#5200)
  • [Test] [MongoDB-CDC] Fix mongodb cdc e2e instability (#5128)
  • [Test] Declare files that will always have UNIX line endings on checkout (#5221)
  • [Test] Fixed a minor bug (#5274)
  • [Test] Fixed IMap file storage e2e bug (#5237)
  • [Test] Split updated modules integration test for part 4 (#5028)
  • [Test] Fix zeta ci error (#5254)

Others

  • Update the project description (#4967)
  • Update version to 2.3.3-SNAPSHOT (#5043)
  • Fix com.google.common.base.Preconditions to seatunnel shade one (#5284)
  • Fix style error (#5280)
  • Fix code style (#5092)

Improve

Core

  • [API & Zeta] Using connector custom serializer encode/decode states (#5238)
  • [API]Improve savemode api (#4767)

Connector

  • [CDC Base] Add some split parameters to the optionRule (#5161)
  • [CDC Base] Support exactly-once of cdc and fix the BinlogOffset comparing bug (#5057)
  • [CDC Base] Add a fast sampling method that supports character types (#5179)
  • [CDC Base] Change driver scope to provider (#5002)
  • [CDC Base] Remove driver for cdc connector (#4952)
  • [Connector-V2] [File] Optimize files commit order (#5045)
  • [Connector-V2] [HiveSink] Remove drop partition when abort (#4940)
  • [Connector-V2] [Jdbc] Remove scheduler in JDBC sink #4736 (#5168)
  • [Connector-V2] [Jdbc] Check url not null throw friendly message (#5097)
  • [Connector-V2] [Jdbc] Support for Decimal types as splict keys (#4634)
  • [Connector-V2] [Mongodb] Sink support transaction update/writing (#5034)
  • [Connector-V2] [Mongodb] Optimize reading logic (#5001)
  • [Connector-V2] [Neo4j] Supports neo4j sink batch write and update docs (#4841)
  • [Connector-V2] [OSS-Jindo] Optimize jindo oss connector (#4964)
  • [Connector-V2] [Tablestore sink] Remove scheduler in Tablestore sink (#5272)

Transform

  • [Transform-V2] [SQL] Support use catalogTableName as SQL expression (#5273)

CI

  • [CI/CD] Remove 'paths-ignore', enable the code style check for markdown files (#5286)

Zeta(ST-Engine)

  • [Zeta] Upgrade the key log output level (#4993)
  • [Zeta] Don't trigger handleSaveMode when restore (#5192)
  • [Zeta] Move checkpoint notify complete in checkpoint stage (#5185)
  • [Zeta] Add sleep for Task to reduce CPU cost (#5117)
  • [Zeta] Add an interface for batch retrieval of JobMetrics (#4576)
  • [Zeta] Move imap storage file dependency packages to submodules (#5218)
  • [Zeta] Improve CheckpointCoordinator log error when report error from task (#178) (#5134)
  • [Zeta] Improve CheckpointCoordinator notify complete when restore (#5136)

Feature

Connector

  • [Connector-V2] [CDC-base] Support tables without primary keys (with unique keys) (#163) (#5150)
  • [Connector-V2] [CDC-base] Support disable/enable exactly once for INITIAL (#4921)
  • [Connector-V2] [CDC-base] Change the time zone to the default time zone (#5030)
  • [Connector-V2] [CDC-base] Support string type shard fields (#5147)
  • [Connector-V2] [Clickhouse] Add clickhouse connector time zone key,default system time zone (#5078)
  • [Connector-V2] [Clickhouse] Clickhouse writes with checkpoints (#4999)
  • [Connector-V2] [Doris] Add doris factory type convertor (#5061)
  • [Connector-V2] [File] Add config of 'file_filter_pattern', which used for filtering files (#5153)
  • [Connector-V2] [File] Add cos source&sink (#4979)
  • [Connector-V2] [Hive] Hive add option because web need (#5154)
  • [Connector-V2] [Jdbc] Add JDBC Catalog auto create table (#4917)
  • [Connector-V2] [Jdbc] Add oceanbase dialect factory (#4989)
  • [Connector-V2] [Jdbc] JDBC source support string type as partition key (#4947)
  • [Connector-V2] [Jdbc-DM] Add DM upsert support (#5073)
  • [Connector-V2] [Kafka] Support read debezium format message from kafka (#5066)
  • [Connector-V2] [Kafka] Connector-kafka source support data conversion extracted by kafka connect source (#4516)
  • [Connector-V2] [Mongodb] Mongodb support cdc sink (#4833)
  • [Connector-V2] [Mongodb-CDC] Support source mongodb cdc (#4923)
  • [Connector-V2] [Mysql-CDC] Conversion of tinyint(1) to bool is supported (#5105)
  • [Connector-V2] [Redis] Support redis to set expiration time (#4975)

Zeta(ST-Engine)

  • [Zeta] [CDC-base] Support schema evolution framework(DDL) (#5125)
  • [Zeta] [REST-API] Add REST API To Submit Job (#5107)
  • [Zeta] Checkpoint support hdfs ha mode (#4942)
  • [Zeta] The expiration time of a historical Job can be config (#5180)
  • [Zeta] Support history service record job execute error (#5114)
  • [Zeta] Configuration files support user variable replacement (#4969)

Transform

  • [Transform-V2] [SQL] Support 'select *' and 'like' clause for SQL Transform plugin (#4991)

CI

  • [CI] Update action config to support run CI on fork repo (#5065)
  • [E2E] [FtpFile] Add ftp file e2e test case (#4647)

Docs

  • [Connector-V2][Clickhouse] Reconstruct the clickhouse connector doc (#5085)
  • [Connector-V2][Doris] Reconstruct the Doris connector document (#4903)
  • [Connector-V2][HDFS] Refactor connector-v2 docs using unified format HDFS (#4871)
  • [Connector-V2][Http] Reconstruct the Http connector document (#4962)
  • [Connector-V2][Hudi] Reconstruct the Hudi connector document (#4905)
  • [Connector-V2][Jdbc] Change the line boundary store value type to BigDecimal (#4900)
  • [Connector-V2][Jdbc-DB2、Jdbc-Vertica] Improve DB2 Source Vertica Source & DB2 Sink Vertica Sink document (#5102)
  • [Connector-V2][Jdbc-Oracle] Add JDBC Oracle Documentation (#5239)
  • [Connector-V2][Kafka] Reconstruct the kafka connector document (#4778)
  • [Connector-V2][MyHours] Reconstruct the MyH...
Read more
Assets 2

2.3.2 Release

16 Jun 08:05
@Hisoka-X Hisoka-X
2.3.2
db79592
Compare
Choose a tag to compare
2.3.2 Release

2.3.2 Release Note

Bug fix

Core

  • [Core] [API] Fixed generic class loss for lists (#4421)
  • [Core] [API] Fix parse nested row data type key changed upper (#4459)

Connector-V2

  • [Json-format] [Canal-Json] Fix json deserialize NPE (#4195)
  • [Connector-V2] [Jdbc] Field aliases are not supported in the query of jdbc source. (#4210)
  • [Connector-V2] [Jdbc] Fix connection failure caused by connection timeout. (#4322)
  • [Connector-V2] [Jdbc] Set default value to false of JdbcOption: generate_sink_sql (#4471)
  • [Connector-V2] [JDBC] Fix TiDBCatalog without open (#4718)
  • [Connector-V2] [Jdbc] Fix XA DataSource crash(Oracle/Dameng/SqlServer) (#4866)
  • [Connector-V2] [Pulsar] Fix the bug that can't consume messages all the time. (#4125)
  • [Connector-V2] [Eleasticsearch] Document description error (#4390)
  • [Connector-V2] [Eleasticsearch] Source deserializer error and inappropriate (#4233)
  • [Connector-V2] [Kafka] Fix KafkaProducer resources have never been released. (#4302)
  • [Connector-V2] [Kafka] Fix the permission problem caused by client.id. (#4246)
  • [Connector-V2] [Kafka] Fix KafkaConsumerThread exit caused by commit offset error. (#4379)
  • [Connector-V2] [kafka] Fix the problem that the partition information can not be obtained when kafka is restored (#4764)
  • [Connector-V2] [SFTP] Fix incorrect exception handling logic (#4720)
  • [Connector-V2] [File] Fix read temp file (#4876)
  • [Connector-V2] [CDC Base] Solving the ConcurrentModificationException caused by snapshotState being modified concurrently. (#4877)
  • [Connector-V2] [Doris] update last checkpoint id when doing snapshot (#4881)
  • [Connector-v2] [kafka] Fix the short interval of pull data settings and revise the format (#4875)
  • [Connector-v2] [RabbitMQ] Fix reduplicate ack msg bug and code style (#4842)
  • [Connector-V2] [Jdbc] Fix the error of extracting primary key column in sink (#4815)
  • [Connector-V2] [Jdbc] Fix reconnect throw close statement exception (#4801)
  • [Connector-V2] [Jdbc] Fix sqlserver system table case sensitivity (#4806)
  • [Connector-v2] [File] Fix configuration file format and error comments (#4762)
  • [Connector-v2] [Jdbc] Fix oracle sql table identifier (#4754)
  • [Connector-v2] [Clickhouse] fix get clickhouse local table name with closing bracket from distributed table engineFull (#4710)

Zeta(ST-Engine)

  • [Zeta] Fix LogicalDagGeneratorTest testcase (#4401)
  • [Zeta] Fix MultipleTableJobConfigParser parse only one transform (#4412)
  • [Zeta] Fix missing common plugin jars (#4448)
  • [Zeta] Fix handleCheckpointError be called while checkpoint already complete (#4442)
  • [Zeta] Fix job error message is not right bug (#4463)
  • [Zeta] Fix finding TaskGroup deployment node bug (#4449)
  • [Zeta] Fix the bug of conf (#4488)
  • [Zeta] Fix Connector load logic from zeta (#4510)
  • [Zeta] Fix conflict dependency of hadoop-hdfs (#4509)
  • [Zeta] Fix TaskExecutionService synchronized lock will not release (#4886)
  • [Zeta] Fix TaskExecutionService will return not active ExecutionContext (#4869)
  • [Zeta] Fix deploy operation timeout but task already finished bug (#4867)
  • [Zeta] Fix restoreComplete Future can't be completed when cancel task (#4863)
  • [Zeta] Fix IMap operation timeout bug (#4859)
  • [Zeta] fix pipeline state not right bug (#4823)
  • [Zeta] Fix the incorrect setting of transform parallelism (#4814)
  • [Zeta] Fix master active bug (#4855)
  • [Zeta] Fix completePendingCheckpoint concurrent action (#4854)
  • [Zeta] Fix engine runtime error (#4850)
  • [Zeta] Fix TaskGroupContext always hold classloader so classloader can't recycle (#4849)
  • [Zeta] Fix task notifyTaskStatusToMaster failed when job not running or failed before run (#4847)
  • [Zeta] Fix cpu load problem (#4828)
  • [zeta] Fix the deadlock issue with JDBC driver loading (#4878)

E2E

  • [E2E] [Kafka] Fix kafka e2e testcase (#4520)
  • [Container Version] Fix risk of unreproducible test cases #4591
  • [E2e] [Mysql-cdc] Removing the excess MySqlIncrementalSourceIT e2e reduces the CI time (#4738)

Improve

Core

  • [Core] [Spark] Push transform operation from Spark Driver to Executors (#4503)
  • [Core] [Starter] Optimize code structure & remove redundant code (#4525)
  • [Core] [Translation] [Flink] Optimize code structure & remove redundant code (#4527)
  • [Core] [Starter] Add check of sink and source config to avoid null pointer exception. (#4734)

Connector-V2

  • [Connector-V2] [CDC] Improve startup.mode/stop.mode options (#4360)
  • [Connector-V2] [CDC] Optimize jdbc fetch-size options (#4352)
  • [Connector-V2] [CDC] Fix chunk start/end parameter type error (#4777)
  • [Connector-V2] [SQLServer] Fix sqlserver catalog (#4441)
  • [Connector-V2] [StarRocks] Improve StarRocks Serialize Error Message (#4458)
  • [Connector-V2] [Jdbc] add the log for sql and update some style (#4475)
  • [Connector-V2] [Jdbc] Fix the table name is not automatically obtained when multiple tables (#4514)
  • [Connector-V2] [S3 & Kafka] Delete unavailable S3 & Kafka Catalogs (#4477)
  • [Connector-V2] [Pulsar] Support Canal Format
  • [Connector-V2] [CDC base] Implement Sample-based Sharding Strategy with Configurable Sampling Rate (#4856)
  • [Connector-V2] [SelectDB] Add a jobId to the selectDB label to distinguish between tasks (#4864)
  • [Connector-V2] [Doris] Add a jobId to the doris label to distinguish between tasks (#4839) (#4853)
  • [Connector-v2] [Mongodb]Refactor mongodb connector (#4620)
  • [Connector-v2] [Jdbc] Populate primary key when jdbc sink is created using CatalogTable (#4755)
  • [Transform-V2] Optimize SQL Transform package and Fix Spark type conversion bug of transform (#4490)

CI

  • [CI] Fix error repository name in ci config files (#4795)

Zeta(ST-Engine)

  • [Zeta] Support run the server through daemon mode (#4161)
  • [Zeta] Change ClassLoader To Improve the SDK compatibility of the client (#4447)
  • [Zeta] Client Support Async Submit Job (#4456)
  • [Zeta] Add more detailed log output. (#4446)
  • [Zeta] Improve seatunnel-cluster.sh (#4435)
  • [Zeta] Reduce CPU Cost When Task Not Ready (#4479)
  • [Zeta] Add parser log (#4485)
  • [Zeta] Remove redundant code (#4489)
  • [Zeta] Remove redundancy code in validateSQL (#4506)
  • [Zeta] Improve JobMetrics fetch performance (#4467)
  • [Zeta] Reduce the operation count of imap_running_job_metrics (#4861)
  • [Zeta] Speed up listAllJob function (#4852)
  • [Zeta] async execute checkpoint trigger and other block method (#4846)
  • [Zeta] Reduce the number of IMAPs used by checkpointIdCounter (#4832)
  • [Zeta] Cancel pipeline add retry to avoid cancel failed. (#4792)
  • [Zeta] Improve Zeta operation max count and ignore NPE (#4787)
  • [Zeta] Remove serialize(deserialize) cost when use shuffle action (#4722)

Feature

Core

  • [Core] [API] Support convert strings as List option (#4362)
  • [Core] [API] Add copy method to Catalog codes (#4414)
  • [Core] [API] Add options check before create source and sink and transform in FactoryUtil (#4424)
  • [Core] [Shade] Add guava shade module (#4358)

Connector-V2

  • [Connector-V2] [CDC] [SQLServer] Support multi-table read (#4377)
  • [Connector-V2] [Kafka] Kafka source supports data deserialization failure skipping (#4364)
  • [Connector-V2] [Jdbc] [TiDB] Add TiDB catalog (#4438)
  • [Connector-V2] [File] Add file excel sink and source (#4164)
  • [Connector-v2] [Snowflake] Add Snowflake Source&Sink connector (#4470)
  • [Connector-V2] [Pular] support read format for pulsar (#4111)
  • [Connector-V2] [Paimon] Introduce paimon connector (#4178)
  • [Connector V2] [Cassandra] Expose configurable options in Cassandra (#3681)
  • [Connector V2] [Jdbc] Supports GEOMETRY data type for PostgreSQL (#4673)
  • [Transform-V2] Add UDF SPI and an example implement for SQL Transform plugin (#4392)
  • [Transform-V2] Support copy field list (#4404)
  • [Transform-V2] Add support CatalogTable for FieldMapperTransform (#4423)
  • [Transform-V2] Add CatalogTable support for ReplaceTransform (#4411)
  • [Transform-V2] Add Catalog support for FilterRowKindTransform (#4420)
  • [Transform-V2] Add support CatalogTable for FilterFieldTransform (#4422)
  • [Transform-V2] Add catalog support for SQL Transform plugin (#4819)

Zeta(ST-Engine)

  • [Zeta] Support for mixing Factory and Plugin SPI (#4359)
  • [Zeta] Add get running job info by jobId rest api (#4140)
  • [Zeta] Add REST API To Get System Monitoring Information (#4315)
  • [Transform V2 & Zeta] Make SplitTransform Support CatalogTable And CatalogTable Evolution (#4396)
  • [Zeta] Move driver into lib directory and change operation count (#4845)
  • [Zeta] Add Metaspace size default value to config file (#4848)
  • [Zeta] Reduce the frequency of fetching data from imap (#4851)
  • [Zeta] Add OSS support for Imap storage to cluster-mode type (#4683)
  • [Zeta] Improve local mode startup request ports (#4660)

Docs

  • [Docs] Optimizes part of the Doris and SelectDB connector documentation (#4365)
  • [Docs] Fix docs code style (#4368)
  • [Docs] Update jdbc doc and kafka doc (#4380)
  • [Docs] Fix max_retries default value is 0. (#4383)
  • [Docs] Fix markdown syntax (#4426)
  • [Docs] Fix Kafka Doc Error Config Key "kafka." (#4427)
  • [Docs] Add Transform to Quick Start v2 (#4436)
  • [Docs] Fix Mysql sink format doc (#4800)
  • [Docs] Add the generate sink sql parameter for the jdbc sink document (#4797)
  • [Docs] Add the generate sink sql parameter And example (#4769)
  • [Docs] Redshift add defaultRowFetchSize (#4616)
  • [Docs] Refactor connector-v2 docs using unified format Mysql (#4590)
Assets 2

2.3.1-release

26 Mar 07:56
@TyrantLucifer TyrantLucifer
2.3.1
3cd51b6
Compare
Choose a tag to compare
2.3.1-release

2.3.1-release

Bug fix

Core

  • [Core] [Shade] [Hadoop] Fix hadoop shade dependency can not be used (#3835)
  • [Core] Fix Handover using linked blocking queue cause the oom (#3469)
  • [Core] Fix a bug of Spark Translation when has a timestamp/date type filed in source (#4226)
  • [Core] Remove unnecessary row conversion (#4335)

Connector-V2

  • [Connecor-V2] [JDBC] Fix the problem that can not throw exception correctly (#3796)
  • [Connector-V2] [JDBC] Fix xa transaction commit failure on pipeline restore (#3809)
  • [Connector-V2] [Clickhouse] Fix the serializable problems of Committer (#3803)
  • [Connector-V2] [Clickhouse] Fix the performance bug (#3910)
  • [Connector-v2] [Clickhouse] Fix clickhouse write cdc changelog update event (#3951)
  • [Connector-V2] [Clickhouse] Clickhouse File Connector failed to sink to table with settings like storage_policy (#4172)
  • [Connector-V2] [Clickhouse] Clickhouse File Connector not support split mode for write data to all shards of distributed table (#4035)
  • [Connector-V2] [File] Fix the error type of option rule path (#3804)
  • [Connector-V2] [File] Text And Json WriteStrategy lost the sinkColumnsIndexInRow (#3863)
  • [Connector-V2] [Kafka] Fix the bug that can not parse offset format (#3810)
  • [Connecor-V2] [Kafka] Fix the default value of commit_on_checkpoint (#3831)
  • [Connector-V2] [Kafka] Json deserialize exception log no content (#3874)
  • [Connector-V2] [Kafka] Fix commit kafka offset bug. (#3933)
  • [Connector-V2] [Kafka] Fix the bug that kafka consumer is not close (#3836)
  • [Connector-V2] [Kafka] Fix config option error (#4244)
  • [Connector-V2] [CDC] Guaranteed to be exactly-once in the process of switching from SnapshotTask to IncrementalTask (#3837)
  • [Connector-V2] [CDC] Fix concurrent modify of splits (#3937)
  • [Connector-V2] [CDC] Fix jdbc sink generate update sql (#3940)
  • [Connector-V2] [CDC] Fix cdc option rule error (#4018)
  • [Connector-V2] [CDC] Fix cdc base shutdown thread not cleared (#4327)
  • [Connector-V2] [ALL] Fix ConcurrentModificationException when snapshotState based on SourceReaderBase (#4011)
  • [Connector-V2] Fix connector source snapshot state NPE (#4027)
  • [Connector-v2] [Pulsar] Fix pulsar option topic-pattern bug (#3989)
  • [Connector-V2] [ElasticSearch] Fix es source no data (#4076)
  • [Connector-V2] [Hive] Fix hive unknownhost (#4141)
  • [Connector-V2] [Pulsar] Fix Pulsar source consumer ack exception (#4237)
  • [Connector-V2] [Maxcompute] Fix failed to parse some maxcompute type (#3894)
  • [Connector-V2] [Doris] Fix Content Length header already present (#4277)
  • [Connector-V2] [ElasticSearch] Fix es field type not support binary (#4274)
  • [Connector-V2] [JDBC] Field aliases are not supported in the query of jdbc source (#4210)
  • [Json-format] [Canal] Fix json deserialize NPE (#4195)

Zeta(ST-Engine)

  • [Zeta] Fix the bug that checkpoint will be continued trigger after job CANCELED (#3808)
  • [Zeta] Fix the bug that the source split can not be deserialized in local mode (#3817)
  • [Zeta] Fix Engine Metrics will lose when Job be canceled. (#3797)
  • [Zeta] Fix CheckpointIDCounter thread not safe problem (#3875)
  • [Zeta] Fix SeatunnelChildFirstClassLoader load jackson error (#3884)
  • [Zeta] Fix local mode can not generate logs (#3917)
  • [Zeta] Fix actionSubtaskState can be null error (#3902)
  • [Zeta] A checkpoint exception may be thrown when the active cancel task terminates with an exception (#3915)
  • [Zeta] Fix NPE when scheduling sub plan fails. (#3909)
  • [Zeta] Fix NPE when starting from checkpoint. (#3904)
  • [Zeta] Fix metrics lose on pipeline restart (#3977)
  • [Zeta] Fix Job will lost control when JobMaster init failed (#4045)
  • [Zeta] Fix ResourceManager Assign Task Not Random (#4078)
  • [Zeta] Fix NullPointerException when registering Reader with the enumerator (#4048)
  • [Zeta] Fix clean TaskGroupContext Error when target node is offline (#4086)
  • [Zeta] Fix Slot Status Not Sync After Restart When All Node Down (#4047)
  • [Zeta] Fix parse job mode bug and improve doc (#4091)
  • [Zeta] Fix Client Have Error Can't be Shutdown (#4099)
  • [Zeta] Fix zeta bugs
  • [Zeta] Fix checkpoint storage namespace (#4260)
  • [Zeta] Fix read checkpoint file data is incomplete (#4263)
  • [Zeta] The pipeline needs to wait for the CheckpointCoordinator to end (#4272)
  • [Zeta] Fix CheckpointCoordinator Can't Trigger Timeout Task (#4276)
  • [Zeta] Fix job crash when run it on Seatunnel Engine cluster (#4299)
  • [Zeta] Fix Default Cluster Not Working In Config File (#3770)
  • [Zeta] Adapt StarRocks With Multi-Table And Single-Table Mode (#4324)
  • [Zeta] Fix TaskExecutionService Deploy Failed The Job Can't Stop (#4265)
  • [Zeta] Fix cancelJob and checkpoint complete error (#4330)

E2E

  • [Connector-V2] [SQLServer-CDC] Fix the bug of SQLServer-CDC it case (#3807)
  • [Connector-V2] [Clickhouse] Fix the bug of clickhouse e2e case (#3985)

Improve

Core

  • [Core] [API] Add parallelism and column projection interface (#3829)
  • [Core] [Connector-V2] Add get source method to all source connector (#3846)
  • [Core] [Shade] [Hadoop] Improve hadoop shade by including classes in package com.google.common.cache.* (#3858)
  • [Core] Use ReadonlyConfig to avoid option being changed (#4056)
  • [Core] Give the maven module a human readable name (#4114)
  • [Core] Unified the checkpoint setting key of Flink (#4296)

Connector-V2

  • [Connector-V2] [MySQL-CDC] Add mysql-cdc source factory (#3791)
  • [Connector-V2] [MySQL-CDC] Ennable binlog watermark compare (#4293)
  • [Connector-V2] [Kafka] Support user-defined client id (#3783)
  • [Connector-V2] [Kafka] Support extract topic from SeaTunnelRow field (#3742)
  • [Connector-V2] [Kafka] Add Kafka catalog (#4106)
  • [Connector-V2] [JDBC] Remove unused options that in jdbc source factory (#3794)
  • [Connector-V2] [JDBC] Support exactly-once semantics for JDBC source connector (#3750)
  • [Connector-V2] [JDBC] Improve option rule (#3802) (#3864)
  • [Connector-V2] [MongoDB] Support use source query in connector (#3697)
  • [Connector-V2] [File] Add the exception stack detail for log output (#3805)
  • [Connector-V2] [File] Improve file connector option rule and document (#3812)
  • [Connector-V2] [File] Support skip number when reading text csv files (#3900)
  • [Connector-V2] [File] Allow the user to set the row delimiter as an empty string (#3854)
  • [Connector-V2] [File] Support compress (#3899)
  • [Connector-V2] [File & Hive] Support kerberos in hive and hdfs file connector (#3840)
  • [Connector-V2] [Hive] Improve config check logic (#3886)
  • [Connector-V2] [Hive] Support assign partitions (#3842)
  • [Connector-V2] [Hive] Support read text table & Column projection (#4105)
  • [Connector-V2] [Clickhouse & Kafka & Rabbitmq & StarRocks] Change connector custom config prefix to map (#3719)
  • [Connector-V2] [Clickhouse] Special characters in column names are supported (#3881)
  • [Connector-V2] [Clickhouse] Remove Clickhouse Fields Config (#3826)
  • [Connector-V2] [Email] Unified exception for email connector (#3898)
  • [Connector-V2] [Iceberg] Unified exception for iceberg source connector (#3677)
  • [Connector-v2] [StarRocks] Support write cdc changelog event(INSERT/UPDATE/DELETE) (#3865)
  • [Connector-V2] [Fake] Improve fake connector (#3932)
  • [Connector-V2] [Fake] Optimizing Data Generation Strategies (#4061)
  • [Connector-V2] [InfluxDB] Unifie InfluxDB source fields to schema (#3897)
  • [Connector-V2] [IoTDB] Unifie IoTDB source fields to schema (#3896)
  • [Transform-V2] Add transform factory test (#3887)
  • [Connector-V2] [SQLServer-CDC] Add sqlserver cdc optionRule (#4019)
  • [Connector-V2] [Elasticsearch] Support https protocol (#3997)
  • [Connector-V2] [Elasticsearch] Add ElasticSearch catalog (#4108)
  • [Connector-V2] [Elasticsearch] Support dsl filter (#4130)
  • [Connector-V2] [S3] Add S3Catalog (#4121)
  • [Connector-V2] [Doris] Refactor some Doris Sink code as well as support 2pc and cdc (#4235)
  • [Connector-V2] [CDC] Optimize options & add docs for compatible_debezium_json (#4351)

CI

  • [CI] Imprve CI/CD process, split all connector it cases to 4 (#3832)
  • [CI] Imprve CI/CD process, split all connector it cases to 5 (#4065)
  • [E2E] Improve CI stability (#4068)
  • [CI] Improve ci steps (#4314) (#4342)

Zeta(ST-Engine)

  • [Zeta] Remove unnecessary dependencies in pom (#3795)
  • [Zeta] Add the restart job logic when all nodes down (#3784)
  • [Zeta] Add the logic that cancel CheckpointCoordinator first before cancel task (#3838)
  • [Zeta] Remove seatunnel-api from engine storage (#3834)
  • [Zeta] Suppress delete data log level (#4001)
  • [Zeta] Set the write data timeout to be configurable (#4059)
  • [Zeta] Job clean before JobMaster future complete (#4087)
  • [Zeta] Add Slot Sequence To Avoid Active Check Error (#4097)
  • [Zeta] Improve Client Job Info Message
  • [Zeta] Client Job Info Message Add Order By Submit Time
  • [Zeta] JVM parameters distinguish between client and server (#4297)

E2E

  • [Connector-V2] [Iceberg] Refactor iceberg connector e2e test cases (#3820)
  • [Connector-V2] [Datahub] Refactor DataHub connector e2e test cases (#3866)
  • [Connector-V2] [MongoDB] Refactor MongoDB connector e2e test (#3819)
  • [Connector-V2] [ES & InfluxDB & Redis] Add a 'nonNull' check to avoid npe when executing 'tearDown'. (#3967)
  • [Connector-V2] [Http] Refactor the e2e test of http (#3969)
  • [Connector-V2] Remove the use of scala.Tuple in the e2e module (#3974)
  • [Zeta] Change E2E To support ClusterFaultToleranceIT (#3976)
  • [Zeta] Statistics server job and system resource usage (#3982)
  • [Transform-v2] Merge e2e tests and config files of SQL transform plugin (#4278)
  • [Connector-V2] [Jdbc] Reactor jdbc e2e with new api, then remove the useless e2e case. For better performance, we split the jdbc e2e module into three modules. (#4165)

Feature

Core

  • [Core] [Transform-V2] Support tr...
Read more
Assets 2

2.3.0-release

30 Dec 06:54
@TyrantLucifer TyrantLucifer
2.3.0
d7280ab
Compare
Choose a tag to compare
2.3.0-release

2.3.0-release

Bug fix

Core

  • [Core] [Starter] Fix the bug of ST log print failed in some jdk versions #3160
  • [Core] [Shell] Fix bug that shell script about downloading plugins does not work #3462

Connector-V2

  • [Connector-V2] [Jdbc] Fix the bug that jdbc source can not be stopped in batch mode #3220
  • [Connector-V2] [Jdbc] Fix the bug that jdbc connector reset in jdbc connector #3670
  • [Connector-V2] [Jdbc] Fix the bug that jdbc connector exactly-once it will throw NullPointerException #3730
  • [Connector-V2] [Hive] Fix the following bugs of hive connector: 1. write parquet NullPointerException 2. when restore write from states getting error file path #3258
  • [Connector-V2] [File] Fix the bug that when getting file system throw NullPointerException #3506
  • [Connector-V2] [File] Fix the bug that when user does not config the fileNameExpression it will throw NullPointerException #3706
  • [Connector-V2] [Hudi] Fix the bug that the split owner of Hudi connector may be negative #3184

ST-Engine

  • [ST-Engine] Fix bug data file name will duplicate when use SeaTunnel Engine #3717
  • [ST-Engine] Fix job restart of all nodes down #3722
  • [ST-Engine] Fix the bug that checkpoint stuck in ST-Engine #3213
  • [ST-Engine] Fix the bug that checkpoint failed in ST-Engine #3769

E2E

  • [E2E] [Spark] Corrected spark version in e2e container #3225

Improve

Core

  • [Core] [Starter] [Flink] Upgrade the method of loading extra jars in flink starter #2982
  • [Core] [Pom] [Package] Optimize package process #3751

Connector-V1

  • [Connector-V1] Remove connector v1 related codes from dev branch #3450

Connector-V2

  • [Connector-V2] Add split templates for all connectors #3335
  • [Connector-V2] [Redis] Support redis cluster mode & user authentication #3188
  • [Connector-V2] [Clickhouse] Support nest type and array type in clickhouse connector #3047
  • [Connector-V2] [Clickhouse] Support geo type in clickhouse connector #3141
  • [Connector-V2] [Clickhouse] Improve double convert that in clickhouse connector #3441
  • [Connector-V2] [Clickhouse] Improve float long convert that in clickhouse connector #3471
  • [Connector-V2] [Kafka] Support setting read start offset or message time in kafka connector #3157
  • [Connector-V2] [Kafka] Support specify multiple partition keys in kafka connector #3230
  • [Connector-V2] [Kafka] Support dynamic discover topic & partition in kafka connector #3125
  • [Connector-V2] [Kafka] Support text format for kafka connector #3711
  • [Connector-V2] [IotDB] Add the parameter check logic for iotDB sink connector #3412
  • [Connector-V2] [Jdbc] Support setting fetch size in jdbc connector #3478
  • [Connector-V2] [Jdbc] Support upsert config in jdbc connector #3708
  • [Connector-V2] [Jdbc] Optimize the commit process of jdbc connector #3451
  • [Connector-V2] [Jdbc] Release jdbc resource when after using #3358
  • [Connector-V2] [Oracle] Improve data type mapping of Oracle connector #3486
  • [Connector-V2] [Http] Support extract complex json string in http connector #3510
  • [Connector-V2] [File] [S3] Support s3a protocol in S3 file connector #3632
  • [Connector-V2] [File] [HDFS] Support setting hdfs-site.xml #3778
  • [Connector-V2] [File] Support file split in file connectors #3625
  • [Connector-V2] [CDC] Support write cdc changelog event in elsticsearch sink connector #3673
  • [Connector-V2] [CDC] Support write cdc changelog event in clickhouse sink connector #3653
  • [Conncetor-V2] [CDC] Support write cdc changelog event in jdbc connector #3444

ST-Engine

  • [ST-Engine] Improve statistic information print format that in ST-Engine #3492
  • [ST-Engine] Improve ST-Engine performance #3216
  • [ST-Engine] Support user-defined jvm parameters in ST-Engine #3307

CI

E2E

  • [E2E] [Flink] Support execute extra commands on task-manager container #3224
  • [E2E] [Jdbc] Increased Jdbc e2e stability #3234

Feature

Core

  • [Core] [Log] Integrate slf4j and log4j2 for unified management logs #3025
  • [Core] [Connector-V2] [Exception] Unified exception API & Unified connector error tip message #3045
  • [Core] [Shade] [Hadoop] Add hadoop shade package for SeaTunnel #3755

Connector-V2

  • [Connector-V2] [Elasticsearch] Add elasticsearch source connector #2821
  • [Connector-V2] [AmazondynamoDB] Add AmazondynamoDB source & sink connector #3166
  • [Connector-V2] [StarRocks] Add StarRocks sink connector #3164
  • [Connector-V2] [DB2] Add DB2 source & sink connector #2410
  • [Connector-V2] [Transform] Add transform-v2 api #3145
  • [Connector-V2] [InfluxDB] Add influxDB sink connector #3174
  • [Connector-V2] [Cassandra] Add Cassandra Source & Sink connector #3229
  • [Connector-V2] [MyHours] Add MyHours source connector #3228
  • [Connector-V2] [Lemlist] Add Lemlist source connector #3346
  • [Connector-V2] [CDC] [MySql] Add mysql cdc source connector #3455
  • [Connector-V2] [CDC] [SqlServer] Add sqlserver cdc source connector #3686
  • [Connector-V2] [Klaviyo] Add Klaviyo source connector #3443
  • [Connector-V2] [OneSingal] Add OneSingal source connector #3454
  • [Connector-V2] [Slack] Add slack sink connector #3226
  • [Connector-V2] [Jira] Add Jira source connector #3473
  • [Connector-V2] [Sqlite] Add Sqlite source & sink connector #3089
  • [Connector-V2] [OpenMldb] Add openmldb source connector #3313
  • [Connector-V2] [Teradata] Add teradata source & sink connector #3362
  • [Connector-V2] [Doris] Add doris source & sink connector #3586
  • [Connector-V2] [MaxCompute] Add MaxCompute source & sink connector #3640
  • [Connector-V2] [Doris] [Streamload] Add doris streamload sink connector #3631
  • [Connector-V2] [Redshift] Add redshift source & sink connector #3615
  • [Connector-V2] [Notion] Add notion source connector #3470
  • [Connector-V2] [File] [Oss-Jindo] Add oss jindo source & sink connector #3456

ST-Engine

  • [ST-Engine] Support print job metrics when job finished #3691
  • [ST-Engine] Add metrics statistic in ST-Engine #3621
  • [ST-Engine] Support IMap file storage in ST-Engine #3418
  • [ST-Engine] Support S3 file system for IMap file storage #3675
  • [ST-Engine] Support save job restart status information in ST-Engine #3637

E2E

  • [E2E] [Http] Add http type connector e2e test cases #3340
  • [E2E] [File] [Local] Add local file connector e2e test cases #3221

Docs

  • [Docs] [Connector-V2] [Factory] Add TableSourceFactory & TableSinkFactor docs #3343
  • [Docs] [Connector-V2] [Schema] Add connector-v2 schema docs #3296
  • [Docs] [Connector-V2] [Quick-Manaul] Add error quick reference manual #3437
  • [Docs] [README] Improve README and refactored other docs #3619
Assets 2

2.3.0-beta-Release

09 Nov 03:09
@CalvinKirs CalvinKirs
2.3.0-beta
f57b897
Compare
Choose a tag to compare
2.3.0-beta-Release

2.3.0 Beta

[Connector V2]

[New Connector V2 Added]

  • [Source] [Kafka] Add Kafka Source Connector (2953)
  • [Source] [Pulsar] Add Pulsar Source Connector (1980)
  • [Source] [S3File] Add S3 File Source Connector (3119)
  • [Source] [JDBC] [Phoenix] Add Phoenix JDBC Source Connector (2499)
  • [Source] [JDBC] [SQL Server] Add SQL Server JDBC Source Connector (2646)
  • [Source] [JDBC] [Oracle] Add Oracle JDBC Source Connector (2550)
  • [Source] [JDBC] [GBase8a] Add GBase8a JDBC Source Connector (3026)
  • [Source] [JDBC] [StarRocks] Add StarRocks JDBC Source Connector (3060)
  • [Sink] [Kafka] Add Kafka Source Connector (2953)
  • [Sink] [S3File] Add S3 File Sink Connector (3119)

[Improve & Bug Fix]

  • [Source] [Fake]

    • [Improve] Supports direct definition of data values(row) (2839)
    • [Improve] Improve fake source connector: (2944)
      • Support user-defined map size
      • Support user-defined array size
      • Support user-defined string length
      • Support user-defined bytes length
    • [Improve] Support multiple splits for fake source connector (2974)
    • [Improve] Supports setting the number of splits per parallelism and the reading interval between two splits (3098)

  • [Source] [Clickhouse]

    • [Improve] Clickhouse Source random use host when config multi-host (3108)

  • [Source] [FtpFile]

    • [BugFix] Fix the bug of incorrect path in windows environment (2980)
    • [Improve] Support extract partition from SeaTunnelRow fields (3085)
    • [Improve] Support parse field from file path (2985)

  • [Source] [HDFSFile]

    • [BugFix] Fix the bug of incorrect path in windows environment (2980)
    • [Improve] Support extract partition from SeaTunnelRow fields (3085)
    • [Improve] Support parse field from file path (2985)

  • [Source] [LocalFile]

    • [BugFix] Fix the bug of incorrect path in windows environment (2980)
    • [Improve] Support extract partition from SeaTunnelRow fields (3085)
    • [Improve] Support parse field from file path (2985)

  • [Source] [OSSFile]

    • [BugFix] Fix the bug of incorrect path in windows environment (2980)
    • [Improve] Support extract partition from SeaTunnelRow fields (3085)
    • [Improve] Support parse field from file path (2985)

  • [Source] [IoTDB]

    • [Improve] Improve IoTDB Source Connector (2917)
      • Support extract timestamp、device、measurement from SeaTunnelRow
      • Support TINYINT、SMALLINT
      • Support flush cache to database before prepareCommit

  • [Source] [JDBC]

    • [Feature] Support Phoenix JDBC Source (2499)
    • [Feature] Support SQL Server JDBC Source (2646)
    • [Feature] Support Oracle JDBC Source (2550)
    • [Feature] Support StarRocks JDBC Source (3060)
    • [Feature] Support GBase8a JDBC Source (3026)

  • [Sink] [Assert]

    • [Improve] 1.Support check the number of rows (2844) (3031):
      • check rows not empty
      • check minimum number of rows
      • check maximum number of rows
    • [Improve] 2.Support direct define of data values(row) (2844) (3031)
    • [Improve] 3.Support setting parallelism as 1 (2844) (3031)

  • [Sink] [Clickhouse]

    • [Improve] Clickhouse Support Int128,Int256 Type (3067)

  • [Sink] [Console]

    • [Improve] Console sink support print subtask index (3000)

  • [Sink] [Enterprise-WeChat]

    • [BugFix] Fix Enterprise-WeChat Sink data serialization (2856)

  • [Sink] [FtpFile]

    • [BugFix] Fix the bug of incorrect path in windows environment (2980)
    • [BugFix] Fix filesystem get error (3117)
    • [BugFix] Solved the bug of can not parse '\t' as delimiter from config file (3083)

  • [Sink] [HDFSFile]

    • [BugFix] Fix the bug of incorrect path in windows environment (2980)
    • [BugFix] Fix filesystem get error (3117)
    • [BugFix] Solved the bug of can not parse '\t' as delimiter from config file (3083)

  • [Sink] [LocalFile]

    • [BugFix] Fix the bug of incorrect path in windows environment (2980)
    • [BugFix] Fix filesystem get error (3117)
    • [BugFix] Solved the bug of can not parse '\t' as delimiter from config file (3083)

  • [Sink] [OSSFile]

    • [BugFix] Fix the bug of incorrect path in windows environment (2980)
    • [BugFix] Fix filesystem get error (3117)
    • [BugFix] Solved the bug of can not parse '\t' as delimiter from config file (3083)

  • [Sink] [IoTDB]

    • [Improve] Improve IoTDB Sink Connector (2917)
      • Support align by sql syntax
      • Support sql split ignore case
      • Support restore split offset to at-least-once
      • Support read timestamp from RowRecord
    • [BugFix] Fix IoTDB connector sink NPE (3080)

  • [Sink] [JDBC]

    • [BugFix] Fix JDBC split exception (2904)
    • [Feature] Support Phoenix JDBC Source (2499)
    • [Feature] Support SQL Server JDBC Source (2646)
    • [Feature] Support Oracle JDBC Source (2550)
    • [Feature] Support StarRocks JDBC Source (3060)

  • [Sink] [Kudu]

    • [Improve] Kudu Sink Connector Support to upsert row (2881)

  • [Sink] [Hive]

    • [Improve] Hive Sink supports automatic partition repair (3133)

[Connector V1]

[New Connector V1 Added]

[Improve & Bug Fix]

  • [Sink] [Spark-Hbase]
    • [BugFix] Handling null values (3099)

[Starter & Core & API]

[Feature & Improve]

  • [Improve] [Sink] Support define parallelism for sink connector (2941)
  • [Improve] [all] change Log to @slf4j (3001)
  • [Improve] [format] [text] Support read & write SeaTunnelRow type (2969)
  • [Improve] [api] [flink] extraction unified method (2862)
  • [Feature] [deploy] Add Helm charts (2903)
  • [Feature] [seatunnel-text-format] (2884)

[Bug Fix]

  • [BugFix] Fix assert connector name error in config/plugin_config file (3127)
  • [BugFix] [starter] Fix connector-v2 flink & spark dockerfile (3007)
  • [BugFix] [core] Fix spark e...
Read more

Contributors

slf4j
Assets 2

2.2.0-beta-Release

03 Oct 09:06
@CalvinKirs CalvinKirs
2.2.0-beta
d42b5e1
Compare
Choose a tag to compare
2.2.0-beta-Release

[Feature & Improve]

  • Connector V2 API, Decoupling connectors from compute engines
    • [Translation] Support Flink 1.13.x
    • [Translation] Support Spark 2.4
    • [Connector-V2] [Fake] Support FakeSource (#1864)
    • [Connector-V2] [Console] Support ConsoleSink (#1864)
    • [Connector-V2] [ElasticSearch] Support ElasticSearchSink (#2330)
    • [Connector-V2] [ClickHouse] Support ClickHouse Source & Sink (#2051)
    • [Connector-V2] [JDBC] Support JDBC Source & Sink (#2048)
    • [Connector-V2] [JDBC] [Greenplum] Support Greenplum Source & Sink(#2429)
    • [Connector-V2] [JDBC] [DM] Support DaMengDB Source & Sink(#2377)
    • [Connector-V2] [File] Support Source & Sink for Local, HDFS & OSS File
    • [Connector-V2] [File] [FTP] Support FTP File Source & Sink (#2774)
    • [Connector-V2] [Hudi] Support Hudi Source (#2147)
    • [Connector-V2] [Icebreg] Support Icebreg Source (#2615)
    • [Connector-V2] [Kafka] Support Kafka Source (#1940)
    • [Connector-V2] [Kafka] Support Kafka Sink (#1952)
    • [Connector-V2] [Kudu] Support Kudu Source & Sink (#2254)
    • [Connector-V2] [MongoDB] Support MongoDB Source (#2596)
    • [Connector-V2] [MongoDB] Support MongoDB Sink (#2649)
    • [Connector-V2] [Neo4j] Support Neo4j Sink (#2434)
    • [Connector-V2] [Phoenix] Support Phoenix Source & Sink (#2499)
    • [Connector-V2] [Redis] Support Redis Source (#2569)
    • [Connector-V2] [Redis] Support Redis Sink (#2647)
    • [Connector-V2] [Socket] Support Socket Source (#1999)
    • [Connector-V2] [Socket] Support Socket Sink (#2549)
    • [Connector-V2] [HTTP] Support HTTP Source (#2012)
    • [Connector-V2] [HTTP] Support HTTP Sink (#2348)
    • [Connector-V2] [HTTP] [Wechat] Support Wechat Source Sink(#2412)
    • [Connector-V2] [Pulsar] Support Pulsar Source (#1984)
    • [Connector-V2] [Email] Support Email Sink (#2304)
    • [Connector-V2] [Sentry] Support Sentry Sink (#2244)
    • [Connector-V2] [DingTalk] Support DingTalk Sink (#2257)
    • [Connector-V2] [IotDB] Support IotDB Source (#2431)
    • [Connector-V2] [IotDB] Support IotDB Sink (#2407)
    • [Connector-V2] [Hive] Support Hive Source & Sink(#2708)
    • [Connector-V2] [Datahub] Support Datahub Sink(#2558)
  • [Catalog] MySQL Catalog (#2042)
  • [Format] JSON Format (#2014)
  • [Spark] [ClickHouse] Support unauthorized ClickHouse (#2393)
  • [Binary-Package] Add script to automatically download plugins (#2831)
  • [License] Update binary license (#2798)
  • [e2e] Improved e2e start sleep (#2677)
  • [e2e] Container only copy required connector jars (#2675)
  • [build] delete connectors*-dist modules (#2709)
  • [build] Dependency management split (#2606)
  • [build] The e2e module don't depend on the connector*-dist module (#2702)
  • [build] Improved scope of maven-shade-plugin (#2665)
  • [build] make sure flatten-maven-plugin runs after maven-shade-plugin (#2603)
  • [Starter] Use the public CommandLine util class to parse the args (#2470)
  • [Spark] [Redis] Self-Achieved Redis Proxy which is not support redis function of "info replication" (#2389)
  • [Flink] [Transform] support multi split,and add custome split function name (#2268)
  • [Test] Upgrade junit to 5.+ (#2305)

[Bugfix]

  • [Starter] Ensure that output paths constructed from zip archive entries are validated to prevent writing files to
    unexpected locations (#2843)
  • [Starter] Let the SparkCommandArgs do not split the variable value with comma (#2523)
  • [Spark] fix the problem of calling the getData() method twice (#2764)
  • [e2e] Fix path split exception in win10,not check file existed (#2715)

[Docs]

  • [Kafka] Update Kafka.md (#2863)
  • [JDBC] Fix inconsistency between document (#2776)
  • [Flink-SQL] [ElasticSearch] Updated prepare section (#2634)
  • [Contribution] add CheckStyle-IDEA Plugin introduction (#2535)
  • [Contribution] Update new-license.md (#2494)
Assets 2

2.1.3 Release

10 Aug 02:16
@Hisoka-X Hisoka-X
2.1.3
f641116
Compare
Choose a tag to compare
2.1.3 Release

[Feature & Improvement]

[Connector][Flink][Fake] Supported BigInteger Type (#2118)

[Connector][Spark][TiDB] Refactored config parameters (#1983)

[Connector][Flink]add AssertSink connector (#2022)

[Connector][Spark][ClickHouse]Support Rsync to transfer clickhouse data file (#2074)

[Connector & e2e][Flink] add IT for Assert Sink in e2e module (#2036)

[Transform][Spark] data quality for null data rate (#1978)

[Transform][Spark] Add a module to set default value for null field #1958

[Chore]a more understandable code,and code warning will disappear #2005

[Spark] Use higher version of the libthrift dependency (#1994)

[Core][Starter] Change jar connector load logic (#2193)

[Core]Add plugin discovery module (#1881)

[BUG]

[Connector][Hudi] Source loads the data twice

[Connector][Doris]Fix the bug Unrecognized field "TwoPhaseCommit" after doris 0.15 (#2054)

[Connector][Jdbc]Fix the data output exception when accessing Hive using Spark JDBC #2085

[Connector][Jdbc]Fix JDBC data loss occurs when partition_column (partition mode) is set #2033

[Connector][Kafka]KafkaTableStream schema json parse #2168

[seatunnel-core] Failed to get APP_DIR path bug fixed (#2165)

[seatunnel-api-flink] Connectors dependencies repeat additions (#2207)

[seatunnel-core] Failed to get APP_DIR path bug fixed (#2165)

[seatunnel-core-flink] Updated FlinkRunMode enum to get the proper help message for run modes. (#2008)

[seatunnel-core-flink]fix same source and sink registerplugin librarycache error (#2015)

[Command]fix commandArgs -t(--check) conflict with flink deployment target (#2174)

[Core][Jackson]fix jackson type convert error (#2031)

[Core][Starter] When use cluster mode, but starter app root dir also should same as client mode. (#2141)

Docs

source socket connector docs update (#1995)

Add uuid, udf, replace transform to doc (#2016)

Update Flink engine version requirements (#2220)

Add Flink SQL module to website. (#2021)

[kubernetes] update seatunnel doc on kubernetes (#2035)

Dependency upgrade

Upgrade common-collecions4 to 4.4

Upgrade common-codec to 1.13

Assets 2

2.1.2 Release

18 Jun 12:50
@Hisoka-X Hisoka-X
2.1.2
cc2bc03
Compare
Choose a tag to compare
2.1.2 Release

[Feature]

  • Add Spark webhook source
  • Support Flink application mode
  • Split connector jar from core jar
  • Add Replace transforms for Spark
  • Add Uuid transform for Spark
  • Support Flink dynamic configurations
  • Flink JDBC source support Oracle database
  • Add Flink connector Http
  • Add Flink transform for register user define function
  • Add Flink SQL Kafka, ElasticSearch connector

[Bugfix]

  • Fixed ClickHouse sink data type convert error
  • Fixed first execute Spark start shell can't run problem
  • Fixed can not get config file when use Spark on yarn cluster mode
  • Fixed Spark extraJavaOptions can't be empty
  • Fixed the "plugins.tar.gz" decompression failure in Spark standalone cluster mode
  • Fixed Clickhouse sink can not work correctly when use multiple hosts
  • Fixed Flink sql conf parse exception
  • Fixed Flink JDBC Mysql datatype mapping incomplete
  • Fixed variables cannot be set in Flink mode
  • Fixed SeaTunnel Flink engine cannot check source config

[Improvement]

  • Update Jackson version to 2.12.6
  • Add guide on how to Set Up SeaTunnel with Kubernetes
  • Optimize some generic type code
  • Add Flink SQL e2e module
  • Flink JDBC connector add pre sql and post sql
  • Use @autoservice to generate SPI file
  • Support Flink FakeSourceStream to mock data
  • Support Read Hive by Flink JDBC source
  • ClickhouseFile support ReplicatedMergeTree
  • Support use Hive sink to save table as ORCFileFormat
  • Support Spark Redis sink custom expire time
  • Add Spark JDBC isolationLevel config
  • Use Jackson replace Fastjson

Contributors

autoservice
Assets 2