Skip to content

Latest commit

 

History

History
333 lines (261 loc) · 16.4 KB

CHANGELOG.md

File metadata and controls

333 lines (261 loc) · 16.4 KB
Raw

Change Log

Unreleased - ReleaseDate

0.5.8 - 2024-08-14

  • Separate qualifier for new and old sync pks (#608)
  • Allow running Seafowl without an object store / catalog config. (#607)
  • Remove bucket from options (#603)
  • Remove prefix from options (#595)
  • Migrate unstable field json repr to a canonical one (#594)
  • Add flag denoting first sync response (#593)
  • Refactor object_store_factory and create StorageLocationInfo (#592)
  • Support empty syncs with no batches provided as payload (#591)
  • Add origin and position to flight handler debug (#590)
  • Patch DF to circumvent an analyzer rule (#586)
  • DataFusion 40 upgrade (#584)
  • Optional sync sequence number (#581)
  • BEAC-167: Move object store configs to object_store_factory (#580)
  • Add criteria for max separate sync calls to apply when flushing (#579)
  • Add memory and local FS store to the object_store_factory builder function (#578)
  • BEAC-167: Create a crate that instantiates an object store based on URL, settings and environment variables (#569)
  • TPC-DS Q75 fix backport (#563)
  • Workflow_dispatch trigger for nightly workflow (#562)
  • Allow using RUST_LOG_STYLE=never to disable ANSI colors (#561)
  • Use the storage location name as the key in the clade schema (#556)
  • Increase retries and add delay backoff to connect to client on tests (#554)
  • Make origin a string (#548)

0.5.7 - 2024-06-21

  • Add flush task (#534)
  • Add sync metrics (#533)
  • Upgrade to DataFusion 39 (#532)
  • Add support for inline-ing metastore over Flight (#531)
  • Sync protocol improvements (#530)
  • Data replication interface (#527)
  • Autodetect bucket region in S3 without custom endpoints (#526)
  • Ignore DeltaTableAlready delta error during conversion of a parquet folder (#525)
  • Make CONVERT TO DELTA overwrite if exists (#523)
  • DataFusion 37.1 upgrade (#520)
  • Add various object store cache Prometheus metrics (#519)
  • Add Prometheus metrics to MemoryManager (#518)

0.5.6 - 2024-04-22

  • Improve Arrow Flight SQL interface logging (#517)
  • Move Delta store creation into a separate ObjectStoreFactory (#516)
  • Best effort chunk coalescing implementation (#515)
  • Use tokio-graceful-shutdown for SIGINT / SIGTERM (#513)
  • Upgrade to DataFusion 36 (#511)
  • Enable using session tokens for the S3 client (#510)
  • Fix condition for caching clade object stores (#509)
  • Add caching on top of clade-provided object stores too (#508)
  • Improve caching object store performance (#507)
  • Make object_store confgi section optional (#506)
  • Add support for dynamic object store usage via clade (#504)

0.5.5 - 2024-02-26

  • Add a http health endpoint (#502)
  • Add support for public S3 buckets (#501)
  • Add gRPC metrics (#500)
  • Implement prometheus metrics (#499)
  • Upgrade to DataFusion 35 and Arrow 50 (#498)
  • Replace log with tracing (#493)
  • Use from_env when creating the DataFusion session config (#490)

0.5.4 - 2024-01-17

  • Upgrade to DataFusion 34 and Arrow 49 (#486)
  • Enable external metastores (#483)
  • Eager dropping of tables and schemas (#481)
  • Add Arrow Flight SQL frontend (#478)

0.5.3 - 2023-12-04

  • Implement CONVERT TO DELTA statement (#473)
  • Support sub-folder paths as data firs in object stores (#472)
  • Introduce Seafowl CLI (#468)
  • Upgrade to DataFusion 32 (#465)

0.5.2 - 2023-09-07

  • Wire up COPY TO statement (#462)
  • Upgrade to DataFusion 30 and Arrow 45 (#461)
  • Enable time travel for write statements with source queries (#460)

0.5.1 - 2023-08-01

  • Upgrade to DataFusion 28 and Arrow 43 (#458)
  • Implement CREATE OR REPLACE FUNCTION statement path (#455)
  • Enable passing of supplied get options via the HTTP object store (#454)

0.5.0 - 2023-07-12

  • Upgrade to DataFusion 27 and Arrow 42 (#453)
  • Load schemas without tables and double-check missing DB ids in a given context (#446)
  • Implement schema coercion and file streaming in the upload endpoint (#439)
  • Return timing data in response headers (#438)

0.4.3 - 2023-06-07

  • Enable creating external tables on cloud object stores (#433)
  • Implement streaming of result rows (#424)

0.4.2 - 2023-05-26

  • Enable specifying the GET endpoint cache-control header via config (#422)
  • Upgrade to DataFusion 25 and Arrow 39 (#415)

0.4.1 - 2023-05-17

  • Upgrade to DataFusion 24 and Arrow 38 (#403)
  • Enable passing query as a path parameter in the GET endpoint (#402)

0.4.0 - 2023-05-11

  • Remove legacy table reading logic (#390)
  • Upgrade DataFusion to 23 and Arrow to 37 (#386)
  • Add support for GCS bucket object stores (#379)

0.3.4 - 2023-04-27

  • Return result type info in the content-type header, and bump object_store crate (#367)

0.3.3 - 2023-04-20

  • Fix table renaming to quoted identifiers with special characters (#362)
  • Upgrade DataFusion to v22.0.0 and Arrow to v36.0.0 (#360)

0.3.2 - 2023-04-04

  • Enable caching of S3 objects in the local FS (#341)
  • Fix the silent panic in the cache eviction hook rendering it useless (#334)

0.3.1 - 2023-03-29

  • Fix unimplemented method in AWS object store, and broaden integration test coverage by extending target object stores to include a MinIO instance (#331)
  • Enable cached GET endpoint for authorized requests, and enable dropping of external tables (#326)

0.3.0 - 2023-03-22

  • Migration of storage layer to delta-rs (#307)
  • Purge legacy logic (#313)
  • Deltify parquet writing logic (#316)
  • Implement VACUUM DATABASE command (#320)
  • Enable VACUUM for Delta tables (#322)

0.2.12 - 2023-02-14

  • Upgrade to DataFusion 18.0.0 (#297)
  • Implement multi-database support for upload endpoint (#291)
  • Implement multi-database support for GET and POST query endpoints (#289)

0.2.11 - 2023-01-27

  • Upgrade to DataFusion post-16.0 (#270)
  • Add an ssl_cert_file option to the config (#278)
  • Migrate to DataFusion's native DML nodes for logical write statement planning (#281)

0.2.10 - 2022-12-30

  • Add region config parameter for Amazon S3 object store (#255)
  • Enable querying external Delta tables in Seafowl (#252)
  • Implement remote table factory (#250)

0.2.9 - 2022-12-23

  • Add support for pushdown in remote tables:
  • Factor out remote tables into a separate crate (#238)
  • Upgrade to DataFusion 15 (#248)

0.2.8 - 2022-11-21

  • Implement table_partitions system table (#214)
  • Add WASI + MessagePack UDF language variant (#149)

0.2.7 - 2022-11-17

  • Import JSON values as strings in CREATE EXTERNAL TABLE (#208)
  • Add support for SQLite in CREATE EXTERNAL TABLE (#200)

0.2.6 - 2022-11-08

  • Update to DataFusion 14 / Arrow 26 (#198)
  • Bugfix: VACUUM with shared partitions (#191)
  • Bugfix: DELETE with certain filters that cover a whole partition (#192)
  • Initial support for other databases in CREATE EXTERNAL TABLE (#190)
    • More documentation pending. Example: CREATE EXTERNAL TABLE t STORED AS TABLE 'public.t' LOCATION 'postgresql://uname:pw@host:port/dbname'

0.2.5 - 2022-11-02

  • Upgrade to DataFusion 13 (784f10bb) / Arrow 25.0.0 (#176)
  • Use ZSTD compression in Parquet files (#182)
  • Fix HTTP external tables using pre-signed S3 URLs (#183)
  • Fix INSERT INTO .. SELECT FROM (#184)
  • Fix some OUTER JOIN issues by using a minimum of 2 target_partitions (#189)

0.2.4 - 2022-10-25

  • Add system.table_versions table for inspecting table history (#168)
  • Add SQLite catalog.read_only option for compatibility with LiteFS replicas (#171)

0.2.3 - 2022-10-21

  • Add support for time travel queries (SELECT * FROM table('2022-01-01T00:00:00')) (#154)
  • Allow overriding SQLite journal mode (#158)

0.2.2 - 2022-10-12

  • Allow using SQL types in WASM UDF definitions (#147)

0.2.1 - 2022-09-30

  • Cached GET API now accepts URL-encoded query text in X-Seafowl-Header (#122)
  • Add support for DELETE statements (#121)
  • Add support for UPDATE statements (#127)

0.2.0 - 2022-09-21

Breaking: Previous versions of Seafowl won't be able to read data written by this version.

  • Fix storage of nullable / list columns (#119)
  • Default columns in CREATE TABLE to nullable (#119)

0.1.1 - 2022-09-16

  • Upgrade to DataFusion 12 (#113)

0.1.0 - 2022-09-12

Fixes

  • Use multi-part uploads to fix the memory usage issue when uploading data to S3 (#99)