Skip to content

Commit

Permalink
Update changelog and attribution files
Browse files Browse the repository at this point in the history
  • Loading branch information
@tlento
tlento committed Aug 3, 2023
1 parent a8a16cb commit 6f3ccbb
Show file tree
Hide file tree
Showing 2 changed files with 126 additions and 819 deletions.
121 changes: 95 additions & 26 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,97 @@
- "Breaking changes" listed under a version may require action from end users or external maintainers when upgrading to that version.
- Do not edit this file directly. This file is auto-generated using [changie](https://github.com/miniscruff/changie). For details on how to document a change, see [the contributing guide](https://github.com/dbt-labs/metricflow/blob/main/CONTRIBUTING.md)

## MetricFlow 0.200.0 - August 02, 2023

### Breaking Changes

- License Change - Version 0 to 0.140.0 was covered by the Affero GPL license. Version 0.150.0 and greater is covered by the BSL license. ([#465](https://github.com/dbt-labs/metricflow/issues/465))
- Removing time_format from DimensionTypeParams ([#494](https://github.com/dbt-labs/metricflow/issues/494))
- Use Templates For Defining Metric Filters ([#505](https://github.com/dbt-labs/metricflow/issues/505))
- Rename Metric.constraint to Metric.filter ([#511](https://github.com/dbt-labs/metricflow/issues/511))
- Deprecate and refactor CLI commands
- Removes async query and query cancel methods from SqlClient protocols ([#577](https://github.com/dbt-labs/metricflow/issues/577))
- Remove time spine introspection and table creation, which may break cumulative metric queries ([#592](https://github.com/dbt-labs/metricflow/issues/592))
- Remove SqlEngineAttributes construct from SqlClient interface in favor of dialect rendering and engine type properties ([#577](https://github.com/dbt-labs/metricflow/issues/577))
- CLI needs to be ran in a dbt project root directory
- Remove expr & ratio metrics and bundle with derived metrics. ([#504](https://github.com/dbt-labs/metricflow/issues/504))
- `explain_get_dimension_values` & `get_dimension_values` take a list of metrics parameters
- Remove MetricFlow config file - all future configuration must originate with the dbt project ([#624](https://github.com/dbt-labs/metricflow/issues/624))
- Update to dbt-semantic-interfaces==0.1.0.dev8. ([#634](https://github.com/dbt-labs/metricflow/issues/634))
- Changed the --group-bys option in mf query to be --group-by
- Add Support for `primary_entity` in Semantic Models ([#694](https://github.com/dbt-labs/metricflow/issues/694))

### Features

- Added new entity calls to CLI/MetricFlowEngine
- Script to Generate Snapshots for Supported SQL Engines. ([#609](https://github.com/dbt-labs/metricflow/issues/609))
- Add dbt adapter support for postgres and enable it for tests ([#578](https://github.com/dbt-labs/metricflow/issues/578))
- Use dbt adapter to run queries and warehouse validations from MetricFlow CLI ([#624](https://github.com/dbt-labs/metricflow/issues/624))
- Enable Snowflake queries in dbt <-> MetricFlow CLI integration ([#579](https://github.com/dbt-labs/metricflow/issues/579))
- Refactor `mf tutorial` to work alongside a dbt project.
- New package `dbt-metricflow` which bundles dbt-core and metricflow and dbt-adapters
- Add Support for Python 3.10 / 3.11 ([#659](https://github.com/dbt-labs/metricflow/issues/659))
- Include metric_time in List Dimensions Output Where Appropriate ([#673](https://github.com/dbt-labs/metricflow/issues/673))
- Enable support for Redshift queries in dbt-metricflow integration ([#582](https://github.com/dbt-labs/metricflow/issues/582))
- Enable Databricks support for the dbt-metricflow integration ([#580](https://github.com/dbt-labs/metricflow/issues/580))
- Enable support for BigQuery for dbt metricflow integration users ([#581](https://github.com/dbt-labs/metricflow/issues/581))

### Fixes

- Removes MySQL from SqlEngine and SqlDialect options since it is not supported. ([#0](https://github.com/dbt-labs/metricflow/issues/0))
- Derived metrics were not respecting the constraint defined in the original input metric's definition.
- Fixes type error in BigQuerySqlExpressionRenderer ([#536](https://github.com/dbt-labs/metricflow/issues/536))
- Fix broken type signature for log_call decorator
- Apply transformations to dbt-generated serialized model to fix issue with query generation ([#624](https://github.com/dbt-labs/metricflow/issues/624))
- Improve error message rendering in MetricFlow CLI ([#646](https://github.com/dbt-labs/metricflow/issues/646))
- Added --version and fix manifest transformer rules for dbt-core-=1.6.0b8 ([#650](https://github.com/dbt-labs/metricflow/issues/650))
- Include granularity suffix on time dimension name rendering for all time dimension granularities
- Clean up list dimensions outputs

### Under the Hood

- Adding Changie ([#457](https://github.com/dbt-labs/metricflow/issues/457))
- Ensure use of ValidationIssue instead of ValidationIssueType. ValidationIssueType was from a time before ValidationIssue classes had proper inheritance, and it's continued use was become problematic for typing.
- Removing `model` from the `ModelValidator.validate` return type. The model isn't altered, and thus doesn't need to be returned.
- Moving AggregationType enum into dbt-semantic-interfaces
- Moving errors relevant to dbt_semantic_interfaces to dbt_semantic_interfaces
- Migrating to RapidFuzz ([#470](https://github.com/dbt-labs/metricflow/issues/470))
- Matching dbt-core issue templates ([#457](https://github.com/dbt-labs/metricflow/issues/457))
- Removing the transform CLA ([#450](https://github.com/dbt-labs/metricflow/issues/450))
- Pinning dbt-core to 1.4 ([#475](https://github.com/dbt-labs/metricflow/issues/475))
- Removing YamlLint ([#472](https://github.com/dbt-labs/metricflow/issues/472))
- Add ObjectToReference class in preparation of removing the .reference calls ([##463,](https://github.com/dbt-labs/metricflow/issues/#463,), [##464](https://github.com/dbt-labs/metricflow/issues/#464))
- Moving all *Reference objects to dbt-semantic-interfaces.
- Add pytest flag to use a persistent source schema for faster repeat testing. ([#482](https://github.com/dbt-labs/metricflow/issues/482))
- Renamed instances of and related to Identifiers to Entities. ([#dbt-semantic-interfaces#9](https://github.com/dbt-labs/metricflow/issues/dbt-semantic-interfaces#9))
- Improves typechecking coverage by updgrading to MyPy 0.942 and removing blanket ignore all imports setting ([#536](https://github.com/dbt-labs/metricflow/issues/536))
- Push mypy to run using local environment packages in pre-commit. Developers should always use a clean virtual environment to ensure consistency with CI. ([#530](https://github.com/dbt-labs/metricflow/issues/530), [#536](https://github.com/dbt-labs/metricflow/issues/536))
- Update mypy to 1.3.0 ([#546](https://github.com/dbt-labs/metricflow/issues/546))
- Migrate from Poetry -> Hatch for Project / Package Management ([#549](https://github.com/dbt-labs/metricflow/issues/549))
- Enable the ability to return only dimensions requested in the query, specifically used for dimension values queries.
- Raising a UnsupportedEngineFeatureError instead of a generic RuntimeError when a data platform doesn't support a feature
- Remove SqlIsolationLevel constructs and other vestigial remnants of defunct SqlClient features ([#577](https://github.com/dbt-labs/metricflow/issues/577))
- Raise a more specific exception when a Metric isn't found during linking.
- Update test environment configuration to allow for more streamlined dependencies
- Remove DDL and other unused methods from SqlClient protocol

### Dependencies

- Switches MetricFlow SemanticManifest dependencies from the local dbt semantic interfaces package to the initial dev release ([#540](https://github.com/dbt-labs/metricflow/pull/540))
- Clean up unused dependencies, relax tabulate version pin ([#545](https://github.com/dbt-labs/metricflow/pull/545))
- Update dbt dependencies to support development on the new integration ([#571](https://github.com/dbt-labs/metricflow/pull/571))
- Move SQLAlchemy and SQL engine dependencies out of the production package ([#672](https://github.com/dbt-labs/metricflow/pull/672))
- Update dependencies and attribution file in preparation for 0.200.0 release ([#703](https://github.com/dbt-labs/metricflow/pull/703))

### Contributors
- [@DevonFulcher](https://github.com/DevonFulcher)
- [@QMalcolm](https://github.com/QMalcolm)
- [@WilliamDee](https://github.com/WilliamDee)
- [@callum-mcdata](https://github.com/callum-mcdata)
- [@courtneyholcomb](https://github.com/courtneyholcomb)
- [@nhandel](https://github.com/nhandel)
- [@plypaul](https://github.com/plypaul)
- [@tlento](https://github.com/tlento)
## MetricFlow [0.140.0] - 2023-01-12

### Breaking Changes
Expand Down Expand Up @@ -55,44 +146,37 @@
## MetricFlow [0.130.1] - 2022-10-19

### Added

- Support for overriding dbt `profile` and `targets` attributes when querying dbt models (@QMalcolm)
- Validation to block use of `DISTINCT` keyword in `COUNT` aggregation expressions, as this can lead to incorrect results if optimized queries relying on partial aggregation attempt to do something like `SUM(counts)` to retrieve a less granular total value. (@tlento)

### Updated

- Made minor improvements to safeguards for internal development (@tlento)


## MetricFlow [0.130.0] - 2022-10-18

### Added

- [#292](https://github.com/transform-data/metricflow/pull/292) This is the initial implementation of the MetricFlow dbt metrics integration. Allows MetricFlow to bootstrap itself natively from a dbt project that has metrics 🎉

### Updated

- Refactored the semantic layer to use the MetricReference object as opposed to MetricSpec (@WilliamDee)

## [0.120.0] - 2022-10-13

### Breaking Changes

- Minor API change on SqlClient protocol could break existing API users writing custom SqlClient classes. MyPy should detect this issue. See [the relevant PR](https://github.com/transform-data/metricflow/pull/282) for details.

### Added

- Support for Databricks! Now you can use Metricflow with your Databricks-backed warehouse! (@courtneyholcomb)
- The ability to define constraints on input measures for ratio and expr metric types. This is a temporary quality of life improvement until full-featured derived metrics (i.e., metrics based on other metrics) are available. (@tlento)
- Support for different time granularities, improved group by expression handling, and corrected constraint handling on semi-additive measures (@WilliamDee)
- Support for `count` as a measure aggregation type. Note this is implemented as an alias around `sum`, so use of the `DISTINCT` keyword in expressions is not supported, and will be blocked via validation in a separate update. Users wishing for a `COUNT(DISTINCT thing)` equivalent should continue to use the `count_distinct` aggregation type. (@WilliamDee)

### Fixed

- Resolved incorrect constraint handling with semi-additive measures (@WilliamDee)
- Eliminated Dataclass deserialization errors on default parameters (@tlento)

### Updated

- Optimized multi-hop join candidate selection (@plypaul)
- Improved error handling and error messages on config validation (@QMalcolm, @tlento)
- Streamlined our project README (@nhandel)
Expand All @@ -101,41 +185,36 @@
## MetricFlow [0.111.1] - 2022-08-30

### Added

- A lightweight Visual Studio Code [extension for authoring configs](https://marketplace.visualstudio.com/items?itemName=Transform.metricflow) with simple support for inline schema validation and auto-completion! Local repo changes limited to JSON schema generation. (@jack-transform)
- Metric model template for Shopify metrics - if you use Shopify, this template can get you up and running for tracking your Shopify metrics! (@hesham-nawaz)

### Fixed

- Fixed warehouse validation failures caused by interaction with measure-specific aggregation times (@QMalcolm)
- Resolved intermittent semantic validation failures due to unpicklable pydantic objects (@tlento)
- Fixed edge case with semi-additive measures failing on certain join dimension connections (@WilliamDee)
- Restructured semantic container depedencies to eliminate certain classes of circular import errors (@tlento)

### Updated

- Replaced Pydantic objects with serializable Dataclasses to improve plan building speed. This can reduce time to generate dataflow plans by 10-20x for complex queries. (@plypaul)
- Refactored validation logic to make it easier to reason about (@QMalcolm)
- Modified inference detection rules to make it more likely to classify low cardinality string columns as categorical dimensions (@tlento)


## MetricFlow [0.111.0] - 2022-08-10

### Added

- Early beta for data source inference - Snowflake users can now run a command to bootstrap data source definitions based on warehouse metadata for tables in a given schema. Check out `mf infer --help` for more details. Feedback welcome! (@serramatutu)
- Support for semi-additive measures - semi-additive measures are measures which can be aggregated across some dimensions, but not others. The most common use case for this is an end-of-period measure, such as a statement balance or end of day inventory hold. As of this release we support semi-additive restrictions bound by any time dimension. See the description on [PR #183](https://github.com/transform-data/metricflow/pull/183) for more details on usage. (@WilliamDee)
- Expanded data warehouse validations - data warehoue validation runs will now check to ensure all measures, identifiers, and dimensions defined in data sources are queryable. (@QMalcolm)

### Fixed

- Resolved serialization error on custom Pydantic objects in the model parser (@tlento)
- Cleaned up error message handling and missing test depencies on model validation (@tlento)
- Eliminated a class of circular import errors by forking reference classes from the specs module (@WilliamDee)
- Fixed error state when specifying a where constraint with metric_time (@plypaul)
- Removed unused jsonschema element (@QMalcolm)

### Updated

- Updated the tutorial to use metric_time instead of arbitrary time dimensions (@plypaul)
- Increased strictness of typechecking enforcement (@tlento)
- Removed unnecessary "all time" WHERE clause rendering for cases where time constraints were not provided (@WilliamDee)
Expand All @@ -145,31 +224,26 @@
## MetricFlow [0.110.1] - 2022-07-25

### Fixed

- Cleaned up package metadata and broken links in external README viewers (@tlento)
- Fixed cases where output columns were rendering in the wrong order (@WilliamDee)

### Updated

- Improved model config schema validation to return all jsonschema errors instead of one at a time (@QMalcolm)
- Streamlined rendered SQL to use BETWEEN expressions for date range checks (@WilliamDee)

## MetricFlow [0.110.0] - 2022-07-21

### Breaking Changes

- Updated query inputs for time series dimensions to use `metric_time` instead of dimension names, since it is now possible for measures to have different time dimensions for time series aggregation. This also removes the restriction that all data sources have the same primary time dimension name. However, users issuing queries might experience exceptions if they are not using `metric_time` as their requested primary time dimension. (@plypaul)
- Added enforcement for new reserved keyword `metric_time` (@tlento)
- Reordered column output to `time dimension, identifiers, dimensions, metrics`, which could break automation relying on order-dependent parsing of CLI output. We encourage affected users to switch to using the API, and to access the resulting data frame with order-independent (i.e., by name) access to column values. (@WilliamDee)
- Removed support for SQLite - expected impact is minimal as the repo has been cut to DuckDB for in memory testing (@plypaul)

### Added

- Support for specifying measure-specific time dimensions for time series aggregations (@plypaul)
- Validation against use of SQL reserved keywords as element names, which previously would have resulted in SQL errors at query time (@QMalcolm)

### Updated

- Improved code structure around the model validation process (@QMalcolm)
- Improved readability of model validation errors in CLI output (@QMalcolm)
- Cleaned up DuckDB documentation (@yanghua)
Expand All @@ -182,26 +256,23 @@
## MetricFlow [0.100.2] 2022-06-24

## Updated

- Updated numpy version to 1.23.0 to fix issues with operations on m1 macbooks (@serramatutu)
- Downgraded DuckDB version to 0.3.4 to fix segfaults observed with multithreaded operations (@plypaul)
- Added YAML linting to model validation process to catch syntax errors more explicitly (@QMalcolm)


## MetricFlow [0.100.1] - 2022-06-23

### Updated

- Fixed CLI support for PostgreSQL (@plypaul)
- Improved validation error messaging (@QMalcolm)

## MetricFlow [0.100.0] - 2022-06-22

### Breaking Changes

- Updated MetricFlow config parameters for BigQuery users. See description on https://github.com/transform-data/metricflow/pull/62 for usage instructions.

### Added

- Metric / dimension SQL validations via DW engine (@QMalcolm)
- Support for PostgreSQL and setup for local testing (@rexledesma)
- Support for DuckDB (@plypaul)
Expand All @@ -210,19 +281,18 @@
- Description and owner fields to model objects (@WilliamDee)

### Updated

- CLI tests without mocks (@WilliamDee)
- Package dependencies to address security issues (@alliehowe)
- Query cancellation behavior on CLI exit (@plypaul)

## [0.93.0] - 2022-04-27

### Added

- Simple Developer API for interacting with MetricFlow engine based on a local config file for Warehouse credentials (@williamdee)
- Config validation for Materialization structs (@lebca)
- Config templates for useful metrics sourced from Salesforce data, for use by end users of MetricFlow (@JStein77)


## MetricFlow [0.92.1] - 2022-04-13

### Added
Expand All @@ -237,7 +307,6 @@
- Typing errors in MeasureReference (@lebca)

### Removed

- Remove legacy interfaces (@andykram)
- Remove legacy time_format parameters from test models (@belindabennett)

Expand Down
Loading

0 comments on commit 6f3ccbb

Please sign in to comment.