🎉 New Source: Salesforce Marketing Cloud (Singer) #9080
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
|
marcosmarxm
changed the title
…irbytehq#8294) * SAT: Add assert that output catalog does not have any in it * SAT: update code after review, add test parametrizes
…st one data point (airbytehq#8272) * Implement vlidatation if each field in a stream has appeared it least once in some record. * Add unit tests for `_validate_empty_streams` TestBasicRead method. * Add validate_data_points basic read input option. * Update `_validate_field_appears_at_least_once_in_stream` to support oneOf/anyOf choices. Add oneOf/anyOf unit tests. * Bump docker version. Update CHANGELOG.md * Fix test_core.py imports.
…q#8329) * ignored fields added * remove 'skip_dynamic_fields' Signed-off-by: Sergey Chvalyuk <[email protected]>
…8354) * Change synchronisationProgress to synchronizationProgress * Change locale key
…irbytehq#8140) * make spec field required * remove spec backfill logic * remove usages of specFetcher.getSpec() * remove unused code and the caching scheduler clients * fix tests to work with fetching specs from definitions * fetch spec from definition in config repository and fix RunMigrationTest * remove unused SpecFetcher methods/tests * run gw format * run gw format * undo change to main method * add back newlines * set additional properties to true on destination definition * remove now-unused VERSION_0_32_0_FORCE_UPGRADE env var
…iant date string (airbytehq#5814) * added format_reference.txt, edited edge_case_messages.txt, edge_case_catalog.json Co-authored-by: Yurii Bidiuk <[email protected]>
* 'Boards' stream returns only specific (board_ids) records * github secret -> google secret Signed-off-by: Sergey Chvalyuk <[email protected]>
* Remove unique key on exploded nested tables * un-nest hint * Regen SQL
* allow retrieving deleted sources and destinations * fix test
Co-authored-by: sherifnada <[email protected]>
Co-authored-by: jrhizor <[email protected]>
* Fix incremental with empty final tables * upgrade docker images * Regen SQL * Bumpversion & format
…ally_large_values (airbytehq#8264) * Commented incremental tests to skip them in SAT * limit the slice range with end_date * added unit tests for stream_slices * changed the connector version * changed reason of the comment yml * use freezgun in unit tests * formatted code * updated version in spec yaml Co-authored-by: Auganbay <[email protected]>
* Make all these public so Cloud can reuse these helpers. * Reorganise the env var page for better readability. * Format. * Reorganise the EnvConfigs class.
* get date-time format form json schema * created universal date-time converter * implemented jsonnode transformation for avro and parquet * removed unneeded dependency from build.gradle * fix checkstyle * add DateTimeUtilsTest * add AvroRecordHelperTest * resolve merge conflicts | fix checkstyle * update LocalTime parsing * added String type to avro schema for Logical Types, removed date-time conversion * fix checkstyle * fix checkstyle * added static String schema, added comments * bump version * Bump GCS version with avro/parquet timestamp conversion * update docs * update docs Co-authored-by: vmaltsev <[email protected]>
) Co-authored-by: ChristopheDuong <[email protected]>
…sts (airbytehq#8840) * fix for jdk 17 * Destination MariaDbColumnstore updated Docker image * update Docker image for ssh tests Co-authored-by: vmaltsev <[email protected]>
…irbytehq#8699) * Add incremental sync support for companies, contact_lists, contacts, deals, line_items, products, quotes, tickets streams * Updated PR number * Fix formating * Fix typo * Commented unsupported tests * Updated to review * Updated abnormal state file * Deleted comment * Updated version in docker * Updated version in spec yaml and seed
…ytehq#8742) * fix schema for 'breakdowns' and 'action_breakdowns fields Signed-off-by: Sergey Chvalyuk <[email protected]>
* fix logs slicing * add unit test * add more tests * bumb version * send last line if there are not trackback or container error
Signed-off-by: Serhii Chvaliuk <[email protected]>
* Prepend all workspaces resources with /workspaces
* update changelog files * remove mentions to oauth
* feat(zendesk): Add Brands and CustomRoles * feat(zendesk): add incremental unsorted cursor stream implement IncrementalUnsortedCursorStream to ticket_metrics * feat(zendesk): use sorted cursor pagination for ticket comments and macros * feat(zendesk): use unsorted cursor stream for groups, group memberships and satisfaction ratings * fix(zendesk): use safe method to get value from nested dict * style(zendesk): reformat using gradlew * fix(zendesk): format created_at and updated_at to date-time format * feat(zendesk): add business hours schedule * bump connector version * bump dockerfile version Co-authored-by: Marcos Marx <[email protected]>
…data. (airbytehq#8649) * feat(zendesk): Add Brands and CustomRoles * feat(zendesk): add incremental unsorted cursor stream implement IncrementalUnsortedCursorStream to ticket_metrics * feat(zendesk): use sorted cursor pagination for ticket comments and macros * feat(zendesk): use unsorted cursor stream for groups, group memberships and satisfaction ratings * fix(zendesk): use safe method to get value from nested dict * style(zendesk): reformat using gradlew * fix(zendesk): format created_at and updated_at to date-time format * feat(zendesk): add business hours schedule * bump connector version * bump dockerfile version * reset * resolve webapp files Co-authored-by: asyarif93 <[email protected]> Co-authored-by: Marcos Marx <[email protected]>
* it's working * clean up process factory * fix and add tests * clarify airbyte_default * fix build * fix kube acceptance test (maybe) * oops * fix output prop issue * fix output propagation regression * fix kube singleton problem * sync passing on kube but getting wrong exit code of 7 * misc * fix port usage * remove host port that causes conflicts * eliminate envconfigs static usage * this took way too long to figure out * get rid of annoying ==== this is new log messages * finally successfully completing syncs * stop using magic strings and clean up logging * misc minor cleanups * fmt * misc * correct * misc fixes * rename + misc * better logs * logging fix 1 * logging fix 1 -- fixed * finally get logging working nicely * add comment for simplification * fmt * misc * fmt * break into separate class * remove comment * remove flaky multi-node testing * try to fix connector build * remove separate node check * switch to new configs type * fix regression from logging config changes * only log path one time * remove misleading setting terminology * fix connector build * fix earlier merge conflict * fix runtime kubernetes bug * fix connector build (again) * greatly simplify logging config by forcing the container-orchestrator to use default (non-job) logging * add secret insertion for orchestrator on kube * fix k8s ports * add four ports * fix logging test regression * temporarily disable kube tests to check logging * improve comments * make Docker run more secure by limiting env vars transferred * re-enable kubernetes tests * fix conflict * fix docker launching * revert temporal hacks * match master * fix typo * remove completed todo * fix conflict * increase memory requirement to something reasonable * Update airbyte-container-orchestrator/Dockerfile Co-authored-by: Davin Chia <[email protected]> * Update airbyte-container-orchestrator/Dockerfile Co-authored-by: Davin Chia <[email protected]> * see if this stabilitizes tests * address review comments * bump new container version * revert temporary addition * change port back to 9000 * make re-initialization actually a no-op * add feature flag * fix version from merging earlier * fix dockerfile * fix connector build * fix * bump node version * fix dockerfile Co-authored-by: Davin Chia <[email protected]>
explore all the nodes to mask secrets, even if they are arrays or nested at a given deep level. It makes what has been made in airbytehq#8859 more generic
* Implement destination null * Update existing testing destinations * Merge in logging consumer * Remove old destination null * Add documentation * Add destination to build and summary * Fix test * Update acceptance test * Log state message * Remove unused variable * Remove extra statement * Remove old null doc * Add dev null destination * Update doc to include changelog for dev null * Format code * Fix doc * Register e2e test destination in seed
* Extend bootloader for Cloud. * Add test that postLoad hook executes.
* Bump Airbyte version from 0.34.1-alpha to 0.34.2-alpha Co-authored-by: Davin Chia <[email protected]>
* Fix documentation url * Update pr url * Update doc * Bump version
* Add skeleton to support all postgres types * Consolidate type tests * Fix corner cases * Bump postgres version * Add tests for time and timetz * Format code * Revert date to timestamp * Update comment * Fix unit tests * 🐛 Jdbc sources: switch from "string" to "array" schema type for columns with JDBCType.ARRAY (airbytehq#8749) * support array for jdbc sources * fixed PR comments, added test cases * added more elements for test case * Fix test case * add array test case for JdbcSourceOperations Co-authored-by: Liren Tu <[email protected]> * Revert changes to support special number values Postgres source cannot handle these special values yet See https://github.com/airbytehq/airbyte/issues/8902 * Revert infinity and nan assertion in unit tests This reverts commit 3bee7d1. * Update documentation * Bump postgres source version in seed Co-authored-by: Yurii Bidiuk <[email protected]>
* move webapp tests module
add line rem line
|
Hey, @schlattk looks something went wrong can you fix it. |
|
hi @harshithmullapudi why is that PR blocked? Is it because of the missing test account or is there another issue? |
|
Hey I could see 2103 commits and I was not able to understand the PR so I couldn't move forward is it possible to fix it ? |
|
@harshithmullapudi I have created a new clean PR, hopefully that works now |
40 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What
adding a new source connector for salesforce marketing cloud (tap-exacttarget)
How
using a singer tap
Recommended reading order
x.javay.python🚨 User Impact 🚨
Are there any breaking changes? What is the end result perceived by the user? If yes, please merge this PR with the 🚨🚨 emoji so changelog authors can further highlight this if needed.
Pre-merge Checklist
Expand the relevant checklist and delete the others.
New Connector
Community member or Airbyter
airbyte_secret./gradlew :airbyte-integrations:connectors:<name>:integrationTest.README.mdbootstrap.md. See description and examplesdocs/SUMMARY.mddocs/integrations/<source or destination>/<name>.mdincluding changelog. See changelog exampledocs/integrations/README.mdairbyte-integrations/builds.mdAirbyter
If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.
/test connector=connectors/<name>command is passing./publishcommand described hereUpdating a connector
Community member or Airbyter
airbyte_secret./gradlew :airbyte-integrations:connectors:<name>:integrationTest.README.mdbootstrap.md. See description and examplesdocs/integrations/<source or destination>/<name>.mdincluding changelog. See changelog exampleAirbyter
If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.
/test connector=connectors/<name>command is passing./publishcommand described hereConnector Generator
-scaffoldin their name) have been updated with the latest scaffold by running./gradlew :airbyte-integrations:connector-templates:generator:testScaffoldTemplatesthen checking in your changesThis change is