Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add a wait before validating ksqlDB recipe tests relying on table output #1329

Merged
merged 1 commit into from
Sep 23, 2022
Merged

add a wait before validating ksqlDB recipe tests relying on table output #1329

merged 1 commit into from
Sep 23, 2022

Conversation

davetroiano
Copy link
Contributor

The handful of tests that query table output tend to be flaky due to the
async nature of ksqlDB. There's no guarantee that table result are
available after corresponding INSERTS. Other recipes that validate
via a stream query like "SELECT ... EMIT CHANGES LIMIT n;" aren't
susceptible to the same test flakiness since they will block on
n rows being emitted..

Description

#1328

Staging Docs

N/A

New tutorial checklist

N/A

@davetroiano davetroiano requested review from a team and colinhicks as code owners July 27, 2022 20:56
@davetroiano davetroiano force-pushed the test-flake-mitigation-for-table-query-validation-recipes branch from 5b6a807 to 5bf1901 Compare July 27, 2022 20:59
@davetroiano
Copy link
Contributor Author

one data point in favor of sleep mitigation: I am able to repro #1328 locally (on 2 of 6 runs) and with this change I haven't hit it in 6 runs

@ybyzek
Copy link
Contributor

ybyzek commented Jul 28, 2022

It would be nice to see these tests pass before committing. If it's blocked by #1327 , perhaps merge and rebase, then rerun this?

@davetroiano
Copy link
Contributor Author

It would be nice to see these tests pass before committing. If it's blocked by #1327 , perhaps merge and rebase, then rerun this?

good plan... FWIW I believe that this would eventually pass if I rerun a few times, because we have had recent builds succeed. But my hope is that once we get these two in, we'll see a downtick in flakes.

@davetroiano davetroiano force-pushed the test-flake-mitigation-for-table-query-validation-recipes branch from 5bf1901 to 2fa7caf Compare July 28, 2022 20:23
@davetroiano davetroiano force-pushed the test-flake-mitigation-for-table-query-validation-recipes branch from 2fa7caf to 0746cce Compare August 26, 2022 12:52
@davetroiano
add a wait before validating ksqlDB recipe tests relying on table output
27ce179 
The handful of tests that query table output tend to be flaky due to the
async nature of ksqlDB. There's no guarantee that table results are
available after corresponding INSERTS. Other recipes that validate
via a stream query like "SELECT ... EMIT CHANGES LIMIT n;" aren't
susceptible to the same test flakiness since they will block on
n rows being emitted.
@davetroiano davetroiano force-pushed the test-flake-mitigation-for-table-query-validation-recipes branch from 0746cce to 27ce179 Compare September 23, 2022 14:30
bbejeck
bbejeck approved these changes Sep 23, 2022
View reviewed changes
Copy link
Contributor

@bbejeck bbejeck left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR @davetroiano

LGTM pending tests passing - hopefully, this will fix the flaky tests

@davetroiano
Copy link
Contributor Author

I stopped build after recipe tests passed

@davetroiano davetroiano merged commit 99e746a into master Sep 23, 2022
@davetroiano davetroiano deleted the test-flake-mitigation-for-table-query-validation-recipes branch September 23, 2022 15:17
davetroiano added a commit that referenced this pull request Sep 23, 2022
@davetroiano @ybyzek
@bbejeck @danicafine @bpurkaystha @nlou9 @viks-confluent @scott-confluent @scoot2006 @dependabot @rafikalid @hartantothio @ashleebeggs @daveklein
Kafka Tutorials Release [September 23, 2022] (#1407)
a8ce1fd 
* Add missing semi colons (#1258)

* Reorder online dating (#1259)

* Reorder messaging modernization (#1260)

* Add Colin to codeowners (#1263)

* Changes for updating tutorials to work with AK 3.1 (#1262)

* Changes for updating tutorial to work with AK 3.1

* Updates gradle fix console consumer script

* Revert to old docker exec command without -T switch

* add waits for streams to start, update AK version, update docker images

* Update deprecated join methods

* Changes for updating gradle version (#1265)

* Remove threads and use MockConsumer.schedulePollTask (#1267)

* Add copy with instructions for recipes (#1271)

* Update promotion title (#1272)

* update ksql with sales_orders_enriched (#1274)

* update ksql with sales_orders_enriched

* remove partition, format

* Rename kafkacat to kcat (#1277)

* Tutorials navigation taxonomy updated (#1264)

* Tutorials navigation taxonomy updated

* Trailing slashes added based on PR comments.

* Dead sub-menu removed

* Few links updated as per the google sheet

Co-authored-by: bpurkaystha <[email protected]>

* Bullets in menu list removed (#1283)

* Tutorials navigation taxonomy updated

* Trailing slashes added based on PR comments.

* Dead sub-menu removed

* Few links updated as per the google sheet

* List does not have any bullets with it

Co-authored-by: bpurkaystha <[email protected]>

* IoT loss tracking ksqlDB tutorial (#1287)

* IoT loss tracking ksqlDB tutorial

* use JSON instead of AVRO in IoT loss tracking recipe

* update comment to trigger build

* fix IoT loss tracking test to ensure only lost devices are marked lost

* Update release process (#1307)

* DP-8085 - Migrate to Semaphore self-hosted agent (#1303)

* DP-8085 - Migrate to Sempahore self-hosted agent

* Update semaphore.yml

Co-authored-by: Sai Vishnu <[email protected]>

* updating swiftype tags -- category -> contentType (#1305)

* updating swiftype tags -- category -> contentType

* updating swiftype tags -- category -> contentType

* remove typo

* updating to use tutorial for both recipes and tutorials

Co-authored-by: scoot2006 <[email protected]>

* Bump kafka from 1.16.3 to 1.17.3 (#1293)

* Bump kafka from 1.16.3 to 1.17.3

Bumps [kafka](https://github.com/testcontainers/testcontainers-java) from 1.16.3 to 1.17.3.
- [Release notes](https://github.com/testcontainers/testcontainers-java/releases)
- [Changelog](https://github.com/testcontainers/testcontainers-java/blob/master/CHANGELOG.md)
- [Commits](testcontainers/testcontainers-java@1.16.3...1.17.3)

---
updated-dependencies:
- dependency-name: org.testcontainers:kafka
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* change deprecated getContainerIpAddress method to recommended getHost

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Dave Troiano <[email protected]>

* increase semaphore timeout to reduce build flakiness (#1322)

* Project generation script fixes (#1320)

* consistent tab spacing in tools scripts

Reformatting was done via `shfmt -i 4 -ln=bash <script>`

* flatten location of confluent.html in kafka-consumer-application

This enables this tutorial to be successfully cloned, since
tools/clone-tutorial.sh expects confluent.html to live directly
inside the tutorial directory. This is the only tutorial structured
in this way.

* non-root harness-runner install (#1331)

* install harness-runner as local user, not system-wide root

* add explicit pyyaml dependency to harness-runner setup.py

This prevents a "ModuleNotFoundError: No module named 'yaml'" error
that is likely only now cropping up with the Python environment
on self-hosted Semaphore agents.

* User feedback added on KT (#1292)

* User feedback added on KT

* - CSS changed

* - CSS updated

* - Feedback button's new design accommodated

* - Code refactored

* - Cleaned up

* - CSS updated

* - cursor pointer added

Co-authored-by: bpurkaystha <[email protected]>

* Use double backslash for escape to fix the build (#1344)

* feat: changes nav item Stream Processing Cookbook to Tutorials (#1346)

Co-authored-by: Khalid RAFIK <[email protected]>

* update jmespath to 1.6.1 (#1352)

* making sure some things arent indexed by swiftype (#1351)

* update deployment instructions (#1355)

* example release PR description so releasers can copy/paste
* change instructions to requiring squash and merge since that's the
  only option allowed

* remove dupe properties explanation (#1359)

* bash syntax highligting (#1357)

* IoT asset tracking ksqlDB tutorial (#1287) (#1309)

* marketing copy edits
* rename "loss tracking" to more familiar "asset tracking"

* adding search link in nav (#1363)

* MINOR: Replace deprectated 'TIMESTAMPTOSTRING' with 'FORMAT_TIMESTAMP' (#1341)

Updated insert statements to use 'UNIX_TIMESTAMP()' to generate dates

This PR proposes to solve the "too old" date issue by using UNIX_TIMESTAMP() to generate dates for inserting the initial data for the tutorial. Additionally, the tutorial uses a deprecated method TIMESTAMPTOSTRING, so it's been replaced with the recommendedFORMAT_TIMESTAMP function.

* KT navigation redesign (#1362)

* Started changing the KT navs

* Nav courses book icon added

* CSS improved

* CSS improved

* - Learn added with the related icons
- Few refactorings of code

* - Few CSS fixed:: opacity removed & display: none introduced
- Dummy `build.html` added for nav

* - Build menu completed

* - Community menu added

* - Docs menu added

* - CSS improved

* - CSS improved

* - Mobile version updated

* - CSS improved for mobile version

* - CSS improved for mobile version

* - CSS improved for mobile version

* - Hamburger menu button hidden in desktop view

* - Tablet & mobile views' problem fixed

* - Quick start button fixed

* - Assets link fixed

* - Padding/font-size issues fixed

* - width fixed according to review from designer

* - New word added conditionally

* - Background circle added on courses header book icon

* - Font size & boldness issue fixed

* - Border radius fixed

* - The alignment fixed

* - The alignment fixed

* - Courses right border radius fixed

* - minimum height constraint removed & few other refactorings of the CSS file

* - Styles changed

* - Unnecessary bottom space removed for mobile views

* - Arrow adjusted

* - CSS updated

* - Courses copy updated

* - Learn copy updates added (CSS will still need to be updated)

* - Few styles improved

* - Copy updates on build & CSS improvements

* - CSS updates

* - Community updates added & CSS improvements

* - Copy updates on Docs

* - CSS improved

* - Icon added

* - New icon added & corresponding changes added

* - CSS improved

* - CSS improved

* - CSS improved

* Courses updated & CSS improved

* Learn updated

* Build updated

* Community updated

* Docs updated

* Border left radius fixed across all the navigations

* Mobile view improved

* Rounded borders added for mobile view

* CSS fixed for tablet view

* Text updated

* CSS updated

* Icon positioning fixed

* Weird symbol ® updated

* Updated nav link

* - CSS & text updates to match the design

* Empty commit

* - Refactored based on PR comments

* - Unused image file removed

* - Refactored based on PR comments

* - Refactored

* - Markup elements fixed

* - Few texts updated & CSS

* - Relative URLs added instead of static URLs for assets

* - Removed unnecessary code

* - Updated the relative URLs

* - Updated the relative URLs

Co-authored-by: bpurkaystha <[email protected]>
Co-authored-by: bpurkaystha <[email protected]>

* - Navigation UI bug fixed for specific viewport width range (#1370)

* Small fixes for Client Section on CCloud, CLI warning (#1372)

* Bump com.google.cloud.tools.jib from 3.2.0 to 3.3.0 (#1367)

Bumps com.google.cloud.tools.jib from 3.2.0 to 3.3.0.

---
updated-dependencies:
- dependency-name: com.google.cloud.tools.jib
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump jackson-core from 2.10.1 to 2.13.4 (#1366)

Bumps [jackson-core](https://github.com/FasterXML/jackson-core) from 2.10.1 to 2.13.4.
- [Release notes](https://github.com/FasterXML/jackson-core/releases)
- [Commits](FasterXML/jackson-core@jackson-core-2.10.1...jackson-core-2.13.4)

---
updated-dependencies:
- dependency-name: com.fasterxml.jackson.core:jackson-core
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump protobuf-java from 3.19.4 to 3.21.6 (#1376)

Bumps [protobuf-java](https://github.com/protocolbuffers/protobuf) from 3.19.4 to 3.21.6.
- [Release notes](https://github.com/protocolbuffers/protobuf/releases)
- [Changelog](https://github.com/protocolbuffers/protobuf/blob/main/generate_changelog.py)
- [Commits](protocolbuffers/protobuf@v3.19.4...v3.21.6)

---
updated-dependencies:
- dependency-name: com.google.protobuf:protobuf-java
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump jackson-annotations from 2.10.1 to 2.13.4 (#1375)

Bumps [jackson-annotations](https://github.com/FasterXML/jackson) from 2.10.1 to 2.13.4.
- [Release notes](https://github.com/FasterXML/jackson/releases)
- [Commits](https://github.com/FasterXML/jackson/commits)

---
updated-dependencies:
- dependency-name: com.fasterxml.jackson.core:jackson-annotations
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* fix sink connector topics to match topic names exactly (#1386)

build failure unrelated, merging as this is a trivial change to commented out recipe code

* Bump lombok from 1.18.22 to 1.18.24 (#1251)

Bumps [lombok](https://github.com/projectlombok/lombok) from 1.18.22 to 1.18.24.
- [Release notes](https://github.com/projectlombok/lombok/releases)
- [Changelog](https://github.com/projectlombok/lombok/blob/master/doc/changelog.markdown)
- [Commits](projectlombok/lombok@v1.18.22...v1.18.24)

---
updated-dependencies:
- dependency-name: org.projectlombok:lombok
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* use SEMAPHORE_GIT_WORKING_BRANCH for deployment paths (#1394)

This is better than SEMAPHORE_GIT_BRANCH for staging PRs because
SEMAPHORE_GIT_BRANCH can be the target (typically master) for PRs.

* Fix for stopping dependabot PRs from deploying (#1390)

* tutorial breadcrumb (#1391)

* Gradle cleanup (#1396)

* install gradle once during build

* standardize examples on latest gradle release (7.5.1)

* use mavenCentral repo instead of deprecated jcenter

* upgrade to gradle 7.5.1

* use `cache has_key` to check whether to install gradle

* convert JavaExec:main to mainClass in build.gradle files

JavaExec:main is deprecated and will be removed in Gradle 8.

* punchlist of recipe fixes (#1393)

* remove `name` property from `CREATE CONNECTOR` statements
* connector name consistency
* whitespace consistency
* topic name casing consistency

* Updating tutorials to use new shadowJar location. (#1365)

* Bump assertj-core from 3.22.0 to 3.23.1 (#1381)

Bumps [assertj-core](https://github.com/assertj/assertj-core) from 3.22.0 to 3.23.1.
- [Release notes](https://github.com/assertj/assertj-core/releases)
- [Commits](assertj/assertj@assertj-core-3.22.0...assertj-core-3.23.1)

---
updated-dependencies:
- dependency-name: org.assertj:assertj-core
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump kantan.csv-enumeratum_2.13 from 0.6.2 to 0.7.0 (#1380)

Bumps [kantan.csv-enumeratum_2.13](https://github.com/nrinaudo/kantan.csv) from 0.6.2 to 0.7.0.
- [Release notes](https://github.com/nrinaudo/kantan.csv/releases)
- [Commits](nrinaudo/kantan.csv@v0.6.2...v0.7.0)

---
updated-dependencies:
- dependency-name: com.nrinaudo:kantan.csv-enumeratum_2.13
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* add a wait before validating ksqlDB recipe tests relying on table output (#1329)

The handful of tests that query table output tend to be flaky due to the
async nature of ksqlDB. There's no guarantee that table results are
available after corresponding INSERTS. Other recipes that validate
via a stream query like "SELECT ... EMIT CHANGES LIMIT n;" aren't
susceptible to the same test flakiness since they will block on
n rows being emitted.

* reorder stream and table inserts in fleet recipe to avoid nulls in join (#1288)

* this mitigates null values from table side of join

* Bump com.github.davidmc24.gradle.plugin.avro from 1.2.1 to 1.5.0 (#1397)

Bumps [com.github.davidmc24.gradle.plugin.avro](https://github.com/davidmc24/gradle-avro-plugin) from 1.2.1 to 1.5.0.
- [Release notes](https://github.com/davidmc24/gradle-avro-plugin/releases)
- [Changelog](https://github.com/davidmc24/gradle-avro-plugin/blob/master/CHANGES.md)
- [Commits](davidmc24/gradle-avro-plugin@1.2.1...1.5.0)

---
updated-dependencies:
- dependency-name: com.github.davidmc24.gradle.plugin.avro
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump avro4s-core_2.13 from 4.0.12 to 4.1.0 (#1399)

Bumps [avro4s-core_2.13](https://github.com/sksamuel/avro4s) from 4.0.12 to 4.1.0.
- [Release notes](https://github.com/sksamuel/avro4s/releases)
- [Commits](https://github.com/sksamuel/avro4s/commits)

---
updated-dependencies:
- dependency-name: com.sksamuel.avro4s:avro4s-core_2.13
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump slf4j-simple from 1.7.36 to 2.0.2 (#1398)

Bumps [slf4j-simple](https://github.com/qos-ch/slf4j) from 1.7.36 to 2.0.2.
- [Release notes](https://github.com/qos-ch/slf4j/releases)
- [Commits](qos-ch/slf4j@v_1.7.36...v_2.0.2)

---
updated-dependencies:
- dependency-name: org.slf4j:slf4j-simple
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump scala-library from 2.13.8 to 2.13.9 (#1402)

Bumps [scala-library](https://github.com/scala/scala) from 2.13.8 to 2.13.9.
- [Release notes](https://github.com/scala/scala/releases)
- [Commits](scala/scala@v2.13.8...v2.13.9)

---
updated-dependencies:
- dependency-name: org.scala-lang:scala-library
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump kafka-avro-serializer from 7.1.0 to 7.1.4 (#1403)

Bumps [kafka-avro-serializer](https://github.com/confluentinc/schema-registry) from 7.1.0 to 7.1.4.
- [Release notes](https://github.com/confluentinc/schema-registry/releases)
- [Commits](https://github.com/confluentinc/schema-registry/commits)

---
updated-dependencies:
- dependency-name: io.confluent:kafka-avro-serializer
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump common-utils from 7.1.0 to 7.1.4 (#1404)

Bumps [common-utils](https://github.com/confluentinc/common) from 7.1.0 to 7.1.4.
- [Release notes](https://github.com/confluentinc/common/releases)
- [Commits](https://github.com/confluentinc/common/commits)

---
updated-dependencies:
- dependency-name: io.confluent:common-utils
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump protoc from 3.20.0 to 3.21.6 (#1374)

Bumps [protoc](https://github.com/protocolbuffers/protobuf) from 3.20.0 to 3.21.6.
- [Release notes](https://github.com/protocolbuffers/protobuf/releases)
- [Changelog](https://github.com/protocolbuffers/protobuf/blob/main/generate_changelog.py)
- [Commits](protocolbuffers/protobuf@v3.20.0...v3.21.6)

---
updated-dependencies:
- dependency-name: com.google.protobuf:protoc
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: Yeva Byzek <[email protected]>
Co-authored-by: Bill Bejeck <[email protected]>
Co-authored-by: Danica Fine <[email protected]>
Co-authored-by: bpurkaystha <[email protected]>
Co-authored-by: bpurkaystha <[email protected]>
Co-authored-by: nlou9 <[email protected]>
Co-authored-by: Sai Vishnu <[email protected]>
Co-authored-by: scott-confluent <[email protected]>
Co-authored-by: scoot2006 <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: khalid RAFIK <[email protected]>
Co-authored-by: Khalid RAFIK <[email protected]>
Co-authored-by: Danica Fine <[email protected]>
Co-authored-by: Hartanto Thio <[email protected]>
Co-authored-by: bpurkaystha <[email protected]>
Co-authored-by: ashleebeggs <[email protected]>
Co-authored-by: Dave Klein <[email protected]>
davetroiano added a commit that referenced this pull request Oct 13, 2022
@davetroiano @ybyzek
@bbejeck @danicafine @bpurkaystha @nlou9 @viks-confluent @scott-confluent @scoot2006 @dependabot @rafikalid @hartantothio @ashleebeggs @daveklein @colinhicks
Kafka Tutorials release [October 13, 2022] (#1422)
3b0a814 
* Add missing semi colons (#1258)

* Reorder online dating (#1259)

* Reorder messaging modernization (#1260)

* Add Colin to codeowners (#1263)

* Changes for updating tutorials to work with AK 3.1 (#1262)

* Changes for updating tutorial to work with AK 3.1

* Updates gradle fix console consumer script

* Revert to old docker exec command without -T switch

* add waits for streams to start, update AK version, update docker images

* Update deprecated join methods

* Changes for updating gradle version (#1265)

* Remove threads and use MockConsumer.schedulePollTask (#1267)

* Add copy with instructions for recipes (#1271)

* Update promotion title (#1272)

* update ksql with sales_orders_enriched (#1274)

* update ksql with sales_orders_enriched

* remove partition, format

* Rename kafkacat to kcat (#1277)

* Tutorials navigation taxonomy updated (#1264)

* Tutorials navigation taxonomy updated

* Trailing slashes added based on PR comments.

* Dead sub-menu removed

* Few links updated as per the google sheet

Co-authored-by: bpurkaystha <[email protected]>

* Bullets in menu list removed (#1283)

* Tutorials navigation taxonomy updated

* Trailing slashes added based on PR comments.

* Dead sub-menu removed

* Few links updated as per the google sheet

* List does not have any bullets with it

Co-authored-by: bpurkaystha <[email protected]>

* IoT loss tracking ksqlDB tutorial (#1287)

* IoT loss tracking ksqlDB tutorial

* use JSON instead of AVRO in IoT loss tracking recipe

* update comment to trigger build

* fix IoT loss tracking test to ensure only lost devices are marked lost

* Update release process (#1307)

* DP-8085 - Migrate to Semaphore self-hosted agent (#1303)

* DP-8085 - Migrate to Sempahore self-hosted agent

* Update semaphore.yml

Co-authored-by: Sai Vishnu <[email protected]>

* updating swiftype tags -- category -> contentType (#1305)

* updating swiftype tags -- category -> contentType

* updating swiftype tags -- category -> contentType

* remove typo

* updating to use tutorial for both recipes and tutorials

Co-authored-by: scoot2006 <[email protected]>

* Bump kafka from 1.16.3 to 1.17.3 (#1293)

* Bump kafka from 1.16.3 to 1.17.3

Bumps [kafka](https://github.com/testcontainers/testcontainers-java) from 1.16.3 to 1.17.3.
- [Release notes](https://github.com/testcontainers/testcontainers-java/releases)
- [Changelog](https://github.com/testcontainers/testcontainers-java/blob/master/CHANGELOG.md)
- [Commits](testcontainers/testcontainers-java@1.16.3...1.17.3)

---
updated-dependencies:
- dependency-name: org.testcontainers:kafka
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* change deprecated getContainerIpAddress method to recommended getHost

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Dave Troiano <[email protected]>

* increase semaphore timeout to reduce build flakiness (#1322)

* Project generation script fixes (#1320)

* consistent tab spacing in tools scripts

Reformatting was done via `shfmt -i 4 -ln=bash <script>`

* flatten location of confluent.html in kafka-consumer-application

This enables this tutorial to be successfully cloned, since
tools/clone-tutorial.sh expects confluent.html to live directly
inside the tutorial directory. This is the only tutorial structured
in this way.

* non-root harness-runner install (#1331)

* install harness-runner as local user, not system-wide root

* add explicit pyyaml dependency to harness-runner setup.py

This prevents a "ModuleNotFoundError: No module named 'yaml'" error
that is likely only now cropping up with the Python environment
on self-hosted Semaphore agents.

* User feedback added on KT (#1292)

* User feedback added on KT

* - CSS changed

* - CSS updated

* - Feedback button's new design accommodated

* - Code refactored

* - Cleaned up

* - CSS updated

* - cursor pointer added

Co-authored-by: bpurkaystha <[email protected]>

* Use double backslash for escape to fix the build (#1344)

* feat: changes nav item Stream Processing Cookbook to Tutorials (#1346)

Co-authored-by: Khalid RAFIK <[email protected]>

* update jmespath to 1.6.1 (#1352)

* making sure some things arent indexed by swiftype (#1351)

* update deployment instructions (#1355)

* example release PR description so releasers can copy/paste
* change instructions to requiring squash and merge since that's the
  only option allowed

* remove dupe properties explanation (#1359)

* bash syntax highligting (#1357)

* IoT asset tracking ksqlDB tutorial (#1287) (#1309)

* marketing copy edits
* rename "loss tracking" to more familiar "asset tracking"

* adding search link in nav (#1363)

* MINOR: Replace deprectated 'TIMESTAMPTOSTRING' with 'FORMAT_TIMESTAMP' (#1341)

Updated insert statements to use 'UNIX_TIMESTAMP()' to generate dates

This PR proposes to solve the "too old" date issue by using UNIX_TIMESTAMP() to generate dates for inserting the initial data for the tutorial. Additionally, the tutorial uses a deprecated method TIMESTAMPTOSTRING, so it's been replaced with the recommendedFORMAT_TIMESTAMP function.

* KT navigation redesign (#1362)

* Started changing the KT navs

* Nav courses book icon added

* CSS improved

* CSS improved

* - Learn added with the related icons
- Few refactorings of code

* - Few CSS fixed:: opacity removed & display: none introduced
- Dummy `build.html` added for nav

* - Build menu completed

* - Community menu added

* - Docs menu added

* - CSS improved

* - CSS improved

* - Mobile version updated

* - CSS improved for mobile version

* - CSS improved for mobile version

* - CSS improved for mobile version

* - Hamburger menu button hidden in desktop view

* - Tablet & mobile views' problem fixed

* - Quick start button fixed

* - Assets link fixed

* - Padding/font-size issues fixed

* - width fixed according to review from designer

* - New word added conditionally

* - Background circle added on courses header book icon

* - Font size & boldness issue fixed

* - Border radius fixed

* - The alignment fixed

* - The alignment fixed

* - Courses right border radius fixed

* - minimum height constraint removed & few other refactorings of the CSS file

* - Styles changed

* - Unnecessary bottom space removed for mobile views

* - Arrow adjusted

* - CSS updated

* - Courses copy updated

* - Learn copy updates added (CSS will still need to be updated)

* - Few styles improved

* - Copy updates on build & CSS improvements

* - CSS updates

* - Community updates added & CSS improvements

* - Copy updates on Docs

* - CSS improved

* - Icon added

* - New icon added & corresponding changes added

* - CSS improved

* - CSS improved

* - CSS improved

* Courses updated & CSS improved

* Learn updated

* Build updated

* Community updated

* Docs updated

* Border left radius fixed across all the navigations

* Mobile view improved

* Rounded borders added for mobile view

* CSS fixed for tablet view

* Text updated

* CSS updated

* Icon positioning fixed

* Weird symbol ® updated

* Updated nav link

* - CSS & text updates to match the design

* Empty commit

* - Refactored based on PR comments

* - Unused image file removed

* - Refactored based on PR comments

* - Refactored

* - Markup elements fixed

* - Few texts updated & CSS

* - Relative URLs added instead of static URLs for assets

* - Removed unnecessary code

* - Updated the relative URLs

* - Updated the relative URLs

Co-authored-by: bpurkaystha <[email protected]>
Co-authored-by: bpurkaystha <[email protected]>

* - Navigation UI bug fixed for specific viewport width range (#1370)

* Small fixes for Client Section on CCloud, CLI warning (#1372)

* Bump com.google.cloud.tools.jib from 3.2.0 to 3.3.0 (#1367)

Bumps com.google.cloud.tools.jib from 3.2.0 to 3.3.0.

---
updated-dependencies:
- dependency-name: com.google.cloud.tools.jib
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump jackson-core from 2.10.1 to 2.13.4 (#1366)

Bumps [jackson-core](https://github.com/FasterXML/jackson-core) from 2.10.1 to 2.13.4.
- [Release notes](https://github.com/FasterXML/jackson-core/releases)
- [Commits](FasterXML/jackson-core@jackson-core-2.10.1...jackson-core-2.13.4)

---
updated-dependencies:
- dependency-name: com.fasterxml.jackson.core:jackson-core
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump protobuf-java from 3.19.4 to 3.21.6 (#1376)

Bumps [protobuf-java](https://github.com/protocolbuffers/protobuf) from 3.19.4 to 3.21.6.
- [Release notes](https://github.com/protocolbuffers/protobuf/releases)
- [Changelog](https://github.com/protocolbuffers/protobuf/blob/main/generate_changelog.py)
- [Commits](protocolbuffers/protobuf@v3.19.4...v3.21.6)

---
updated-dependencies:
- dependency-name: com.google.protobuf:protobuf-java
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump jackson-annotations from 2.10.1 to 2.13.4 (#1375)

Bumps [jackson-annotations](https://github.com/FasterXML/jackson) from 2.10.1 to 2.13.4.
- [Release notes](https://github.com/FasterXML/jackson/releases)
- [Commits](https://github.com/FasterXML/jackson/commits)

---
updated-dependencies:
- dependency-name: com.fasterxml.jackson.core:jackson-annotations
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* fix sink connector topics to match topic names exactly (#1386)

build failure unrelated, merging as this is a trivial change to commented out recipe code

* Bump lombok from 1.18.22 to 1.18.24 (#1251)

Bumps [lombok](https://github.com/projectlombok/lombok) from 1.18.22 to 1.18.24.
- [Release notes](https://github.com/projectlombok/lombok/releases)
- [Changelog](https://github.com/projectlombok/lombok/blob/master/doc/changelog.markdown)
- [Commits](projectlombok/lombok@v1.18.22...v1.18.24)

---
updated-dependencies:
- dependency-name: org.projectlombok:lombok
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* use SEMAPHORE_GIT_WORKING_BRANCH for deployment paths (#1394)

This is better than SEMAPHORE_GIT_BRANCH for staging PRs because
SEMAPHORE_GIT_BRANCH can be the target (typically master) for PRs.

* Fix for stopping dependabot PRs from deploying (#1390)

* tutorial breadcrumb (#1391)

* Gradle cleanup (#1396)

* install gradle once during build

* standardize examples on latest gradle release (7.5.1)

* use mavenCentral repo instead of deprecated jcenter

* upgrade to gradle 7.5.1

* use `cache has_key` to check whether to install gradle

* convert JavaExec:main to mainClass in build.gradle files

JavaExec:main is deprecated and will be removed in Gradle 8.

* punchlist of recipe fixes (#1393)

* remove `name` property from `CREATE CONNECTOR` statements
* connector name consistency
* whitespace consistency
* topic name casing consistency

* Updating tutorials to use new shadowJar location. (#1365)

* Bump assertj-core from 3.22.0 to 3.23.1 (#1381)

Bumps [assertj-core](https://github.com/assertj/assertj-core) from 3.22.0 to 3.23.1.
- [Release notes](https://github.com/assertj/assertj-core/releases)
- [Commits](assertj/assertj@assertj-core-3.22.0...assertj-core-3.23.1)

---
updated-dependencies:
- dependency-name: org.assertj:assertj-core
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump kantan.csv-enumeratum_2.13 from 0.6.2 to 0.7.0 (#1380)

Bumps [kantan.csv-enumeratum_2.13](https://github.com/nrinaudo/kantan.csv) from 0.6.2 to 0.7.0.
- [Release notes](https://github.com/nrinaudo/kantan.csv/releases)
- [Commits](nrinaudo/kantan.csv@v0.6.2...v0.7.0)

---
updated-dependencies:
- dependency-name: com.nrinaudo:kantan.csv-enumeratum_2.13
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* add a wait before validating ksqlDB recipe tests relying on table output (#1329)

The handful of tests that query table output tend to be flaky due to the
async nature of ksqlDB. There's no guarantee that table results are
available after corresponding INSERTS. Other recipes that validate
via a stream query like "SELECT ... EMIT CHANGES LIMIT n;" aren't
susceptible to the same test flakiness since they will block on
n rows being emitted.

* reorder stream and table inserts in fleet recipe to avoid nulls in join (#1288)

* this mitigates null values from table side of join

* Bump com.github.davidmc24.gradle.plugin.avro from 1.2.1 to 1.5.0 (#1397)

Bumps [com.github.davidmc24.gradle.plugin.avro](https://github.com/davidmc24/gradle-avro-plugin) from 1.2.1 to 1.5.0.
- [Release notes](https://github.com/davidmc24/gradle-avro-plugin/releases)
- [Changelog](https://github.com/davidmc24/gradle-avro-plugin/blob/master/CHANGES.md)
- [Commits](davidmc24/gradle-avro-plugin@1.2.1...1.5.0)

---
updated-dependencies:
- dependency-name: com.github.davidmc24.gradle.plugin.avro
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump avro4s-core_2.13 from 4.0.12 to 4.1.0 (#1399)

Bumps [avro4s-core_2.13](https://github.com/sksamuel/avro4s) from 4.0.12 to 4.1.0.
- [Release notes](https://github.com/sksamuel/avro4s/releases)
- [Commits](https://github.com/sksamuel/avro4s/commits)

---
updated-dependencies:
- dependency-name: com.sksamuel.avro4s:avro4s-core_2.13
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump slf4j-simple from 1.7.36 to 2.0.2 (#1398)

Bumps [slf4j-simple](https://github.com/qos-ch/slf4j) from 1.7.36 to 2.0.2.
- [Release notes](https://github.com/qos-ch/slf4j/releases)
- [Commits](qos-ch/slf4j@v_1.7.36...v_2.0.2)

---
updated-dependencies:
- dependency-name: org.slf4j:slf4j-simple
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump scala-library from 2.13.8 to 2.13.9 (#1402)

Bumps [scala-library](https://github.com/scala/scala) from 2.13.8 to 2.13.9.
- [Release notes](https://github.com/scala/scala/releases)
- [Commits](scala/scala@v2.13.8...v2.13.9)

---
updated-dependencies:
- dependency-name: org.scala-lang:scala-library
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump kafka-avro-serializer from 7.1.0 to 7.1.4 (#1403)

Bumps [kafka-avro-serializer](https://github.com/confluentinc/schema-registry) from 7.1.0 to 7.1.4.
- [Release notes](https://github.com/confluentinc/schema-registry/releases)
- [Commits](https://github.com/confluentinc/schema-registry/commits)

---
updated-dependencies:
- dependency-name: io.confluent:kafka-avro-serializer
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump common-utils from 7.1.0 to 7.1.4 (#1404)

Bumps [common-utils](https://github.com/confluentinc/common) from 7.1.0 to 7.1.4.
- [Release notes](https://github.com/confluentinc/common/releases)
- [Commits](https://github.com/confluentinc/common/commits)

---
updated-dependencies:
- dependency-name: io.confluent:common-utils
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump protoc from 3.20.0 to 3.21.6 (#1374)

Bumps [protoc](https://github.com/protocolbuffers/protobuf) from 3.20.0 to 3.21.6.
- [Release notes](https://github.com/protocolbuffers/protobuf/releases)
- [Changelog](https://github.com/protocolbuffers/protobuf/blob/main/generate_changelog.py)
- [Commits](protocolbuffers/protobuf@v3.20.0...v3.21.6)

---
updated-dependencies:
- dependency-name: com.google.protobuf:protoc
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Added click tracking (#1339)

* Added click tracking

* Added comment and remove jquery dependency

* Prioritize segment

* Bump ksql-udf from 5.4.7 to 5.4.10 (#1409)

Bumps ksql-udf from 5.4.7 to 5.4.10.

---
updated-dependencies:
- dependency-name: io.confluent.ksql:ksql-udf
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump protobuf-gradle-plugin from 0.8.18 to 0.8.19 (#1405)

Bumps [protobuf-gradle-plugin](https://github.com/google/protobuf-gradle-plugin) from 0.8.18 to 0.8.19.
- [Release notes](https://github.com/google/protobuf-gradle-plugin/releases)
- [Commits](google/protobuf-gradle-plugin@v0.8.18...v0.8.19)

---
updated-dependencies:
- dependency-name: com.google.protobuf:protobuf-gradle-plugin
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump kantan.csv-java8_2.13 from 0.6.2 to 0.7.0 (#1382)

Bumps [kantan.csv-java8_2.13](https://github.com/nrinaudo/kantan.csv) from 0.6.2 to 0.7.0.
- [Release notes](https://github.com/nrinaudo/kantan.csv/releases)
- [Commits](nrinaudo/kantan.csv@v0.6.2...v0.7.0)

---
updated-dependencies:
- dependency-name: com.nrinaudo:kantan.csv-java8_2.13
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump gson from 2.9.0 to 2.9.1 (#1387)

Bumps [gson](https://github.com/google/gson) from 2.9.0 to 2.9.1.
- [Release notes](https://github.com/google/gson/releases)
- [Changelog](https://github.com/google/gson/blob/master/CHANGELOG.md)
- [Commits](google/gson@gson-parent-2.9.0...gson-parent-2.9.1)

---
updated-dependencies:
- dependency-name: com.google.code.gson:gson
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump avro from 1.11.0 to 1.11.1 (#1406)

Bumps avro from 1.11.0 to 1.11.1.

---
updated-dependencies:
- dependency-name: org.apache.avro:avro
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Revert "Add Colin to codeowners (#1263)" (#1417)

This reverts commit af8b4ae.

* Confluent Parallel Consumer tutorial (#1350)

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: Yeva Byzek <[email protected]>
Co-authored-by: Bill Bejeck <[email protected]>
Co-authored-by: Danica Fine <[email protected]>
Co-authored-by: bpurkaystha <[email protected]>
Co-authored-by: bpurkaystha <[email protected]>
Co-authored-by: nlou9 <[email protected]>
Co-authored-by: Sai Vishnu <[email protected]>
Co-authored-by: scott-confluent <[email protected]>
Co-authored-by: scoot2006 <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: khalid RAFIK <[email protected]>
Co-authored-by: Khalid RAFIK <[email protected]>
Co-authored-by: Danica Fine <[email protected]>
Co-authored-by: Hartanto Thio <[email protected]>
Co-authored-by: bpurkaystha <[email protected]>
Co-authored-by: ashleebeggs <[email protected]>
Co-authored-by: Dave Klein <[email protected]>
Co-authored-by: Colin Hicks <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@bbejeck bbejeck bbejeck approved these changes

@colinhicks colinhicks Awaiting requested review from colinhicks

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@davetroiano @ybyzek @bbejeck