Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Output the aggregate result to TiDB #324

Open
wants to merge 1 commit into
base: 5.4.1-post
Choose a base branch
from

Conversation

Ishiihara
Copy link

This is a POC to result to TiDB.

Here are the steps to run the example:

  1. Follow the instructions in tiup.io to deploy a TiDB cluster.
  2. Connect to TiDB
    $mysql --host 127.0.0.1 --port 4000 -u root
  3. Create a table under the test database:
    mysql>USE test;
    mysql>CREATE TABLE wordcount (word VARCHAR(255) NOT NULL, wordcount BIGINT, PRIMARY KEY (word));
  4. Download Kafka, unzip it and in the Kafka directory, start Zookeeper and Kafka
    $bin/zookeeper-server-start.sh config/zookeeper.properties
    $bin/kafka-server-start.sh config/server.properties
  5. in the Kafka directory, create topics:
    $bin/kafka-topics.sh--create --topic streams-plaintext-input --zookeeper localhost:2181 --partitions 1 --replication-factor 1
    $bin/kafka-topics --create --topic streams-wordcount-output --zookeeper localhost:2181 --partitions 1 --replication-factor 1
  6. In the kafka-streams-examples directory, build and package the application
    $ mvn -DskipTests=true clean package
  7. In the kafka-streams-examples directory, run the application
    $java -cp target/kafka-streams-examples-5.4.1-standalone.jar \ io.confluent.examples.streams.WordCountLambdaExample
  8. In the Kafka directory, use the console producer to produce to the input topic
    $bin/kafka-topics.sh --create --topic streams-plaintext-input \ --zookeeper localhost:2181 --partitions 1 --replication-factor 1
  9. Inspect the result
    mysql> SELECT * from wordcount;

@ghost
Copy link

ghost commented Mar 9, 2020

It looks like @Ishiihara hasn't signed our Contributor License Agreement, yet.

The purpose of a CLA is to ensure that the guardian of a project's outputs has the necessary ownership or grants of rights over all contributions to allow them to distribute under the chosen licence.
Wikipedia

You can read and sign our full Contributor License Agreement here.

Once you've signed reply with [clabot:check] to prove it.

Appreciation of efforts,

clabot

@abbccdda
Copy link

Thanks for the PR. But I don't think this has anything to do with TiDB? Any MySQL client could run with it?

@OneCricketeer
Copy link

Agreed. It'd be best to use Kafka Connect JDBC sink for such a use case

Copy link

cla-assistant bot commented Jul 11, 2024

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.


Liquan Pei seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants