Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add twitterapi.io search plugin #1641

Open
wants to merge 22 commits into
base: develop
Choose a base branch
from

Conversation

kaitoInfra
Copy link

@kaitoInfra kaitoInfra commented Jan 1, 2025

This commit adds a new plugin for searching Twitter content without login. The plugin supports complex queries and integrates with the existing API.

Relates to:

Closes #1431

Risks

Low: 1.The plugin has been extensively tested on Apify with millions of executions, ensuring stability and reliability.
2.After adding the current plugin, there is no place to call it yet. The parts involving other modules will be submitted later. For now, please review this plugin first.

Background
What does this PR do?
This PR introduces a new plugin for fetching Twitter data reliably and efficiently. It addresses the limitations of using personal Twitter accounts for scraping, such as rate limits and account bans, by leveraging a proven solution that has already processed millions of requests on Apify.

What kind of change is this?

Features: Adds a new plugin for Twitter data fetching.

AI agents require large amounts of high-quality, timely data to function effectively. The intelligence of an AI agent depends on two key factors:

The capabilities of the underlying AI model.

The quality, quantity, and timeliness of the data it processes.

Using personal Twitter accounts for scraping data poses significant risks, including account bans and rate limits. This plugin provides a reliable, scalable, and high-performance solution for fetching Twitter data, which has already been validated through millions of executions on Apify.

Additionally, I am offering free usage credits to help developers get started quickly and contribute to the project.

Documentation changes needed?
I added a new environment variable and have updated the documentation in all relevant languages. I also explained the purpose of this variable.
TWITTER_API_IO_KEY=
#A third-party Twitter API key for data retrieval. This shared key is for testing only - get your personal key at https://twitterapi.io to avoid QPS limitations

In the src/tests directory under my newly added component folder, you can find all the unit tests for the APIs. I ran the command pnpm --filter @elizaos/plugin-twitter-search test and confirmed that all unit tests passed successfully.

Detailed testing steps
pnpm --filter @elizaos/plugin-twitter-search test

Verify that the plugin can fetch Twitter data without login.

Test the plugin's performance under high concurrency.

Ensure the plugin handles rate limits and errors gracefully.

Discord username: [kaitoeasyapi]
Twitter: kaitoEasyApi
Apify Twitter Actor: https://apify.com/kaitoeasyapi/twitter-x-data-tweet-scraper-pay-per-result-cheapest

This commit adds a new plugin for searching Twitter content without login.
The plugin supports complex queries and integrates with the existing API.
Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @kaitoInfra! Welcome to the ai16z community. Thanks for submitting your first pull request; your efforts are helping us accelerate towards AGI. We'll review it shortly. You are now a ai16z contributor!

@kaitoInfra kaitoInfra changed the base branch from main to develop January 1, 2025 18:25
@odilitime odilitime added the Plugin_new Mark PRs that are a new plugin label Jan 1, 2025
@odilitime odilitime changed the title Add Twitter search plugin feat: Add twitterapi.io search plugin Jan 1, 2025
@lalalune
Copy link
Member

lalalune commented Jan 1, 2025

nice

@kaitoInfra
Copy link
Author

@odilitime Hi, I've resolved the conflicts in my PR. Could you please help review and merge this PR when you have time? Thank you! 🙏

odilitime
odilitime previously approved these changes Jan 5, 2025
@AIFlowML AIFlowML mentioned this pull request Jan 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Plugin_new Mark PRs that are a new plugin
Projects
None yet
Development

Successfully merging this pull request may close these issues.

DenyLoginSubtask - TWITTER
3 participants