Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG]: vdb_upload pipeline should be stripping html tags from content #1666

Closed
2 tasks done
dagardner-nv opened this issue Apr 24, 2024 · 1 comment · Fixed by #1670
Closed
2 tasks done

[BUG]: vdb_upload pipeline should be stripping html tags from content #1666

dagardner-nv opened this issue Apr 24, 2024 · 1 comment · Fixed by #1670
Assignees
@dagardner-nv
Labels
bug Something isn't working sherlock Issues/PRs related to Sherlock workflows and components

Comments

@dagardner-nv
Copy link
Contributor

Version

24.06

Which installation method(s) does this occur on?

Source

Describe the bug.

I'm seeing tags like <p>, <strong>, and <h2> tags along with CSS class, id and target attributes in the summary field which is contributing to issue #1650

Minimum reproducible example

python examples/llm/main.py --log_level=warning vdb_upload pipeline --stop_after=4000 --enable_cache

Relevant log output

Click here to see error details
[Paste the error here, it will be hidden by default]

Full env printout

Click here to see environment details
[Paste the results of print_env.sh here, it will be hidden by default]

Other/Misc.

No response

Code of Conduct

  • I agree to follow Morpheus' Code of Conduct
  • I have searched the open bugs and have found no duplicates for this bug report
@dagardner-nv dagardner-nv added the bug Something isn't working label Apr 24, 2024
@dagardner-nv dagardner-nv added the sherlock Issues/PRs related to Sherlock workflows and components label Apr 24, 2024
@dagardner-nv
Copy link
Contributor Author

It looks like the web scraper stage is stripping tags, however these tags are coming in from the RSS feed.

@dagardner-nv dagardner-nv self-assigned this Apr 26, 2024
@dagardner-nv dagardner-nv moved this from Todo to In Progress in Morpheus Boards Apr 26, 2024
@dagardner-nv dagardner-nv mentioned this issue Apr 29, 2024
Merged
@rapids-bot rapids-bot bot closed this as completed in #1670 May 1, 2024
rapids-bot bot pushed a commit that referenced this issue May 1, 2024
@dagardner-nv
Strip HTML & XML tags from RSS feed input (#1670)
9d3de8a 
* Optionally strip HTML & XML tags embedded in RSS feeds

Requires PR #1665 to be merged first
Closes #1666

## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md).
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.

Authors:
  - David Gardner (https://github.com/dagardner-nv)
  - Michael Demoret (https://github.com/mdemoret-nv)

Approvers:
  - Michael Demoret (https://github.com/mdemoret-nv)

URL: #1670
@github-project-automation github-project-automation bot moved this from In Progress to Done in Morpheus Boards May 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dagardner-nv dagardner-nv

Labels
bug Something isn't working sherlock Issues/PRs related to Sherlock workflows and components
Projects
Morpheus Boards
Status: Done
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Strip HTML & XML tags from RSS feed input dagardner-nv/Morpheus
1 participant
@dagardner-nv