Skip to content

Commit

Permalink
fix: title not found
Browse files Browse the repository at this point in the history
Co-authored-by: CodiumAI-Agent <[email protected]>
  • Loading branch information
david20571015 and CodiumAI-Agent authored Aug 26, 2024
1 parent 258da38 commit 4a90e2d
Showing 1 changed file with 4 additions and 1 deletion.
5 changes: 4 additions & 1 deletion sync_crawler/crawlers/ltn_crawler.py
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,10 @@ def _crawl_news(self, metadata: LtnNewsMetadata) -> News:

soup = bs4.BeautifulSoup(response.text, "html.parser")

title = soup.select("[itemprop=articleBody] > h1")[0].text.strip()
title_elements = soup.select("[itemprop=articleBody] > h1")
if not title_elements:
raise ValueError("Title element not found.")
title = title_elements[0].text.strip()
content = [
tag.text.strip()
for tag in soup.select(
Expand Down

0 comments on commit 4a90e2d

Please sign in to comment.