Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update/Fix for the Issue: Not able to scrape jobs list because of depreciated selenium function #190

Open
HaniaArif opened this issue Sep 14, 2023 · 0 comments

Comments

@HaniaArif
Copy link

Code used:

  from linkedin_scraper import JobSearch, actions
  from selenium import webdriver
  
  driver = webdriver.Chrome()
  email = "[email protected]"
  password = "password123"
  actions.login(driver, email, password) # if email and password isnt given, it'll prompt in terminal
  input("Press Enter")
  job_search = JobSearch(driver=driver, close_on_complete=False, scrape=False)
  # job_search contains jobs from your logged in front page:
  # - job_search.recommended_jobs
  # - job_search.still_hiring
  # - job_search.more_jobs
  
  job_listings = job_search.search("Machine Learning Engineer") # returns the list of `Job` from the first page

Error:

Traceback (most recent call last):
File "C:/Users/mz/Downloads/linkedin_test.py", line 25, in
job_listings = job_search.search("Machine Learning Engineer") # returns the list of Job from the first page
File "C:\Users\mz\AppData\Local\Programs\Python\Python310\lib\site-packages\linkedin_scraper\job_search.py", line 89, in search
job = self.scrape_job_card(job_card)
File "C:\Users\mz\AppData\Local\Programs\Python\Python310\lib\site-packages\linkedin_scraper\job_search.py", line 39, in scrape_job_card
company = base_element.find_element_by_class_name("artdeco-entity-lockup__subtitle").text
AttributeError: 'WebElement' object has no attribute 'find_element_by_class_name'

Solution:

In linkedin_scraper\job_search.py do the following updates:
At line 39 and 40 change:
company = base_element.find_element_by_class_name("artdeco-entity-lockup__subtitle").text
location = base_element.find_element_by_class_name("job-card-container__metadata-wrapper").text
to this:
company = base_element.find_element(By.CLASS_NAME,"artdeco-entity-lockup__subtitle").text
location = base_element.find_element(By.CLASS_NAME,"job-card-container__metadata-wrapper").text

Result:

Code works and scrapes data for the job as follows:
[<Job (Senior) Software Engineer – ML/DL Algorithms for Autonomous Driving (m/f/d) Embedded AI Solutions>, <Job Computer Vision Engineer Motive>, <Job Senior iOS Developer Coder Crew>, <Job AI-Augmented Software Developer, Trilogy (Remote) - $200,000/year USD Crossover>, <Job ML Engineer/Data Scientist Turing>, <Job Machine Learning/Data Science Engineer Arbisoft>, <Job Full Stack Developer Voyager Global Mobility>, <Job Principal AI Engineer CureMD>, <Job Big Data Engineer Daraz>, <Job Senior Software Engineer Stewart Pakistan Private Limited>, <Job Machine Learning Engineer Turing>, <Job Senior Data Engineer Systems Limited>, <Job Senior Solutions Engineer, ML/AI DigitalOcean>, <Job DevOps Engineer A4767 Nisum>, <Job Lead Data Engineer Teradata>, <Job Senior Big Data Engineer Daraz>]

@HaniaArif HaniaArif changed the title Not able to scrape jobs list because of depreciated selenium function Update/Fix for the Issue: Not able to scrape jobs list because of depreciated selenium function Sep 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants
@HaniaArif and others