Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue Resolved and Library Updated #39

Open
B16f00t opened this issue Dec 3, 2024 · 3 comments
Open

Issue Resolved and Library Updated #39

B16f00t opened this issue Dec 3, 2024 · 3 comments

Comments

@B16f00t
Copy link

B16f00t commented Dec 3, 2024

Description
I'm encountering an issue with the dnsdumpster library while attempting to perform a data lookup. The error occurs when running the following code:

from dnsdumpster.DNSDumpsterAPI import DNSDumpsterAPI
results = DNSDumpsterAPI().search('microsoft.com')
print(results)

Error Message
The generated error is as follows:

Traceback (most recent call last):
File "my_script.py", line 2, in
results = DNSDumpsterAPI().search('microsoft.com')
File "...\site-packages\dnsdumpster\DNSDumpsterAPI.py", line 72, in search
csrf_middleware = soup.findAll('input', attrs={'name': 'csrfmiddlewaretoken'})[0]['value']
IndexError: list index out of range

Observations

The issue seems related to the csrfmiddlewaretoken attribute not being found in the server's response. This might indicate a change in the HTML structure of the website the library relies on.
I tried making a direct request to the website, and it appears the HTML format may have changed.

Request
Is there an update to the library or a temporary fix for this issue?

@iPCNet
Copy link

iPCNet commented Dec 10, 2024

Same issue over here. DNSDumpster has updated their UI and other stuff. That's probably the cause of the issue.
The csrftoken seems to be no longer in the HTML source. Just removing it results in a 403 Status Code. So the new "session-check" must be evaluated (had no time yet to check/fix it myself... just trying to help out over here)

@B16f00t
Copy link
Author

B16f00t commented Dec 10, 2024

Hi everyone,

I’ve completely reworked the library to address the issues caused by recent changes in DNSDumpster’s interface. The script is now fully functional and has been updated to align with the new HTML structure and session requirements.
Main Updates:

Removed reliance on the csrfmiddlewaretoken and implemented a new approach to handle session authentication.
Improved error handling to prevent IndexError and other potential issues.
Added new features:
    Results can now be saved in JSON format, along with support for downloading the network map as a PNG and the host list as an XLS file.
    The script includes commented code for testing and debugging purposes.

Additionally, I’ve attached the updated script as a compressed file in this thread, and I’ll be submitting a pull request soon to share the changes with everyone.

Best regards,

DNSDumpsterAPI.zip

@B16f00t B16f00t changed the title IndexError: list index out of range Issue Resolved and Library Updated Dec 10, 2024
@iPCNet
Copy link

iPCNet commented Dec 11, 2024

Check... Works on my machine :)
Thanks a lot!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants