How do companies like Recorded Future and their competitors scrape and index Dark Web data? While I understand they use NLP to processes and categorize the data, how do they get it in the first place? For example, do they use scripts that work in a similar fashion as ones that would scrape the Clear Net? Do Dark Web (Tor) hidden services employ things like the robots exclusion standard? I’m probably just over thinking this…

Share This Discussion

Leave a Comment

Note: By filling this form and submitting your commen, you acknowledge, agree and comply with our terms of service. In addition you acknowledge that you are willingly sharing your email address with AiOWikis and you might receive notification emails from AiOWikis for comment notifications. AiOWiksi guarantees that your email address WILL NOT be used for advertisement or email marketting purposes.

This site uses Akismet to reduce spam. Learn how your comment data is processed.