top of page
Search

Dark Web Searching

Updated: May 11

The dark web is a subset of the internet that is accessed via special means, such as a TOR browser, and not immediately available from the clear net. The term dark web and darknet are often used interchangeably. For reference during this article, we will refer to the darknet as the network infrastructure, such as the TOR network or I2P network, and dark web as the content aspect that is accessed and viewed by users.


There are a lot of great resources that explain what the dark web is, where it originated from and the nefarious activity that occurs there on a daily basis. This article is focused on identifying safe access options and then the multiple search options available using freely available dark web search engines that crawl the dark web.


Investigators looking to conduct traditional search techniques on the dark web need to operate in a safe manner and be aware of the variation in results that are presented by different search engines & also actors who are active in different types of darknets.



Different "Dark Nets"

Most dark web articles refer to The Onion Router (TOR) as it is the most popular and researched. However, it is important to note that there are many darknets and below is an example of three common ones:

Another common darknet is Zeronet. Each has different access requirements or methods.


Different darknet details:


Focusing on TOR, the browser bundle to connect can be downloaded here: https://www.torproject.org/download/


Simply accessing TOR from your standard machine is not advised due to possible security implications. For a lot of users, they will favor ease-of-use over security and connect directly from their standard workstation, but this has serious security considerations. The TOR browser is built on Firefox as a base, and therefore it is subject to the same vulnerabilities that Firefox has. Whilst the Firefox team might patch vulnerabilities regularly, there can be a delay for the update to reach the TOR bundle and therefore exposure users to risks. Given the nature of the content & site hosts on the dark web, this should be a critical consideration so as to not compromise your machine from both an attribution or malware perspective.


It is recommended to apply safe connection methods so as to protect your attribution and host machine from compromise.


Safe Browsing Options

There are many opinions and options for how to access darknets. Below is a simple chart for three options that you can use when connecting to a darknet to provide a safer level of protection. Each has varying barriers to entry and users will have different requirements, budgets, or considerations as part of their connection approach.


Option 1:

Option 2:

  • Install and configure a local virtual machine using a platform such as VirtualBox (https://www.virtualbox.org/) and downloading pre-configured VM's (such as the TL VM: https://www.tracelabs.org/initiatives/osint-vm) or installing an operating system from scratch

  • Install TOR/darknet access on the virtual machine

  • Configure a VPN on your standard workstation

  • Connect to the darknet from within the virtual machine

Option 3:

  • Provision a standalone research laptop/computer (consider using bootable operating systems such as Tails for lower-attribution)

  • Configure a VPN on your research laptop

  • Install TOR/darknet access natively on the research laptop

  • Connect to the darknet natively from your research laptop


Disclaimer: anything you view on the darknet that is rendered locally can still be stored in local caches on any of the options above. You must consider the legal aspects of what you are viewing in the context of your respective governing laws. OSINT Combine takes no responsibility for the content viewed or access methods detailed above.



Dark Web Searching

The dark web is crawled and indexed from numerous non-standard providers, i.e. your traditional search engines such as Google and Bing will not crawl .onion sites on the TOR network. However, proxied TOR sites, being those which use TOR2WEB type services to allow users to view dark web sites from their standard clear web browser, are regularly indexed on Google, although it is not advisable to access these through a proxy for attribution reasons.


Search engines routinely change .onion addresses or go up/down. If any of the following links become unavailable, or you are looking for search engine URLs, a great site is https://onion.live/ - simply search for the search engine name here to find its URL.


Alternatively, there are many "hidden" wiki sites that provide a catalog of resources that also include active search engine URLs. Below is a sample of some of the search engines available.


Dark Web Search Engines: (descriptions obtained from the host websites)


Not all search engines are created equal. They vary significantly with results (due to the sites that are crawled with their platforms), advanced search options (such as boolean or multilingual searching), and intent. Investigators will ideally look across multiple search engines as part of their investigation and compare results, or target their searching to search engines that provide a particular service.


Onion Watching

Some investigators will have a requirement to identify and monitor new .onion sites as they arise. This could be to observe patterns, identify new vectors, or simply to create additional pipelines of new .onion URLs to feed into custom crawling engines for advanced users.


There are three good resources that can support this requirement:



Information Slippage

Investigating people on the dark web usually comes down to attribution between the surface and dark web through information slippage. This is where the same attributable markers, e.g. usernames, PGP keys, cryptocurrency addresses, are used by actors on both the surface and dark web.


When trying to attribute users participating in transactions or activity on the dark web, the information slippage is more often tied to poor habits. Below is a diagram that outlines how information propagated and shared between the surface and dark web can be used conceptually to conduct attribution:

A key consideration is the social network analysis aspect once you have identified markers on the surface web. Social groups cluster based around interests or direct associations, mapping out networks, and then conducting language and image analysis of content posted can provide valuable clues to help with attribution. There is no silver bullet and plenty of false positives, so being thorough and diligent is important in your investigation.


When trying to deanonymize and identify hosts of sites, this may require more technical investigations into SSL certificates. An introductory guide to this can be found at Hunchly (https://www.hunch.ly/resources/Hunchly-Dark-Web-Setup.pdf)


Light Up the Dark (Web)- Uncover More with NexusXplore NexusXplore is the world’s premier OSINT platform, delivering scale, efficiency, and speed to the modern analyst working in today's complex information landscape. NexusXplore contains a comprehensive dark web search and investigative functionality, allowing analysts to shine a light into the deepest and darkest recesses of the online environment.

NexusXplore’s dark web capability provides analysts with the following benefits:

  • Quickly and safely investigate the dark web environment with a single button-click. NexusXplore removes the need for dedicated laptops, misattribution infrastructure or configured virtual machines, which take time to establish and maintain.

  • Seamlessly pivot from dark to deep or surface web within the same pane of glass to identify information slippage and drag dark web actors into the light – for example, investigating linked Telegram channels, or identifying further online presence and biographical information tied to user handles.

  • Access historical dark web posts and pages, allowing for access to valuable information which may have since been taken down or altered.

  • Advanced filtering options, boolean-enabled search functionality, and translation capability allows rapid identification of actionable information.

  • A true dark web search functionality: explore the lesser-known dark nets such as i2p and FreeNet which are often overlooked in investigations due to the higher barrier to entry.

With NexusXplore, you have two different search options at your fingertips:

  1. Text-only: cut through the noise and reduce your team’s exposure to vicarious trauma by retrieving sanitised, text-only results

  2. Live Tor browsing: investigate dark web actors and networks in their native habitat safely and securely using our integrated, sandboxed, and anonymised Tor portal

This article aimed to provide readers with a basic understanding of how to conduct dark web searches using only free resources. One of the primary takeaways from this discussion is the importance of pursuing various search options to increase the likelihood of uncovering relevant leads. Additionally, it is crucial to establish an efficient workflow to prevent feeling overwhelmed by the vast amount of information available on the dark web.


By keeping these key points in mind, individuals can conduct successful investigations even without access to paid resources. However, if you want to know more about how NexusXplore can build safety, scale and efficiency into your dark web investigations and research, please contact us for a discussion and demonstration of our world-leading OSINT solution.


Alternatively, if you are interested in more detailed dark web training, please take a look at our online, self-paced OSINT Combine Academy at https://academy.osintcombine.com or contact us to learn about our bespoke training offerings.

bottom of page