A Guide to Open-Source Intelligence (OSINT)

Alec Smith and Stevie Cook

Source Link

OSINT can be traced back to WWII, where allies used newspapers and radio broadcasts for espionage and strategic intelligence gathering (source). Since the establishment of the digital age, the application of OSINT has exponentially evolved. OSINT is no longer exclusive to military and government organisations, with security professionals across multiple sectors harnessing OSINT to inform decision-makers and counter malign activity from threat actors.

1. What is OSINT?

OSINT is the collection and analysis of Publicly Available Information (PAI). OSINT can be derived from various sources such as; media (newspapers, radio, television), social media platforms, information made publicly available by request, academic journals, public events.

2. Why is OSINT important?

OSINT is available to everyone, making it an economically-viable alternative to other more sophisticated intelligence mediums (such as IMINT and GEOINT). It’s also incredibly agile; able to provide real-time event monitoring, and can be quickly refocused as a collection medium with little-to-no impact.

OSINT has also become pivotal in cybersecurity. Security professionals use OSINT to identify weaknesses in friendly networks, which can then be remediated to prevent exploitation by adversaries and threat actors (source). The most common techniques are Penetration Testing (PenTesting) and Ethical Hacking. In this capacity, OSINT also allows cybersecurity professionals to develop a comprehensive threat intelligence to better understand and counter threat actors, their tactics and targets (source).

3. How to apply OSINT

Using search engines, such as Google, will only provide oversight of 5% of internet sources. This area is known as the Surface Web, but there are additional layers such as the Deep Web and the Dark Web. Certain OSINT platforms are able to draw data and information across all 3 layers of the web (source), however, there is no single OSINT tool completely effective as a standalone capability. OSINT tools can identify and index raw data so that it’s quickly searchable and filtered for relevancy (source).

3.1 Overview of OSINT Tools

The application of OSINT is dependent on the desired end-state. There are countless tools available, but here are just a few examples of how you can use various OSINT platforms to glean the relevant information (source).

3.2 Relative Advantages, Disadvantages

In the end, each source available to use has relative advantages and disadvantages. For example, in the realm of flight tracking alone, there are multiple tools we can use to investigate questionable activity in the air. ADSB Exchange, Flight Radar 24, and Radarbox are all popular platforms that allow users to track commercial aircraft in real time. While they share some similarities, there are also several differences between these services that may make one more suitable for certain OSINT investigations than others.ADSB Exchange is a non-profit organization that relies on a network of volunteer feeders to collect and share aircraft tracking data. As such, it tends to have a wider coverage area and more detailed information on military and government aircraft than some of its commercial competitors. However, the data may not be as up-to-date as some other services, and the user interface is less polished.

Flight Radar 24 is a commercial platform that provides real-time flight tracking information for a wide range of aircraft worldwide. It has a sleek user interface, frequent updates, and a range of features that make it easy to filter and analyze the data. However, some users have expressed concern over the accuracy and completeness of the data, particularly in regions with limited coverage or high levels of air traffic.

FR24’s Radar ViewRadarbox is another commercial platform that offers real-time flight tracking information, with a focus on data accuracy and reliability. It includes features such as real-time weather overlays, advanced filtering options, and a comprehensive database of aircraft and airports. However, it may be more expensive than some of its competitors, and its coverage area may be more limited in some regions.

Ultimately, the choice of platform will depend on the specific needs of the OSINT investigation, as well as the user’s personal preferences and budget. It may be useful to test out multiple platforms and compare their features and data quality before making a final decision.

3.2 Practical Application of OSINTDetermine your objectives: Before you start any OSINT investigation, it is important to define your objectives and the questions you want to answer. This will help you stay focused and avoid getting sidetracked by irrelevant information.

Identify your sources: There are a wide range of online sources available for OSINT investigations, including social media platforms, search engines, and public records databases. It is important to identify which sources are relevant to your investigation, and to be aware of the strengths and limitations of each.

Use specialized tools: There are a variety of specialized OSINT tools available that can help streamline the process of gathering and analyzing information. These tools can include everything from web scrapers to network analysis tools.

Verify your information: It is important to verify any information you gather through OSINT, as there is always a risk of misinformation or disinformation. This can involve cross-checking information with multiple sources, or using specialized fact-checking tools.

Stay within ethical and legal boundaries: When conducting OSINT investigations, it is important to respect the privacy of individuals and organizations, and to stay within ethical and legal boundaries. This can involve following data protection regulations, avoiding unauthorized access to private data, and being aware of the potential consequences of your investigations.

4. Tips and tricks for OSINT

Responsible online footprint: all online activity leaves a digital footprint, such as your browser details and IP address, which can be traced back to you. However, there are mechanisms which can mitigate against this. Whilst OSINT is readily available to intelligence analysts, this also means OSINT is also available to threat actors. Minimising your online footprint will also help with tackling counter-collection by a source, whereby programmers of an OSINT tool may be able to collect data on your searches and / or omit information from your searches. There are ways to reduce and protect your digital footprint, starting with (source):Establishing a Virtual Private Network (VPN) to protect your Internet Protocol (IP) address from hackers and advertisement companies. You can check how private or public your IP address is via showmyip.com.Most search engines also have the capability to enable an “incognito mode” to protect you from website cookies. It’s also wise to regularly clear your browser cache. Using fake social media accounts to conduct OSINT investigations and using fake identity generators.

4.1 Framework:

With an overwhelming amount of information available at your fingertips, it’s important to establish a collection and analysis framework to streamline the intelligence process, whilst also preventing burnout. The best way to do this is to define a goal and outline the all the questions you must answer in order to achieve your goal (source).

5. Common mistakes to avoid when using OSINTOSINT (Open Source Intelligence) vs OSINF (Open Source Information): The internet is the largest information resource available to intelligence analysts. However, this information can only evolve into intelligence if it provides analytical value, such as the forecasting of an event, assessment of current situation or corroborating other intelligence disciplines (ie, IMINT, GEOINT, SIGINT, HUMINT). Don’t fall into the trap of regurgitating PAI and assuming this is intelligence. Use our SAT article as guide for effective intelligence analysis.Source credibility, biases and disinformation: unfortunately, not every source is immune to a form of bias (as covered in our SAT article) and disinformation is becoming ever more prevalent online. It’s becoming increasingly difficult to navigate through PAI in order to find credible OSINT, so here’s some practical guidelines you can use (source);Analyse the source: are they obviously partisan or politically motivated? Analyse the sources historical accuracy: is this a new source or do they have a long history in providing credible information?Corroborate: cross-reference your search with other sources, don’t just rely on the first source you find. AI: using this technological advancement can assist in filtering through the “noise” to automatically differentiate between multiple source.

6. Tools and Resources for OSINT

Here is a list of online tools for OSINT that includes IP lookup, web search, public records, flight tracking, and satellite imagery, categorized separately:

IP Lookup:Shodan

Whois Lookup

IPinfo.io

WhatIsMyIPAddress.com

IP2Location

Web Search:Google Search

Bing Search

DuckDuckGo

Yandex

Wayback Machine

Public Records:OpenCorporates

SEC EDGAR

PACER

FRED (Federal Reserve Economic Data)

Govinfo

Flight Tracking:Flight Radar 24

ADSB Exchange

RadarBox

FlightAware

Planefinder

Satellite Imagery:Google Earth

Bing Maps

Mapbox Satellite

Planet Explorer

Sentinel Hub EO Browser

Other OSINT Tools:OSINT Framework

Maltego

Tineye

Reverse Image Search Engines (Google Images, Bing Images)

ZoomEye

FOCA (Fingerprinting Organizations with Collected Archives)

This list is not exhaustive, and there are many other online tools and resources available for OSINT investigations. It is important to keep in mind the ethical considerations and legal restrictions associated with OSINT investigations, and to ensure that any information gathered is obtained legally and used in a responsible and ethical manner. A goof general rule of thumb is that if the police are unable to gather some form of information without a warrant, you as a civilian are certainly not able to do so either.

7. Frequently Asked Questions About OSINT

So who uses OSINT? OSINT is used by a wide range of individuals and organizations including government agencies, law enforcement, corporations, journalists, and researchers. It is particularly useful for those who need to gather information quickly, efficiently, and cost-effectively.

7.1 Security Concerns

Another important question which arises from OSINT is what are the security risks involved with OSINT? There are certainly some security risks involved with Open Source Intelligence (OSINT), and it’s important to be aware of them when conducting OSINT activities. Here are some examples of security risks associated with OSINT:Exposure of Sensitive Information: One of the biggest risks associated with OSINT is the exposure of sensitive information. This can include personal information, trade secrets, or other confidential data that may be inadvertently shared during the course of OSINT activities.Malware and Phishing Attacks: OSINT can involve visiting a wide range of websites, some of which may contain malware or phishing attacks. This can result in the compromise of personal or sensitive information, or the introduction of malware onto the user’s device.Social Engineering Attacks: OSINT can be used to gather information about individuals or organizations that can be used to conduct social engineering attacks. For example, an attacker may use information gathered from social media to craft a convincing phishing email.Legal Issues: While OSINT involves gathering information from publicly available sources, there are still legal risks associated with the practice. For example, accessing certain types of information may violate privacy laws or other regulations.Reputation Damage: Conducting OSINT activities can result in unintended consequences, such as damage to an individual or organization’s reputation. This can occur if information is misinterpreted or shared without proper context.

To mitigate these risks, it’s important to follow best practices when conducting OSINT, such as verifying sources, analyzing data carefully, and protecting sensitive information. It’s also important to use reliable and secure tools and resources when conducting OSINT activities. Finally, it’s important to stay up-to-date on the latest security threats and vulnerabilities associated with OSINT to ensure that risks are minimized as much as possible. A virtual private network can go a long way in mitigating these risks.

8. Advanced techniques for OSINT

The best way to develop and refine your OSINT skillset is to test yourself with challenges, such as these;

You can challenge yourself to find one additional piece of information during a search. For example, if you’re applying for a job, try and find an old version of their website or an old phone number for the company.

Another more niche aspect of OSINT is paying attention to smaller details such as the position of the sun in order to verify the time photo or video were taken. For this, we can use a tool known as SunCalc.

9. Case studies for OSINT

If you’re a Netflix subscriber, “Don’t F*ck With Cats” is one of the best OSINT films around. The 2019 documentary reveals how online sleuths used OSINT tools, such as geolocating social media posts and cross-referencing metadata with Google Maps, to locate a murderer (source, source).

Bellingcat are often regarded as the “gold standard” for OSINT. They regularly publish articles following their online investigations and provide useful advice for amateur OSINTers. Of particular interest is their 10 month joint investigation in uncovering a Russian spy. Using OS data, public archives, FOIA data from Peru, leaked Russian databases and interviews with individuals who had unsuspectingly befriended the GRU agent, Bellingcat and associates discovered how the GRU agent charmed her way into NATO circles in Italy (source).

10. Conclusion

With an increasingly congested information environment, the scope for OSINT has also grown. Immerse yourself into OSINT by conducting searches and developing your own investigation, carefully growing your toolkit as you become more familiar with the multitude of OSINT tools available to you. Ensure you’re cognisant of your own digital footprint and make every effort to mitigate against tracking, hacking and malign actors.