“You can’t defend what you can’t see. Threat hunting is how you bring threats into the light.”
What is Threat Hunting?
Threat Hunting is a proactive cybersecurity practice that involves the systematic and continuous search across networks, endpoints, servers, and data repositories to identify and isolate threats that have evaded automated detection mechanisms. It goes beyond passive monitoring by engaging experienced analysts in the deliberate pursuit of indicators of compromise (IOCs), tactics, techniques, and procedures (TTPs), and behavioral anomalies. This proactive approach is grounded in the assumption that no environment is completely secure and that adversaries may already be operating undetected within the organization.
Unlike traditional security monitoring, which relies on predefined rules, signatures, and alerts, threat hunting is hypothesis-driven. Analysts craft hypotheses based on threat intelligence, recent attack trends, or observed anomalies, and then explore system and event data to confirm or refute the presence of malicious activity.
Why is Threat Hunting Important?
Advanced Persistent Threats (APTs): These are stealthy, well-funded, and highly skilled adversaries—often linked to nation-state actors—who aim to infiltrate and remain hidden in a network for extended periods. Traditional tools may miss their subtle footprints, but hunters can identify patterns of persistence and lateral movement that indicate their presence.
Living-Off-the-Land (LoL) Techniques: Modern attackers increasingly use legitimate system tools (like PowerShell, WMI, mshta.exe) to conduct malicious operations, making their activities difficult to distinguish from normal behaviour. Threat hunters analyze behavioural context to expose such misuse.
Limitations of Traditional Security Tools:
SIEMs generate alerts based on known patterns but may not identify new or tailored attacks.
Antivirus solutions rely on known signatures and may not catch polymorphic or fileless malware.
Firewalls focus on ingress/egress filtering and are ineffective against internal lateral movement or credential abuse.
Zero-Day and Fileless Attacks: These attacks exploit unknown vulnerabilities or execute entirely in memory without leaving traditional forensic artifacts. Threat hunting uses behavioural analytics and heuristic investigation to uncover such threats before they cause significant harm.
Core Philosophy: "Assume Breach"
This fundamental principle acknowledges that preventive controls will never be 100% effective. By assuming that a compromise has already occurred, threat hunters shift focus from perimeter defense to in-depth, lateral detection. This change in mindset transforms security operations from passively waiting for alerts to actively seeking adversaries, thereby reducing dwell time and minimizing potential damage.
Old Guard vs New Age: Traditional Detection Meets Threat Hunting
Complementary Practice
Threat hunting augments existing detection by finding unknown threats, fine-tuning detection rules, and feeding back insights for automated prevention.
Threat Hunting Methodology
Threat hunting follows a structured process that guides security analysts from idea to action. The goal is to uncover hidden threats, validate their presence, and feed actionable intelligence back into the security ecosystem.
1. Hypothesis Creation
What it is:
The starting point of every hunt is the formation of a hypothesis — an educated assumption about how an adversary might behave in your environment. This can be based on:
Current threat intelligence
MITRE ATT&CK TTPs (Tactics, Techniques, and Procedures)
Anomalies seen in the environment
Insights from previous incidents
Example Hypotheses:
An attacker may be using PowerShell to perform lateral movement after initial compromise.
There may be attempts to dump LSASS memory to extract credentials using living-off-the-land binaries.
Sources for Hypothesis Ideas:
Threat intelligence platforms (MISP, Threat Intelligence Blogs)
MITRE ATT&CK Navigator
Microsoft Threat Analytics in Defender XDR
Industry reports (e.g., Mandiant M-Trends, Verizon DBIR)
2. Data Collection
What it is:
Once you have a hypothesis, you collect data to support or disprove it. This involves gathering telemetry from a variety of sources. In the Microsoft ecosystem, telemetry is vast and highly correlated.
Data Sources:
Example: PowerShell Activity
DeviceProcessEvents
| where FileName =~ “powershell.exe”
| where ProcessCommandLine has_any (“Invoke-Command”, “EncodedCommand”, “IEX”)
| project Timestamp, DeviceName, AccountName, ProcessCommandLine
3. Data Analysis
What it is:
Data analysis involves querying, filtering, correlating, and visualizing collected telemetry to test the hypothesis. This step aims to separate normal behaviour from potential threat activity.
What to Look For:
Anomalous Behaviour: New or rare processes, strange parent-child relationships.
Unauthorized Access: Account logins from unusual geolocations or devices.
TTP Signatures: Actions that align with known attack behaviours in MITRE ATT&CK.
Tools for Analysis:
Microsoft Sentinel (KQL queries, Analytics Rules)
Defender for Endpoint (Advanced Hunting)
Power BI or Jupyter Notebooks for advanced correlation/visualization
Example: Rare Process Tree
DeviceProcessEvents
| summarize Count = count() by InitiatingProcessFileName, FileName
| sort by Count ascThis query shows rare child-parent process pairs — potential signs of abuse.
4. Investigation
What it is:
Investigation takes leads from the analysis phase and digs deeper to determine whether they represent real threats or benign anomalies.
Steps:
Pivot into contextual logs: Look at host-based evidence, timeline of actions, and affected users.
Check file hashes and IPs: Use threat intel to validate indicators.
Perform triage: Determine impact and urgency.
Microsoft Tools for Investigation:
Defender for Endpoint: Full process tree visualization, file behaviour
Microsoft Sentinel: Incident page, related entities, timeline analysis
Defender for Identity: Shows lateral movement paths and exposed credentials
Example: Investigating Credential Dumping
You find that a process accessed lsass.exe — you investigate:
Was it initiated by a legitimate security tool or a suspicious binary?
Did the binary come from a known software vendor?
Was the account associated with the action privileged?
Example: LSASS Access
DeviceProcessEvents
| where FileName in~ ("procdump.exe", "rundll32.exe")
| where ProcessCommandLine has "lsass"
| project Timestamp, DeviceName, AccountName, ProcessCommandLine5. Response and Feedback
What it is:
The outcome of a hunt should always loop back into the security ecosystem for improvement. Whether the hypothesis is confirmed or not, the insights help sharpen future detection and reduce dwell time.
Key Activities:
Document findings: Describe what was found, how, and why.
Update detection rules: Create new alerts in Sentinel or tune existing ones.
Share IOCs and TTPs: Feed into Threat Intelligence platforms or SOAR tools.
Inform playbooks: Enrich incident response and automation routines.
Example: Convert Hunt to MDE Use case
If your PowerShell hunt surfaces genuine threats, turn the KQL query into an automated usecase
Hunting Styles Decoded
Threat hunting can take multiple forms, each driven by a different set of inputs or strategies. Here are the main types of threat hunting, explained with practical examples and use cases.
1. Intel-Based Hunting
What it is: Intel-based hunting is reactive but valuable. It starts with known indicators of compromise (IOCs) and threat actor behaviours provided by threat intelligence feeds.
Sources:
MISP, AlienVault OTX, Research Blogs
Use Case: A feed reveals a new C2 domain linked to a nation-state campaign.
Hunting Action: Search DNS logs and proxy data to detect any communication attempts to that domain.
Example: Connection to Malicious Domain
DeviceNetworkEvents
| where RemoteUrl in~ ('malicious-domain1.com', 'malicious-domain2.net')
| project DeviceName, RemoteUrl, InitiatingProcessFileNameGoal: Check whether known threats have already targeted or breached your environment.
2. TTP-Based Hunting
What it is: Focused on behaviours, not just signatures. TTP-based hunting uses the MITRE ATT&CK framework to explore tactics and techniques used by adversaries.
Example Techniques:
Credential Dumping
Persistence via Registry Run Keys
Lateral Movement via SMB
Use Case: You want to hunt for evidence of credential dumping via LSASS memory access.
Example:
DeviceProcessEvents
| where ProcessCommandLine has "lsass"
| where FileName in~ ("procdump.exe", "taskmgr.exe", "rundll32.exe")Goal: Surface attacker behaviours that map to known adversary playbooks.
3. Anomaly-Based Hunting
What it is: Anomaly-based hunting involves building baselines of what’s considered “normal” behaviour in your environment and flagging deviations.
Examples:
A user logs in from Russia for the first time.
A new binary executes under svchost.exe.
Example:
SigninLogs
| summarize Logins = count() by UserPrincipalName, LocationChallenge: Requires an understanding of “normal” — which varies per organization.
Goal: Identify previously unknown threats by spotting suspicious behaviour patterns.
4. Machine Learning-Driven Hunting
What it is: Uses statistical models, clustering, or anomaly detection algorithms to find outliers in massive datasets. Typically paired with expert validation.
Example Use Case: Unsupervised model detects a group of endpoints showing unusual SMB traffic volumes after business hours.
Microsoft Tools:
Microsoft Sentinel UEBA (User and Entity Behaviour Analytics)
Defender XDR anomaly models (e.g., unusual process launches, token theft)
Analyst Task: Investigate model-flagged outliers for true positives.
Benefit: Scales across data and highlights subtle signals.
Goal: Spot emerging threats not yet documented in threat intel or known attack patterns.
Data Sources and Tooling
Key Data Sources:
Endpoint Data: Process, Network and File Telemetry
Network Traffic: Firewall logs, DNS logs, proxy logs.
Authentication Logs: Active Directory, Azure AD
Cloud Logs: Office 365 logs, AWS CloudTrail.
Tools Commonly Used:
Microsoft Defender XDR: Unifies data from endpoints, Office 365, and Defender for Identity.
Microsoft Sentinel: SIEM platform with powerful analytics and hunting capabilities.
Velociraptor: Endpoint visibility and artifact collection.
ELK Stack: Open-source log aggregation and hunting platform.
Threat Intel Platforms: MISP, ThreatConnect, OpenCTI
Building a Threat Hunting Program
Developing an effective threat hunting program requires careful planning, the right mix of talent, robust tooling, and continuous measurement. Below are the key pillars to help you build a mature and repeatable threat hunting practice.
1. Define Scope and Objectives
Key Questions:
What types of threats are you trying to uncover? (e.g., insider threats, APTs, credential theft)
Which assets are your crown jewels? (e.g. domain controllers)
Who are your highest-risk users? (e.g. executives, privileged accounts)
Example Objective: "Hunt for signs of lateral movement originating from compromised service accounts in our Azure AD environment."
Benefits: Defining scope ensures hunts are targeted, manageable, and measurable.
2. Assemble a Hunting Team
Roles and Responsibilities:
Threat Hunters: Lead investigations and develop hunting hypotheses.
Threat Intelligence Analysts: Feed current TTPs, APT reports, and campaign data into hunt plans.
Data Scientists (optional): Create ML models for anomaly detection and correlate massive datasets.
Collaboration Tip: Hold weekly hunt planning meetings to align team priorities with business and threat landscape.
3. Establish Data Infrastructure
Core Capabilities:
Centralized Logging: Collect logs from endpoints (Defender for Endpoint), network devices (firewalls, proxies), and cloud sources (Azure, O365).
Data Normalization: Standardize fields (e.g., IPs, usernames, timestamps) for cross-source correlation.
Historical Retention: Maintain at least 90-180 days of data to support retrospective analysis.
Example Tools:
Microsoft Sentinel for long-term log ingestion and correlation.
Azure Data Explorer or Log Analytics Workspace for efficient KQL queries.
Why It Matters: Without accessible, clean data, even the best hunting hypotheses will fail.
4. Use MITRE ATT&CK Framework
Purpose: Use ATT&CK to design structured and repeatable hunts.
Example Mapping:
T1059 – Command and Scripting Interpreter
T1086 – PowerShell
T1071 – Application Layer Protocol
Practical Use: Create a heatmap or matrix of coverage based on techniques relevant to your environment and threat actors.
KQL Example:
DeviceProcessEvents
| where ProcessCommandLine has "Invoke-WebRequest"
| project Timestamp, DeviceName, ProcessCommandLine, InitiatingProcessFileNameInsight: This query targets misuse of PowerShell to download payloads.
5. Measure Effectiveness
Key Metrics:
Number of Hunts Conducted: Indicates operational maturity.
Time to Detection (TTD): Tracks how long adversaries dwell before being identified.
Number of New IOCs Hunted: Reflects the number of IOC’s hunted for in the environment
Threat Hunting with Microsoft Tools
Microsoft’s security stack offers integrated, scalable, and powerful platforms for conducting advanced threat hunting. When used in tandem, Microsoft Sentinel and Microsoft Defender XDR provide a rich set of telemetry, analytical tools, and automation capabilities.
Microsoft Sentinel
Microsoft Sentinel is a cloud-native SIEM platform that supports:
Built-in Hunting Queries: Sentinel includes ready-to-use hunting queries categorized by MITRE ATT&CK techniques. For example:
SecurityEvent | where EventID == 4688 | where CommandLine has "EncodedCommand" | project TimeGenerated, Account, CommandLine, Computer
This detects suspicious PowerShell activity with base64 encoding (T1059, T1086).
Custom KQL Queries: You can write tailored queries to hunt specific behaviours or indicators.
Notebook Integration: Integrate Jupyter Notebooks to enrich hunts with visualization, anomaly detection, and ML models.
Automated Alert Creation: Convert successful hunting queries into analytics rules that trigger alerts or playbooks automatically.
Microsoft Defender for Endpoint
MDE is a powerful EDR tool that offers deep visibility into endpoint activity:
Timeline View: Visualizes process creation, network connections, file modifications, and registry changes on a per-host basis. Ideal for tracing back activity of malware or suspicious scripts.
Threat Analytics: Supplies curated reports on trending vulnerabilities and attacks, helping hunters prioritize.
Attack Surface Reduction Rules: Policies that block vulnerable behaviours, with telemetry to guide hunting (e.g., blocking Office macros).
Microsoft Defender for Identity (MDI)
Defender for Identity focuses on Active Directory threats:
Lateral Movement Paths (LMPs): Maps potential routes an attacker might exploit from one account to domain admin.
Abnormal Domain Activity: Flags behaviours like pass-the-ticket or unusual Kerberos activity.
Example Use Case: A hunter notices multiple failed logins from a workstation and uses MDI to confirm the account was used laterally.
Challenges and Best Practices in Threat Hunting
Key Challenges in Threat Hunting
Data Overload
Security environments today generate massive volumes of data. Without effective normalization and correlation, analysts face alert fatigue and risk missing true positives hidden in noise. For example, SIEMs like Sentinel may process millions of logs per day across endpoints, firewalls, and cloud services—overwhelming unprepared teams.
Lack of Behavioural Baselines
Anomaly-based hunting depends on understanding what is "normal" in the environment. Without defined baselines, identifying deviations becomes subjective. For instance, if remote PowerShell access is rare in a company, a sudden spike might be worth investigating—unless that's expected during system maintenance.
Skills Gap
Threat hunting requires cross-domain expertise—threat intelligence, log analytics (e.g., KQL in Sentinel), system forensics, and attacker tradecraft (e.g., MITRE ATT&CK). Finding analysts who combine these competencies is difficult. Many SOCs struggle to develop in-house talent without formalized training paths.
Best Practices for Effective Threat Hunting
Correlate Across Multiple Data Sources
Use data from endpoints (MDE), identities (MDI), email (Defender for Office 365), and cloud (Defender for Cloud Apps) to strengthen findings.
Example: A phishing email (Defender for O365) drops a payload that triggers unusual network connections (MDE), followed by domain reconnaissance (MDI).
Treat Hunts Like Mini Research Projects
Each hunt should be:
Documented: Hypothesis, queries, results.
Reviewed: Lessons learned.
Operationalized: Turned into detection rules or playbooks.
This transforms tribal knowledge into organizational memory.
Stay Aligned with Current Threats via CTI
Subscribe to commercial and open-source Cyber Threat Intelligence feeds (e.g. MISP, OTX). Align your hunting with:
Trending malware families (e.g., Qakbot, Agent Tesla)
Emerging attack patterns (e.g., FakeCaptcha)
Automate Repetitive Successes
If a hunting query is validated more than once:
Convert it into a rule.
Automate response via playbooks (e.g., isolate device, disable account).
The Horizon of Hunting: Innovations and What Lies Ahead
Emerging Trends in Threat Hunting
Automation and AI-Augmented Hunting
Artificial Intelligence and Machine Learning are transforming how threat hunting is conducted. Tools now incorporate behavioural analytics to highlight outliers—giving analysts a head start in triaging vast datasets. However, AI is not a replacement for human intuition. Instead, it augments decision-making by providing context, clustering anomalies, and suggesting hunting paths.
Example: Microsoft Sentinel’s UEBA (User and Entity Behaviour Analytics) can flag unusual login times or access patterns, which analysts can then investigate using KQL.
Threat Hunting-as-a-Service (THaaS)
With the global skills shortage, many organizations are turning to managed service providers or MDRs (Managed Detection and Response) who offer hunting as a specialized service. THaaS brings:
Expert-level talent
Shared visibility into threats across industries
This democratizes advanced detection capabilities without requiring in-house expertise.
Tighter Integration with Threat Intel and SOAR
Modern hunting workflows are increasingly integrated with:
Threat Intelligence Platforms (TIPs) for enriching hypotheses.
Security Orchestration, Automation, and Response (SOAR) tools for automatic evidence collection and remediation.
Example: A successful hunt in Defender XDR can trigger an action like:
Isolation of the endpoint
Final Thoughts
Threat hunting is a unique blend of science and art. The science lies in understanding telemetry, writing precise queries, and analyzing behaviour. The art lies in curiosity, creativity, and intuition.
As attackers evolve—using stealthy techniques and AI—defenders must evolve faster. Threat hunting empowers defenders to stay ahead by actively surfacing malicious activity.
Whether you’re a large enterprise SOC or a small MSSP, proactive hunting adds deep value:
It reduces dwell time.
Uncovers stealthy attacks before damage.
Builds threat awareness across the team.