Setting the Scene
Creating threat detection analytics is a massive topic. To keep things focused, I’m stripping away most of the ‘how-to’ of logic development to highlight one critical problem: un-unified schemas.
In this article I will discuss a very high level version of the detection workflow — enough to demonstrate how un-unified schemas create significant friction, even within a small fraction of the threat detection process.
Malicious Actors
When looking at the creation of threat detection logic, the first question I often ask myself is ‘who is going to attack us?’. You can generally split the answer into two groups — Advanced Persistent Threats (APTs) and everyone else.
When you come to track your groups and attack methods, it quickly becomes apparent that the threat landscape is both vast and diverse. When profiling APTs, the analysis is typically segmented by sector, as these actors often specialise in specific areas. For instance:
- APT5: Telecommunications, Defence, and Aerospace.
- APT39: Travel, IT Services, and Healthcare.
- APT19: Legal, Investment, and Energy sectors.
By identifying which groups historically target your specific industry, you can proactively profile their unique Tactics, Techniques, and Procedures (TTPs). This intelligence-led approach provides a solid foundation for prioritising detection development.
The ‘everyone else’ category consists of actors who are largely industry agnostic. Rather than targeting specific sectors, they prioritise the path of least resistance — unpatched public-facing applications, vulnerable infrastructure, anything that provides easy initial access.
Attack Surface
The next question is ‘how will they attack us?’. By auditing internal application stacks and infrastructure, companies can accurately define the organisation’s attack surface. This mapping provides a clear scope of the exploitable assets and helps prioritise where detection logic is most critical.
Exposure
Here companies are able to combine the malicious actors (the ‘who’) with the attack surface (the ‘how’), and can accurately define their total exposure. This intersection provides the clarity needed to prioritise detection efforts where they are most critically required.
Monitoring Scope
Modern cyberattacks are rarely isolated events — they are multi-stage chains. Once a malicious actor gains a foothold, they move through a sequence of phases, from initial access to lateral movement and data exfiltration.
To build good detection logic and ensure the correct controls are in place, companies should translate the TTPs outlined in frameworks like MITRE ATT&CK into possible attack vectors as a baseline. Each stage of an attack manifests differently depending on the resource being targeted. By identifying which applications — Office 365, AWS, local file servers — are involved in each phase of the lifecycle, companies can ensure detection logic covers the entire breadth of the attack surface.
The Core Challenge
In summary, the challenge is three-fold:
- Companies must defend their ever-expanding infrastructure against a vast landscape of malicious actors using a constantly evolving set of TTPs.
- Security must maintain visibility across the whole estate. Every new resource, application, or asset has a different schema — meaning detection logic is slow to create, inefficient to maintain, and impossible to scale.
- Detecting, responding, automating, and logging an incident becomes so much harder when the underlying data is fragmented.
Traditional Approach
Understanding the need for unified schemas requires looking at how we traditionally handle logs. Logging and monitoring has always been a trade-off between technical capability and budget. The logic seems simple — more monitoring equals less risk. However, the ‘ingest everything’ approach is an expensive and inefficient way to tackle SecOps, and it is the primary reason our data has become so unmanageable.
Here is what un-unified schemas cost each role in the detection pipeline:
- Detection Engineer: Spending excessive time re-writing the same use cases for new applications rather than focusing on innovation. Every new log source requires custom-coded logic, even for standard attacks like brute force.
- SOAR Engineer: Fragile automation playbooks that break whenever a vendor changes a field name. Limited workflow budgets are wasted building duplicate versions of the same process for different tools.
- ML Engineer: 90% of their time is consumed teaching the model that different field names mean the same thing. Whenever you onboard a new application, you have to do it all again — an endlessly repetitive task.
- SOC Analyst: During a live incident, the pressure to manually map an attacker’s lateral movement across different applications leads to inefficiency and lost time.
Working in this type of SOC is hard. Analysts face some of the highest burnout rates in the industry, largely because they sit at the end of the onboarding pipeline — the point where every un-unified schema and fragmented log finally collides. This isn’t a necessary struggle; it is a symptom of broken data.
Derwent Labs Approach
Traditional detection often fails because it happens in silos. Our approach solves this by unifying schemas to create a solid foundation for teams to build upon. When every log source speaks a common language, you can detect and respond to complex attack chains quicker and at a reduced cost. This unification turns fragmented events into a clear narrative, ensuring that your analysts see the whole story of an attack — rather than isolated, un-unified alerts.
Here is what the same pipeline looks like with a unified schema:
- Detection Engineer: Write once and detect everywhere. By using normalised fields, they can point their existing analytics at new data sources instantly, eliminating the need to start from zero.
- SOAR Engineer: One universal playbook can handle an ‘Account Lockout’ across the entire estate. This reduces technical debt and allows the engineer to cover 100% of the infrastructure without exceeding workflow limits.
- ML Engineer: Standardised data allows for true behavioural baselines. The ML can finally connect the dots between disparate events automatically — turning the ML model into a functional, predictive detection engine.
- SOC Analyst: A common language allows for seamless pivoting and clear dashboards. The analyst can focus 100% of their cognitive load on the attacker’s behaviour rather than fighting with the data structure, significantly lowering Mean Time to Triage.
Final Thoughts
Threat detection is the core of a SOC. It is the catalyst for every action, from initial triage to automated response. Everything else we build — our playbooks, our metrics, our machine learning models — rests entirely on the quality of these detections.
By moving toward unified schemas, we aren’t just sanitising log inputs — we are ensuring that the core of our security operations is built on a solid foundation.
In a world of evolving threats, the most effective SOC won’t be the one with the most data, or the newest tools. It will be the one with the most unified vision.
We are so confident that this is the missing link for your SecOps that we will show you exactly how to do this for your environment, completely free. No catch, no consultation fee — we want to show how good detection logic can be when all data is unified.