This is not like any other blog you’ve read online. It is a comprehensive blog series that was written carefully with my personal analysis and trail/error kind of experience. You will get deep insights on the internal components of the product so you can know how it thinks. Moreover, I have worked on architecture diagrams that were inspired from my understanding (not found elsewhere online). So keep reading the whole blog series and let me know your feedback.

Read other parts here:

P1: Microsoft Defender for Endpoint – Architecture

P2: MS Defender for Security Strategy & Role of AI

P3: MS Defender for Endpoint – Threat and Vulnerability Management (TVM)

P4: MS Defender for Endpoint – Attack Surface Reduction ASR

P5: Microsoft Defender Antivirus Internal Mechanics

P6: Microsoft Defender Endpoint Detection & Response (EDR)

DISCLAIMER: This content was written for the “Microsoft 365 Security for IT PRO 2020/2021” Edition which talks in great details about the entire security stack for Microsoft 365. Newer version of the book is now released and can be accessed here. I encourage you to download the book to get updated content of defender for endpoint and many other M365 security products.

Introduction

Microsoft Defender for Endpoint is an enterprise endpoint security platform designed to help enterprises prevent, detect, investigate, and respond to advanced threats. It is different than the typical signature-based anti-malware solutions out there as it contains sensors to collect and process behavioral signals from operating systems and uses machine learning to detect suspicious behavior.

Microsoft Defender for Endpoint has many different capabilities and features, including:

• It is embedded into Windows 10 (no additional agent to deploy)
• Support for Windows 7/8 and non-Windows operating systems like Linux, macOS, Android and iOS
• Anti-tampering
• Endpoint Detect and Response (EDR)
• Integration with Microsoft Endpoint Manager
• Vulnerability analysis
• Configuration score capability
• Cross-suite integrations
• Built-in data separation and RBAC
• Deep data collection (up to 6 month of data storage)
• Native integration with Azure AD Conditional Access

In this blog series, I am going to talk about the product from different perspectives. To set the stage for upcoming sections, I will start by exploring the architecture of Microsoft Defender for Endpoint and how all of its components work together to deliver maximum protection.

With that in mind, I will then highlight the main capabilities and features of Microsoft Defender for Endpoint. You will learn that there are a lot of features packed inside the product and that the cloud plays a big role in the overall protection story.

Before going into the details of how each component works, I want to help you understand the need for this product in your organization, and how it fits within a broader endpoint security strategy. You will learn that each component of the product serves a specific goal and covers an important risk.

I will also talk about how Microsoft Defender for Endpoint interacts and integrates with other Microsoft security solutions to provide a layered security approach. This might make you re-consider how you plan your defense in depth-strategy, so that it fits today’s new cybersecurity landscape.

Finally, I will also tackle the topic of machine learning. As new security defenses rely heavily on the use of automation and machine learning to defend against sophisticated attacks, it is important that you grasp the concepts and how they are used within the product itself.

What Exactly it is?

Microsoft Defender for Endpoint contains many components (as you can see in the figure below) that work together to offer a unique endpoint protection platform (EPP) for your enterprise.

According to Gartner, an EPP is a solution deployed on endpoint devices to prevent file-based malware, malicious scripts, and memory-based threats. In August 2019, Gartner named Microsoft a leader in Endpoint Protection Platforms (EPP) magic quadrant. Check Microsoft’s blog here for more details.

When someone asks me what this product is, I tend to use the below picture and explain the different components listed below. It is not one thing, but multiple components. Each component works with the other to provide comprehensive protection. The cloud is essential peace as you will learn later in this blog, in which signaling and further investigation are carried in the cloud.

What took me time to understand is that Microsoft Defender for Endpoint when deployed in a Windows box, will integrate and use other Windows services and components. For example, Microsoft Defender for Endpoint will use the built-in Microsoft Defender Antivirus for real-time malware protection instead of re-inventing the wheel. It will also integrate with the built-in Attack Surface Reduction capabilities in Windows and add more reporting on the defender dashboard about ASR rules. As you go through my blog series, just remember the following points:

• This product is made of different components. It is not just an EDR or antimalware.
• The product uses the cloud for various reasons. This does not mean it will not function if cloud connectivity is off (more on this later), neither that you need to upgrade your internet link accordingly.
• The product once deployed on Windows, will talk to the different built-in Windows security components and integrate with them, such as Windows Update, Attack Surface Reduction ASR, Defender Antivirus and more.

So what are the different components that this product consists of? If you want to present one slide about this product, then the below picture is all what you need. The product has capabilities that can be classified to (pre-breach, pre-execution, post-execution, and more):

• Hardening and increasing your security posture (pre-breach):
  • Threat and Vulnerability Management (TVM): identifying, assessing, and remediating endpoint weaknesses.
  • Attack Surface Protection: helps in reducing your attack surface and is considered a key protection element especially for disconnected clients (without internet connectivity).
• Real-time protection (pre-execution):
  • Next Generation Protection: behavioral-based heuristics and real-time antivirus solution powered by Microsoft Defender Antivirus.
• Detecting and responding to attacks (post-breach/post-execution)
  • Endpoint Detection and Response (EDR): advanced attack detections and visibility into the full scope of a breach.
  • Auto Investigation and Remediation (AIR): automatically remediate compromised assets.
• Advanced capabilities:
  • Advanced Hunting: for discovering hidden threats and proactively looking for threats.
  • Threat Analytics: stay informed with rich threat intelligence.
  • Threat Experts: get help from world class experts at Microsoft.
• Management and administration:
  • Microsoft Defender portal: monitors and assists in responding to alerts of potential advanced persistent threat activity or data breaches.

Microsoft Defender for Endpoint Architecture

Microsoft Defender for Endpoint is a lot more than a traditional antivirus product. It is a comprehensive solution to protect, detect, automate the investigation of, and respond to threats on endpoints. It is core part of Microsoft 365 Defender.

At first, it might be overwhelming to keep up with all the product components and its capabilities because there are just so many of them: Attack Surface Reduction, Next Generation Protection, Threat and Vulnerability Management, Auto Investigation and Remediation, Threat Experts, the centralized management portal, and more.

To help you understand how the product really works, we first need to understand the different components of the product, what the role of the cloud is, and how Microsoft Defender for Endpoint integrates with other Microsoft security solutions.

Microsoft Defender for Endpoint Components

This is where things become interesting. All what I wanted is to draw a big picture in my head about what this product does and how it thinks. It took me a while indeed. Think of the product as a combination of three components:

Endpoint: security components at the endpoint.

Cloud: Microsoft Defender for Endpoint customer tenant in the cloud (accessible via the Defender portal).

API and Integration: signal sharing with other Microsoft security solutions or third-party systems.

Combining the cloud capabilities with security components that are built-in Windows 10 is what makes this product stand out in the market. This brings up the better together-story we will discuss soon. The platform can inspect a file, send a sample of the file to the cloud, run the file in a sandbox environment in the cloud, and then return the truth about the file back to the endpoint.

In Figure below, you can see the different components of Microsoft Defender for Endpoint. At the far left is the endpoint. Microsoft Defender for Endpoint sensors are built-in to Windows 10; you don’t need to deploy additional agent which significantly reduces the management overhead.

Microsoft Defender for Endpoint uses a lot of the Windows 10 built-in security components for better protections such as:

Microsoft Defender Antivirus. A core component that is used for real-time protection and cloud-based protection. This component includes local ML models, heuristics, behavioral analysis and more. If you blocked a file from within the Microsoft Defender for Endpoint management portal, then Microsoft Defender Antivirus is the service that performs the blocking action.

Attack Surface Reduction. These are technologies that help you reduce the attack surface in your environment (such as Network Protection, ASR rules, Application Control and more).

Built-in Firewall. Filter network data transmissions to and from your Windows systems.

Windows Updates. Used to install security updates and updated definitions.

Note: Although we are talking about Windows 10 in this section, Microsoft Defender for Endpoint is currently also supported on macOS, Linux, Android and iOS.

Sensors. Are built-in Windows 10. They collect cyber telemetry from the endpoint and send it to Microsoft Defender for Endpoint cloud protection services.

Security events and information from Microsoft Defender Antivirus, Attack Surface Reduction, built-in firewall, Windows update and other cyber telemetry from the built-in sensors are sent to your Microsoft Defender for Endpoint tenant in the cloud where information is stored and processed.

The endpoint itself has a lot of security capabilities (heuristics, local ML models and more). Even if it is disconnected from the Internet for some time, you still have good protection. However, Internet connectivity is essential to get the maximum protection offered by Microsoft Defender for Endpoint.

Air-gapped networks: For some devices, direct connectivity to the Internet isn’t always possible or allowed. Even if no direct Internet connectivity is available, Microsoft Defender for Endpoint can provide additional protection as most features can work without access to the cloud as well – though there are limitations. More information on how to deal with such scenarios is described in the following whitepaper: https://techcommunity.microsoft.com/t5/microsoft-defender-atp/protecting-disconnected-devices-with-microsoft-defender-atp/ba-p/500341

Cloud Power Protection

Keeping the above in mind, a totally disconnected endpoint will never have the same level of protection as an Internet-connected endpoint.  Microsoft Defender for Endpoint depends on many cloud features such as cloud-delivered protection and signal sharing for better visibility. No matter how powerful the machine is, there are things that only the cloud can do such as big data analysis, detonation-based ML models, and sharing signals with the Microsoft Intelligent Security Graph (ISG) for more informed decisions.

• There are three scenarios when a client needs to communicate with the cloud:
• When the endpoint encounters a file that it can’t decide for whether it is malicious or not. For example, when a file is seen for the first time. In this case the following happens:
  • The endpoint sends metadata about the file to the cloud.
  • The endpoint might send the file itself to the cloud where it is analyzed further. That analysis might include detonating the file in a sandbox, for example.
• When sending security events and cyber telemetry from sensors and other built-in security components to the cloud.
• When security teams trigger endpoints to perform certain actions such as requesting an investigation package, isolating the device from the network, or running a full on-demand antivirus scan.

Sensors

Each Windows 10 endpoint has built-in sensors that are used by Microsoft Defender for Endpoint. These sensors collect and process signals from the operating systems and use machine learning to detect suspicious behavior (not only relying on signatures). The main sensors used by Microsoft Defender for Endpoint are:

• The Threat and Vulnerability sensor that collects information about Windows 10 configuration, installed software and patches information.
• The EDR behavioral sensor that collects data that can be used for detecting and investigating attacks.

In addition to data that comes from these sensors, each endpoint gathers security-related events from different Windows 10 security components such as:

• Attack surface reduction events
• Exploit protection events
• Hardware-based isolation events
• Application control events
• Network protection events
• Firewall events
• Browser protection events
• Microsoft Defender Antivirus events
• EDR response controller events
• Update service events

Telemetry from sensors and all other security events are then sent to your Microsoft Defender for Endpoint tenant in the cloud. Each customer gets a separate and isolated Microsoft Defender for Endpoint tenant. That data is only accessible by customers through Azure Active Directory and access is audited.

Cloud components

The cloud portion of Microsoft Defender for Endpoint contains many components, including:

• Microsoft Defender Security Portal: the portal used by security admins to access their Microsoft Defender for Endpoint data and to interact with endpoints, investigate potential threats, respond to a breach, or improve their security posture. The security team can also trigger endpoints to perform actions like collecting suspicious sample files, isolating endpoints from the network, or run a full on-demand antivirus scan. Many sub-components also live here (alerts, hunting, reporting, events, actions, TVM, and more).
• Machine Learning and Security Analytics: this is the brain of Microsoft Defender for Endpoint as it includes big data analysis, machine learning models, heuristics, behavioral analysis, detonation-based analysis, and anomaly detection algorithms that detect suspicious and attack-related events in the sensor data. It also includes integration with the Microsoft Security Graph for unique optics to translate behavioral signals into actionable insights, detections, and recommended actions.
• Real-time detections and Non real-time detections
• Detonation Chamber:  a sandbox environment for deep file analysis where customers upload suspicious files and get a readable report back. This is an incredibly powerful, yet safe way to investigate potential threats.
• AutoIR: The Automated Investigation and Response engine that can start automated threat investigations and, if a threat is found, immediately perform the necessary remediation steps.
• Notification services
• APIs: collected data is sent to the Security Graph where it can then be sent out to other security products. MDTP also has its own API which can be used to interact with the service. Amongst other things, the API can be used to read data, stream events from Microsoft Defender for Endpoint  to a SIEM, or perform specific actions on a device, like offboarding it from the service.

For example, sensors at the endpoint can capture whenever a process connects to a web server which drops and then launches an application. Microsoft Defender for Endpoint uses that data and combines it with signals from the Microsoft Intelligent Security Graph to trigger detections. Microsoft Defender for Endpoint augments sensor data with a variety of information about the web server, including IP address reputation as well as Microsoft Defender SmartScreen reputation.

The graph can expand further to cover file prevalence as well as files with similar network activity and other shared behavior. By referencing contextual information available through the Intelligent Security Graph, Microsoft Defender for Endpoint can deliver more reliable verdicts.

Microsoft Defender for Endpoint is also a key component of Microsoft 365 Defender, which combines other Microsoft security services such as Microsoft Defender for Identity, Microsoft Defender for Office 365, and Microsoft Cloud App Security to provide an integrated experience. There are also additional integrations with Azure Sentinel, Azure Defender (formerly known as Azure Security Center), Microsoft Information Protection, and Microsoft Endpoint Manager.

Aside from the built-in integrations, you can integrate Microsoft Defender for Endpoint with your existing security infrastructure by using the rich set of APIs. For example, you can connect your SIEM or ticketing system to your Microsoft Defender for Endpoint tenant or enable your own custom security workflows. You can even add your own threat intelligence by using the API or the console itself.

The importance of connectivity

As outlined in the previous section, parts of the detections of Microsoft Defender for Endpoint rely on the ability to upload signals (telemetry) to the cloud platform so that it can be analyzed for (suspicious) behavior. When no connectivity is available, Microsoft Defender for Endpoint falls back to the offline capabilities, such as the next-gen protection offered by Windows Defender.

In the period a device is offline, telemetry data is cached locally on the device. Once the device is back online, that cached information will be sent to the cloud platform for analysis. That way activities can still be correlated, albeit after the fact. Needless to say: whereas EDR detections typically happen within a matter of seconds to minutes, which is the time it takes to upload data to the Microsoft Defender platform, there won’t be any detections (by the platform) whilst the device is offline. For maximum protection, you would want a device to be online as much as possible.

Unfortunately, attackers are also aware that connectivity – or a lack thereof – can help them to bypass detections, even if it were for only a moment. Consider the following: an attacker somehow manages to successfully gain access to a device. Before launching other attacks, they attempt to create firewall rules which block outbound access to the Microsoft Defender platform. By doing so, they prevent the process(es) to communicate with the online platform, and thus block uploading telemetry data. In the process, they neutralize the advanced detections and increase the chances of remaining undetected. By tailoring the firewall rules, attackers can ensure that only Microsoft Defender for Endpoint is impacted; other outbound connectivity remains unaffected, making it harder for the user to detect that something is amiss.

If anything, the above scenario is a good example of why a defense-in-depth strategy is required. And even though Microsoft Defender for Endpoint has anti-tamper protection capabilities, it doesn’t prevent from (locally) updating firewall rules. Luckily, Microsoft added logic into Windows Defender anti-virus, which will pick up on those changes, throw a toast notification and raise an alert in Microsoft Defender for Endpoint.

Notice the remark about rebooting the computer: in order for Defender to “remediate” the firewall changes, you’ll have to reboot te machine. If the user hasn’t already based on the toast, it would be worth checking as well, just to make sure.

As illustrated in the image below, in the Microsoft 365 Security Center an alert will be raised with the name ‘BlockMsav’ malware was prevented. Unfortunately, the alert doesn’t provide a lot of context about why the alert was raised. Luckily, along with the alert, you’ll find an Incident as illustrated below:

The way that Microsoft Defender for Endpoint reports on offline devices doesn’t make things any easier; at least not when it comes to distinguishing devices that are legitimately offline, vs. devices that have been tampered with. Whenever a device has been offline for 7 consecutive days, its status will update from active to inactive in the portal, regardless of whether this was because the device was turned off, or just stopped reporting in because firewall rules blocked activity.

One way to ensure that attackers can’t abuse the local firewall to prevent Microsoft Defender from communicating with the cloud platform is to monitor the creation of (local) firewall rules. Whenever a change is made to the local firewall, an event 2004 will be posted in the “Microsoft-Windows-Windows Firewall With Advanced Security” event log:

There’s different way you could go about to collect this information. For example, you could use Windows Event Log Forwarding to collect these events remotely and alert whenever a firewall rule change is detected, especially if one involves the blocking of processes related to Microsoft Defender for Endpoint. Another way would be to use the Microsoft Monitoring Agent to collect specific event log locations and send them to an Azure Log Analytics workspace, where these event can be analyzed.

Finally, if you are using a device management solution like Microsoft Intune, you could use the last check-in date of the device to see if a device has been online with Intune, but not with Microsoft Defender for Endpoint. The challenge with this approach is that devices only check-in with Microsoft Intune once in a while (8 hours by default), and thus using this method will set you back several hours. Furthermore, a device that checks in to Microsoft Intune but not Microsoft Defender for Endpoint is not necessarily a compromised device. A discrepancy might just trigger further investigation to figure out if something is really amiss. However, none of these methods are 100% safe: if an attacker can add firewall rules to block traffic from Microsoft Defender for Endpoint, what is there to prevent them to do the same for e.g., the management channel, event log forwarding, etc…?         

Customer Tenant and Data

Built on the Microsoft Azure cloud, each customer has a dedicated Microsoft Defender for Endpoint tenant that is separated from other customers. This means that customers data is isolated and secured within their tenants. That data is only accessible by customers through Azure Active Directory authentication and access is audited. Microsoft Defender for Endpoint data is kept for a maximum of six months. This makes it convenient in case you want to investigate an attack that happened months back.

Retention period: Not all information is kept for 6 months. For example, advanced hunting data is only accessible for up to thirty days. A device’s timeline, on the other hand, can go back six months.

Data isolation and location

Since each customer has a separate Microsoft Defender for Endpoint tenant, data is separated and isolated from other tenants through access authentication and logical segregation. The data itself is stored in one of Microsoft Azure data centers in the European Union, the United Kingdom, or in the United States. Note that, once configured, you cannot change the location where your data is stored.

Microsoft also encrypts your data using 256-bit encryption at the minimum and data is encrypted at rest and in flight.

Note: For more information on the Microsoft Defender for Endpoint certification reports, see Microsoft Trust Center.

Why is data stored in the cloud?

As you know by now, Microsoft Defender for Endpoint relies on the power of the cloud to provide maximum protection. This requires storing security events and cyber telemetry for various reasons, such as:

• To identify indicators of attack (IOAs) – also referred to as Threat Indicators (TIs) – in your organization.
• For triggering alerts when attacks happen in your organization.
• For the Microsoft Defender portal to display different views for various entities such as machines, files, and more. This also powers the investigation engine used to explore the presence of security threats in your organization.
• To power the Advanced Hunting capabilities in Microsoft Defender for Endpoint.

What data is stored in the cloud?

Microsoft Defender for Endpoint stores data about files, process data (running processes), registry data, network data (IP address and ports) and machine details. The information is stored in the backend and used by various features. A large part of the data that is stored is available through the Advanced Hunting feature where you can query data based on the source or type of information. For example, data about files encountered on devices is stored in the DeviceFileEvents table, information about processes in DeviceProcessEvents, and so on.

Read More

Part 2 is now available here.

About this Microsoft Defender for Endpoint Blog Series

During the years, I have worked with many security and Infrastructure services, and I usually don’t find good information in the web on how a product or service works. For me to master a service, I need to learn how it thinks, the internal mechanics, and even how the product group who designed it really thought about different features.

So, I started blogging years back to reflect my understanding and help others find useful information that is not found elsewhere on the internet (at least in one place) and direct from my experience.

This blog series is written after careful consideration and will help you imagine how Defender for Endpoint works from the bottom up. I rarely have time to blog these days, so I might not update the blog on new features. However, the content here will give the information you need to build on top.

CREDITS Big thanks to my friend and fellow Microsoft MVP and RD: Ahmad Nabil who helped me put such content and the Microsoft 365 Security for IT PRO book family who helped in reviewing and editing this chapter. Newer version of the book is available here with updated content and valuable content about other Microsoft 365 security services. Download the new book here.

The post P1: Microsoft Defender for Endpoint – Architecture appeared first on Ammar Hasayen.