☁️
CTHFM: Azure
  • Welcome
  • Getting Started
    • Account Setup
      • Account Creation Instructions
  • Azure Fundamentals
    • Azure Fundamentals Overview
      • Azure Documentation
      • Azure Entra
      • Azure Hierarchy
      • Identities
      • Azure Entra Roles
      • Azure RBAC
      • Azure Licensing Overview
        • Azure Entra ID Premium Licenses Comparison
      • Azure Shared Responsibility Model
      • Azure Frameworks
      • Azure Security Services
      • Conditional Access Policies
        • How Conditional Access Policies Work
        • Benefits of Conditional Access
        • Best Practices for Conditional Access
        • Conditions for Conditional Access
        • Conditional Access Controls
        • Sign-In Risk and Identity Protection
        • Conditional Access Session Control
        • Name Locations IP Location
      • Azure Quotas
      • Azure Tokens
        • Access Tokens
        • ID Tokens
        • Refresh Tokens
          • Invalidate Refresh Tokens
        • Primary Refresh Tokens
        • Continuous Access Evaluation (CAE)
        • Token Protection: Conditional Access (Public Preview)
  • Azure CLI
    • Introduction to Azure CLI
    • Installing Azure CLI
    • CLI Commands
    • Resource Group Management
    • Creating and Managing Resources with Azure CLI
    • Managing IAM
    • Azure CLI Automation
    • Monitoring and Troubleshooting in Azure CLI
    • Azure CLI Overview
      • Azure CLI: Linux
      • Azure CLI: Windows
      • Azure CLI: MacOS
  • KQL
    • KQL Overview
      • KQL Introduction
      • Data Types
      • Quick Reference
      • Render
      • Basic Queries
  • Powershell
    • PowerShell for Azure
      • Powershell Documentation
      • Powershell Basics
      • Understanding Powershell Variables
      • Understanding Cmdlets
      • Powershell Console & ISE
      • Powershell: Entra Module
        • Module Reference & Getting Started
      • Powershell: Azure Module
        • Installation Instructions
        • Powershell Azure CLI: Windows
        • Powershell Azure CLI: Linux
        • Powershell Azure CLI: MacOS
        • Example Azure Cmdlets
  • Bicep
    • Bicep File Structure
    • Variable Types
  • Azure Logging References
    • Logging
      • Log Retention Strategies
      • Azure Log Types
      • Azure Activity Logs
        • Administrative Event Schema
        • Service Health Notification Schema
        • Resource Health
        • Alert Category
        • Autoscale
        • Security
        • Recommendation
        • Policy
      • Entra ID Logging
        • Identity Based Logs
          • Audit Logs
          • Sign-In Logs
            • AADNonInteractiveUserSignInLogs
            • AADManagedIdentitySignInLogs
            • AADServicePrincipalSignInLogs
            • First Party Sign-In Activity
          • Provisioning Logs (AADProvisioningLogs)
          • Microsoft Graph Activity
          • Identity Protection
            • Risk Detections
            • AADRiskyServicePrincipals
            • AADRiskyUsers
            • AADUserRiskEvents
            • AADServicePrincipalRiskEvents
        • Additional Entra ID Logs
      • Azure Key Vault
        • Azure Key Vault Logging Overview
      • Network Watcher
        • RBAC Permissions
        • Flow Log Types
          • NSG Flow Log Schema
          • VNET Flow Log Schema
        • Enabling Logs
          • NSG Flow Logs
          • VNET Flow Logs
        • Packet Capture
          • Packet Capture: VM
          • Packet Capture: Scale Sets
      • Compute Resources
        • Azure Monitor Agent
        • VM Insights
          • VM Insights Tables
      • Storage Accounts
        • Storage Account Logging
          • File
            • Enable StorageFileLogs
            • StorageFileLogs
          • Blob
            • Enable Blob Logging
            • StorageBlobLogs
          • Queue
            • Enable Queue Logging
            • StorageQueueLogs Table
          • Table
            • Enable Table Logging
            • StorageTableLogs Table
      • Azure App Service
        • Log Types
        • Enabling Logging
      • Azure Monitor
        • Resource Logs
          • Resource Log Top Level Documentation
        • Log Analytics Workspace
          • Setup
        • Workbooks
        • Dashboards
        • Alerts
        • Azure Monitor Documentation
      • Defender for Cloud
      • Intune
      • Sysmon
      • Purview Audit Log Schema
      • Kubernetes Audit Log (AKS)
  • Threat Hunting
    • Threat Hunting in Azure
      • Threat Hunting Introduction
      • Threat Hunting Process
        • Hypothesis Generation
        • Investigation
        • Identification
        • Resolution & Follow Up
      • Pyramid of Pain
      • Azure Threat Hunting Ideas
      • Hands On Threat Hunting Examples
      • OSINT Feeds
  • Sigma
    • Sigma Rule Structure
  • Microsoft Defender TI
    • Microsoft Defender Threat Intelligence
      • Data Sets
      • Reputational Scoring
      • Analyst Insights
      • Microsoft Defender TI: Copilot Integration
  • MITRE Att&ck
    • MITRE Att&ck
      • MITRE Att&ck Concepts
      • MITRE Data Sources
      • MITRE Att&ck Mitigations
      • MITRE Att&ck: Azure
        • MITRE Att&CK: Azure Security Control Mapping
  • Microsoft Resources
    • Microsoft Incident Response Ninja Hub
    • Microsoft Defender XDR Ninja Hub
  • Azure Threat Research Matrix (ATRM)
  • Security Research & Resources
    • Azure Goat
    • Azure Security Research
      • Azure Related CVEs
  • Defender XDR
    • Defender XDR Overview
    • Defender XDR Licensing
    • Defender XDR Default Retention
    • Defender XDR Advanced Hunting Table Schemas
    • Automated Response Requirements
    • Supported Response Actions
  • Azure Sentinel
    • Sentinel Overview
    • Azure Sentinel Deployments
    • Supported Data
    • Workbook, Playbook, Notebook Comparison
    • Sentinel Workbooks
    • Entities
    • User and Entity Behavior Analytics
    • Anomaly Detection
    • Mult-Stage Attack Detection
    • Sentinel: Az CLI
  • Microsoft Defender
    • Microsoft Defender for Cloud References
    • Defender for Cloud: Az CLI
  • Azure Policy
    • Azure Policy
    • Azure Policy Components
    • Azure Policy Rules
    • Scope Azure Policy
    • Policy Assignments
    • Policy Effect
    • Initiative Definition
    • Policy Parameters
    • Remediation Task Structure
    • Use Cases for Azure Policy
    • Azure Policy: Az CLI
  • Intune
    • Intune Overview
    • Intune Licensing
    • Intune API Permission Scopes
    • Intune Sample Script Resources
  • Intune Logging
    • Configure Logging
    • Logging Schema References
    • Intune Queries and Resources
  • Windows Host Security
    • Windows System Architecture and OS Fundamentals
    • SysInternals
    • Basic vs Advanced Security Auditing
    • Sysmon
  • Adversary Emulation
    • AzureHound
    • AADInternals
      • Install
    • RoadTools
      • Install
    • Oh365UserFinder
    • GraphRunner
  • Incident Response
    • Incident Response
      • Azure IR Program Development Cheat Sheet
      • Azure IR Playbooks (MS Guidance)
      • Ransomware (MS Guidance)
  • Automation
    • Automation Overview
    • Logic Apps
      • How Logic Apps Work
      • Logic App Types
      • Triggers
      • Connectors
      • Conditional Logic and Control Flow
      • APIs in Logic Apps
      • Handling Large Workflows with Stateful Logic Apps
      • External Service Integration
      • Securing, Managing, and Scaling Azure Logic Apps
      • Logic Apps: Az CLI
  • Packet Analysis
    • Wireshark Cheatsheet
    • TShark Cheatsheet
    • TCPDUMP Cheatsheet
    • Protocol Analysis Basics
    • HTTP Response Code Cheatsheet
    • RFC Protocol Mappings
    • PCAP Acquisition
  • Detection Lab
    • Detection Lab Introduction
    • Account Creation Instructions
    • Enable MFA Within Azure Tenant
    • Create an Azure Admin With Cloud Shell
    • Setup and Install Instructions
  • VSCode and Code Setup
  • Deploying Code
  • Enabling Logs for Log Analytics Workspace
  • Logging Into Windows VM
  • Verifying Logs in Log Analytics Workspace
  • Creating Detections: Azure Monitor
  • Cost Management: Billing Alarms
Powered by GitBook
On this page
  • 1. Overview of Stateful vs. Stateless Logic Apps
  • Use Cases for Stateful Logic Apps in Threat Hunting:
  • 2. When to Use Stateful Workflows
  • Example Use Cases:
  • 3. Creating a Stateful Logic App
  • Steps to Create a Stateful Logic App:
  • 4. Handling Long-Running Processes with Stateful Workflows
  • Example: Automating Incident Follow-up and Remediation
  • 5. Error Handling in Stateful Logic Apps
  • Example: Handling API Errors in Long-Running Workflows
  • 6. Managing Large-Scale Data Processing
  • Example: Correlating Logs from Multiple Sources
  • 7. Best Practices for Stateful Logic Apps
  1. Automation
  2. Logic Apps

Handling Large Workflows with Stateful Logic Apps

When building complex threat-hunting or incident response workflows, you may encounter scenarios where the workflow needs to run for an extended period, maintain state between actions, or process large amounts of data. Azure Logic Apps offers stateful workflows that are designed to manage such scenarios efficiently.

1. Overview of Stateful vs. Stateless Logic Apps

Azure Logic Apps can be created in two modes: Stateful and Stateless.

  • Stateless Logic Apps: Execute faster, but they don't preserve the state between actions. They are suited for real-time and short-running processes, such as simple alerts or small workflows that need quick execution.

  • Stateful Logic Apps: Keep track of state across different steps in the workflow and are ideal for complex, long-running workflows. They store the history of each step, making it easier to manage errors, retries, and decision-making over extended periods.

Use Cases for Stateful Logic Apps in Threat Hunting:

  • Handling long-running processes, such as waiting for external systems to respond or awaiting human intervention.

  • Complex workflows involving multiple systems that need to maintain context and track intermediate results.

  • Large-scale threat-hunting queries that gather and process data over extended periods, such as correlating logs from multiple sources or performing daily batch processing of incidents.

2. When to Use Stateful Workflows

Stateful Logic Apps are especially useful in scenarios where the workflow spans multiple steps or external systems, or when the workflow requires manual intervention at certain points.

Example Use Cases:

  • Waiting for Incident Resolution: You may want to trigger a workflow to handle a security incident but keep the workflow running until the incident is marked as resolved in your incident management system.

  • Handling Asynchronous API Calls: When interacting with APIs that may take time to respond, a stateful Logic App can maintain the workflow until a response is received.

  • Automating Large-Scale Threat Correlation: When correlating security logs from multiple sources, you may need to maintain state across different queries and handle large datasets without losing context.

3. Creating a Stateful Logic App

When creating a new Logic App in Azure, you can choose between a Consumption (Stateless) Logic App or a Standard (Stateful) Logic App. The Standard Logic Apps have built-in support for stateful workflows.

Steps to Create a Stateful Logic App:

  1. Create a New Standard Logic App:

    • In the Azure Portal, select Create a Resource.

    • Search for Logic App (Standard).

    • Choose the region, name, and resource group for your Logic App.

    • Select Review + Create and then Create.

  2. Design the Stateful Workflow:

    • Once the Logic App is created, open the Logic Apps Designer and start building the workflow.

    • Add actions like waiting for incident resolution, calling APIs, or gathering data from external sources.

  3. Use Delay Actions:

    • If your workflow involves long-running tasks (e.g., waiting for an incident to be closed or waiting for a manual review), you can use the Delay action to pause the workflow for a specific period before resuming.

    • Example: Add a Delay action to check every hour if an incident has been resolved.

  4. Use Parallel Branching:

    • Stateful Logic Apps support parallel branching, which allows you to run different parts of the workflow simultaneously.

    • This is useful for performing multiple threat-hunting tasks in parallel, such as querying multiple log sources for suspicious activity.

4. Handling Long-Running Processes with Stateful Workflows

One of the key advantages of stateful workflows is their ability to handle long-running processes while maintaining the state of the workflow.

Example: Automating Incident Follow-up and Remediation

Consider a scenario where you need to automatically follow up on a security incident reported in your incident management system, but the incident might take several hours or even days to resolve. You want your Logic App to continue running, checking the incident status at regular intervals until it is closed.

Steps:

  1. Trigger: Start the Logic App when an incident is created in the incident management system (e.g., ServiceNow or Jira).

  2. Delay Action: Add a Delay action to wait for a specified period (e.g., 1 hour).

  3. Query Incident Status: After the delay, query the incident management system to check if the incident has been resolved.

  4. Loop Until Resolution: Use an Until control to keep repeating the delay and query actions until the incident is marked as resolved.

  5. Take Final Action: Once the incident is resolved, execute the next steps, such as closing the incident in the system or notifying the security team.

This type of workflow could run for hours or days, and the state of the workflow (i.e., the fact that it's waiting for resolution) is preserved across all these steps.

5. Error Handling in Stateful Logic Apps

Stateful Logic Apps offer advanced error-handling capabilities, allowing you to manage failures and retries across different steps in the workflow.

Example: Handling API Errors in Long-Running Workflows

In long-running workflows, it’s common to encounter transient issues, such as API timeouts or network failures. Stateful Logic Apps allow you to handle these errors effectively by storing the state of the workflow and providing options to retry actions.

Steps for Handling API Errors:

  1. Add an HTTP Action: Make an API call to an external threat intelligence platform or security tool.

  2. Configure Error Handling:

    • Click on the ellipsis (three dots) of the HTTP action.

    • Select Configure run after, and define what should happen if the action fails (e.g., retry, log error, or notify).

  3. Use Retry Policies: Define retry policies for actions that might fail. For example, if the API request fails due to a timeout, configure the Logic App to retry the request a few times before marking it as failed.

6. Managing Large-Scale Data Processing

Stateful Logic Apps are ideal for processing large volumes of data, such as when you need to ingest and correlate logs from multiple sources or scan through large datasets during threat-hunting operations.

Example: Correlating Logs from Multiple Sources

You can build a workflow that queries multiple log sources, such as Azure Monitor, Office 365 logs, and third-party SIEMs, and then correlates the data to identify suspicious patterns.

Steps:

  1. Trigger: Set the workflow to trigger at regular intervals (e.g., daily) or in response to an event (e.g., when a specific threat indicator is found).

  2. Query Multiple Log Sources:

    • Use built-in connectors or HTTP actions to query logs from different sources.

  3. Correlate Data:

    • Store the results of each query in variables and correlate the data to detect suspicious activity.

  4. Store Results: Save the correlated results in a database or alert the security team for further investigation.

7. Best Practices for Stateful Logic Apps

  • Use Delay Actions and Loops Carefully: When building long-running workflows, be mindful of how often actions are triggered. Use delay actions to avoid overwhelming systems with too many requests.

  • Enable Error Handling and Retries: Always configure error-handling mechanisms and retries to ensure that the workflow can recover from transient errors.

  • Monitor Workflow Performance: Stateful Logic Apps store the history of each step, allowing you to monitor performance and troubleshoot issues more easily. Use Azure Monitor to track workflow performance and ensure workflows run as expected.

  • Optimize for Cost: While stateful Logic Apps provide powerful capabilities, they can incur higher costs due to their storage of state. Be mindful of your use of storage and compute resources.

Last updated 8 months ago