tutorials-and-examples/feature-tutorials/AzureSentinelAPIs.ipynb (1,882 lines of code) (raw):

{ "cells": [ { "cell_type": "markdown", "metadata": { "toc": true }, "source": [ "<h1>Table of Contents<span class=\"tocSkip\"></span></h1>\n", "<div class=\"toc\"><ul class=\"toc-item\"><li><span><a href=\"#Azure-Sentinel-API-Calls\" data-toc-modified-id=\"Azure-Sentinel-API-Calls-1\">Microsoft Sentinel API Calls</a></span><ul class=\"toc-item\"><li><ul class=\"toc-item\"><li><span><a href=\"#Description\" data-toc-modified-id=\"Description-1.0.1\">Description</a></span></li><li><span><a href=\"#Installation-and-imports\" data-toc-modified-id=\"Installation-and-imports-1.0.2\">Installation and imports</a></span></li><li><span><a href=\"#Authentication\" data-toc-modified-id=\"Authentication-1.0.3\">Authentication</a></span></li></ul></li></ul></li></ul></div>" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Microsoft Sentinel API Calls\n", "\n", "MSTICpy versions > 0.8.5\n", "\n", "### Description\n", "\n", "This Notebook provides an example of using the Microsoft Sentinel API features of MSTICpy in order retrieve specific data from Microsoft Sentinel\n", "\n", "### Installation and imports" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%pip install --upgrade msticpy[azsentinel]" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "ExecuteTime": { "end_time": "2020-10-28T00:46:14.769939Z", "start_time": "2020-10-28T00:46:12.006888Z" } }, "outputs": [], "source": [ "from msticpy.data.azure_sentinel import AzureSentinel\n", "import msticpy.nbtools.nbwidgets as widgets\n", "from msticpy.data import data_obfus as mask" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Authentication\n", "The first step to be able to use the features is to call the AzureSentinel class and connect to it. Authentication uses the standardized Azure authentication options of using environment variables, Azure CLI credentials, Managed Identities, and interactive logons." ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "ExecuteTime": { "end_time": "2020-10-28T00:46:21.283503Z", "start_time": "2020-10-28T00:46:16.357954Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Attempting to sign-in with environment variable credentials...\n" ] } ], "source": [ "azs = AzureSentinel()\n", "azs.connect()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Once connected we need to select a Microsoft Sentinel workspace to get details from. The easies way to do this is with the get_subscriptions() and get_sentinel_workspaces() functions to select the subscription and workspace you with to connect to. If you already know which workspace you wish to connect to you can skip straight to the other functions and enter these details." ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "ExecuteTime": { "end_time": "2020-10-28T01:39:48.623598Z", "start_time": "2020-10-28T01:39:48.258598Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Attempting to sign-in with environment variable credentials...\n", "obfuscating columns:\n", "Display Name, \n", "done\n", "Select a subscription:\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "830b967ed650424692fa54a55234a9a9", "version_major": 2, "version_minor": 0 }, "text/plain": [ "VBox(children=(Text(value='', description='Filter:', style=DescriptionStyle(description_width='initial')), Sel…" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "<hr>" ], "text/plain": [ "<IPython.core.display.HTML object>" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Query for our subscriptions\n", "subs = azs.get_subscriptions()\n", "subs = subs.mp_obf.obfuscate(column_map={\"Display Name\": \"str\"})\n", "\n", "# Display subscriptions (masked names) in a pick list\n", "print(\"Select a subscription:\")\n", "sub = widgets.SelectItem(\n", " item_list=subs['Display Name'].to_list(),\n", " auto_display=True\n", ")" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "ExecuteTime": { "end_time": "2020-10-28T01:44:53.170848Z", "start_time": "2020-10-28T01:44:47.548676Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Finding Microsoft Sentinel Workspaces...\n", "Attempting to sign-in with environment variable credentials...\n", "Select a Microsoft Sentinel Workspace:\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "c24a0d08d4614340907a64b1039eadf6", "version_major": 2, "version_minor": 0 }, "text/plain": [ "VBox(children=(Text(value='', description='Filter:', style=DescriptionStyle(description_width='initial')), Sel…" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "<hr>" ], "text/plain": [ "<IPython.core.display.HTML object>" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Get the subscription ID\n", "sub_id = subs[subs['Display Name'] == sub.value].iloc[0]['Subscription ID']\n", "# Query for workspaces in that subscription\n", "workspaces = azs.get_sentinel_workspaces(sub_id = sub_id)\n", "# Display workspaces in a list\n", "print(\"Select a Microsoft Sentinel Workspace:\")\n", "ws = widgets.SelectItem(\n", " item_dict=workspaces,\n", " auto_display=True\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now that we have selected our workspace we can call various functions to get details about content in the workspace. These are typically returned as DataFrames. Below we get a list of hunting queries configured in our workspace." ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "ExecuteTime": { "end_time": "2020-10-28T01:45:10.962506Z", "start_time": "2020-10-28T01:45:09.792474Z" } }, "outputs": [ { "data": { "text/html": [ "<div>\n", "<style scoped>\n", " .dataframe tbody tr th:only-of-type {\n", " vertical-align: middle;\n", " }\n", "\n", " .dataframe tbody tr th {\n", " vertical-align: top;\n", " }\n", "\n", " .dataframe thead th {\n", " text-align: right;\n", " }\n", "</style>\n", "<table border=\"1\" class=\"dataframe\">\n", " <thead>\n", " <tr style=\"text-align: right;\">\n", " <th></th>\n", " <th>type</th>\n", " <th>properties.Category</th>\n", " <th>properties.DisplayName</th>\n", " <th>properties.Query</th>\n", " <th>properties.Version</th>\n", " <th>properties.Tags</th>\n", " <th>properties.FunctionAlias</th>\n", " <th>properties.FunctionParameters</th>\n", " </tr>\n", " </thead>\n", " <tbody>\n", " <tr>\n", " <th>2</th>\n", " <td>Microsoft.OperationalInsights/savedSearches</td>\n", " <td>Hunting Queries</td>\n", " <td>Powershell</td>\n", " <td>SecurityEvent\\r\\n| where ParentProcessName con...</td>\n", " <td>2</td>\n", " <td>NaN</td>\n", " <td>NaN</td>\n", " <td>NaN</td>\n", " </tr>\n", " <tr>\n", " <th>5</th>\n", " <td>Microsoft.OperationalInsights/savedSearches</td>\n", " <td>Hunting Queries</td>\n", " <td>Anomalous AAD Account Creation</td>\n", " <td>\\nBehaviorAnalytics\\n| where ActionType == \"Ad...</td>\n", " <td>2</td>\n", " <td>[{'Name': 'description', 'Value': ''}, {'Name'...</td>\n", " <td>NaN</td>\n", " <td>NaN</td>\n", " </tr>\n", " <tr>\n", " <th>7</th>\n", " <td>Microsoft.OperationalInsights/savedSearches</td>\n", " <td>Hunting Queries</td>\n", " <td>Entropy for Processes for a given Host</td>\n", " <td>\\n// May need to reduce the number of days if ...</td>\n", " <td>2</td>\n", " <td>[{'Name': 'description', 'Value': ''}, {'Name'...</td>\n", " <td>NaN</td>\n", " <td>NaN</td>\n", " </tr>\n", " <tr>\n", " <th>9</th>\n", " <td>Microsoft.OperationalInsights/savedSearches</td>\n", " <td>Hunting Queries</td>\n", " <td>RareDNSLookupWithDataTransfer</td>\n", " <td>\\nlet lookbackint = 7;\\nlet lookupThreshold = ...</td>\n", " <td>2</td>\n", " <td>[{'Name': 'description', 'Value': ''}, {'Name'...</td>\n", " <td>NaN</td>\n", " <td>NaN</td>\n", " </tr>\n", " <tr>\n", " <th>12</th>\n", " <td>Microsoft.OperationalInsights/savedSearches</td>\n", " <td>Hunting Queries</td>\n", " <td>Least Common Processes by Command Line</td>\n", " <td>\\nlet Allowlist = dynamic (['foo.exe', 'baz.ex...</td>\n", " <td>2</td>\n", " <td>[{'Name': 'description', 'Value': ''}, {'Name'...</td>\n", " <td>NaN</td>\n", " <td>NaN</td>\n", " </tr>\n", " </tbody>\n", "</table>\n", "</div>" ], "text/plain": [ " type properties.Category \\\n", "2 Microsoft.OperationalInsights/savedSearches Hunting Queries \n", "5 Microsoft.OperationalInsights/savedSearches Hunting Queries \n", "7 Microsoft.OperationalInsights/savedSearches Hunting Queries \n", "9 Microsoft.OperationalInsights/savedSearches Hunting Queries \n", "12 Microsoft.OperationalInsights/savedSearches Hunting Queries \n", "\n", " properties.DisplayName \\\n", "2 Powershell \n", "5 Anomalous AAD Account Creation \n", "7 Entropy for Processes for a given Host \n", "9 RareDNSLookupWithDataTransfer \n", "12 Least Common Processes by Command Line \n", "\n", " properties.Query properties.Version \\\n", "2 SecurityEvent\\r\\n| where ParentProcessName con... 2 \n", "5 \\nBehaviorAnalytics\\n| where ActionType == \"Ad... 2 \n", "7 \\n// May need to reduce the number of days if ... 2 \n", "9 \\nlet lookbackint = 7;\\nlet lookupThreshold = ... 2 \n", "12 \\nlet Allowlist = dynamic (['foo.exe', 'baz.ex... 2 \n", "\n", " properties.Tags \\\n", "2 NaN \n", "5 [{'Name': 'description', 'Value': ''}, {'Name'... \n", "7 [{'Name': 'description', 'Value': ''}, {'Name'... \n", "9 [{'Name': 'description', 'Value': ''}, {'Name'... \n", "12 [{'Name': 'description', 'Value': ''}, {'Name'... \n", "\n", " properties.FunctionAlias properties.FunctionParameters \n", "2 NaN NaN \n", "5 NaN NaN \n", "7 NaN NaN \n", "9 NaN NaN \n", "12 NaN NaN " ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "queries = azs.get_hunting_queries(ws.value)\n", "queries.head().drop(columns=[\"id\", \"etag\", \"name\"])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Hunting queries return the raw queries associated with them, this allows us to pass the query directly to a QueryProvider in order to get the results of the hunting query within the notebook." ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "ExecuteTime": { "end_time": "2020-10-28T00:52:19.961283Z", "start_time": "2020-10-28T00:52:16.730666Z" } }, "outputs": [ { "data": { "application/javascript": [ "try {IPython.notebook.kernel.execute(\"NOTEBOOK_URL = '\" + window.location + \"'\");} catch(err) {;}" ], "text/plain": [ "<IPython.core.display.Javascript object>" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/javascript": [ "try {IPython.notebook.kernel.execute(\"NOTEBOOK_URL = '\" + window.location + \"'\");} catch(err) {;}" ], "text/plain": [ "<IPython.core.display.Javascript object>" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/javascript": [ "try {IPython.notebook.kernel.execute(\"NOTEBOOK_URL = '\" + window.location + \"'\");} catch(err) {;}" ], "text/plain": [ "<IPython.core.display.Javascript object>" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "<div>\n", "<style scoped>\n", " .dataframe tbody tr th:only-of-type {\n", " vertical-align: middle;\n", " }\n", "\n", " .dataframe tbody tr th {\n", " vertical-align: top;\n", " }\n", "\n", " .dataframe thead th {\n", " text-align: right;\n", " }\n", "</style>\n", "<table border=\"1\" class=\"dataframe\">\n", " <thead>\n", " <tr style=\"text-align: right;\">\n", " <th></th>\n", " <th>TimeGenerated</th>\n", " <th>EventID</th>\n", " <th>Computer</th>\n", " <th>SubjectUserSid</th>\n", " <th>Account</th>\n", " <th>Weight</th>\n", " <th>AdjustedProcessEntropy</th>\n", " <th>FullDecimalProcessEntropy</th>\n", " <th>Process</th>\n", " <th>NewProcessName</th>\n", " <th>CommandLine</th>\n", " <th>ParentProcessName</th>\n", " <th>TotalProcessCountOnHost</th>\n", " <th>ProcessCountOnHost</th>\n", " <th>DistinctComputersWithProcessCount</th>\n", " <th>timestamp</th>\n", " <th>HostCustomEntity</th>\n", " <th>AccountCustomEntity</th>\n", " </tr>\n", " </thead>\n", " <tbody>\n", " <tr>\n", " <th>0</th>\n", " <td>2020-11-21 21:18:51.317000+00:00</td>\n", " <td>4688</td>\n", " <td>WinAttackSim</td>\n", " <td>S-1-5-18</td>\n", " <td>WORKGROUP\\WinAttackSim$</td>\n", " <td>10.743361</td>\n", " <td>10.743361</td>\n", " <td>0.001074</td>\n", " <td>Defrag.exe</td>\n", " <td>C:\\Windows\\System32\\Defrag.exe</td>\n", " <td>C:\\windows\\system32\\defrag.exe -c -h -k -g -$</td>\n", " <td>C:\\Windows\\System32\\svchost.exe</td>\n", " <td>12688</td>\n", " <td>1</td>\n", " <td>1</td>\n", " <td>2020-11-21 21:18:51.317000+00:00</td>\n", " <td>WinAttackSim</td>\n", " <td>WORKGROUP\\WinAttackSim$</td>\n", " </tr>\n", " <tr>\n", " <th>1</th>\n", " <td>2020-11-23 01:31:49.930000+00:00</td>\n", " <td>4688</td>\n", " <td>WinAttackSim</td>\n", " <td>S-1-5-18</td>\n", " <td>WORKGROUP\\WinAttackSim$</td>\n", " <td>10.743361</td>\n", " <td>10.743361</td>\n", " <td>0.001074</td>\n", " <td>SppExtComObj.Exe</td>\n", " <td>C:\\Windows\\System32\\SppExtComObj.Exe</td>\n", " <td>C:\\windows\\system32\\SppExtComObj.exe -Embedding</td>\n", " <td>C:\\Windows\\System32\\svchost.exe</td>\n", " <td>12688</td>\n", " <td>1</td>\n", " <td>1</td>\n", " <td>2020-11-23 01:31:49.930000+00:00</td>\n", " <td>WinAttackSim</td>\n", " <td>WORKGROUP\\WinAttackSim$</td>\n", " </tr>\n", " <tr>\n", " <th>2</th>\n", " <td>2020-11-20 17:18:36.960000+00:00</td>\n", " <td>4688</td>\n", " <td>WinAttackSim</td>\n", " <td>S-1-5-18</td>\n", " <td>WORKGROUP\\WinAttackSim$</td>\n", " <td>10.743361</td>\n", " <td>10.743361</td>\n", " <td>0.001074</td>\n", " <td>makecab.exe</td>\n", " <td>C:\\Windows\\System32\\makecab.exe</td>\n", " <td>\"C:\\windows\\system32\\makecab.exe\" C:\\windows\\L...</td>\n", " <td>C:\\Windows\\WinSxS\\amd64_microsoft-windows-serv...</td>\n", " <td>12688</td>\n", " <td>1</td>\n", " <td>1</td>\n", " <td>2020-11-20 17:18:36.960000+00:00</td>\n", " <td>WinAttackSim</td>\n", " <td>WORKGROUP\\WinAttackSim$</td>\n", " </tr>\n", " <tr>\n", " <th>3</th>\n", " <td>2020-11-21 21:18:51.303000+00:00</td>\n", " <td>4688</td>\n", " <td>WinAttackSim</td>\n", " <td>S-1-5-18</td>\n", " <td>WORKGROUP\\WinAttackSim$</td>\n", " <td>10.743361</td>\n", " <td>10.743361</td>\n", " <td>0.001074</td>\n", " <td>rundll32.exe</td>\n", " <td>C:\\Windows\\System32\\rundll32.exe</td>\n", " <td>C:\\windows\\system32\\rundll32.exe Windows.Stora...</td>\n", " <td>C:\\Windows\\System32\\svchost.exe</td>\n", " <td>12688</td>\n", " <td>1</td>\n", " <td>1</td>\n", " <td>2020-11-21 21:18:51.303000+00:00</td>\n", " <td>WinAttackSim</td>\n", " <td>WORKGROUP\\WinAttackSim$</td>\n", " </tr>\n", " <tr>\n", " <th>4</th>\n", " <td>2020-11-21 21:18:51.310000+00:00</td>\n", " <td>4688</td>\n", " <td>WinAttackSim</td>\n", " <td>S-1-5-18</td>\n", " <td>WORKGROUP\\WinAttackSim$</td>\n", " <td>10.743361</td>\n", " <td>10.743361</td>\n", " <td>0.001074</td>\n", " <td>tzsync.exe</td>\n", " <td>C:\\Windows\\System32\\tzsync.exe</td>\n", " <td>C:\\windows\\system32\\tzsync.exe</td>\n", " <td>C:\\Windows\\System32\\svchost.exe</td>\n", " <td>12688</td>\n", " <td>1</td>\n", " <td>1</td>\n", " <td>2020-11-21 21:18:51.310000+00:00</td>\n", " <td>WinAttackSim</td>\n", " <td>WORKGROUP\\WinAttackSim$</td>\n", " </tr>\n", " <tr>\n", " <th>5</th>\n", " <td>2020-11-18 01:18:36.913000+00:00</td>\n", " <td>4688</td>\n", " <td>WinAttackSim</td>\n", " <td>S-1-5-18</td>\n", " <td>WORKGROUP\\WinAttackSim$</td>\n", " <td>39.820861</td>\n", " <td>19.910430</td>\n", " <td>0.001991</td>\n", " <td>lpremove.exe</td>\n", " <td>C:\\Windows\\System32\\lpremove.exe</td>\n", " <td>C:\\windows\\system32\\lpremove.exe</td>\n", " <td>C:\\Windows\\System32\\svchost.exe</td>\n", " <td>12688</td>\n", " <td>2</td>\n", " <td>1</td>\n", " <td>2020-11-18 01:18:36.913000+00:00</td>\n", " <td>WinAttackSim</td>\n", " <td>WORKGROUP\\WinAttackSim$</td>\n", " </tr>\n", " <tr>\n", " <th>6</th>\n", " <td>2020-11-21 03:18:49.223000+00:00</td>\n", " <td>4688</td>\n", " <td>WinAttackSim</td>\n", " <td>S-1-5-18</td>\n", " <td>WORKGROUP\\WinAttackSim$</td>\n", " <td>39.820861</td>\n", " <td>19.910430</td>\n", " <td>0.001991</td>\n", " <td>lpremove.exe</td>\n", " <td>C:\\Windows\\System32\\lpremove.exe</td>\n", " <td>C:\\windows\\system32\\lpremove.exe</td>\n", " <td>C:\\Windows\\System32\\svchost.exe</td>\n", " <td>12688</td>\n", " <td>2</td>\n", " <td>1</td>\n", " <td>2020-11-21 03:18:49.223000+00:00</td>\n", " <td>WinAttackSim</td>\n", " <td>WORKGROUP\\WinAttackSim$</td>\n", " </tr>\n", " <tr>\n", " <th>7</th>\n", " <td>2020-11-23 01:31:49.957000+00:00</td>\n", " <td>4688</td>\n", " <td>WinAttackSim</td>\n", " <td>S-1-5-20</td>\n", " <td>WORKGROUP\\WinAttackSim$</td>\n", " <td>39.820861</td>\n", " <td>19.910430</td>\n", " <td>0.001991</td>\n", " <td>slui.exe</td>\n", " <td>C:\\Windows\\System32\\slui.exe</td>\n", " <td>\"C:\\windows\\System32\\SLUI.exe\" RuleId=502ff3ba...</td>\n", " <td>C:\\Windows\\System32\\SppExtComObj.Exe</td>\n", " <td>12688</td>\n", " <td>2</td>\n", " <td>1</td>\n", " <td>2020-11-23 01:31:49.957000+00:00</td>\n", " <td>WinAttackSim</td>\n", " <td>WORKGROUP\\WinAttackSim$</td>\n", " </tr>\n", " <tr>\n", " <th>8</th>\n", " <td>2020-11-23 01:31:54.340000+00:00</td>\n", " <td>4688</td>\n", " <td>WinAttackSim</td>\n", " <td>S-1-5-20</td>\n", " <td>WORKGROUP\\WinAttackSim$</td>\n", " <td>39.820861</td>\n", " <td>19.910430</td>\n", " <td>0.001991</td>\n", " <td>slui.exe</td>\n", " <td>C:\\Windows\\System32\\slui.exe</td>\n", " <td>\"C:\\windows\\System32\\SLUI.exe\" RuleId=379cccfb...</td>\n", " <td>C:\\Windows\\System32\\SppExtComObj.Exe</td>\n", " <td>12688</td>\n", " <td>2</td>\n", " <td>1</td>\n", " <td>2020-11-23 01:31:54.340000+00:00</td>\n", " <td>WinAttackSim</td>\n", " <td>WORKGROUP\\WinAttackSim$</td>\n", " </tr>\n", " </tbody>\n", "</table>\n", "</div>" ], "text/plain": [ " TimeGenerated EventID Computer SubjectUserSid \\\n", "0 2020-11-21 21:18:51.317000+00:00 4688 WinAttackSim S-1-5-18 \n", "1 2020-11-23 01:31:49.930000+00:00 4688 WinAttackSim S-1-5-18 \n", "2 2020-11-20 17:18:36.960000+00:00 4688 WinAttackSim S-1-5-18 \n", "3 2020-11-21 21:18:51.303000+00:00 4688 WinAttackSim S-1-5-18 \n", "4 2020-11-21 21:18:51.310000+00:00 4688 WinAttackSim S-1-5-18 \n", "5 2020-11-18 01:18:36.913000+00:00 4688 WinAttackSim S-1-5-18 \n", "6 2020-11-21 03:18:49.223000+00:00 4688 WinAttackSim S-1-5-18 \n", "7 2020-11-23 01:31:49.957000+00:00 4688 WinAttackSim S-1-5-20 \n", "8 2020-11-23 01:31:54.340000+00:00 4688 WinAttackSim S-1-5-20 \n", "\n", " Account Weight AdjustedProcessEntropy \\\n", "0 WORKGROUP\\WinAttackSim$ 10.743361 10.743361 \n", "1 WORKGROUP\\WinAttackSim$ 10.743361 10.743361 \n", "2 WORKGROUP\\WinAttackSim$ 10.743361 10.743361 \n", "3 WORKGROUP\\WinAttackSim$ 10.743361 10.743361 \n", "4 WORKGROUP\\WinAttackSim$ 10.743361 10.743361 \n", "5 WORKGROUP\\WinAttackSim$ 39.820861 19.910430 \n", "6 WORKGROUP\\WinAttackSim$ 39.820861 19.910430 \n", "7 WORKGROUP\\WinAttackSim$ 39.820861 19.910430 \n", "8 WORKGROUP\\WinAttackSim$ 39.820861 19.910430 \n", "\n", " FullDecimalProcessEntropy Process \\\n", "0 0.001074 Defrag.exe \n", "1 0.001074 SppExtComObj.Exe \n", "2 0.001074 makecab.exe \n", "3 0.001074 rundll32.exe \n", "4 0.001074 tzsync.exe \n", "5 0.001991 lpremove.exe \n", "6 0.001991 lpremove.exe \n", "7 0.001991 slui.exe \n", "8 0.001991 slui.exe \n", "\n", " NewProcessName \\\n", "0 C:\\Windows\\System32\\Defrag.exe \n", "1 C:\\Windows\\System32\\SppExtComObj.Exe \n", "2 C:\\Windows\\System32\\makecab.exe \n", "3 C:\\Windows\\System32\\rundll32.exe \n", "4 C:\\Windows\\System32\\tzsync.exe \n", "5 C:\\Windows\\System32\\lpremove.exe \n", "6 C:\\Windows\\System32\\lpremove.exe \n", "7 C:\\Windows\\System32\\slui.exe \n", "8 C:\\Windows\\System32\\slui.exe \n", "\n", " CommandLine \\\n", "0 C:\\windows\\system32\\defrag.exe -c -h -k -g -$ \n", "1 C:\\windows\\system32\\SppExtComObj.exe -Embedding \n", "2 \"C:\\windows\\system32\\makecab.exe\" C:\\windows\\L... \n", "3 C:\\windows\\system32\\rundll32.exe Windows.Stora... \n", "4 C:\\windows\\system32\\tzsync.exe \n", "5 C:\\windows\\system32\\lpremove.exe \n", "6 C:\\windows\\system32\\lpremove.exe \n", "7 \"C:\\windows\\System32\\SLUI.exe\" RuleId=502ff3ba... \n", "8 \"C:\\windows\\System32\\SLUI.exe\" RuleId=379cccfb... \n", "\n", " ParentProcessName TotalProcessCountOnHost \\\n", "0 C:\\Windows\\System32\\svchost.exe 12688 \n", "1 C:\\Windows\\System32\\svchost.exe 12688 \n", "2 C:\\Windows\\WinSxS\\amd64_microsoft-windows-serv... 12688 \n", "3 C:\\Windows\\System32\\svchost.exe 12688 \n", "4 C:\\Windows\\System32\\svchost.exe 12688 \n", "5 C:\\Windows\\System32\\svchost.exe 12688 \n", "6 C:\\Windows\\System32\\svchost.exe 12688 \n", "7 C:\\Windows\\System32\\SppExtComObj.Exe 12688 \n", "8 C:\\Windows\\System32\\SppExtComObj.Exe 12688 \n", "\n", " ProcessCountOnHost DistinctComputersWithProcessCount \\\n", "0 1 1 \n", "1 1 1 \n", "2 1 1 \n", "3 1 1 \n", "4 1 1 \n", "5 2 1 \n", "6 2 1 \n", "7 2 1 \n", "8 2 1 \n", "\n", " timestamp HostCustomEntity AccountCustomEntity \n", "0 2020-11-21 21:18:51.317000+00:00 WinAttackSim WORKGROUP\\WinAttackSim$ \n", "1 2020-11-23 01:31:49.930000+00:00 WinAttackSim WORKGROUP\\WinAttackSim$ \n", "2 2020-11-20 17:18:36.960000+00:00 WinAttackSim WORKGROUP\\WinAttackSim$ \n", "3 2020-11-21 21:18:51.303000+00:00 WinAttackSim WORKGROUP\\WinAttackSim$ \n", "4 2020-11-21 21:18:51.310000+00:00 WinAttackSim WORKGROUP\\WinAttackSim$ \n", "5 2020-11-18 01:18:36.913000+00:00 WinAttackSim WORKGROUP\\WinAttackSim$ \n", "6 2020-11-21 03:18:49.223000+00:00 WinAttackSim WORKGROUP\\WinAttackSim$ \n", "7 2020-11-23 01:31:49.957000+00:00 WinAttackSim WORKGROUP\\WinAttackSim$ \n", "8 2020-11-23 01:31:54.340000+00:00 WinAttackSim WORKGROUP\\WinAttackSim$ " ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from msticpy.data.data_providers import QueryProvider\n", "from msticpy.common.wsconfig import WorkspaceConfig\n", "qry_prov = QueryProvider('LogAnalytics')\n", "wkspace = WorkspaceConfig()\n", "qry_prov.connect(wkspace.code_connect_str)\n", "qry_prov.exec_query(queries['properties.Query'].iloc[2])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can also get a list of configured alert rules:" ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "ExecuteTime": { "end_time": "2020-10-28T00:50:07.155005Z", "start_time": "2020-10-28T00:50:06.191005Z" } }, "outputs": [ { "data": { "text/html": [ "<div>\n", "<style scoped>\n", " .dataframe tbody tr th:only-of-type {\n", " vertical-align: middle;\n", " }\n", "\n", " .dataframe tbody tr th {\n", " vertical-align: top;\n", " }\n", "\n", " .dataframe thead th {\n", " text-align: right;\n", " }\n", "</style>\n", "<table border=\"1\" class=\"dataframe\">\n", " <thead>\n", " <tr style=\"text-align: right;\">\n", " <th></th>\n", " <th>type</th>\n", " <th>kind</th>\n", " <th>properties.severity</th>\n", " <th>properties.query</th>\n", " <th>properties.queryFrequency</th>\n", " <th>properties.queryPeriod</th>\n", " <th>properties.triggerOperator</th>\n", " <th>properties.triggerThreshold</th>\n", " <th>properties.suppressionDuration</th>\n", " <th>properties.suppressionEnabled</th>\n", " <th>...</th>\n", " <th>properties.description</th>\n", " <th>properties.tactics</th>\n", " <th>properties.alertRuleTemplateName</th>\n", " <th>properties.lastModifiedUtc</th>\n", " <th>properties.customFields.Filename</th>\n", " <th>properties.customFields.Reason</th>\n", " <th>properties.productFilter</th>\n", " <th>properties.severitiesFilter</th>\n", " <th>properties.displayNamesFilter</th>\n", " <th>properties.displayNamesExcludeFilter</th>\n", " </tr>\n", " </thead>\n", " <tbody>\n", " <tr>\n", " <th>0</th>\n", " <td>Microsoft.SecurityInsights/alertRules</td>\n", " <td>Scheduled</td>\n", " <td>Medium</td>\n", " <td>let failureCountThreshold = 5;\\nlet successCou...</td>\n", " <td>P1D</td>\n", " <td>P1D</td>\n", " <td>GreaterThan</td>\n", " <td>0.0</td>\n", " <td>PT5H</td>\n", " <td>False</td>\n", " <td>...</td>\n", " <td>Identifies evidence of brute force activity ag...</td>\n", " <td>[CredentialAccess]</td>\n", " <td>28b42356-45af-40a6-a0b4-a554cdfd5d8a</td>\n", " <td>2020-11-17T08:15:49.636781Z</td>\n", " <td>NaN</td>\n", " <td>NaN</td>\n", " <td>NaN</td>\n", " <td>NaN</td>\n", " <td>NaN</td>\n", " <td>NaN</td>\n", " </tr>\n", " <tr>\n", " <th>1</th>\n", " <td>Microsoft.SecurityInsights/alertRules</td>\n", " <td>Scheduled</td>\n", " <td>Medium</td>\n", " <td>let timeframe = 1d;\\n//Set a threshold of fail...</td>\n", " <td>P1D</td>\n", " <td>P1D</td>\n", " <td>GreaterThan</td>\n", " <td>0.0</td>\n", " <td>PT5H</td>\n", " <td>False</td>\n", " <td>...</td>\n", " <td>This query creates a list of IP addresses with...</td>\n", " <td>[InitialAccess, CredentialAccess]</td>\n", " <td>ba144bf8-75b8-406f-9420-ed74397f9479</td>\n", " <td>2020-11-11T05:16:14.5036485Z</td>\n", " <td>FileName</td>\n", " <td>Reason</td>\n", " <td>NaN</td>\n", " <td>NaN</td>\n", " <td>NaN</td>\n", " <td>NaN</td>\n", " </tr>\n", " <tr>\n", " <th>2</th>\n", " <td>Microsoft.SecurityInsights/alertRules</td>\n", " <td>Scheduled</td>\n", " <td>Medium</td>\n", " <td>let timeframe = 1d;\\nSecurityEvent\\n| where Ti...</td>\n", " <td>P1D</td>\n", " <td>P1D</td>\n", " <td>GreaterThan</td>\n", " <td>0.0</td>\n", " <td>PT5H</td>\n", " <td>False</td>\n", " <td>...</td>\n", " <td>Checks for event id 1102 which indicates the s...</td>\n", " <td>[DefenseEvasion]</td>\n", " <td>80da0a8f-cfe1-4cd0-a895-8bc1771a720e</td>\n", " <td>2020-11-11T01:46:53.4905768Z</td>\n", " <td>NaN</td>\n", " <td>NaN</td>\n", " <td>NaN</td>\n", " <td>NaN</td>\n", " <td>NaN</td>\n", " <td>NaN</td>\n", " </tr>\n", " <tr>\n", " <th>3</th>\n", " <td>Microsoft.SecurityInsights/alertRules</td>\n", " <td>Scheduled</td>\n", " <td>Medium</td>\n", " <td>AzureActivity\\n| take 1\\n| extend IPCustomEnti...</td>\n", " <td>P1D</td>\n", " <td>P14D</td>\n", " <td>GreaterThan</td>\n", " <td>0.0</td>\n", " <td>PT5H</td>\n", " <td>False</td>\n", " <td>...</td>\n", " <td>This analytic matches Azure Activity logs to k...</td>\n", " <td>[Impact]</td>\n", " <td>None</td>\n", " <td>2020-11-04T22:43:33.9845152Z</td>\n", " <td>NaN</td>\n", " <td>NaN</td>\n", " <td>NaN</td>\n", " <td>NaN</td>\n", " <td>NaN</td>\n", " <td>NaN</td>\n", " </tr>\n", " <tr>\n", " <th>4</th>\n", " <td>Microsoft.SecurityInsights/alertRules</td>\n", " <td>Scheduled</td>\n", " <td>Medium</td>\n", " <td>let timeframe = 1d;\\nSecurityEvent\\n| where Ti...</td>\n", " <td>P1D</td>\n", " <td>P1D</td>\n", " <td>GreaterThan</td>\n", " <td>0.0</td>\n", " <td>PT5H</td>\n", " <td>False</td>\n", " <td>...</td>\n", " <td>Checks for event id 1102 which indicates the s...</td>\n", " <td>[DefenseEvasion]</td>\n", " <td>80da0a8f-cfe1-4cd0-a895-8bc1771a720e</td>\n", " <td>2020-11-11T07:19:24.7658031Z</td>\n", " <td>NaN</td>\n", " <td>NaN</td>\n", " <td>NaN</td>\n", " <td>NaN</td>\n", " <td>NaN</td>\n", " <td>NaN</td>\n", " </tr>\n", " </tbody>\n", "</table>\n", "<p>5 rows × 29 columns</p>\n", "</div>" ], "text/plain": [ " type kind properties.severity \\\n", "0 Microsoft.SecurityInsights/alertRules Scheduled Medium \n", "1 Microsoft.SecurityInsights/alertRules Scheduled Medium \n", "2 Microsoft.SecurityInsights/alertRules Scheduled Medium \n", "3 Microsoft.SecurityInsights/alertRules Scheduled Medium \n", "4 Microsoft.SecurityInsights/alertRules Scheduled Medium \n", "\n", " properties.query \\\n", "0 let failureCountThreshold = 5;\\nlet successCou... \n", "1 let timeframe = 1d;\\n//Set a threshold of fail... \n", "2 let timeframe = 1d;\\nSecurityEvent\\n| where Ti... \n", "3 AzureActivity\\n| take 1\\n| extend IPCustomEnti... \n", "4 let timeframe = 1d;\\nSecurityEvent\\n| where Ti... \n", "\n", " properties.queryFrequency properties.queryPeriod properties.triggerOperator \\\n", "0 P1D P1D GreaterThan \n", "1 P1D P1D GreaterThan \n", "2 P1D P1D GreaterThan \n", "3 P1D P14D GreaterThan \n", "4 P1D P1D GreaterThan \n", "\n", " properties.triggerThreshold properties.suppressionDuration \\\n", "0 0.0 PT5H \n", "1 0.0 PT5H \n", "2 0.0 PT5H \n", "3 0.0 PT5H \n", "4 0.0 PT5H \n", "\n", " properties.suppressionEnabled ... \\\n", "0 False ... \n", "1 False ... \n", "2 False ... \n", "3 False ... \n", "4 False ... \n", "\n", " properties.description \\\n", "0 Identifies evidence of brute force activity ag... \n", "1 This query creates a list of IP addresses with... \n", "2 Checks for event id 1102 which indicates the s... \n", "3 This analytic matches Azure Activity logs to k... \n", "4 Checks for event id 1102 which indicates the s... \n", "\n", " properties.tactics properties.alertRuleTemplateName \\\n", "0 [CredentialAccess] 28b42356-45af-40a6-a0b4-a554cdfd5d8a \n", "1 [InitialAccess, CredentialAccess] ba144bf8-75b8-406f-9420-ed74397f9479 \n", "2 [DefenseEvasion] 80da0a8f-cfe1-4cd0-a895-8bc1771a720e \n", "3 [Impact] None \n", "4 [DefenseEvasion] 80da0a8f-cfe1-4cd0-a895-8bc1771a720e \n", "\n", " properties.lastModifiedUtc properties.customFields.Filename \\\n", "0 2020-11-17T08:15:49.636781Z NaN \n", "1 2020-11-11T05:16:14.5036485Z FileName \n", "2 2020-11-11T01:46:53.4905768Z NaN \n", "3 2020-11-04T22:43:33.9845152Z NaN \n", "4 2020-11-11T07:19:24.7658031Z NaN \n", "\n", " properties.customFields.Reason properties.productFilter \\\n", "0 NaN NaN \n", "1 Reason NaN \n", "2 NaN NaN \n", "3 NaN NaN \n", "4 NaN NaN \n", "\n", " properties.severitiesFilter properties.displayNamesFilter \\\n", "0 NaN NaN \n", "1 NaN NaN \n", "2 NaN NaN \n", "3 NaN NaN \n", "4 NaN NaN \n", "\n", " properties.displayNamesExcludeFilter \n", "0 NaN \n", "1 NaN \n", "2 NaN \n", "3 NaN \n", "4 NaN \n", "\n", "[5 rows x 29 columns]" ] }, "execution_count": 14, "metadata": {}, "output_type": "execute_result" } ], "source": [ "alert_rules = azs.get_alert_rules(ws.value)\n", "alert_rules.head().drop(columns=[\"id\", \"etag\", \"name\"])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can also get a list of saved bookmarks. To see the events these bookmarks relate to you can pass the query value to a QueryProvider." ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "ExecuteTime": { "end_time": "2020-10-28T00:50:17.332381Z", "start_time": "2020-10-28T00:50:14.780804Z" } }, "outputs": [ { "data": { "text/html": [ "<div>\n", "<style scoped>\n", " .dataframe tbody tr th:only-of-type {\n", " vertical-align: middle;\n", " }\n", "\n", " .dataframe tbody tr th {\n", " vertical-align: top;\n", " }\n", "\n", " .dataframe thead th {\n", " text-align: right;\n", " }\n", "</style>\n", "<table border=\"1\" class=\"dataframe\">\n", " <thead>\n", " <tr style=\"text-align: right;\">\n", " <th></th>\n", " <th>type</th>\n", " <th>properties.displayName</th>\n", " <th>properties.created</th>\n", " <th>properties.updated</th>\n", " <th>properties.createdBy.objectId</th>\n", " <th>properties.createdBy.email</th>\n", " <th>properties.createdBy.name</th>\n", " <th>properties.updatedBy.objectId</th>\n", " <th>properties.updatedBy.email</th>\n", " <th>properties.updatedBy.name</th>\n", " <th>...</th>\n", " <th>properties.labels</th>\n", " <th>properties.query</th>\n", " <th>properties.queryResult</th>\n", " <th>properties.queryStartTime</th>\n", " <th>properties.queryEndTime</th>\n", " <th>properties.incidentInfo.incidentId</th>\n", " <th>properties.incidentInfo.title</th>\n", " <th>properties.incidentInfo.relationName</th>\n", " <th>properties.incidentInfo.severity</th>\n", " <th>properties.notes</th>\n", " </tr>\n", " </thead>\n", " <tbody>\n", " <tr>\n", " <th>0</th>\n", " <td>Microsoft.SecurityInsights/Bookmarks</td>\n", " <td>mercury IP</td>\n", " <td>2020-11-18T09:26:54.1605891+00:00</td>\n", " <td>2020-11-18T09:26:54.1605891+00:00</td>\n", " <td>e0139aae-7811-40ca-abc6-3fcb79140a6b</td>\n", " <td>Tim.Burrell@microsoft.com</td>\n", " <td>Tim Burrell (MSTIC)</td>\n", " <td>e0139aae-7811-40ca-abc6-3fcb79140a6b</td>\n", " <td>Tim.Burrell@microsoft.com</td>\n", " <td>Tim Burrell (MSTIC)</td>\n", " <td>...</td>\n", " <td>[]</td>\n", " <td>print \"192.168.15.6\" \\n</td>\n", " <td>{\"print_0\":\"192.168.15.6\",\"__entityMapping\":{\"...</td>\n", " <td>2020-11-17T09:26:33.557+00:00</td>\n", " <td>2020-11-18T09:26:33.557+00:00</td>\n", " <td>None</td>\n", " <td>None</td>\n", " <td>None</td>\n", " <td>None</td>\n", " <td>NaN</td>\n", " </tr>\n", " <tr>\n", " <th>1</th>\n", " <td>Microsoft.SecurityInsights/Bookmarks</td>\n", " <td>test 1</td>\n", " <td>2020-11-18T15:25:01.1843361+00:00</td>\n", " <td>2020-11-18T15:25:01.1843361+00:00</td>\n", " <td>b3a76793-1a0d-4bfe-95f6-96919d4b9acf</td>\n", " <td>bnick@microsoft.com</td>\n", " <td>Ben Nick</td>\n", " <td>b3a76793-1a0d-4bfe-95f6-96919d4b9acf</td>\n", " <td>bnick@microsoft.com</td>\n", " <td>Ben Nick</td>\n", " <td>...</td>\n", " <td>[fluffyDogCampaign]</td>\n", " <td>let auditLookback = 14d;\\n// Setting threshold...</td>\n", " <td>{\"InitiatedBy\":\"seb@seccxp.ninja\",\"IpAddress\":...</td>\n", " <td>NaN</td>\n", " <td>NaN</td>\n", " <td>None</td>\n", " <td>None</td>\n", " <td>None</td>\n", " <td>None</td>\n", " <td>this looks suspicious</td>\n", " </tr>\n", " <tr>\n", " <th>2</th>\n", " <td>Microsoft.SecurityInsights/Bookmarks</td>\n", " <td>failed logons - decb171c8160 (1)</td>\n", " <td>2020-11-19T11:26:31.3053573+00:00</td>\n", " <td>2020-11-19T11:26:31.3053573+00:00</td>\n", " <td>518a3ca6-44f0-4ac7-8179-97d18e48d65c</td>\n", " <td>pascals@microsoft.com</td>\n", " <td>Pascal Sauliere</td>\n", " <td>518a3ca6-44f0-4ac7-8179-97d18e48d65c</td>\n", " <td>pascals@microsoft.com</td>\n", " <td>Pascal Sauliere</td>\n", " <td>...</td>\n", " <td>[]</td>\n", " <td>// Event: An account failed to log on\\nSecurit...</td>\n", " <td>{\"TenantId\":\"8ecf8077-cf51-4820-aadd-14040956f...</td>\n", " <td>NaN</td>\n", " <td>NaN</td>\n", " <td>None</td>\n", " <td>None</td>\n", " <td>None</td>\n", " <td>None</td>\n", " <td>NaN</td>\n", " </tr>\n", " <tr>\n", " <th>3</th>\n", " <td>Microsoft.SecurityInsights/Bookmarks</td>\n", " <td>Rare Audit activity initiated by App - cbade9...</td>\n", " <td>2020-11-11T18:39:16.6537628+00:00</td>\n", " <td>2020-11-11T18:39:16.6537628+00:00</td>\n", " <td>f6b78447-93dc-4041-a22a-6eb1c34265e2</td>\n", " <td>Umesh.Nagdev@microsoft.com</td>\n", " <td>Umesh Nagdev</td>\n", " <td>f6b78447-93dc-4041-a22a-6eb1c34265e2</td>\n", " <td>Umesh.Nagdev@microsoft.com</td>\n", " <td>Umesh Nagdev</td>\n", " <td>...</td>\n", " <td>[]</td>\n", " <td>let current = 1d;\\nlet auditLookback = 14d;\\nl...</td>\n", " <td>{\"InitiatedByApp\":\"Microsoft Azure AD Group-Ba...</td>\n", " <td>2020-11-10T18:39:01.061+00:00</td>\n", " <td>2020-11-11T18:39:01.061+00:00</td>\n", " <td>None</td>\n", " <td>None</td>\n", " <td>None</td>\n", " <td>None</td>\n", " <td>NaN</td>\n", " </tr>\n", " <tr>\n", " <th>4</th>\n", " <td>Microsoft.SecurityInsights/Bookmarks</td>\n", " <td>ThreatIntelligenceIndicator - 4193cb45b90a (2)</td>\n", " <td>2020-11-11T16:08:45.6964987+00:00</td>\n", " <td>2020-11-11T16:08:45.6964987+00:00</td>\n", " <td>525c09b5-61ef-4e10-8150-b44c97ead3a1</td>\n", " <td>Andrew.Blumhardt@microsoft.com</td>\n", " <td>Andrew Blumhardt</td>\n", " <td>525c09b5-61ef-4e10-8150-b44c97ead3a1</td>\n", " <td>Andrew.Blumhardt@microsoft.com</td>\n", " <td>Andrew Blumhardt</td>\n", " <td>...</td>\n", " <td>[]</td>\n", " <td>ThreatIntelligenceIndicator</td>\n", " <td>{\"TenantId\":\"8ecf8077-cf51-4820-aadd-14040956f...</td>\n", " <td>2020-11-10T16:08:26.089+00:00</td>\n", " <td>2020-11-11T16:08:26.089+00:00</td>\n", " <td>None</td>\n", " <td>None</td>\n", " <td>None</td>\n", " <td>None</td>\n", " <td>My Bookmark</td>\n", " </tr>\n", " </tbody>\n", "</table>\n", "<p>5 rows × 21 columns</p>\n", "</div>" ], "text/plain": [ " type \\\n", "0 Microsoft.SecurityInsights/Bookmarks \n", "1 Microsoft.SecurityInsights/Bookmarks \n", "2 Microsoft.SecurityInsights/Bookmarks \n", "3 Microsoft.SecurityInsights/Bookmarks \n", "4 Microsoft.SecurityInsights/Bookmarks \n", "\n", " properties.displayName \\\n", "0 mercury IP \n", "1 test 1 \n", "2 failed logons - decb171c8160 (1) \n", "3 Rare Audit activity initiated by App - cbade9... \n", "4 ThreatIntelligenceIndicator - 4193cb45b90a (2) \n", "\n", " properties.created properties.updated \\\n", "0 2020-11-18T09:26:54.1605891+00:00 2020-11-18T09:26:54.1605891+00:00 \n", "1 2020-11-18T15:25:01.1843361+00:00 2020-11-18T15:25:01.1843361+00:00 \n", "2 2020-11-19T11:26:31.3053573+00:00 2020-11-19T11:26:31.3053573+00:00 \n", "3 2020-11-11T18:39:16.6537628+00:00 2020-11-11T18:39:16.6537628+00:00 \n", "4 2020-11-11T16:08:45.6964987+00:00 2020-11-11T16:08:45.6964987+00:00 \n", "\n", " properties.createdBy.objectId properties.createdBy.email \\\n", "0 e0139aae-7811-40ca-abc6-3fcb79140a6b Tim.Burrell@microsoft.com \n", "1 b3a76793-1a0d-4bfe-95f6-96919d4b9acf bnick@microsoft.com \n", "2 518a3ca6-44f0-4ac7-8179-97d18e48d65c pascals@microsoft.com \n", "3 f6b78447-93dc-4041-a22a-6eb1c34265e2 Umesh.Nagdev@microsoft.com \n", "4 525c09b5-61ef-4e10-8150-b44c97ead3a1 Andrew.Blumhardt@microsoft.com \n", "\n", " properties.createdBy.name properties.updatedBy.objectId \\\n", "0 Tim Burrell (MSTIC) e0139aae-7811-40ca-abc6-3fcb79140a6b \n", "1 Ben Nick b3a76793-1a0d-4bfe-95f6-96919d4b9acf \n", "2 Pascal Sauliere 518a3ca6-44f0-4ac7-8179-97d18e48d65c \n", "3 Umesh Nagdev f6b78447-93dc-4041-a22a-6eb1c34265e2 \n", "4 Andrew Blumhardt 525c09b5-61ef-4e10-8150-b44c97ead3a1 \n", "\n", " properties.updatedBy.email properties.updatedBy.name ... \\\n", "0 Tim.Burrell@microsoft.com Tim Burrell (MSTIC) ... \n", "1 bnick@microsoft.com Ben Nick ... \n", "2 pascals@microsoft.com Pascal Sauliere ... \n", "3 Umesh.Nagdev@microsoft.com Umesh Nagdev ... \n", "4 Andrew.Blumhardt@microsoft.com Andrew Blumhardt ... \n", "\n", " properties.labels properties.query \\\n", "0 [] print \"192.168.15.6\" \\n \n", "1 [fluffyDogCampaign] let auditLookback = 14d;\\n// Setting threshold... \n", "2 [] // Event: An account failed to log on\\nSecurit... \n", "3 [] let current = 1d;\\nlet auditLookback = 14d;\\nl... \n", "4 [] ThreatIntelligenceIndicator \n", "\n", " properties.queryResult \\\n", "0 {\"print_0\":\"192.168.15.6\",\"__entityMapping\":{\"... \n", "1 {\"InitiatedBy\":\"seb@seccxp.ninja\",\"IpAddress\":... \n", "2 {\"TenantId\":\"8ecf8077-cf51-4820-aadd-14040956f... \n", "3 {\"InitiatedByApp\":\"Microsoft Azure AD Group-Ba... \n", "4 {\"TenantId\":\"8ecf8077-cf51-4820-aadd-14040956f... \n", "\n", " properties.queryStartTime properties.queryEndTime \\\n", "0 2020-11-17T09:26:33.557+00:00 2020-11-18T09:26:33.557+00:00 \n", "1 NaN NaN \n", "2 NaN NaN \n", "3 2020-11-10T18:39:01.061+00:00 2020-11-11T18:39:01.061+00:00 \n", "4 2020-11-10T16:08:26.089+00:00 2020-11-11T16:08:26.089+00:00 \n", "\n", " properties.incidentInfo.incidentId properties.incidentInfo.title \\\n", "0 None None \n", "1 None None \n", "2 None None \n", "3 None None \n", "4 None None \n", "\n", " properties.incidentInfo.relationName properties.incidentInfo.severity \\\n", "0 None None \n", "1 None None \n", "2 None None \n", "3 None None \n", "4 None None \n", "\n", " properties.notes \n", "0 NaN \n", "1 this looks suspicious \n", "2 NaN \n", "3 NaN \n", "4 My Bookmark \n", "\n", "[5 rows x 21 columns]" ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "bkmarks = azs.get_bookmarks(ws.value)\n", "bkmarks.head().drop(columns=[\"id\", \"etag\", \"name\"])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can also interact with Incidents via the API to get a set of all incidents, or a single incident:" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "data": { "text/html": [ "<div>\n", "<style scoped>\n", " .dataframe tbody tr th:only-of-type {\n", " vertical-align: middle;\n", " }\n", "\n", " .dataframe tbody tr th {\n", " vertical-align: top;\n", " }\n", "\n", " .dataframe thead th {\n", " text-align: right;\n", " }\n", "</style>\n", "<table border=\"1\" class=\"dataframe\">\n", " <thead>\n", " <tr style=\"text-align: right;\">\n", " <th></th>\n", " <th>id</th>\n", " <th>name</th>\n", " <th>etag</th>\n", " <th>type</th>\n", " <th>properties.title</th>\n", " <th>properties.severity</th>\n", " <th>properties.status</th>\n", " <th>properties.owner.objectId</th>\n", " <th>properties.owner.email</th>\n", " <th>properties.owner.assignedTo</th>\n", " <th>...</th>\n", " <th>properties.additionalData.commentsCount</th>\n", " <th>properties.additionalData.alertProductNames</th>\n", " <th>properties.additionalData.tactics</th>\n", " <th>properties.firstActivityTimeGenerated</th>\n", " <th>properties.lastActivityTimeGenerated</th>\n", " <th>properties.relatedAnalyticRuleIds</th>\n", " <th>properties.incidentUrl</th>\n", " <th>properties.description</th>\n", " <th>properties.firstActivityTimeUtc</th>\n", " <th>properties.lastActivityTimeUtc</th>\n", " </tr>\n", " </thead>\n", " <tbody>\n", " <tr>\n", " <th>0</th>\n", " <td>/subscriptions/d1d8779d-38d7-4f06-91db-9cbc8de...</td>\n", " <td>aabf6bcd-4134-b07b-1152-040aa0cdf069</td>\n", " <td>\"0402f99f-0000-0100-0000-5fbd43d50000\"</td>\n", " <td>Microsoft.SecurityInsights/Incidents</td>\n", " <td>Time series anomaly detection for total volume...</td>\n", " <td>High</td>\n", " <td>New</td>\n", " <td>None</td>\n", " <td>None</td>\n", " <td>None</td>\n", " <td>...</td>\n", " <td>2</td>\n", " <td>[Microsoft Sentinel]</td>\n", " <td>[Exfiltration]</td>\n", " <td>2020-11-24T17:10:35.7652885Z</td>\n", " <td>2020-11-24T17:10:35.7652885Z</td>\n", " <td>[/subscriptions/d1d8779d-38d7-4f06-91db-9cbc8d...</td>\n", " <td>https://portal.azure.com/#asset/Microsoft_Azur...</td>\n", " <td>NaN</td>\n", " <td>NaN</td>\n", " <td>NaN</td>\n", " </tr>\n", " <tr>\n", " <th>1</th>\n", " <td>/subscriptions/d1d8779d-38d7-4f06-91db-9cbc8de...</td>\n", " <td>e917efd0-331d-48b7-81d7-6205cee787f5</td>\n", " <td>\"0302de84-0000-0100-0000-5fbd23f30000\"</td>\n", " <td>Microsoft.SecurityInsights/Incidents</td>\n", " <td>XASE SENSITIVITY TEST</td>\n", " <td>Medium</td>\n", " <td>New</td>\n", " <td>None</td>\n", " <td>None</td>\n", " <td>None</td>\n", " <td>...</td>\n", " <td>0</td>\n", " <td>[Microsoft Sentinel]</td>\n", " <td>[]</td>\n", " <td>2020-11-24T15:17:06.8646498Z</td>\n", " <td>2020-11-24T15:17:06.8646498Z</td>\n", " <td>[/subscriptions/d1d8779d-38d7-4f06-91db-9cbc8d...</td>\n", " <td>https://portal.azure.com/#asset/Microsoft_Azur...</td>\n", " <td>LOWER CASE</td>\n", " <td>2020-11-24T14:55:03.95Z</td>\n", " <td>2020-11-24T14:55:03.95Z</td>\n", " </tr>\n", " <tr>\n", " <th>2</th>\n", " <td>/subscriptions/d1d8779d-38d7-4f06-91db-9cbc8de...</td>\n", " <td>81d59f6e-988f-4758-a2d2-90886befccab</td>\n", " <td>\"03029c83-0000-0100-0000-5fbd23d10000\"</td>\n", " <td>Microsoft.SecurityInsights/Incidents</td>\n", " <td>Case Sensitivity test UPPER</td>\n", " <td>Medium</td>\n", " <td>New</td>\n", " <td>None</td>\n", " <td>None</td>\n", " <td>None</td>\n", " <td>...</td>\n", " <td>0</td>\n", " <td>[Microsoft Sentinel]</td>\n", " <td>[]</td>\n", " <td>2020-11-24T15:16:33.5131821Z</td>\n", " <td>2020-11-24T15:16:33.5131821Z</td>\n", " <td>[/subscriptions/d1d8779d-38d7-4f06-91db-9cbc8d...</td>\n", " <td>https://portal.azure.com/#asset/Microsoft_Azur...</td>\n", " <td>sdff</td>\n", " <td>2020-11-24T14:57:49.43Z</td>\n", " <td>2020-11-24T14:57:49.43Z</td>\n", " </tr>\n", " <tr>\n", " <th>3</th>\n", " <td>/subscriptions/d1d8779d-38d7-4f06-91db-9cbc8de...</td>\n", " <td>b68151e7-890f-48aa-befb-3de2bc987557</td>\n", " <td>\"03022274-0000-0100-0000-5fbd222f0000\"</td>\n", " <td>Microsoft.SecurityInsights/Incidents</td>\n", " <td>Potential Password Spray</td>\n", " <td>Medium</td>\n", " <td>New</td>\n", " <td>None</td>\n", " <td>None</td>\n", " <td>None</td>\n", " <td>...</td>\n", " <td>0</td>\n", " <td>[Microsoft Sentinel]</td>\n", " <td>[Persistence]</td>\n", " <td>2020-11-24T15:09:35.0020779Z</td>\n", " <td>2020-11-24T15:09:35.0020779Z</td>\n", " <td>[/subscriptions/d1d8779d-38d7-4f06-91db-9cbc8d...</td>\n", " <td>https://portal.azure.com/#asset/Microsoft_Azur...</td>\n", " <td>Description with a link</td>\n", " <td>2020-11-24T10:04:32.5297051Z</td>\n", " <td>2020-11-24T15:04:32.5297051Z</td>\n", " </tr>\n", " <tr>\n", " <th>4</th>\n", " <td>/subscriptions/d1d8779d-38d7-4f06-91db-9cbc8de...</td>\n", " <td>49f91f51-30ce-4028-9117-96ca3debbe14</td>\n", " <td>\"0302f05f-0000-0100-0000-5fbd204e0000\"</td>\n", " <td>Microsoft.SecurityInsights/Incidents</td>\n", " <td>Case Sensitivity test UPPER</td>\n", " <td>Medium</td>\n", " <td>New</td>\n", " <td>None</td>\n", " <td>None</td>\n", " <td>None</td>\n", " <td>...</td>\n", " <td>0</td>\n", " <td>[Microsoft Sentinel]</td>\n", " <td>[]</td>\n", " <td>2020-11-24T15:01:33.9949456Z</td>\n", " <td>2020-11-24T15:01:33.9949456Z</td>\n", " <td>[/subscriptions/d1d8779d-38d7-4f06-91db-9cbc8d...</td>\n", " <td>https://portal.azure.com/#asset/Microsoft_Azur...</td>\n", " <td>sdff</td>\n", " <td>2020-11-24T14:41:32.13Z</td>\n", " <td>2020-11-24T14:41:32.13Z</td>\n", " </tr>\n", " </tbody>\n", "</table>\n", "<p>5 rows × 27 columns</p>\n", "</div>" ], "text/plain": [ " id \\\n", "0 /subscriptions/d1d8779d-38d7-4f06-91db-9cbc8de... \n", "1 /subscriptions/d1d8779d-38d7-4f06-91db-9cbc8de... \n", "2 /subscriptions/d1d8779d-38d7-4f06-91db-9cbc8de... \n", "3 /subscriptions/d1d8779d-38d7-4f06-91db-9cbc8de... \n", "4 /subscriptions/d1d8779d-38d7-4f06-91db-9cbc8de... \n", "\n", " name \\\n", "0 aabf6bcd-4134-b07b-1152-040aa0cdf069 \n", "1 e917efd0-331d-48b7-81d7-6205cee787f5 \n", "2 81d59f6e-988f-4758-a2d2-90886befccab \n", "3 b68151e7-890f-48aa-befb-3de2bc987557 \n", "4 49f91f51-30ce-4028-9117-96ca3debbe14 \n", "\n", " etag \\\n", "0 \"0402f99f-0000-0100-0000-5fbd43d50000\" \n", "1 \"0302de84-0000-0100-0000-5fbd23f30000\" \n", "2 \"03029c83-0000-0100-0000-5fbd23d10000\" \n", "3 \"03022274-0000-0100-0000-5fbd222f0000\" \n", "4 \"0302f05f-0000-0100-0000-5fbd204e0000\" \n", "\n", " type \\\n", "0 Microsoft.SecurityInsights/Incidents \n", "1 Microsoft.SecurityInsights/Incidents \n", "2 Microsoft.SecurityInsights/Incidents \n", "3 Microsoft.SecurityInsights/Incidents \n", "4 Microsoft.SecurityInsights/Incidents \n", "\n", " properties.title properties.severity \\\n", "0 Time series anomaly detection for total volume... High \n", "1 XASE SENSITIVITY TEST Medium \n", "2 Case Sensitivity test UPPER Medium \n", "3 Potential Password Spray Medium \n", "4 Case Sensitivity test UPPER Medium \n", "\n", " properties.status properties.owner.objectId properties.owner.email \\\n", "0 New None None \n", "1 New None None \n", "2 New None None \n", "3 New None None \n", "4 New None None \n", "\n", " properties.owner.assignedTo ... properties.additionalData.commentsCount \\\n", "0 None ... 2 \n", "1 None ... 0 \n", "2 None ... 0 \n", "3 None ... 0 \n", "4 None ... 0 \n", "\n", " properties.additionalData.alertProductNames \\\n", "0 [Microsoft Sentinel] \n", "1 [Microsoft Sentinel] \n", "2 [Microsoft Sentinel] \n", "3 [Microsoft Sentinel] \n", "4 [Microsoft Sentinel] \n", "\n", " properties.additionalData.tactics properties.firstActivityTimeGenerated \\\n", "0 [Exfiltration] 2020-11-24T17:10:35.7652885Z \n", "1 [] 2020-11-24T15:17:06.8646498Z \n", "2 [] 2020-11-24T15:16:33.5131821Z \n", "3 [Persistence] 2020-11-24T15:09:35.0020779Z \n", "4 [] 2020-11-24T15:01:33.9949456Z \n", "\n", " properties.lastActivityTimeGenerated \\\n", "0 2020-11-24T17:10:35.7652885Z \n", "1 2020-11-24T15:17:06.8646498Z \n", "2 2020-11-24T15:16:33.5131821Z \n", "3 2020-11-24T15:09:35.0020779Z \n", "4 2020-11-24T15:01:33.9949456Z \n", "\n", " properties.relatedAnalyticRuleIds \\\n", "0 [/subscriptions/d1d8779d-38d7-4f06-91db-9cbc8d... \n", "1 [/subscriptions/d1d8779d-38d7-4f06-91db-9cbc8d... \n", "2 [/subscriptions/d1d8779d-38d7-4f06-91db-9cbc8d... \n", "3 [/subscriptions/d1d8779d-38d7-4f06-91db-9cbc8d... \n", "4 [/subscriptions/d1d8779d-38d7-4f06-91db-9cbc8d... \n", "\n", " properties.incidentUrl properties.description \\\n", "0 https://portal.azure.com/#asset/Microsoft_Azur... NaN \n", "1 https://portal.azure.com/#asset/Microsoft_Azur... LOWER CASE \n", "2 https://portal.azure.com/#asset/Microsoft_Azur... sdff \n", "3 https://portal.azure.com/#asset/Microsoft_Azur... Description with a link \n", "4 https://portal.azure.com/#asset/Microsoft_Azur... sdff \n", "\n", " properties.firstActivityTimeUtc properties.lastActivityTimeUtc \n", "0 NaN NaN \n", "1 2020-11-24T14:55:03.95Z 2020-11-24T14:55:03.95Z \n", "2 2020-11-24T14:57:49.43Z 2020-11-24T14:57:49.43Z \n", "3 2020-11-24T10:04:32.5297051Z 2020-11-24T15:04:32.5297051Z \n", "4 2020-11-24T14:41:32.13Z 2020-11-24T14:41:32.13Z \n", "\n", "[5 rows x 27 columns]" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "incidents = azs.get_incidents(res_id=ws.value)\n", "display(incidents.head())" ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "data": { "text/html": [ "<div>\n", "<style scoped>\n", " .dataframe tbody tr th:only-of-type {\n", " vertical-align: middle;\n", " }\n", "\n", " .dataframe tbody tr th {\n", " vertical-align: top;\n", " }\n", "\n", " .dataframe thead th {\n", " text-align: right;\n", " }\n", "</style>\n", "<table border=\"1\" class=\"dataframe\">\n", " <thead>\n", " <tr style=\"text-align: right;\">\n", " <th></th>\n", " <th>id</th>\n", " <th>name</th>\n", " <th>etag</th>\n", " <th>type</th>\n", " <th>properties.title</th>\n", " <th>properties.severity</th>\n", " <th>properties.status</th>\n", " <th>properties.owner.objectId</th>\n", " <th>properties.owner.email</th>\n", " <th>properties.owner.assignedTo</th>\n", " <th>...</th>\n", " <th>properties.incidentNumber</th>\n", " <th>properties.additionalData.alertsCount</th>\n", " <th>properties.additionalData.bookmarksCount</th>\n", " <th>properties.additionalData.commentsCount</th>\n", " <th>properties.additionalData.alertProductNames</th>\n", " <th>properties.additionalData.tactics</th>\n", " <th>properties.firstActivityTimeGenerated</th>\n", " <th>properties.lastActivityTimeGenerated</th>\n", " <th>properties.relatedAnalyticRuleIds</th>\n", " <th>properties.incidentUrl</th>\n", " </tr>\n", " </thead>\n", " <tbody>\n", " <tr>\n", " <th>0</th>\n", " <td>/subscriptions/d1d8779d-38d7-4f06-91db-9cbc8de...</td>\n", " <td>aabf6bcd-4134-b07b-1152-040aa0cdf069</td>\n", " <td>\"0402f99f-0000-0100-0000-5fbd43d50000\"</td>\n", " <td>Microsoft.SecurityInsights/Incidents</td>\n", " <td>Time series anomaly detection for total volume...</td>\n", " <td>High</td>\n", " <td>New</td>\n", " <td>None</td>\n", " <td>None</td>\n", " <td>None</td>\n", " <td>...</td>\n", " <td>4601</td>\n", " <td>1</td>\n", " <td>0</td>\n", " <td>2</td>\n", " <td>[Microsoft Sentinel]</td>\n", " <td>[Exfiltration]</td>\n", " <td>2020-11-24T17:10:35.7652885Z</td>\n", " <td>2020-11-24T17:10:35.7652885Z</td>\n", " <td>[/subscriptions/d1d8779d-38d7-4f06-91db-9cbc8d...</td>\n", " <td>https://portal.azure.com/#asset/Microsoft_Azur...</td>\n", " </tr>\n", " </tbody>\n", "</table>\n", "<p>1 rows × 24 columns</p>\n", "</div>" ], "text/plain": [ " id \\\n", "0 /subscriptions/d1d8779d-38d7-4f06-91db-9cbc8de... \n", "\n", " name \\\n", "0 aabf6bcd-4134-b07b-1152-040aa0cdf069 \n", "\n", " etag \\\n", "0 \"0402f99f-0000-0100-0000-5fbd43d50000\" \n", "\n", " type \\\n", "0 Microsoft.SecurityInsights/Incidents \n", "\n", " properties.title properties.severity \\\n", "0 Time series anomaly detection for total volume... High \n", "\n", " properties.status properties.owner.objectId properties.owner.email \\\n", "0 New None None \n", "\n", " properties.owner.assignedTo ... properties.incidentNumber \\\n", "0 None ... 4601 \n", "\n", " properties.additionalData.alertsCount \\\n", "0 1 \n", "\n", " properties.additionalData.bookmarksCount \\\n", "0 0 \n", "\n", " properties.additionalData.commentsCount \\\n", "0 2 \n", "\n", " properties.additionalData.alertProductNames \\\n", "0 [Microsoft Sentinel] \n", "\n", " properties.additionalData.tactics properties.firstActivityTimeGenerated \\\n", "0 [Exfiltration] 2020-11-24T17:10:35.7652885Z \n", "\n", " properties.lastActivityTimeGenerated \\\n", "0 2020-11-24T17:10:35.7652885Z \n", "\n", " properties.relatedAnalyticRuleIds \\\n", "0 [/subscriptions/d1d8779d-38d7-4f06-91db-9cbc8d... \n", "\n", " properties.incidentUrl \n", "0 https://portal.azure.com/#asset/Microsoft_Azur... \n", "\n", "[1 rows x 24 columns]" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "incident = azs.get_incident(incident_id = incidents.iloc[0]['name'] , res_id=ws.value)\n", "display(incident)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You can also interact with an incident - adding comments or changing properties such as severity or status:" ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Comment posted.\n" ] } ], "source": [ "azs.post_comment(incident_id = incident.iloc[0]['name'], comment=\"This is a test comment\", res_id=ws.value)" ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Incident updated.\n" ] } ], "source": [ "azs.update_incident(incident_id = incident.iloc[0]['name'], update_items={\"severity\":\"High\"}, res_id=ws.value)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "hide_input": false, "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.9" }, "latex_envs": { "LaTeX_envs_menu_present": true, "autoclose": false, "autocomplete": true, "bibliofile": "biblio.bib", "cite_by": "apalike", "current_citInitial": 1, "eqLabelWithNumbers": true, "eqNumInitial": 1, "hotkeys": { "equation": "Ctrl-E", "itemize": "Ctrl-I" }, "labels_anchors": false, "latex_user_defs": false, "report_style_numbering": false, "user_envs_cfg": false }, "toc": { "base_numbering": 1, "nav_menu": {}, "number_sections": false, "sideBar": true, "skip_h1_title": false, "title_cell": "Table of Contents", "title_sidebar": "Contents", "toc_cell": true, "toc_position": {}, "toc_section_display": true, "toc_window_display": false }, "varInspector": { "cols": { "lenName": 16, "lenType": 16, "lenVar": 40 }, "kernels_config": { "python": { "delete_cmd_postfix": "", "delete_cmd_prefix": "del ", "library": "var_list.py", "varRefreshCmd": "print(var_dic_list())" }, "r": { "delete_cmd_postfix": ") ", "delete_cmd_prefix": "rm(", "library": "var_list.r", "varRefreshCmd": "cat(var_dic_list()) " } }, "types_to_exclude": [ "module", "function", "builtin_function_or_method", "instance", "_Feature" ], "window_display": false }, "widgets": { "application/vnd.jupyter.widget-state+json": { "state": {}, "version_major": 2, "version_minor": 0 } } }, "nbformat": 4, "nbformat_minor": 4 }