by Dr. Radut

Migrating Log Analytics tables between workspaces

Published by Laurie Rhodes -

I've been working extensively with Azure Monitor table schemas recently. In preparation for the deprecation of the legacy API data ingestion method for Log Analytics workspaces, I needed a simple method to recreate custom log tables as Data Collection Rule (DCR)-based tables while also migrating them to a new workspace.

The solution turned out to be straightforward, utilising some basic PowerShell functions I've previously shared on this site.  These modules are unique in that they automatically derive the current Microsoft API versions based on the Azure object ID.  This capability allows for easy "Get" and "Push" operations of Azure objects to and from JSON representations.

You can find the modules here: https://github.com/LaurieRhodes/AZRest.

For this task, I'll migrate an old custom table from a Sentinel workspace.

 

I will use two functions.  Get-AzureObject to create a JSON file for editing, and Push-AzureObject to write the updated AMA table to a new workspace.

#Get an Authorised Header

$authHeader = Get-Header -scope "azure"  -Tenant "laurierhodes.info" -AppId "XXXXXXXXXXXXXXXXXXXXXXXXXXXXX" `
                         -secret "XXXXXXXXXXXXXXXXXXXXXXXXXXXXX"

# Retrieve an up to date list of namespace versions (once per session)

if (!$AzAPIVersions){$AzAPIVersions = Get-AzureAPIVersions -header $authHeader -SubscriptionID "XXXXXXXXXXXXXXXXXXXXXXXXXXXX"}

$id='/subscriptions/XXXXXXXX-XXXXXXX/resourcegroups/sentinel/providers/microsoft.operationalinsights/workspaces/asesentinel/tables/AzureCompliance_CL'

$object = $null
$object =    Get-Azureobject -AuthHeader $authHeader -apiversions $AzAPIVersions -id $id

Out-File -FilePath "C:\temp\AzureCompliance_CL.json" -InputObject (convertto-json -InputObject $object -Depth 10) -Force

The element to note is that every table in Log Analytics is referenced as:
/providers/microsoft.operationalinsights/workspaces/<workspace>/tables/<table>

I need the schema for a specific table (AzureCompliance_CL) which I simply tack on to the end of the Get-AzureObject call. This will write the JSON file to a location I can edit it from.

The schema in question is a legacy or "Classic" type. To migrate it, I need to change the table to a "DataCollectionRuleBased" type and then redeploy it to Azure.

Every object in Azure is identified by its ID string. By altering that ID, the Push-AzureObject function can be used to migrate the updated table into a Log Analytics workspace that exists in a different Subscription or Resource Group.

Using the same PowerShell session, I can push the altered schema back to the cloud.

$file = "C:\temp\AzureCompliance_CL.json" 

Get-jsonfile -Path $file | Push-Azureobject -authHeader $authHeader -apiversions $AzAPIVersions

I can now see the migrated DCR-based table in the new workspace.

 

 

Tags
PowerShell