2010-03-05

How to Throttle Removing ESX Snapshots

Recently we had a planned maintenance window that required shutting down power to the entire building. As part of the procedure we created snapshots of each VM before the powerdown. With PowerCLI this is really easy

get-vm | new-snapshot -name "before power down" -description "powerdown" -runasync -confirm:$false

 
Of course there is a whole methodology of how to perform this shutdown - what comes before what and what should be powered down in which order, but that is not the issue at hand.

After powering everything back up and verifying that all was ok I wanted to remove the snapshots. Now since I had given the snapshot a name it is easy to remove that specific snapshot.

$snapshots = get-vm | get-snapshot -name "before power down"
foreach ($item in $snapshots) {
 remove-snaphot $item -confirm:$false
}

 

OK so that started to remove the snapshots - but one at a time
WHICH …. WAS…. SLOW ……………………………………….

So a quick change to the code adding the runasync switch

$snaps = get-vm | get-snapshot -name "before power down"
foreach ($item in $snapshots) {
 remove-snaphot $item -confirm:$false -runasync
}

 

And it started to run them all one after the other. Great!

Or was it ??? within a short time the host choked! trying to remove all the snapshots at once - simultaneously - it was not happy.

In the end I ended up looping though the array manually. But after consulting with the PowerCLI master Luc Dekens who helped me out tonight with finding a way to throttle the process. Here is is below

$maxtasks = 5

$snaps = get-vm | get-snapshot -name "before power down"

$i = 0
while($i -lt $snaps.Count){
 Remove-Snapshot -Snapshot $snaps[$i] -RunAsync -Confirm:$false
 $tasks = Get-Task -Status "Running" | where {$_.Name -eq "RemoveSnapshot_Task"}
 while($tasks.Count -gt ($maxtasks-1)) {
 sleep 30
 $tasks = Get-Task -Status "Running" | where {$_.Name -eq "RemoveSnapshot_Task"}
     }
     $i++
}


First I I relied on the same $snaps variable as in the previous example to get all the relevant snapshots

I then ran through a loop from $i=0 until the number of snapshots and removed the snapshot

I then continue the loop and retrieve the number of running "RemoveSnapshot" tasks. As long as there are more than 4 - then the script will sleep, once there are less than 4 it will increment $i by 1 and will continue to remove the next snapshot.

I am sure you can use this logic for other kind of tasks as well.

Thanks again Master Luc for your help!