#AWS Glacier & SNS #PowerShell cmdlets

So, yeah, I finally finished writing this. In terms of coding projects, this one has been the most challenging for a number of reasons. First, AWS Glacier is cold storage, which means that vault inventories and archive retrievals have (approximately) a four hour wait before further action can be taken. After the job completes, it only stays visible for 24 hours. As you might imagine, I’ve lost track of how many times I’ve kicked off jobs for testing purposes, and then never got back to it due to family or work obligations. Second, those family and work obligations have been getting in the way. Most of the time, after the work day is done and the kids are asleep (providing I’m not on the road for work) I simply don’t want to sit in front of my laptop again. My brain is fried and wants a “bourbon, neat” for its troubles. 🙂

“Why did you do this?”, you might ask, and I would tell that 1.) I’m a nerd that likes to do nerdly things in order to keep the nerdly brain cells happy and well-exercised, and 2.) Glacier is a hell of a cool solution for my particular use case. Namely, “What the bloody hell do I do with 100’s of GBs (and more) of digital family photos and videos?!”

Usually, a service like Mozy/BackBlaze/Carbonite would be a suitable solution, but I store everything on a NAS and those silly companies usually pick NAS as the dividing line between consumer and business. The NAS I speak of is an Iomega ix-200, and if you’ve ever used one, you’re probably painfully aware that it’s a bit of a dog on the performance tip, so doing some fancy/lame thing like copying files off the NAS just to sync “to the cloud!” will eat up tons of local disk and be horribly slow. Slow as in, whenever it finishes, the zombie apocalypse will be upon us and I won’t care about digital photos anymore, just eating brains and trying to stay out of Rick Grimes‘ way.

Based on the above, even if I use the ever-loving hell out of Glacier, with the currently available, pricing I’ll never use more than a couple dollars a month. Pretty awesome, I think. OK, you’ve tolerated me long enough.

The code is up on Ye Olde Github here: https://github.com/sixfootdad/AWS-Glacier-SNS-PowerShell.

Be sure to check out the README and holler if you have any querstions.

Search-Cloud is cooler than you might think

PowerCLI 5.0.1 delivered new vCloud Director functionality including a cmdlet named Search-Cloud. If you’ve taken a look at Get-Help Search-Cloud, you’ll see a parameter named -QueryType used to “Specify what types of objects you want to search for”. That’s great, but what types of objects *can* you search for? The help, sadly, isn’t very helpful.

There are a couple different ways to find out. One of them is specific to PowerCLI 5.0.1 and is the Cmdlet Reference. By drilling into the All Cmdlets section and locating Search-Cloud, you’ll see that the parameter named QueryType is of the type QueryType, which is a .NET Enum. Click on the QueryType and you’ll see all the cool things that you can search for using Search-Cloud. Very nice!



There are a few other ways to figure out what the type of parameter -QueryType is, and the expected values. One of them is quite simple, use an invalid -QueryType.

PS C:\>; Search-Cloud -QueryType me
Search-Cloud : Cannot bind parameter 'QueryType'. Cannot convert value "me" to type "VMware.VimAutomation.Cloud.View
s.QueryType" due to invalid enumeration values. Specify one of the following enumeration values and try again. The p
ossible enumeration values are "User, AdminVAppNetwork, AdminUser, BlockingTask, Cell, Host, AdminCatalogItem, Vm, A
dminCatalog, Group, AllocatedExternalAddress, OrgVdcResourcePoolRelation, AdminVApp, DvSwitch, Organization, VAppTem
plate, AdminAllocatedExternalAddress, OrgNetwork, Catalog, VirtualCenter, ResourcePool, Right, StrandedUser, Portgro
up, AdminVM, Media, AdminVAppTemplate, AdminOrgVdc, OrgVdc, CatalogItem, Event, VApp, AdminOrgNetwork, VAppNetwork,
AdminShadowVM, Datastore, ProviderVdc, DatastoreProviderVdcRelation, AdminMedia, ExternalNetwork, NetworkPool, Admin
Group, Task, AdminTask, VAppOrgNetworkRelation, Role, ProviderVdcResourcePoolRelation".
At line:1 char:24
+ Search-Cloud -QueryType <;<;<;<;  me     + CategoryInfo          : InvalidArgument: (:) [Search-Cloud], ParameterBindingException     + FullyQualifiedErrorId : CannotConvertArgumentNoMessage,VMware.VimAutomation.Cloud.Commands.Cmdlets.SearchCloud 

Another would be to find the ParameterType, which shows us it’s VMware.VimAutomation.Cloud.Views.QueryType. This can be accomplished in steps:

 PS C:\>; Get-Command search-Cloud | Select-Object -ExpandProperty Parameters

Key                                Value
---                                -----
Name                               System.Management.Automation.ParameterMetadata
Filter                             System.Management.Automation.ParameterMetadata
QueryType                          System.Management.Automation.ParameterMetadata
Property                           System.Management.Automation.ParameterMetadata
Server                             System.Management.Automation.ParameterMetadata
Verbose                            System.Management.Automation.ParameterMetadata
Debug                              System.Management.Automation.ParameterMetadata
ErrorAction                        System.Management.Automation.ParameterMetadata
WarningAction                      System.Management.Automation.ParameterMetadata
ErrorVariable                      System.Management.Automation.ParameterMetadata
WarningVariable                    System.Management.Automation.ParameterMetadata
OutVariable                        System.Management.Automation.ParameterMetadata
OutBuffer                          System.Management.Automation.ParameterMetadata

And then:

PS C:\>; (Get-Command search-Cloud | Select-Object -ExpandProperty Parameters).QueryType

Name            : QueryType
ParameterType   : VMware.VimAutomation.Cloud.Views.QueryType
ParameterSets   : {[__AllParameterSets, System.Management.Automation.ParameterSetMetadata]}
IsDynamic       : False
Aliases         : {}
Attributes      : {__AllParameterSets, System.Management.Automation.ValidateNotNullOrEmptyAttribute}
SwitchParameter : False


PS C:\>; (Get-Command search-Cloud | Select-Object -ExpandProperty Parameters).QueryType.ParameterType

IsPublic IsSerial Name                      BaseType
-------- -------- ----                      --------
True     True     QueryType                 System.Enum

Now that we know it’s an enum, we can do a couple different things. The code below will list out all the names in the enum.


We’ll get effectively the same information back in this way, too.

[VMware.VimAutomation.Cloud.Views.QueryType] | Get-Member -Static -MemberType Property

PowerCLI: Remove all vmnics from a vSwitch

You can tell I’m getting rusty with the PowerCLI, which is something I intend to fix. In the meantime, I needed a PowerCLI way to remove all vmnics from a host’s vSS. If you take a look at the PowerCLI cmdlet documentation, you’ll find the syntax for Set-VirtualSwitch. It’s pretty simple; whatever NIC you pass along with the -Nic parameter overwrites whatever’s there already.


Get-VirtualSwitch -VMhost host1.lab.local -Name vSwitch1 | Set-VirtualSwitch -Nic vmnic4

The above code will replace whatever vmnics are currently connected to the vSwitch with vmnic4. But what if you want to remove all the nics from the vSwitch? The answer is simple; pass an empty array to the -Nic parameter.

$nic = @() //creates an empty array called $nic

Get-VirtualSwitch -VMhost host1.lab.local -Name vSwitch1 | Set-VirtualSwitch -Nic $nic


What’s new in PowerCLI 5?

Shay Levy recently wrote an article in PowerShell Magazine that leveraged PowerShell to determine the differences in PowerShell 2 vs PowerShell 3. I took his script, made a few small modifications, and used it in the same way to determine the differences between PowerCLI 4.1 U1 and PowerCLI 5.0.

I also added the extra step of retrieving the cmdlet’s synopsis as retrieved from Get-Help. Unfortunately, there are a number of PowerCLI 5 cmdlets that don’t have a synopsis, and in those instances I chose to go with the descriptions. These aren’t formatted as nicely as the synopses are, but it still gives you the ability to see what each new cmdlet does at a glance. The results are for all modules that come with the PowerCLI 5 installer: VMware.Deploy.Automation, VMware.ImageBuilder, VMware.VimAutomation.Core, and VMware.VimAutomation.License.

First up, the code:

# run in v2, export all core cmdlets, name and parameters
Get-Command -Module *vmware* | Select-Object -Property Name,@{Name='Parameters';Expression={$_.Parameters.Keys}} | Export-Clixml .\v2.xml

# run in v3, export all core cmdlets, name and parameters
Get-Command -Module *vmware* | Select-Object -Property Name,@{Name='Parameters';Expression={$_.Parameters.Keys}} | Export-Clixml .\v3.xml

# run either in v2 or v3 console
$v2 = Import-CliXml .\v2.xml | Sort-Object -Property Name
$v3 = Import-CliXml .\v3.xml | Sort-Object -Property Name

Compare-Object $v2 $v3 -Property Name -IncludeEqual -PassThru | ForEach-Object {

    $Command = $_

    if($_.SideIndicator -eq '==')
        $Command = $_

        $cv2 = $v2 | Where-Object {$_.Name -eq $Command.Name} | Select-Object -ExpandProperty Parameters
        $cv3 = $v3 | Where-Object {$_.Name -eq $Command.Name} | Select-Object -ExpandProperty Parameters

        $compare = Compare-Object $cv2 $cv3

                $NewParameters = $compare | Where-Object {$_.SideIndicator -eq '=>'} | ForEach-Object {$_.InputObject + ' (+)'}
                $RemovedParameters = $compare | Where-Object {$_.SideIndicator -eq '<='} | ForEach-Object {$_.InputObject + ' (-)'}

                "$($command.Name) (!)"
                $NewParameters + $RemovedParameters | Sort-Object | ForEach-Object { "`t$_"}
    elseif($_.SideIndicator -eq '=>')
        "$($Command.name) (+)"
		if ( (Get-Help $Command.Name).Synopsis -eq "") { (Get-Help $Command.Name).Description} else {(Get-Help $Command.Name).Synopsis}
        "$($Command.name) (-)`n"
} | Out-File .\diff.txt

And now, the results (plus some formatting to increase legibility):

Add-VMHost (!)
Server (+)

Connect-VIServer (!)
AllLinked (+)

Export-VApp (!)
Format (+)

Get-AlarmDefinition (!)
Id (+)

Get-Annotation (!)
Name (+)

Get-CDDrive (!)
Id (+)
Name (+)

Get-CustomAttribute (!)
Id (+)

Get-Datastore (!)
Id (+)

Get-EsxCli (!)
VMHost (+)

Get-FloppyDrive (!)
Id (+)
Name (+)

Get-Folder (!)
Type (+)

Get-HardDisk (!)
Id (+)
Name (+)

Get-NetworkAdapter (!)
Id (+)
Name (+)

Get-NicTeamingPolicy (!)
Server (+)

Get-PassthroughDevice (!)
Id (+)

Get-ScsiController (!)
Id (+)
Name (+)

Get-Snapshot (!)
Id (+)

Get-Stat (!)
Server (+)

Get-UsbDevice (!)
Id (+)
Name (+)

Get-VIPrivilege (!)
Id (+)

Get-VIRole (!)
Id (+)

Get-VirtualPortGroup (!)
Id (+)

Get-VirtualSwitch (!)
Id (+)

Get-VMGuestNetworkInterface (!)
Name (+)

Get-VMHostModule (!)
VMHost (+)

Get-VMHostNetworkAdapter (!)
Id (+)

Get-VMHostProfile (!)
Id (+)

Import-VApp (!)
Force (+)

Move-Inventory (!)
Server (+)

New-Cluster (!)
Server (+)

New-Datacenter (!)
Server (+)

New-Datastore (!)
FileSystemVersion (+)Local (-) Cifs (-) Username (-) Password (-)

New-Folder (!)
Server (+)

New-HardDisk (!)
AdvancedOption (+)

New-Template (!)
Datastore (+)
DiskStorageFormat (+)
Template (+)
TemplateFilePath (+)
VMHost (+)

New-VirtualPortGroup (!)
Server (+)

New-VM (!)
AdvancedOption (+)

New-VMHostProfile (!)
CompatibilityMode (+)

Remove-Inventory (!)
Server (+)

Remove-VIProperty (!)
VIProperty (+)

Remove-VirtualSwitch (!)
Server (+)

Remove-VMHostAccount (!)
Server (+)

Set-Datastore (!)
CongestionThresholdMillisecond (+)
StorageIOControlEnabled (+)

Set-DrsRule (!)
Server (+)

Set-HardDisk (!)
ResizeGuestPartition (+)
Server (+)

Set-NetworkAdapter (!)
Server (+)

Set-PowerCLIConfiguration (!)
DisplayDeprecationWarnings (+)

Set-VIPermission (!)
Server (+)

Set-VirtualSwitch (!)
Server (+)

Set-VMResourceConfiguration (!)
DiskLimitIOPerSecond (+)

Add-DeployRule (+)
Adds one or more rules to the rule set.

Add-EsxSoftwareDepot (+)
Adds an ESX software depot or offline depot ZIP file to the current PowerCLI session. Use this cmdlet right after you start Image Builder by adding the Image Builder snap-in. The cmdlet imports metadata from the specified depots. You can then create new image profiles and generate ISOs from the image profiles and VIBs in the depots. The metadata from the depots is kept in memory until the PowerCLI session ends. When you run the cmdlet, Image Builder downloads and parses the metadata from the depot and analyzes the VIBs for dependencies.

The depot will be added to the list of depots maintained in the implicit session variable $DefaultSoftwareDepots that stores all software depots (similar to $DefaultVIServers).

If the software depot was previously added, metadata from the depot will be downloaded again.

Add-EsxSoftwarePackage (+)
Adds new VIBs to an image profile or updates existing VIBs in an image profile. For each package, if it is newer or older than an existing package in the profile, the existing package will be replaced. Image Builder performs all image profile validation tests on the modified image profile.

The output of this cmdlet may be piped into any other cmdlet taking image profiles as input.

Note that the image profile object that is passed in will have its VibList modified. Please see the New-EsxImageProfile cmdlet for details about persistence of image profiles.

Modification is not possible if the ReadOnly property of an ImageProfile is true.

Apply-ESXImageProfile (+)
Associates the specified image profile with the specified ESXi system.

Compare-EsxImageProfile (+)
Returns an ImageProfileDiff object that specifies whether two profiles have the same VIB list and acceptance level. If two  profiles are different, the ImageProfileDiff object specifies the differences.

Copy-DeployRule (+)
Clones an existing rule.

Export-EsxImageProfile (+)
Exports an Image Profile object as either an ESXi ISO image that can be booted up and used as an ESXi installer, or an offline depot ZIP file that contains metadata plus the VIB packages. In both cases, downloads the VIB binaries and validates the VIB signatures.

You can perform the following tasks with the offline depot ZIP file:

*Import the ZIP into VMware Update Manager for patch remediation
*Download the ZIP to an ESXi host and used with esxcli for installation
*Re-import the ZIP into ImageBuilder itself using Add-EsxSoftwareDepot

You can specify either  -ExportToIso or -ExportToBundle but not both.

Get-DatastoreCluster (+)
Retrieves datastore clusters.

Get-DeployRule (+)
Gets a DeployRule object.

Get-DeployRuleSet (+)
Gets the current working rule set or the current active rule set.

Get-EsxImageProfile (+)
Lists the image profiles from software depots as well as image profiles created by the user. The output is in table form by default. Pass the image profile in the -ImageProfile parameter to the New-EsxImageProfile, Export-EsxImageProfile, Set-EsxImageProfile, Add-EsxSoftwarePackage, Remove-EsxSoftwarePackage, and Compare-EsxImageProfile cmdlets in one of these ways:

*Use the name of the image profile
*Pipe the output of this cmdlet into the cmdlets See each cmdlet’s examples for details.

Get-EsxSoftwareChannel (+)
Get-EsxSoftwareChannel [[-SoftwareDepot] <SoftwareDepot[]>] [-Verbose] [-Debug] [-ErrorAction <ActionPreference>] [-WarningAction <ActionPreference>] [-ErrorVariable <String>] [-WarningVariable <String>] [-OutVariable <String>] [-OutBuffer <Int32>]

Get-EsxSoftwarePackage (+)
Returns a list of SoftwarePackage (VIB) objects from all the connected depots, filtered by one or more options. The output is in table form by default. You can pass VIB package names and versions to the New-EsxImageProfile, Set-EsxImageProfile, Add-EsxSoftwarePackage, and Remove-EsxSoftwarePackage cmdlets via the -SoftwarePackage parameter.

The output of this cmdlet can also be piped into Add-EsxSoftwarePackage and Remove-EsxSoftwarePackage.  See the examples under those cmdlets.

Get-LicenseDataManager (+)
Returns the vSphere LicenseDataManager objects for the specified vSphere servers.

Get-VIAccount (+)
Retrieves the accounts from the ESX or vCenter Server.

Get-VIProperty (+)
Retrieves extended object properties.

Get-VMHostAttributes (+)
Gets the identifying attributes of a host.

Get-VMHostAuthentication (+)
Retrieves authentication information for the specified hosts.

Get-VMHostImageProfile (+)
Gets the image profile associated with the specified ESXi host.

Get-VMHostMatchingRules (+)
Retrieves the list of rules in the rule set that match a specified host.

Get-VMHostProfileRequiredInput (+)
Performs a check whether the available information is sufficient to apply a host profile.

Move-VApp (+)
Moves the specified virtual appliances to a new location.

New-DeployRule (+)
Creates a new rule.

New-EsxImageProfile (+)
Creates an image profile on the client machine, either by cloning or from scratch. Image Bulder performs the complete set of image profile validation tests on the new image profile and displays all errors found during validation. You pass the new image profile into the  Set-EsxImageProfile, Export-EsxImageProfile, Add-EsxSoftwarePackage, and Remove-EsxSoftwarePackage cmdlets in the -ImageProfile parameter as follows:

*Use the image profile name, as specified in the -Name parameter.
*Pipe the output of New-EsxImageProfile to the commandlet.

The resulting image profile will be preserved for the current session only. To preserve an image profile across sessions, use the Export-EsxImageProfile cmdlet. The created image profile does not have to be assigned to a variable; it is preserved in memory automatically and will be listed with the Get-EsxImageProfile cmdlet.

Specify either -NewProfile or  -CloneProfile, but not both.

If you want to edit an image profile published in a depot, you must clone the image profile before you edit it. When you clone an image profile, specify a Name parameter for the clone.

Remove-DeployRule (+)
Removes a rule from the working rule set.

Remove-EsxSoftwareDepot (+)
Disconnects the current PowerCLI session from the specified software depots. Updates the $DefaultSoftwareDepots session variable. The depots and their VIBs and image profiles will no longer be available.

If you do not specify the -SoftwareDepot parameter, PowerCLI prompts for an answer. You can type in the URL of the depot you wish to disconnect from.

Remove-EsxSoftwarePackage (+)
Removes existing packages from an image profile. The output of this cmdlet is an image profile object that you can pipe into any other cmdlet taking image profiles as input.

You cannot modify an image profile if its ReadOnly property is true.

ImageBuilder runs the modified image profile  through all image profile validation tests. The modified image profile must have at least a boot kernel so that it can boot. You cannot remove a package if another package depends on it.

Repair-DeployImageCache (+)
Repairs the image cache. Use this cmdlet only when working with VMware Technical Support.

Repair-DeployRuleSetCompliance (+)
Remediate any non-compliant associations discovered by Test-DeployRuleSetCompliance

Set-DeployRule (+)
Updates an existing rule.

Set-DeployRuleSet (+)
Sets the list of rules in the working rule set.

Set-EsxImageProfile (+)
Set-EsxImageProfile is used to modify a user-created image profile. The cmdlet returns the modified ImageProfile object, which can be piped into any other cmdlet taking image profiles as input. For details on persistence, see New-EsxImageProfile.

If the ReadOnly property of the image profile is true, you cannot modify the image profile.

Image Builder performs all image profile validation tests on the modified image profile.

The Name and Vendor of the modified image profile must be unique. If you attempt to change the Name and Vendor, and the new Name and Vendor are the same as an existing Name and Vendor, an error result.

Set-VMHostAuthentication (+)
Modifies the host authentication information.

Switch-ActiveDeployRuleSet (+)
Activates a rule set.

Test-DeployRuleSetCompliance (+)
Checks whether hosts are compliant with rules.

Quick tip: PowerShell here-strings don’t like whitespace

Today I was writing a bit of raw XML to be used as a SOAP message. Being a person who enjoys writing easy to read code, I declared my variable using here-string notation, pasted in the appropriate XML between the @” and “@, making sure to indent everything along the way including the last “@.

Then when I saved my work and tried to run the script, I kept getting an error claiming the “@ terminator was missing. Fast forward a couple minutes of head scratching, checking, double-checking , and triple-checking that there was, indeed, a double-quote followed by an at symbol. Finally I figured it out. The here-string’s “@ terminator needs to be on its own line, with no preceding white-space/tabs/whatever.

And there we have it: my forehead-slapping moment, transformed into a helpful PowerShell quick tip. 🙂