29 November 2011

Speed Up First PowerCLI 5 cmdlet -- Precompile XMLSerializers

Updated for v5.1 - 02 Oct 2012 -- see below

On 30 Jun 2011 there was a post on the official PowerCLI Blog about How to speed-up the execution of the first PowerCLI cmdlet.  They talked about how the first PowerCLI cmdlet run in a PowerShell session is considerably slower than all other PowerCLI cmdlet calls in the same session, and how that was "due to the fact that the .NET framework compiles the underlying code on first use."

So, they listed out the commands for precompiling some assemblies, so that the first PowerCLI cmdlet call would not be extra slow due to just-in-time compilation.  The commands add the compiled code to the "native image cache" on the local machine, and so only need performed once for the computer on which they are run. Note:  that is once per computer per version, so when a new PowerCLI version comes out and you install the new version, you would compile the assembly for the new version.

I wanted to keep the speed alive when I upgraded to PowerCLI v5, but found no mention of doing this for the new release.  So, I looked into it, and the commands for precompiling the XmlSerializers used by PowerCLI are roughly the same as those found at the PowerCLI Blog post, just with updating the version-specific items:
PS vN:\> C:\Windows\Microsoft.NET\Framework\v2.0.50727\ngen.exe install "VimService50.XmlSerializers, Version=5.0.0.0, Culture=neutral, PublicKeyToken=10980b081e887e9f"

And, if running on a 64-bit OS, also run:
PS vN:\> C:\Windows\Microsoft.NET\Framework64\v2.0.50727\ngen.exe install "VimService50.XmlSerializers, Version=5.0.0.0, Culture=neutral, PublicKeyToken=10980b081e887e9f"

These can both be run from PowerShell or a cmd prompt -- the Native Image Generator (ngen.exe) is doing the work.  The VimService50.XmlSerializers info can be found in the assembly folder at C:\Windows\assembly\.  There, one can get its Version and Public Key Token info used in the commands.

* Update - 02 Oct 2012:
With the release of PowerCLI v5.1 Release 1, there is a new XmlSerializers version to pre-compile to keep the speed streak alive.  This new one is named VimService51.XmlSerializers, and the version is "5.1.0.0".  So, here is another example of the speed difference of before and after using ngen.exe to precompile the XmlSerializers, along with the updated commandlines to run:

Speed results before:
PS vN:\> Measure-Command {Connect-VIServer myVcenter.dom.com}
...
TotalSeconds      : 14.4464282
TotalMilliseconds : 14446.4282

PS vN:\> Measure-Command {Get-VMHost}
...
TotalSeconds      : 11.859625
TotalMilliseconds : 11859.625

PS vN:\> Measure-Command {Get-VMHost}
...
TotalSeconds      : 0.0677823
TotalMilliseconds : 67.7823
The pre-compile commands for PowerCLI v5.1:
PS vN:\> C:\Windows\Microsoft.NET\Framework\v2.0.50727\ngen.exe install "VimService51.XmlSerializers, Version=5.1.0.0, Culture=neutral, PublicKeyToken=10980b081e887e9f"
 
If running on a 64-bit OS, also run:
PS vN:\> C:\Windows\Microsoft.NET\Framework64\v2.0.50727\ngen.exe install "VimService51.XmlSerializers, Version=5.1.0.0, Culture=neutral, PublicKeyToken=10980b081e887e9f"
 

Speed results after precompiling:
PS vN:\> Measure-Command {Connect-VIServer myVcenter.dom.com}
...
TotalSeconds      : 2.6289308
TotalMilliseconds : 2628.9308

PS vN:\> Measure-Command {Get-VMHost}
...
TotalSeconds      : 0.4143238
TotalMilliseconds : 414.3238

PS vN:\> Measure-Command {Get-VMHost}
...
TotalSeconds      : 0.0890656
TotalMilliseconds : 89.0656
End of update - 02 Oct 2012

Further info abut using NGen can be found at the MSDN site http://msdn.microsoft.com/en-us/library/6t9t5wcf%28v=VS.100%29.aspx.  There you can read all about:
  • how the precompilation helps with faster application startup 
    • loads faster, requires smaller initial working set
  • displaying the Native Image Cache (seeing one/all compiled native images)
  • uninstalling native images that have been compiled/installed
  • a "Summary of Usage Considerations" -- pros and cons of using native images
The speed increases for the first PowerCLI cmdlet call in a PowerShell session after compiling were on par with those reported in the PowerCLI blog post for previous versions.  Definitely worth running, to help keep away the slow-sies (SAS)!

18 comments:

  1. Thanks for posting this. I never realised untill now, that for a new version of PowerCLI you have to run these commands again for the new version.

    ReplyDelete
  2. You can use my PowerShell Install-PowerCLIXmlSerializer function to pre-compile the PowerCLI XML Serializers every time you install a new PowerCLI version:
    http://rvdnieuwendijk.com/2012/02/18/function-to-speed-up-the-execution-of-the-first-powercli-cmdlet/

    ReplyDelete
    Replies
    1. Very nice, Robert. Thanks for writing that function -- useful each time there is a new version of the XMLSerializers, or when installing PowerCLI on a new machine (when there are many versions of the XMLSerializers to compile). Thanks for sharing here!

      Delete
  3. Awesome article, really helpful! It made the startup over 5 times as fast, so I'm a happy camper.

    ReplyDelete
    Replies
    1. Thanks, Akos. Glad that it helped. Nice speed-up!

      Delete
  4. Thanks for the updated post. Should you also un-install the earlier i.e. pre 5.1 serializers or are they still required?
    e.g. on my machine
    >ls c:\windows\assembly\GAC_MSIL\VimService*.XmlSerializers

    VimService25.XmlSerializers
    VimService40.XmlSerializers
    VimService41.XmlSerializers
    VimService50.XmlSerializers
    VimService51.XmlSerializers

    ReplyDelete
    Replies
    1. Hello, Tim-
      You are welcome for the update. As for uninstalling earlier serializers versions, that is a good question. My understanding has been that leaving them is acceptable, that doing so should harm nothing.

      Delete
  5. Hello

    After I've installed Powershell 3.0 and PowerCLI v5.1 Release 2 this stopper working, was working great with Realease 1.
    I've run the commands as admin again, but it didn't help.
    I'm doing something wrong or do I need to run a different command?

    Regards
    Genoan

    ReplyDelete
    Replies
    1. Hello, Genoan-

      Hmm. I assume that you are getting an error when trying to run the command -- is that true? If so, what error message(s) do you get? And, is this on the same machine on which the commands previously worked? I am getting at: does this machine still have the 2.0 version of the .NET Framework installed (and, so, is that path to ngen.exe still valid)?

      Delete
  6. This comment has been removed by the author.

    ReplyDelete
  7. I have seen some latency caused by running under the 64bit version of powershell. This lag (16 seconds) only affected the first connect-viserver command, and subsequent commands where executed in an acceptable amount of time (1-4 seconds).

    I did validate that all vmware software was precompiled with the 64bit version of ngen.

    Eventually I found somewhere that VMWare acknowledged they needed to work on their 64bit software. I worked around the problem by explicitly launching the 32bit version of powershell (C:\Windows\SysWOW64\WindowsPowerShell\v1.0\powershell.exe). By using this work around you might not have access to the new workflow functionality built into powershell v3.

    ReplyDelete
  8. @Genoan

    I'm seeing this issue, too, with PowerShell 3.0 (.NET 4.0) and PowerCLI 5.1. After some searching, I noticed that the native images for .NET 4.0 aren't getting compiled, i.e. no Vim* directories in the native image directories for .NET 4.0:

    ls C:\Windows\assembly\NativeImages_v4*/Vim*

    But there's a trick to make ngen work here: Force the CLR to 4.0, then use ngen and disable the force of .NET 4.0 again:


    reg add hklm\software\microsoft\.netframework /v OnlyUseLatestCLR /t REG_DWORD /d 1
    reg add hklm\software\wow6432node\microsoft\.netframework /v OnlyUseLatestCLR /t REG_DWORD /d 1

    C:\Windows\Microsoft.NET\Framework\v4.0.30319\ngen.exe install "VimService51.XmlSerializers, Version=5.1.0.0, Culture=neutral, PublicKeyToken=10980b081e887e9f"
    C:\Windows\Microsoft.NET\Framework64\v4.0.30319\ngen.exe install "VimService51.XmlSerializers, Version=5.1.0.0, Culture=neutral, PublicKeyToken=10980b081e887e9f"

    reg delete hklm\software\microsoft\.netframework /v OnlyUseLatestCLR /f
    reg delete hklm\software\wow6432node\microsoft\.netframework /v OnlyUseLatestCLR /f


    You might want to compile VimService51, too, i.e.


    C:\Windows\Microsoft.NET\Framework64\v4.0.30319\ngen.exe install "VimService51.XmlSerializers, Version=5.1.0.0, Culture=neutral, PublicKeyToken=10980b081e887e9f"


    (I've tried a lot of things, so send me an email if that doesn't work. I might have missed something.) After that you should see Vim* directories in the native image directory:


    PS C:\Users\Administrator> ls C:\Windows\assembly\NativeImages_v4*/Vim*


    Directory: C:\Windows\assembly\NativeImages_v4.0.30319_64


    Mode LastWriteTime Length Name
    ---- ------------- ------ ----
    d---- 25.04.2013 17:00 VimService51
    d---- 25.04.2013 17:05 VimService51.XmlSer#


    This reduced my Connect-Viserver time to a few seconds.

    However, I believe .NET 4.0 is not even supported by VMware. But we're using the SANsymphony-V cmdlets in the same scripts, so we don't have much of a choice (.NET 4.0 64bit only there.)

    Mike Gerber
    LEITWERK AG
    mgerber@leitwerk.de

    ReplyDelete
    Replies
    1. The third ngen call should have said:

      C:\Windows\Microsoft.NET\Framework64\v4.0.30319\ngen.exe install "VimService51, Version=5.1.0.0, Culture=neutral, PublicKeyToken=10980b081e887e9f"

      Mike Gerber
      LEITWERK AG
      mgerber@leitwerk.de

      Delete
  9. Any suggestion on what to try if this fix produces no measurable change in powerCLI connection speed? I am running powercli 5.5 and changed the lines to VimServer55 and the Version=5.5.0.0 and still see the same ~52 second connection speed. Tried it for .net 2 and .net 4 on 64-bit OS Server 2012 R2.

    ReplyDelete
    Replies
    1. Hello, CBaum-

      Hm, connection speed, eh? This pre-compiling business was to help with the speed of the first PowerCLI cmdlet after having connected to some VIServer (not with the Connect-VIServer call itself, AFAIK).

      However, there has been some recent conversation about the speed of Connect-VIServer itself. Do you happen to be using passthrough authentication (you are not specifying credentials via -Credential or -User params to Connect-VIServer)?

      If so, how does the speed compare to when you _do_ specify credentials (even if the same credentials as the user as which you are running the PowerShell session)?

      I ask as I have seen a wide difference as of late when connecting in these two different ways. The PowerCLI team has logged this behavior as a bug to investigate (as of about Sep 2014). Let us know

      Delete
  10. i have the same problem with powershell.
    i reduced the time from 1 minutes to 17 seconds
    (my problem is in the Add-PSSnapin "VMware.VimAutomation.Core" step).
    any ides?

    ReplyDelete
    Replies
    1. I have the same problem. To resolve it I have turned off in "Internet Options" snap-in checking of certificates and also a have precompiled all VMware Serializers as described. No results. The problem still the same.

      And I think that there is no solution for speed-up that pssnapins.

      Delete

Note: Only a member of this blog may post a comment.