Logon Duration Research: AppX Packages

Analyze Logon DurationAnalyze Logon Duration (ALD)AppXCitrixMicrosoft
Trentent Tye
October 19, 2020
Duration: 11 minutes
Subscribe to our blog
RSS feed
In the last installment in this series, we explored the way Microsoft has designed Windows to have its AppX packages operate in different ways, even if the user context—logging on to the system and the differing logon durations amongst the operating systems—is the same. The wildly different configurations of Windows have led to some intriguing questions for Citrix, VMware, and Microsoft EUC administrators, chiefly, “Do I need what this feature is offering?” Multi-session support is new to Windows 10, but has been on Server operating systems for a long time. Are your multi-session users happy with the experience they get on the Windows Server OS? If they are happy or productive, why would a full-fledged Windows Desktop need all the other stuff? What would the drawbacks be if you removed all the superfluous features from the Windows Desktop OSes? Before we get out over the tips of our skis, though, let’s start by establishing a baseline.  As an experiment, I installed six different versions of Windows:
  • Windows 10 1809 Enterprise
  • Windows 10 1809 Enterprise—Long Term Service Channel
  • Windows 10 2004 Enterprise
  • Windows 10 2004 Enterprise N
  • Windows 10 2004 Enterprise for Virtual Desktops
  • Windows 2019 Server Standard
I chose these operating systems for some specific reasons.

Windows 10 1809 Enterprise: Long-Term Service Channel

The LTSC version of Windows 10 (supposedly) comes with reduced AppX packages. I’ll validate if that is true, plus the differences between Win10 1809 LTSC and the other OSes.

Windows 10 1809 Enterprise

If the LTSC version has reduced AppX packages, we should be able to measure the impact by comparing it to the “SAC” 1809 version to see what additional packages it has and what they are contributing.

Windows 10 2004 Enterprise

Arguably the latest and greatest version of Windows (as of this writing). I profiled the AppX packages, but we can take a look at the additional packages it contains, compared to its peers.

Windows 10 2004 Enterprise N

This is the “Media Player Free” version of Windows 10. This version automatically comes with at least one less piece of software (compared to the non-N versions). Specifically, I’m curious to see if other AppX packages are removed here, as well, and about how Microsoft chose to handle the file-type associations.

Windows 10 2004 Enterprise for Virtual Desktops

As examined in my last post, Windows 10 Enterprise for Virtual Desktops makes some major changes to the way many AppX packages and file associations are made available. Microsoft significantly reduces the count on both when compared to its Enterprise peer. 

Windows 2019 Server Standard with RDSH enabled

The gold standard: server operating systems contain much-reduced AppX packages, since most are not useful in server environments.  Since loading AppX packages is a CPU-intensive task, I decided to test with 1vCPU, 2vCPU, and 4vCPU configurations. Observations showed that AppX packages load asynchronously, so the more CPU you have, the less contention and the faster loading happens. Reducing the number of CPUs should increase the duration, as contention is higher; not all packages will get a time slice of the CPU while the operation is processing. Finally, since CPU is such an important part of this test, it’s important to disclose the types of processors on which I ran the tests. In this case, I used 12-core, 24-thread Ryzen 9 3900X 3.8Ghz processors. On each of the operating systems, I measured the following data:
  • Number of AppX packages
  • Number of AppX Provisioned Packages
  • Number of File Associations
  • Number of File Associations to AppX packages
For this test, I installed each operating system directly from its release ISO and applied Windows Updates. Each was joined to the domain in a OU with no group policy objects applied, since we just want to test the AppX package impact. Power settings were set to “High Performance” or “Ultimate Performance” both in Windows and on the Hypervisor. The Windows 2019 server operating system had Remote Desktop Session Host role installed. Before I get into all that, though, I want to share some things I found intriguing during the setup phase.

Windows 10 EVD doesn’t have “Ultimate Performance”—only “High Performance”Windows 10 EVD doesn’t have Ultimate Performance — only High Performance

Windows 10 EVD will prompt you for a reason as to why you want to shut down or restart the machine. Windows 10 EVD prompt to shut down or restart machine. Memory Compression is enabled on all Windows 10 machines (including EVD), but is disabled on Server machines. I observed that the Windows 10—non-EVD—machines generated CPU load when idling. This was the result of WinSAT.exe running and appeared ~25 mins after boot-up. As such, I waited an hour to run any/all tests to ensure no outside processes were interfering. Additionally, machines set up for multi-session (Windows 10 EVD and Server 2019) had a process execute [at the ten uptime mark] that consumed significant CPU for an extended period. For Server 2019, it consumed CPU for seven minutes, and on Windows 10 EVD it consumed CPU for one minute.  The CPU consumption for that was mscorsw.exe, which is .NET Framework optimization. Optimizer utilities, such as Citrix Optimizer, VMware Operating System Optimization Tool (OSOT), and Base Image Script Framework (BISF), should remove this load. Since my setup has minimally configured operating systems, this was something I needed to consider, so it wouldn’t impact the performance of my test logons. I captured the early uptime process with ControlUp, so I could see this in real time. See this video: In Windows 10 1809 Enterprise, the OneDriveSetup.exe utility launched on logon and consumed huge amounts of CPU for significant periods of time; the LTSC version of 1809 did not.

1vCPU Logon Duration:

1vCPU Logon Duration   Averaged out over three runs:
Operating System Logon Duration AppX Package Load Time
1809 Enterprise 27 seconds 21.3
1809 Enterprise – LTSC 26 seconds 21.2
2004 Enterprise 29 seconds 23.0
2004 Enterprise N 27 seconds 21.3
2004 Enterprise for Virtual Desktops 26 seconds 21.1
2019 Server Standard 10 seconds 6.8
 

2vCPU Logon Duration:

2vCPU Logon Duration         Averaged out over 3 runs:
Operating System Logon Duration AppX Package Load Time
1809 Enterprise 25 seconds 21.0
1809 Enterprise – LTSC 24 seconds 21.1
2004 Enterprise 25 seconds 21.2
2004 Enterprise N 25 seconds 21.0
2004 Enterprise for Virtual Desktops 23 seconds 19.6
2019 Server Standard 7 seconds 3.7
 

4vCPU Logon Duration:

4vCPU Logon Duration         Averaged out over 3 runs:  
Operating System Logon Duration AppX Package Load Time
1809 Enterprise 24 seconds 20.9
1809 Enterprise – LTSC 24 seconds 21.0
2004 Enterprise 24 seconds 21.0
2004 Enterprise N 24 seconds 21.0
2004 Enterprise for Virtual Desktops 22 seconds 19.1
2019 Server Standard 6 seconds 3.0
  Intriguingly, there appears to be an overhead that increasing the number of CPUs can’t push past.  To gather more data, I logged in and queried for the number of provisioned packages, as well as the number of “AppX” packages, then looked at my AppX registry key AND the event viewer to see how many of each package was reported. Looking at the counts of each operating system:  
Operating System Get-AppxProvisionedPackage -online Get-AppXPackage AppX packages as counted by the registry AppX packages as counted event 327
1809 Enterprise 41 105 77 77
1809 Enterprise – LTSC 0 33 33 33
2004 Enterprise 40 115 79 79
2004 Enterprise N 29 104 68 68
2004 Enterprise for Virtual Desktops 40 127 79 79
2019 Server Standard 0 28 28 28
NOW we’re getting some interesting oddities! Windows 10 1809 E LTSC has close to the same number of packages as Server 2019 standard, but has significantly longer AppX package load times (each of these packages has their load time tracked in the Microsoft-Windows-AppReadiness/Admin log). The list of packages [added to your “task list”] to execute is ID 240. Generating the list of tasks on my system spanned around ~0.5ms per task. For the 68 packages in 2004 Enterprise N, it took 28 ms to generate the list of packages to be installed (execution is measured by event 213). I created a script action to output my logon events to measure how long they took. Windows 10 AppX packages installation time My logon time was 11:00:43; installation of the packages started ~0.6 seconds after logon and finished ~7.4 seconds after that. Total CPU duration was 11.07 seconds for loading of the packages. Sorting by duration revealed some outliers that should be considered. A far cry from the ~68 packages that get loaded, the output here showed just 20 packages. That’s because Microsoft has deemed that some packages aren’t required to be processed at logon, but can be processed later or at app launch. Some (all?) of these packages are processed asynchronously, but I don’t know if that’s all at once or throttled (like 4 at a time). The reported start/stop for this event was 21 seconds. We’ve now accounted for ~8 seconds, so where are the other 13? AppX packages processed during logon Before we get into that, let’s update our chart with the number of packages that were processed during logon.  
Operating System Get-AppxProvisionedPackage -online Get-AppXPackage AppX packages as counted by the registry AppX packages as counted by eventid 327 AppX packages actually processed at logon
1809 Enterprise 41 105 77 77 12
1809 Enterprise – LTSC 0 33 33 33 6
2004 Enterprise 40 115 79 79 20
2004 Enterprise N 29 104 68 68 20
2004 Enterprise for Virtual Desktops 40 127 79 79 21
2019 Server Standard 0 28 28 28 5
  Let’s also update this chart with the package durations:  
Operating System AppX Package Load Time AppX Package processing time (seconds)
1809 Enterprise 20.9 4.5
1809 Enterprise – LTSC 21.0 0.7
2004 Enterprise 21.0 6.1
2004 Enterprise N 21.0 7.4
2004 Enterprise for Virtual Desktops 19.1 7.5
2019 Server Standard 3.0 0.6
  Finally, let’s update the packages evaluated at logon for each OS  
1809 E 1809 LTSC 2004 E
Microsoft.AAD.BrokerPlugin_cw5n1h2txyewy Microsoft.AAD.BrokerPlugin_cw5n1h2txyewy Microsoft.549981C3F5F10_8wekyb3d8bbwe
Microsoft.MicrosoftEdge_8wekyb3d8bbwe Microsoft.Windows.CloudExperienceHost_cw5n1h2txyewy Microsoft.AAD.BrokerPlugin_cw5n1h2txyewy
Microsoft.Windows.CloudExperienceHost_cw5n1h2txyewy Microsoft.Windows.ContentDeliveryManager_cw5n1h2txyewy Microsoft.BioEnrollment_cw5n1h2txyewy
Microsoft.Windows.ContentDeliveryManager_cw5n1h2txyewy Microsoft.Windows.Cortana_cw5n1h2txyewy Microsoft.DesktopAppInstaller_8wekyb3d8bbwe
Microsoft.Windows.Cortana_cw5n1h2txyewy Microsoft.Windows.ShellExperienceHost_cw5n1h2txyewy Microsoft.MicrosoftEdge_8wekyb3d8bbwe
Microsoft.Windows.Photos_8wekyb3d8bbwe windows.immersivecontrolpanel_cw5n1h2txyewy Microsoft.Windows.CloudExperienceHost_cw5n1h2txyewy
Microsoft.Windows.ShellExperienceHost_cw5n1h2txyewy Microsoft.Windows.ContentDeliveryManager_cw5n1h2txyewy
Microsoft.WindowsCamera_8wekyb3d8bbwe Microsoft.Windows.OOBENetworkCaptivePortal_cw5n1h2txyewy
microsoft.windowscommunicationsapps_8wekyb3d8bbwe Microsoft.Windows.OOBENetworkConnectionFlow_cw5n1h2txyewy
Microsoft.WindowsStore_8wekyb3d8bbwe Microsoft.Windows.Photos_8wekyb3d8bbwe
Microsoft.XboxIdentityProvider_8wekyb3d8bbwe Microsoft.Windows.Search_cw5n1h2txyewy
windows.immersivecontrolpanel_cw5n1h2txyewy Microsoft.Windows.ShellExperienceHost_cw5n1h2txyewy
Microsoft.Windows.StartMenuExperienceHost_cw5n1h2txyewy
Microsoft.WindowsCamera_8wekyb3d8bbwe
microsoft.windowscommunicationsapps_8wekyb3d8bbwe
Microsoft.WindowsStore_8wekyb3d8bbwe
Microsoft.XboxIdentityProvider_8wekyb3d8bbwe
MicrosoftWindows.Client.CBS_cw5n1h2txyewy
MicrosoftWindows.UndockedDevKit_cw5n1h2txyewy
windows.immersivecontrolpanel_cw5n1h2txyewy
2004 EN< 2004 EVD 2019 STD
Microsoft.549981C3F5F10_8wekyb3d8bbwe Microsoft.549981C3F5F10_8wekyb3d8bbwe Microsoft.AAD.BrokerPlugin_cw5n1h2txyewy
Microsoft.AAD.BrokerPlugin_cw5n1h2txyewy Microsoft.AAD.BrokerPlugin_cw5n1h2txyewy Microsoft.Windows.CloudExperienceHost_cw5n1h2txyewy
Microsoft.BioEnrollment_cw5n1h2txyewy Microsoft.BioEnrollment_cw5n1h2txyewy Microsoft.Windows.Cortana_cw5n1h2txyewy
Microsoft.DesktopAppInstaller_8wekyb3d8bbwe Microsoft.DesktopAppInstaller_8wekyb3d8bbwe Microsoft.Windows.ShellExperienceHost_cw5n1h2txyewy
Microsoft.MicrosoftEdge_8wekyb3d8bbwe Microsoft.MicrosoftEdge_8wekyb3d8bbwe windows.immersivecontrolpanel_cw5n1h2txyewy
Microsoft.Windows.CloudExperienceHost_cw5n1h2txyewy Microsoft.VCLibs.140.00_8wekyb3d8bbwe
Microsoft.Windows.ContentDeliveryManager_cw5n1h2txyewy Microsoft.Windows.CloudExperienceHost_cw5n1h2txyewy
Microsoft.Windows.OOBENetworkCaptivePortal_cw5n1h2txyewy Microsoft.Windows.ContentDeliveryManager_cw5n1h2txyewy
Microsoft.Windows.OOBENetworkConnectionFlow_cw5n1h2txyewy Microsoft.Windows.OOBENetworkCaptivePortal_cw5n1h2txyewy
Microsoft.Windows.Photos_8wekyb3d8bbwe Microsoft.Windows.OOBENetworkConnectionFlow_cw5n1h2txyewy
Microsoft.Windows.Search_cw5n1h2txyewy Microsoft.Windows.Photos_8wekyb3d8bbwe
Microsoft.Windows.ShellExperienceHost_cw5n1h2txyewy Microsoft.Windows.Search_cw5n1h2txyewy
Microsoft.Windows.StartMenuExperienceHost_cw5n1h2txyewy Microsoft.Windows.ShellExperienceHost_cw5n1h2txyewy
Microsoft.WindowsCamera_8wekyb3d8bbwe Microsoft.Windows.StartMenuExperienceHost_cw5n1h2txyewy
microsoft.windowscommunicationsapps_8wekyb3d8bbwe Microsoft.WindowsCamera_8wekyb3d8bbwe
Microsoft.WindowsStore_8wekyb3d8bbwe microsoft.windowscommunicationsapps_8wekyb3d8bbwe
Microsoft.XboxIdentityProvider_8wekyb3d8bbwe Microsoft.WindowsStore_8wekyb3d8bbwe
MicrosoftWindows.Client.CBS_cw5n1h2txyewy Microsoft.XboxIdentityProvider_8wekyb3d8bbwe
MicrosoftWindows.UndockedDevKit_cw5n1h2txyewy MicrosoftWindows.Client.CBS_cw5n1h2txyewy
windows.immersivecontrolpanel_cw5n1h2txyewy MicrosoftWindows.UndockedDevKit_cw5n1h2txyewy
windows.immersivecontrolpanel_cw5n1h2txyewy
  From just eye-balling it, Windows 1809 LTSC should have the same (or extremely close to) performance as Server 2019. The actual processing time differs by only 100 or so milliseconds, but the actual logon differs by 18 seconds! So, now we have an idea what some out of the box Windows operating systems have for a baseline performance. In our next article, we’ll dive further into the gaps of time that AppX reports and start on optimizations.

Trentent Tye

Trentent Tye, a Tech Person of Interest, is based out of Canada and its many, many feet of snow. FUN FACT: Trentent came to ControlUp because, as a former customer, the product impacted his life in so many positive ways—from reducing stress, time to remediation, increased job satisfaction, and more—he had to be our evangelist. Now an integral part of ControlUp’s Product Marketing Team, he educates our customers, pours his heart and soul into the product, and generally makes ControlUp a better place. Trentent recently moved to be closer to family. He does not recommend moving during a pandemic.