Among the trickier things for IT administrators to nail down are the causes of slow logons and how to fix those problems quickly. That’s why ControlUp created the “Analyze Logon Duration” (ALD) script. And now, we’ve made five key changes to the ALD script that will make spotting problems—and remedying them quickly—simple. In this series, we’ll go over the enhancements and what additional information we can glean from them.

The five changes:

1. Citrix Resultant Set of Policies (RSoP) analysis

2. Resultant Set of Policies Logging (RSoP Logging) 

3. Windows Management Instrumentation (WMI) Filter duration analysis

4. Individual AppX Package Load Times

5. Loopback Processing Mode state

About ControlUp Analyze Logon Duration

Out of the box, ControlUp offers some amazing technology for reporting on logon durations for users. Overall Logon Duration, Profile Load Time, Group Policy Load Time, Desktop Load Time and a column for unknown logon duration contributors are all calculated and displayed—in real time—right after a user has logged on.

This is extraordinarily helpful for identifying low-hanging fruit optimization opportunities.

Originally introduced in 2015, ControlUp (along with help from our community members—particularly Guy Leech) has been continually improving and increasing the functionality of the Analyze Logon Duration script, which can break logons down into granular phases, based on technology or logon processes.

But ALD can only report back on the things it understands. 

See those two large gaps of time on the far right of the image? 33.9 seconds and 20.9 seconds?

When ControlUp customers have unexplained logon duration gaps, they can reach out to us to help them assist in troubleshooting the root cause of these gaps. From here, ControlUp Professional Services can assist in identifying the technology or reason for the gaps. This article will go into the technical details of the gap in this screenshot, what it is we discovered, and how ALD reports it after we added this technology into the script.

What is the AppX – Load Packages phase?

The ControlUp Analyze Logon Duration script can report on the duration of the AppX – Load Packages phase. 

To analyze sessions in your environment, right-click on a session, select “Script Actions,” “Analyze Logon Duration,” and then “OK.”

Alternatively, you can use the ControlUp Virtual Expert™️ by clicking on the “menu” icon next to “Logon Duration” and selecting “Analyze Logon Duration.”

Easy right? The results appear like this:

I’ve previously covered AppX packages in these two articles:

AppX Packages Slowing You Down?

Logon Duration Research: AppX Packages

In a nutshell, AppX packages are Windows applications that are either built into the operating system or distributed by the Windows Store. Some vendors provide applications in AppX format (like NVIDIA with their “new” NVIDIA Control Panel). There are some security and end-user benefits of the format—namely, packages like this can have an experience not unlike what you get with smartphone or tablet operating systems. Applications can be easily repaired, removed, or installed without causing major disruptions to the OS underneath.

The way AppX packages work, though—especially the built-in operating system packages—can negatively affect logon times. Some packages do some configuration tasks during logon and the whole AppX portion blocks explorer.exe from continuing to the desktop until the AppX portion completes.

Also, working with customers, I’ve invested time in investigating the performance impact of AppX packages. The most frequent question I get is “can we break this down further?”

The short answer is YES, but will this information help? It might.

The biggest challenge with the AppX packages phase is that it’s undocumented and Microsoft has decided to have useless eye-candy added on that adds significantly to the logon time. So, what do we see with ALD?

Peeling back the curtain

What can we see with this information?

We can see that packages appear to load at a maximum of two at a time. To visualize this, I’ve converted the AppX packages loaded during logon to a GANTT-style chart:

The resolution I have is 0.1 seconds, which causes some of the package-loading operations to appear to overlap. This makes sorting out a bottleneck a bit more difficult, since the loading occurs asynchronously with a maximum of two packages operating per instance. 

The “Shell : AppX – Load Packages” phase started at 12:17:50.4, but the first package didn’t start loading until 12:17:50.8. Just 0.4 seconds! Not bad, not bad. But what about the end?

The last AppX package started loading at 12:17:56.0 and took 0.719 seconds, finishing at 12:17:56.7 seconds, but the AppX – Load Package doesn’t end until 13 seconds later. Why?

Well, I did say the AppX packages loading process is a black box, didn’t I?

What I’ve been able to observe is that AppX packages do some additional work during or after ‘‘loading the package.” 

What they do:

Setting AppDefaults occurs during this phase and can significantly slow down laying out the start menu.

How does this impact logon duration?

When it comes to Logon Duration, nothing about AppX packages seems to be positive. AppX packages require significant processing time, additional actions are done, and animations with slow fades and timed delays simply add to the logon duration. These delays are more noticeable on desktop-based operating systems. AppX packages loading in those operating systems have a different process than the multi-user operating systems and it appears—by design—to be slower and more involved.

Based on this article, you might be able to consider some optimization opportunities. The lowest- (and easiest) hanging fruit is removing the animation, which adds somewhere between eight and 15 seconds. The next is removing as many packages as possible that might contribute to logon times. The last would be playing with some of the AppX loading procedures while it loads packages. 

For instance, it’s been observed that AppX packages make changes to the Windows firewall service and push notification services.

What would happen if we disabled those services? Would AppX notice the service was non-responsive and skip its configuration tasks? Or would it attempt to start them, thereby adding more delays?

I have done some work exploring some optimization options across multiple operating systems and here are my findings:

Optimization #1: Disable “Show first sign-in animation”

“Hi”

“We’re getting everything ready for you”

“This might take several minutes”

“Almost there”

These screens are part of the “first sign-in animation.” 

One of the more intriguing things was the massive performance differences between Server 2019 and the ‘Desktop’ class operating systems. This is what the logon process looks like between the two:

“Disabling” First sign-in animation has this effect on desktop OS-es:

Significant! It appeared that the first sign-in animation caused a significant delay. I suspect Microsoft hard-coded the animation to run at the end of the process, which added quite a few seconds.

The result was that we reduced logon times significantly and the AppX – Load Packages phase:

Disabled – “First sign-in animation”

Enabled – “First sign-in animation”

How does this compare to the Server OS?

Server OS is still an additional seven seconds faster.

What is the impact across the various operating systems?

Big performance improvement by simply disabling first sign-in animation.

Optimization #2: Removing AppX Packages

The simplest thing is to simply remove as many AppX packages as possible.

The commands are simple enough:

Get-AppxPackage -AllUsers | Remove-AppxPackage

Get-AppXProvisionedPackage -Online | Remove-AppxProvisionedPackage -Online

What’s the impact after executing removals?

Number of packages BEFORE and AFTER removal

Which packages were removed?

Red are packages that were removed and did not process on logon.

And the impact to logon performance?

Minor-to-moderate gains, for sure. Logons were still in the ten-second mark range, sadly.

Optimization #3: EnableFastFirstSignIn
(Not recommended – this information is exploratory in nature only!)

I was alerted to this feature by @znackattack and he wrote [on his blog] about how to enable it.

Microsoft recognized the slower logon faults of Windows 10 and enabled a feature for MDMs to provision a package that can speed up your logons. Aptly named “EnableFastFirstSignIn,” it promises:

“This policy is intended for use on Shared PCs to enable a quick first sign-in experience for a user. It works by automatically connecting new, non-admin Azure Active Directory (Azure AD) accounts to the pre-configured candidate local accounts.”

I was curious about how it achieved its goal of a “quick” first sign-in experience, but was wary about how it actually worked. Connecting to an existing profile would speed up logons, as numerous things get written into your profile when you logon. If they are pre-created then Windows will, theoretically, just skip those steps. But how does all this technology connect?

There is some investigation to do as the explanation of this provisioning policy explanation is incredibly vague and isn’t much more than just saying it works by using “magic.” We like magic. And unicorns. But neither helps in the real world.

There is some description to the policies that “Shared PC Mode” enables, which is documented here.

I wanted to trace each configuration step of the provisioning policy, but I didn’t find an easy way of sussing this out. I created the ppkg files, as described by Nicke, and mounted them (thanks to this hint from deploymentresearch.com), but the XML files didn’t provide many clues around what they accomplished. I did, however, find a .provxml within that gave me some hints:

If you read the “Shared PC Mode” documentation, it lists dozens of policies that get applied by “SharedPCMode.” Something else has to read these values and then execute some commands elsewhere.

In order to understand what each step did, I broke out my trusty old Process Monitor and looked for Write category activities while I applied the provisioning package.

I then filtered on provtool.exe and I got a list of all the registry keys set by EnableFastFirstSignIn

The registry keys are listed here:

Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows\CurrentVersion\AppModel\StateManager]

“AllowSharedLocalAppData”=dword:00000001

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Appx\AppxAllUserStore\OverrideConfig\Microsoft.Windows.ContentDeliveryManager_cw5n1h2txyewy]

“SetupPhase”=dword:00000010

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Appx\AppxAllUserStore\OverrideConfig\Microsoft.Windows.ContentDeliveryManager_cw5n1h2txyewy]

“SetupPhase”=dword:00000010

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Appx\AppxAllUserStore\OverrideConfig\Microsoft.Windows.Cortana_cw5n1h2txyewy]

“SetupPhase”=dword:00000010

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Appx\AppxAllUserStore\OverrideConfig\Microsoft.Windows.Photos_8wekyb3d8bbwe]

“SetupPhase”=dword:00000010

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Appx\AppxAllUserStore\OverrideConfig\Microsoft.WindowsCamera_8wekyb3d8bbwe]

“SetupPhase”=dword:00000010

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Appx\AppxAllUserStore\OverrideConfig\Microsoft.WindowsStore_8wekyb3d8bbwe]

“SetupPhase”=dword:00000010

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Appx\AppxAllUserStore\OverrideConfig\Microsoft.XboxIdentityProvider_8wekyb3d8bbwe]

“SetupPhase”=dword:00000010

[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Biometrics]

“Enabled”=dword:00000000

[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Biometrics\Credential Provider]

“Enabled”=dword:00000000

“Domain Accounts”=dword:00000000

[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows\OneDrive]

“DisableFileSyncNGSC”=dword:00000001

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\PolicyManager\current\device\Authentication]

“EnableFastFirstSignIn”=dword:00000001

“EnableFastFirstSignIn_ProviderSet”=dword:00000001

“EnableFastFirstSignIn_WinningProvider”=”f09f8485-2178-4fa7-9a0c-eeb556b63756”

[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows\FileHistory]

“Disabled”=dword:00000001

[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows\Personalization]

“NoLockScreenSlideshow”=dword:00000001

[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\PassportForWork]

“Enabled”=dword:00000000

[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\PassportForWork\Remote]

“Enabled”=dword:00000000

[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows\System]

“AllowDomainPINLogon”=dword:00000000

“BlockDomainPicturePassword”=dword:00000001

“DisableLockScreenAppNotifications”=dword:00000001

“BlockUserFromShowingAccountDetailsOnSignin”=dword:00000001

“EnumerateLocalUsers”=dword:00000000

[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows\PreviewBuilds]

“AllowBuildPreview”=dword:00000000

“EnableConfigFlighting”=dword:00000000

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\SharedPC]

“SharedPCMode”=dword:00000001

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\SharedPC\AccountManagement]

“AccountModel”=dword:00000001

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\SharedPC\NodeValues]

“08”=dword:00000001

“01”=dword:00000001

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Authentication\Credential Providers\{09C77DF4-B76F-4875-9781-FB692504F39A}]

@=”Windows.SharedPC.CredentialProvider”

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Authentication\Credential Provider Filters\{923B6A38-CA24-4FB7-8914-DD55838B30E1}]

@=”Windows.SharedPC.CredentialProvider.Filter”

[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows\Explorer]

“ShowLockOption”=dword:00000000

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System]

“ConsentPromptBehaviorUser”=dword:00000000

“EnableFirstLogonAnimation”=dword:00000000

“HideFastUserSwitching”=dword:00000001

“dontdisplaylastusername”=dword:00000001

“DisableAutomaticRestartSignOn”=dword:00000001

What’s the impact of adding these registry keys?

What we see is improvements, and percentagewise, the changes have a fair size impact for some (EVD), but not for others.

One of the things I was interested in was the EnableFastSignIn policy applied some registry keys to some of the AppXPackages.  This is what that key looked like:

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Appx\AppxAllUserStore\OverrideConfig\Microsoft.Windows.ContentDeliveryManager_cw5n1h2txyewy]

“SetupPhase”=dword:00000010

It appears to be moving the setup phase of these packages from stage “6” or “7” to stage “10.”  Stage six or seven appears during logon, while here Stage ten is first run after logon.

The next chart shows in black packages that were processed, in red are packages removed by me previously, blue are packages no processed during logon due to the SetupPhase override.

Red are packages that were removed previously, blue were not processed on login after applying EnableFastSignIn and black were processed packages.

Exploring this SetupPhase value, it seems like it has not been updated since the document was created. Looking at all the loaded packages now, I think you might be able to add *more* of these packages to override. Do you need the old, deprecated Edge loading on logon? What about the OOBE packages? Undocked Dev Kit? Do you use Windows Hello with your VDI? No? Then you can probably shift BioEnrollment and if you’re not using Azure AD or Office 365, then the AAD.BrokerPlugin and CloudExperienceHost can probably go, too.

Naturally, when you’re making modifications of this magnitude it’s important to test and test and test! Although the names of these packages imply their function, it’s possible that the packages are required and some operations in Windows might fail without them. When you think of saving 5 or 10 seconds per user vs the complexity of troubleshooting; you’ll need to weigh your efforts carefully.

For additional information on what can impact logons check out the ControlUp blogs and check out the rest of our Analyze Logon Duration deep dive series:

• Better Than Ever: 5 Key Updates to ControlUp Analyze Logon Duration

• 5 Key Updates to ControlUp Analyze Logon Duration: WMI Filter Duration Analysis

• 5 Key Updates to ControlUp Analyze Logon Duration: RSoP Logging

• 5 Key Updates to ControlUp Analyze Logon Duration: Citrix RSoP

• 5 Key Updates to ControlUp Analyze Logon Duration: Loopback Processing Phase

Trentent Tye

Trentent Tye, a Tech Person of Interest, is based out of Canada and its many, many feet of snow. FUN FACT: Trentent came to ControlUp because, as a former customer, the product impacted his life in so many positive ways—from reducing stress, time to remediation, increased job satisfaction, and more—he had to be our evangelist. Now an integral part of ControlUp’s Product Marketing Team, he educates our customers, pours his heart and soul into the product, and generally makes ControlUp a better place. Trentent recently moved to be closer to family. He does not recommend moving during a pandemic.