Those Magnificent Drivers in Their Flying Microsoft Store, Or How I Flew From The Enrollment Status Page To Paris in 25 hours 11 minutes

Last Updated on September 21, 2022 by rudyooms

After you have read this enormous blog title, I guess you will know this blog will be about some weird behavior during the ESP. I couldn’t find any real documentation about the ESP, mentioning that this could occur.

I will divide this blog into multiple parts.

  1. The Problem.
  2. The Three ESP Stages
  3. The ESP and the Required Apps
  4. Troubleshooting.
  5. Fixing it Before Enrollment
  6. Fixing it During Enrollment

1. The Problem

Before I am going to tell you more about the Enrollment Status Page (ESP), I am going to show you what weird problem we encountered.

The customer had ordered 100 brand new Dell notebooks. We made sure all the notebooks were enrolled with Windows Autopilot for pre-provisioned deployment. So the only thing left for us to do was to hand out the devices and finish the last step: Account setup.

And you can guess what happened… things all went south… About 1/5 of the devices were stuck on this screen. It’s telling us: Joining your organization’s network is completed but the rest of it is stuck on identifying!

After staring at this screen with the account setup for 20 minutes, we knew there was something wrong.  So… it succeeded to join the organization network, but it could not succeed in identifying the other parts?

Because we don’t have any Wi-Fi/certificates or apps targeted to a user, we know these aren’t the issue… But what’s left?  It looks like the only thing left was the security policies?

So far as I know the Account ESP step doesn’t track security policies and only one subkey is created under ESPTrackingInfo\Diagnostics\ExpectedPolicies. This is a dummy policy called EntDMID. So, getting much information about what’s happening during this step could be difficult.

But it still hangs on identifying. What’s happening? Let’s take a closer look at the 3 ESP stooges first

2. The Three ESP Stooges

Oh… wait did I say the 3 stooges? I meant the 3 ESP stages!

The Enrolment Status Page (ESP) keeps track of the device’s configuration progress and will show you the progress. When you configured the ESP, most of the time you could be pretty sure the device was ready for use when it finished the ESP successfully. We can divide the ESP in 3 parts

  • Device preparation
  • Device setup
  • Account setup

1. Device Preparation

1.1 Securing the Hardware

When you use Autopilot with user-driven mode this step isn’t required but when you are using self-deploying mode or Windows Autopilot for pre-provisioned deployment, it will be required.

At this first step in the device preparation, the device completes the Trusted Platform Module (TPM) attestation process and sends its hardware hash to Azure AD to prove its identity, so most of the time when your TPM has not the proper version or does not support attestation this stage would fail immediately.

1.2. Azure Ad Join Process.

The device will join the Azure Ad, it will make use of the token it received in the first step. Please note when you are using user-driven mode this step is done before the ESP starts. It will be completed when the user has entered their credentials. And just like with the TPM, it’s required in self-deploying mode and white-glove deployment.

1.3. Intune (MDM) enrolment

At this step, the device is enrolled into Intune. So please make sure, just like with the first step your TPM is ready to go and up to date! Otherwise, you could end up with some errors you will need to troubleshoot.

1.4. Prepare the device for MDM

When the device arrives at this step, the device is going to calculate all the policies and apps that are required to be tracked. From Windows 10, version 1903 the device will also create the tracking policy for the SideCar agent and starts installing the Intune Management Extension(IME) and other device assigned MSI Apps

2. Device setup

You will notice it will start with the identifying stage. At this moment the device will also start to execute the available PowerShell scripts that were assigned to the device.

Please Note: PowerShell scripts aren’t tracked during the ESP!

The device is going to create a tracking policy for this phase and calculate if all apps and policies are targeted to the device context and trigger the installation.

2.1 Security policies

The ESP does show the installation status (1/1) but it does NOT track any security policies which are deployed to the device context. So it should be done within a few moments.

2.2. Certificate profiles

During this step, it will try to install the SCEP certificates which are deployed in the device context.

2.3. Network Profiles

Only Wi-Fi profiles that are deployed in device context are installed. If the Wi-Fi profiles can’t be installed, the device will keep trying until a time-out occurs.

2.4. Applications

At this step, all your Line of Business (LOB), Win32 Apps, and Microsoft apps that are assigned to devices instead of users are installed. You will notice it will keep track of all the apps you configured as required on the ESP page. Please read part 4 if you want to know more about the “required” apps

Please note: Microsoft tells us to not mix Win32apps and LOB apps.

First, we need to know what part is responsible for which type of app.

The OMA-DM agent is responsible for the LOB/Msi installations

The Intune Management Agent is responsible for the installations of the Win32 Apps.

But both need to access the same trusted installer. So, this could fail. But not mixing up these apps? That could be very hard. In my opinion, try it out for yourself. And please try not to assign all apps as required but as available. So, you could install these apps yourself.

3. Account setup

Again, just like with the device setup you will notice it starts in the identifying stage. The device is going to create the tracking policy for this phase, calculate all the apps and policies targeted to the user context and trigger the installation.

3.1. Join the Organization

When you make use of a hybrid Azure Ad Autopilot deployment at this point the device should have been Azure Ad Joined. It could take up to 40 minutes to sync the device from the on-premise AD with azure ad connect.  When Azure Ad Connect didn’t yet sync the device, the primary refresh token is not obtained, and the user could not communicate with the service. When it fails to communicate it also fails to evaluate the targeted apps and policies. If this is the case, the account setup will be stuck on identifying until the ESP times out.

Maybe you want to use skipuserstatuspage when you are using Hybrid Azure Ad Autopilot?

3.2. Security Profiles

The ESP also doesn’t track any security policies deployed to the user context. But this doesn’t mean the policies are not installed! These policies are going to be installed in the background. You will always see (1 of 1) completed in the UI

3.3. Certificate profiles

During this step, it will try to install the SCEP certificates which are deployed in the user context.

3.4. Network Profiles

Only Wi-Fi profiles that are deployed in user context are installed. If the Wi-Fi profiles can’t be installed, the device will keep trying until a time-out occurs.

3.5.Applications

At this step, all of your Line of Business (LOB), Win32 Apps, and Microsoft apps that are assigned to users instead of devices are installed. You will notice it will keep track of all the apps you configured as required on the ESP page.

3. The ESP and the Required Apps

As I was showing you in the latest part of this blog, you could also configure some Required apps during the device and user ESP. Let me tell you more about this topic before we continue because this could give us issues right?

When we look at the ESP settings you’ll notice some additional settings. It really depends on what situation you are in, but you could allow the user to reset the device, or you could decide if you want to allow the users to still use the device when an error occurs.

One of the best parts (and the most misunderstood part) is the: “block device use until their required apps are installed”.

As shown above, I needed to place a big red cross in the text I wrote earlier about the “ESP Required Apps”. As it seems that behavior is somehow changed without me knowing it.

To determine if that behavior changed, I added a lot of Win32 Apps and configured them all as required apps! Besides adding a lot of required Win32 apps, I also removed OneDrive and Office Win32 App from the required apps list so I only had 5 required Win32 Apps left.

After enrolling the device with Autopilot and watching the Intune Management Extension log it showed me it was only going to install those 5 Win32 apps during the Device ESP Phase

Please note that: The Make Me Admin app, was an MSI file (Yeah I know) so that was already installed at the same moment the IME service was installed. The same goes for the Office CSP if you had configured it!

But still, I wasn’t convinced, so I also opened Procmon to determine the incoming files in the Intune Management Extension Content folder during the Autopilot enrollment!

I did so by settings up a few filter rules, in Procmon first! As shown below I am making sure I am only getting output in which the patch contains the content incoming folder.

After my Procmon was good to go, I started the Autopilot enrollment. As shown below, these were the only BIN files that arrived in the IME Content Incoming folder…. nothing more!!!!

When translating the GUID’s to the apps, these 5 bin files were corresponding to the required apps we configured and the Win32Apps that were mentioned in the IME LOG! For example, the d7377849…. app corresponds to the SolarWinds Agent Win32App we configured.

I guess we need to trust our own eyes because no other Win32 Apps got installed during the ESP. This behavior definitely changed somewhere in time.!!! But let’s move on, so I could start some troubleshooting!

4. Troubleshooting

When you normally need to start some troubleshooting during the Device ESP, the first thing you would do… is press Shift + F10 to open a cmd and elevate yourself. If your device is stuck at the Account phase, you still need to press Shift+F10 but you don’t need to enter the credentials, just click on cancel and you can open the Start Menu by pressing the Windows logo on your keyboard.

Of course, we began to search the event logs to see if we could find any errors which could be related to this problem. First, we opened the ModernDeployment-Diagnostics-Provider\Autopilot event log but found nothing useful there. No warnings/No errors.

So up we go to the DeviceManagement-Enterprise-Diagnostic-Provider Event log

The only thing we noticed was that after a couple of minutes when the account setup stage started, we received this error above.

It’s hard to tell if this had anything to do with the account setup page that got stuck. But searching this error showed me there could be a problem with the network connection. Let’s keep that in mind and let’s go further.

As you probably know, the ESP does not track PowerShell Scripts. So, to be sure we checked if there were PowerShell scripts assigned to users. But again, no PowerShell scripts could cause this issue.

Maybe the ESP is completed but could not communicate with the service. If we want to have more information about the ESP, please open the registry and open:  HKLM\Software\microsoft\Windows\Autopilot\EnrollmentstatusTracking

The issue occurs at the account setup page so I guess we can skip the device part of the enrollmentstatustracking.

The same goes for the ESPTrackingInfo. This part is used to store diagnostic information for all applications and policies that are tracked by the enrolment status page. It will also show you the timestamps for when these apps and policies were installed.

This subkey below is created during the account setup phase if the device setup phase is completed successfully. It contains the installation state of Win32 apps that are deployed in the user context and the creation status of the tracking policy for the account setup phase.

So, it looks like everything is fine. I went back to the Event log and opened the system event log. Normally you don’t care about WindowsUpdate notifications in the event log until I noticed an error.

In a few seconds I was questioning myself, why was Windows Update launched, did I forget something? We didn’t create a Powershell script or a Win32 app to trigger this. I even noticed WindowsUpdate was triggering driver updates? It was updating the WiFi driver and the Intel Software which comes with it.  This could have caused all network connections to disconnect.

I didn’t have the opportunity to create some event log screenshots about when the intel driver was updated. But opening MMC and device management showed me the same thing. I could switch back to an older WiFi driver.

That’s odd because I was not expecting drivers to be updated. We made sure we did not allow driver updates with Windows Update for Business.

And if am not mistaken: updates download automatically and then install during automatic maintenance when the device isn’t in use or running on battery power. If you want to read more about this, please check this blog:

So WUfB should not trigger this, what else could be triggering this? I wanted to know if my desktop and everything else was ready, so I decided to kill the Microsoft-Account process.

And look at the first thing I noticed!

The Microsoft Store is triggering those updates! Did I forget something? I can’t remember if I have read this in any official documentation. Of course, we connected the Microsoft Business Store to make sure the Company Portal Online Version would be installed on the devices.

This installation of the Company Portal was also shown in the event log.

After some googling, I could not find anything useful. Except for an article about updating UWP apps with the Microsoft store.  This part below I found most interesting.

Comparing it to our situation. I guess when we entered the username and password to start enrolling the device it also silently signed into the Microsoft Store to trigger the Company portal installation and triggered the driver updates and UWP apps.

Please note: If you are requiring compliant devices to have access to all Cloud Apps, the device needs to be compliant before it could access the Microsoft Store. Please read this blog to get a better understanding of this flow

Requiring Online Microsoft Store Apps and the Autopilot ESP (call4cloud.nl)

But let’s continue by looking at the Store Event log because our device is compliant and has access to the Microsoft Store!

Store App Event log

Device Setup Manager Event log

Windows Update Client Event log

I guess that somehow caused the Account Setup ESP Stage to get stuck! To be sure, I reinstalled the device again and went through the whole autopilot white-glove… excuse me… Windows Autopilot for pre-provisioned deployment, the first thing I did was open the device manager.

In the beginning, it was missing a lot, and I mean a lot, of drivers. The device was installed with the latest Windows 10 version available, but I guess there were still a lot of drivers missing. Within a few moments, the Microsoft store triggered the installation/updating of the drivers.

And after a few minutes, all the drivers were updated and installed and the Microsoft Store made sure we have all the Apps corresponding to it.

5. Fixing it Before Enrolling

You really don’t want to stop the Microsoft Store from auto-updating all the apps and drivers. Do you really want to have an outdated company portal app on all your devices? Disabling the whole Microsoft Store is also not a good option. But I have been talking about this subject in an older blog.

So I guess we have only one option left and it’s the same as Microsoft is advising when you need to skip the account setup part when you use azure hybrid autopilot.

We only need to create a CSP to make sure this stage is skipped:

After selecting the Custom Template, you could configure the SkipUserStatusPage OMA-URI as shown below

OMA-URI

./Device/Vendor/MSFT/DMClient/Provider/MS DM Server/FirstSyncStatus/SkipUserStatusPage

Data Type: Boolean

Value: True

6. Fixing it During enrollment

When you have a device that ends up with the same issue and hangs on identifying even when it’s already done we could kill the Microsoft Account page right?

If we decide to close/kill the Microsoft Account process we still end up with the same window when we reboot the device and that is something we don’t want! To fix this issue you could add the SkipuserStatusPage setting yourself by using this PowerShell script

$Name = get-childitem -path HKLM:\software\microsoft\enrollments\ -Recurse | where { $_.Property -match 'SkipUserStatusPage' }
if ($Name)
{
$Converted = Convert-Path $Name.PSPath
New-ItemProperty -Path Registry::$Converted -Name SkipUserStatusPage  -Value 4294967295 -PropertyType DWORD -Force | Out-Null
}

Conclusion

After digging into finding some evidence that the Microsoft Store can be used for updates, I came across this one article:

Security Update Guide – Loading – Microsoft

So, instead of telling myself, that there is no way the Microsoft Store could trigger it… I guess it can…

New trending GIF on Giphy | Ted gif, What gif, Gif

Normally we don’t skip the user status page instead, we make sure the user account page is not shown when another user logs in to the same device. But in this situation… we rather skip the page instead of disabling the store!

Want to read more? Try out a couple of these blogs:

One thought on “Those Magnificent Drivers in Their Flying Microsoft Store, Or How I Flew From The Enrollment Status Page To Paris in 25 hours 11 minutes

Leave a Reply

Your email address will not be published.

69  +    =  76