Call4Cloud

IMECache: Attack of the Cleaner

In this blog, I will guide you through the whole Win32 App installation process and each step of the IME flow. After I am done showing you the whole IME flow, I will also show you how you could troubleshoot and solve some installation errors.

Sneak Peak alert! How to rerun the failed Win32 apps installations instantly!

Please note that this particular problematic app is deployed with the use of the ServiceUI tool.

I will divide this blog into multiple parts

  1. Win32 App IME Installation Phases
  2. Background info and the Serviceui tool
  3. The Problem
  4. Triggering the Reinstallation
  5. The Exit Codes
  6. Digging Further
  7. The Solution

1.Win32 App IME Installation Phases

When you want to monitor or troubleshoot a Win32 app installation you will need to take a look at the Intune Management Extension log file:

C:\programdata\microsoft\intunemanagementextension\logs\IntuneManagementextension.log

You can open this log file with notepad, but I am recommending the CMTrace tool from the config manager toolkit. It’s way easier to read the log with this tool!

Download System Center 2012 R2 Configuration Manager Toolkit from Official Microsoft Download Center

To understand how apps are being deployed to the device we need to take a look at each IME phase.

1.1. Pre-Detection and Applicability Phase

Before the IME could even begin to install the nice Win32App, it first needs to get its detection rules. Otherwise, how could it know if maybe the app is already installed? As shown below when we want to install an app from the Company Portal the IME will make sure it gets the policies.

When looking a little bit closer at the policy above, you will notice some nice things in it

  • The first thing you could notice is the mention of the “Intent“. I made the app available so guess what the Intent should be

1 –> Available
3 –> Required
4 –> Uninstall

  • The second part you could have noticed in the policy, that it also contains the needed detection rules.

Now IME knows what to check for it fires up the ProcessDetectionRules and will start the detection of the App, by checking the rules it got.

After IME (SideCarFileDetectionManager) successfully determined that the Win32App isn’t already installed (ApplicationDetected: False), it can begin to check the Applicability.

By checking the applicability It will check the requirements you “could” have specified when you created the Win32app earlier on. For example which OS version you are requiring or the Required FreeSpace.

1.2. Downloading Phase

When the applicability checks out, the IME can start downloading the intunewinfile (*.bin) from the Intune service (*.manage.microsoft.com) to the device.  The file download will be placed in the  C:\Program Files (x86)\Microsoft Intune Management Extension\Content\Incoming

Luckily this time the Intune Device certificate is available! When it is not available or expired you could get some weird behavior. I dedicated a blog to this some time ago.

1.3. Decrypting and Verifying Phase

After the download is ready it will verify and decrypt the .bin file into a zip file in the staging folder.

C:\Program Files (x86)\Microsoft Intune Management Extension\staging\  and from the staging folder is will be unzipped to the c:\windows\IMECache\GUID folder.

1.4. Installing Phase

IME will start and retry to install the Win32App, 3 times for 5 minutes each 24 hours. IME will launch the Win32Appinstaller and will try to install the Win32app from the IMECache folder the data was extracted to earlier.

1.5.Detection Phase

IME will try to detect if the installation is successful by checking the detection rules you specified in the Win32App detection rules in Intune.

ApplicationDetected: False –> I guess that one speaks for itself!!

More about this in 4.2 ExitCode and STDOUT part

1.6.Reporting Phase

The final phase of the app installation is the reporting phase, IME will send the AppResults back to the service *.manage.microsoft.com

I updated this Flow below because some information wasn’t correct. The data will be decrypted and moved to the IMECache folder instead of the staged folder. Of course, I notified Microsoft about this little flaw.

2. Background info and the Serviceui Tool

We were asked to publish a new app for all users, called Eplan. When Installing this specific app some user interaction is required. When you need to have interaction from the user context with the system context the only option you have is to use serviceui.exe

By default, the Win32 apps which are being installed by IME, are installed by default in the System Context (0)

Please Note: A regular user session in a normal situation, has no permission whatsoever to view any messages or prompts from the app installation that is running in the system context.

Of course, you could change the install behavior in which context the app must be installed

When your users are all Local admin’s be my guest to change it to “users” but if you are reading my blogs often, you will know I really don’t like Local Admins. So changing this option to User is not going to work.

Serviceui.exe to the rescue. Serviceui.exe makes sure when the app is being installed in the system context it can detect the user session and allow to interact with it.

https://www.microsoft.com/en-us/download/details.aspx?id=54259

A while ago, I created a blog about how to do this

3. The Problem

Now we know how the whole app deployment works, let’s take a look at the issue itself. Creating an App with serviceui.exe could be done within a few minutes. We have done this literally a thousand times…. But this one was a little bit different

After the app was visible in the company app we tried to install it in our Sandbox Intune Enrolled device. If you want to read more about Sandbox, please visit this blog

After a few minutes, the installing phase begins. After pressing next, next, selecting some specifically required settings, and clicking on next, this error was shown. Error 1311. Source File not found

That’s strange because the installation file which is executed was copied to the IMECache folder in the decrypting phase?

4. Triggering the Reinstallation

Of course, we need to trigger the installation again and again to determine the root issue of this weird issue. Again I will divide this part into multiple subparts.

So what options do we have when we want to retry the failed app to reinstall? Waiting 24 hours before the GRS / “Global Retry Schedule” will kick in is just plain awful

  1. Waiting
  2. Company Portal
  3. The Registry
  4. A PowerShell Tool

4.1. Waiting

We could wait 24 hours and hopefully, the GRS would kick in to try to reinstall the app someday, sometime?

I guess that’s not the best option we got! But if you don’t care about waiting… this option is the easiest one!

4.2. Company Portal

Of course, everyone uses the Wonderfull Company App and everyone has also assigned the Apps as available so end users can install the apps or reinstall the apps on their own?. When the apps are also made available, we could trigger a reinstallation from the Company App?

Even when your device is “shared” you could initialize a reinstall?

Please note: When triggering the reinstall, first the detection phase will be launched and you guess what happens when it detects the program! The reinstallation will be skipped!

When you don’t want to wait until the company portal app gives you the possibility to reinstall the app as shown above (sorry for the Dutch words) there is still another method available to speed things up!

4.3. The Registry

The best way you could trigger the immediate reinstallation of a Win32App that failed to install is by opening the registry and browsing the IntuneManagementExtension key

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\IntuneManagementExtension\Win32Apps\{SID}\{App Id GUID}

First, you will need to find the Win32 App you are after. But to do so, you will need the have the right App ID. Again you have multiple options to find the needed GUID!

  • You could start looking in the Intune Management Extension event log. As we noticed earlier, it will mention the download and the corresponding App ID
  • The second option you have to find this GUID, is by opening the Endpoint Manager and clicking on the App that needs to be reinstalled. When you selected the App you can make a copy of the GUID in the URL after /appId/.

When we have the App id we are good to go. Just search for it and open the corresponding registry key. When you are sure the app is failing (as an example exit code 1603) you need to remove the whole subkey “159aeebd-fe8a-4571-98ef-de8c72ff5ed5” just like I have marked with a nice red square and restart the Intune Management Extension.

If you don’t want to wait, just delete the whole AppId key it’s also enough to modify the “DownloadStartTimeUTC” in the past 24 hours and wait for the installation will be retried by the GRS schedule.

4.4 A PowerShell Tool

Removing the registry key could take up some of your time and we all know time is valuable, so why not use a tool to do so?

Mathias Melkerson is creating a wonderful tool to do so, when it’s published I will post a link to it

[Intune Operational Tips] – Retry Win32 failed Apps in Microsoft Intune – YouTube

5. The Exit Codes

Now we know how we could trigger the reinstallation, it’s become way easier to look at the issue. But as shown in part 4, we need to understand the Exit codes, Error Codes, and STDOUT first.

As you have maybe noticed in the picture above, the ExitCode. This exit code will be used to determine if the Win32App has been installed successfully. When looking at the Win32App installation flow, you will notice that this is done at the Post Detection Phase as I showed you earlier.

But how does it detect if the Win32App has been successfully installed? It all depends on how you configured your detection rules while you uploaded your nice Win32App! Like an example, a simple File-Based detection rule to make sure OneDrive (x64) is detected after installation

There are many options to choose from, MIS, Registry, File/folder, or just your own custom-cooked PowerShell script to detect the app after and before installation.

PowerShell Detection Rule

When looking at PowerShell script the STDOUT comes into play. First, let’s take a look at how it looks inside the IME log when the install fails

LOG[[Win32App] Checked Powershell script exitCode: -1 EnforceSignatureCheck: 0 RunAs32Bit: 0 InstallExRunAs: 0, result of applicationDetected: False]

When the app is installed the Intune Agent will check the results/exit code from the script. The Intune agent will read the values that were written to the STDOUT stream. When the PowerShell detection script exits with everything except a nice zero value, the script will fail and the Win32 App will be marked as not detected.

If the PowerShell script exists with a nice zero value (0) everything is okay and the Win32 App will get marked as detected and installed. But if the script exits with 1, everything is NOT okay! When this exit code 1 is given when using proactive remediations, the remediation part will kick in

File Detection Rule

That’s somehow a little bit easier… it just checks if the path exists you configured in the detection rule like I am showing below

If everything checks out it will save the Installation Status Service report in the registry:

Computer\HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\IntuneManagementExtension\SideCarPolicies\StatusServiceReports\

6. Taking a look at what’s Happening

Now let’s go further after we reinitialized the installation again and check if the contents of the bin/intunewinfile were the same.  

We detected no differences…. Let’s check what has been copied to the IMECache folder

Looking at the IMECache folder above… it is missing a lot of files… almost every file which is needed to install the app is missing?

We double-checked the intunemanagement log to be sure we didn’t miss any errors. I was expecting the error 0x87D300C9 with the corresponding message: “The unmonitored process is progress, however it may timeout”. When receiving this message, it could be due to some misconfiguration in the install parameters.

But unfortunately, no usable errors for us this time. To be sure, it was not the device itself we also tried it on multiple clean installed VM’s, Sandboxes and retried the installation multiple times.

So what’s happening here?

No PathTooLongException:  The folder is not over the 248 characters

No Defender issue: we made sure we excluded these folders:

  • c:\windows\IMECache
  • c:\program files (x86)\microsoft intune management extension\content folder

To be sure the exclusion was added we performed a get-mppreference to get the exclusions. As shown below, the folders are added successfully!

*Source : Troubleshoot Win32 apps in Microsoft Intune | Microsoft Docs

-When you are packaging Win32 apps, please make sure you have the latest version installed! Because looking at the release notes of the Win32 Content Prep Tool / IntuneWinApp tool I noticed this:

In version 1.7 some issues were fixed with Unicode support.

  • But unfortunetly updating the tool and recreating the app didn’t fixed it, so let’s dig in some more. When looking closer and refreshing the IMECache folder every few seconds, we noticed the files were removed even while the installation was not yet completed…?

Let’s open the Intune log again. After scrolling down the log we noticed this StatusService error a few minutes while it was installing the App.

The installation is a success but the detection rules just failed… and within 7 seconds the Intune Management Extension (IME) starts to delete the temporary IMECache folder.

Of course, this is default behavior because the app was installed successfully! Let’s take a look at the first installation window and the last error prompt.

That makes totally sense!!! The first installation is the 5.70 version, when this installation is completed successfully it will send a good Exit code and the IME will start cleaning up but the installation is not done yet, it still needs version 5.70 sp1!  The 5.70 sp1 installations will start, but it is missing all the files and will throw an error and revert the installation!

7. The solution

Okay, now we know it is default behavior how are we going to solve this? The solution is very simple! We need to make sure we combine all the files into a zip file. When the deploy application is called upon we need to make sure it first extracts all the files and when all the files are extracted it will start the installation.

7.1. Zipping it:

Instead of adding all the separate files and folders inside the app\files folder, we need to zip them.

7.2. Changing the deploy-application.ps1

We need to make sure we add an additional install task: Expand-Archive and change the execute-process parameter to match the setup file from the zip file.

7.3. Testing the Win32App

Create a new Intunewin app and upload it to Intune. While installing the app, watch the IMECache folder… instead of extracting all the files, it extracts the zip file.

It will start extracting the files to the specified folder and it will install successfully because this folder is not emptied after the first installation is completed!.

Conclusion

Again… another deep dive into troubleshooting Win32app installations. Some time ago I also did a blog about troubleshooting the problem with the RunOnce key

It’s very important you know how to troubleshoot win32 apps and it’s even more important to know how to read the intune management log.

2 thoughts on “IMECache: Attack of the Cleaner

  1. Hi there I read the article I will try the things you mentioned in this article I hope my app installed as I am a new IT technician I am still learning. Thank you for your work

Leave a Reply

Your email address will not be published. Required fields are marked *

71  +    =  79