I Killed My Endpoint Privilege Management Enrollment, Hung Her on a Meathook, and Now I Have a Three Picture Deal at MMP-C

Patch My Pc | install & update thousands of apps

This blog will be just me looking at the delivery of the EPMagent in my own troubleshooting style. While doing so I got focused on even something more important!! I will divide this blog into multiple parts

Please note this blog is based on some fine-ass OSINT…. Not 100% sure if it’s totally accurate so changes can be made along the way. As I still need to look up more details, this is just version 1.0 of this blog

1. Introduction

I guess a brief introduction would be nice, right? In this blog, I intended to look at what would happen when you have activated Endpoint Privilege Management(EPM) and configured some policies in your tenant.

Afbeelding met tekst  Automatisch gegenereerde beschrijving

After we have pressed “activate” the required DeviceHealthMonitoring policies will be delivered to your device.

With the DeviceHealthMonitoring policies being delivered to the device, the EPM Agent AKA MEMEPMAGENT magically appears on your device. Before going into many details, let’s take a look at what happens.

2. Installation in 3 seconds

When looking at the EPM installation we will notice that just before the EPM agent gets installed, “something” will trigger the EPM installation. The enrollment starts with the “Discovery Phase”.

In this discovery request, we will notice the Enrollment URL it needs to get to start the “Enrollment”. With the enrollment URLs in our pocket, it will try to fetch the Policies.

Repeat those 2 names… Discovery and getting policies… sounds familiar right?

As shown above, it kinda reminds me of the MDE enrollment? So I guess after the policies we should get the Binary Security Token? As shown below, after the device received the policies it would indeed receive the Security Token. Which is in fact the ???????

With the enrollment done, the SideCar Agent/ IME EPMAgent installation will be executed by the same CSP that also triggers the IME installation when the device gets enrolled into Intune

./Device/Vendor/MSFT/EnterpriseDesktopAppManagement/MSI

Afbeelding met tekst  Automatisch gegenereerde beschrijving

We will spot the same CSP when looking at the DeviceManagement-Enterprise-Diagnostics-Provider Event log

Afbeelding met tekst  Automatisch gegenereerde beschrijving

Of course, when looking at the EnterpriseDesktopAppMagement registry key, we can spot the same location being mentioned in the LocUri

Afbeelding met tekst  Automatisch gegenereerde beschrijving

Besides this LocURI we will also notice the real download location of the MSI file

https://epmagentpeeusstore.blob.core.windows.net/epmagentpeeus/epmagentmsi/6.2303.42.1000-1/EPMAgent.msi

After the discovery and enrollment phase, the CSP triggered the download. With the download in our pocket, it’s time for Msiexec to start installing this MSI file from the system its Local Appdata folder. It’s all system!!!

If everything checks out we have our wonderful Microsoft EPM Agent Service installed

Afbeelding met tekst  Automatisch gegenereerde beschrijving

With the installation of the Microsoft EPM Agent service, I guess the Blog is done!! Let’s drink some beer!

3. First look with Procmon?

I guess our journey is far from over. Let’s resume our journey to the center of EPM, by looking at the Procmon trace I got. Please Note: Besides this Procmon trace, I also got a nice Fiddler trace and some good old WPR tracing. So you are up for something nice!

Afbeelding met tekst  Automatisch gegenereerde beschrijving

In the Procmon trace, I filtered out the garbage and within a few seconds, I noticed the Epmservice.exe en taskhostw.exe every few seconds. Thread Create, thread exit. That can’t be a coincidence. The Taskhostw.exe, could indicate a scheduled task, and again looking back at the MDE enrollment that does ring a bell!

Afbeelding met tekst  Automatisch gegenereerde beschrijving

I guess that doesn’t sound that stupid. So let’s look at the Enterprisemgt scheduled tasks folder and the corresponding enrollments registry key. As in both locations, we should spot something funny?

Afbeelding met tekst  Automatisch gegenereerde beschrijving

Huhhh 2 Intune enrollments? that’s odd. That doesn’t sound good? Even the device name and EntDMID doesn’t match with the Device ID in Intune.? I know a thing or two about troubleshooting Intune issues and having a lingering Intune enrollment is bad ! (or isn’t it?)

I was curious if this enrollment also was levering the OMADMCLIENT.exe to sync stuff. Let’s switch back to our Procmon trace to find out. We will notice that with enrollment in progress, it indeed also launches the omadmclient.exe with some additional parameters

When looking at the additional parameters we will notice that the serverid corresponds to the Microsoft Device Management Enrollment… If we take a look at that main enrollment key we could also spot some “weird new” URLs

Afbeelding met tekst  Automatisch gegenereerde beschrijving

Okay we got ourselves an aadresourceid mentioning checkin.dm.microsoft and a nice discoveryservice URL, mentioning enrollment.dm.microsoft.com (instead of enrollment.manage.microsoft.com )

I guess with the time we are spending in the registry let’s take a look at the corresponding OMADM accounts registry key. In it, we will have another registry key sslclientcertsearchcriterea

Sounds familiar right? This SSL Client Cert setting is used to find the correct Certificate to be used for communication. So with this registry key, there should also be a Certificate? As shown below, yes! Besides the Intune device certificate, we have an additional certificate.

The Intune MDM certificate secures communication with Intune, so I assume this new Microsoft Device Management Device Certificate also secures communication with “something”???? Not Intune, I guess. I decided to take a look at my adjusted WPR file and the ETL trace it created.

4. The WPR Trace

So I combined some stuff from different trace providers and just put them all in a proper time frame

So before the MDMAppInstaller installs the EPMAgent, it first starts the enrollment (StartEnrollmentEvent) into something called MMPC? And after it got enrolled into MMPC, it got the device management certificate we noticed earlier. So I guess this device management certificate is used to secure communication with MMPC?

The MMPCENROLL part made me curious so I started opening the famous DLLs and EXE files that were updated when installing the 2023-03 preview update on my Windows 10 device.

Afbeelding met tekst  Automatisch gegenereerde beschrijving

omadmclient.exe, omadmprc.exe, deviceenroller.exe, mdmregistration.dll, omadmpapi.dll

5. MMP-C

MMP-C…. It could ring a bell or two because it sounds like Defender. It could be the Microsoft Malware Protection Center, but as that name is already taken, it has to be something else. I guess it must be important as it is mentioned in almost every DLL or other changed Executable file I open.

Afbeelding met tekst  Automatisch gegenereerde beschrijving

In a stupid attempt, I tried to use Google and chatgpt. Somehow all my searches ended up in a complete failure

Complete-failure GIFs - Get the best GIF on GIPHY

But I decided to spend some more time looking at the WPR trace I got. Looking back at my WPR trace and just looking at each tiny step it mentions Microsoft Device Management together with “Microsoft Management Platform Complete”

Mmm…okay… that sounds like a possible name except for the “complete” part. Besides the mention of MMP-C, I also stumbled upon something I need to take a closer look at in the future when digging into the EPMAgent itself

Afbeelding met tekst  Automatisch gegenereerde beschrijving

As shown above, the DeclaredConfiguration is truly something I am not familiar with right now, so I decided to continue and take a look at the enrollment with Fiddler.

6. Fiddler

When looking at the Fiddler trace I got, once the EPMagent was installed (with the use of the exported Microsoft Device Management Certificate) I noticed some more details

As shown above, a nice ticket/token to authenticate at the checkin.dm.microsoft.com service we noticed earlier in the Enrollment registry. Could this be the “MMPC” I stumbled upon?

Besides the checkinuri, we will again notice the “DeclaredConfigurations”. These configurations are also stored in the registry.

Please Check my blogs about the WIN DC / Declared Configurations to find out what they are and why they are way, way better

https://call4cloud.nl/2023/04/declared-configuration-epm/

Mmm… okay that’s still not much to go with. At that exact point in time, I wanted to throw in the towel but then I suddenly noticed the POST in Fiddler

WCOS Checking… as in Windows Core OS checkin?

Okay, so besides the MMPC we also got WCOS.(please read my blog about this) Keep that part in mind for now and let’s dig deeper

7. Enrollments key

So with some more new names and acronyms added to our brains, I switched back to the registry while looking at the two enrollments. The Intune enrollment suddenly had one nice addition.

A new folder, LinkedEnrollment, was created in the Enrollment registry key. As shown below, this LinkedEnrollment also contained some additional keys.

The picture above is telling us that our Intune enrollment is “Linked” with the Microsoft Device Management enrollment. Besides this information, it also shows us the “Enrollstatus” and something called MMPCLocked. I guess our Intune Enrollment is locked into the Microsoft Management Platform?

When taking a closer look at the deviceenroller.exe … Brainfart!! …

Brainfart mode ON

Am I even allowed to talk about the deviceenroller.exe???? As the first rule about the device enroller is…. We don’t speak about the deviceenroller.exe

Brainfart mode OFF

When digging into a weird-ass named executable file, something caught my attention. “SetLinkedEnrollment”

Afbeelding met tekst  Automatisch gegenereerde beschrijving

Luckily (sometimes you need some luck) when using Google this time, searching for LinkedEnrollment, pointed me to a nice MS Doc.

Afbeelding met tekst  Automatisch gegenereerde beschrijving

As shown above, this document mentions the fact that it will trigger a silent MMP-C enrollment by using the AAD device Token from the AADJ device. Luckily this “keyword” also pops up in some DLLs.

Afbeelding met tekst  Automatisch gegenereerde beschrijving

When spending some more time in IDA and looking at where it leads me, every time I end up with the same mention of “Dual Enrollment”. Maybe this could be a wild goose chase……but looking at the name “Dual Enrollment” we indeed noticed 2 enrollments. Double / 2 / Dual?

8. What is in the Name Part 1

But I am not giving up on my wild goose chase, so I spend some more time looking at stuff, and then I tried something else. Googling MMPC or WCOS alone gives you all kinds of results but putting them together showed me something beautiful

A LinkedIn profile from a principal engineer working at Microsoft.…. When looking at the details of the profile and her experience, we will notice some keywords in it

Afbeelding met tekst  Automatisch gegenereerde beschrijving

MMPC, WCOS, and reporting services… And the title…. Microsoft Managed Platform Cloud.

*If you are interested in part 2 of this journey please read this blog

MMP-C | MMPC | Acronym | Microsoft | Platform | Cloud (call4cloud.nl)

So it seems we have 2 enrollments on 1 device. One enrollment communicates with Intune and the linked enrollment with the Microsoft Managed Platform Cloud. But it seems that there is also some communication between them. How else does Intune know how to push the CSP to install the EPMAgent and where did you configure the EPM settings? Intune!!!

9. Digging Deeper

After a good night’s sleep (NOT)…..

I decided to reinstall my device/vm but this time again without the 2023-03 preview and enrolled the device with Autopilot. When the enrollment was done, I took a look at the EnterpriseMgmt Tasks and noticed these tasks

I downloaded the 2023-03 preview update and rebooted the VM, this time in a VLAN, so I could open FIddler and capture some traffic. I opened the EnterpriseMGT tasks, and first, I noticed no additional task had been created.

Before removing the VLAN, I made sure I got the Intune certificate exported and attached to Fiddler

Using Fiddler to decode Intune MDM or Graph traffic (call4cloud.nl)

With everything ready to go I removed the VLAN and “synced” the device. Within a minute or 2, the device started the “Discovery phase“. It did so by reaching out to the discovery URL (Discovery.dm.microsoft.com).

After the device was able to reach out to the discovery URL, an additional Task Schedule was delivered: “Schedule created by dm client for dual enrollment...to MMPC

This scheduled task triggers the deviceenroller.exe with the /c /EnrollMmpc parameters! This task will stay on the device until its job is done! When you have Fiddler in its way or SSL inspection… the enrollment will fail! As shown below, you will receive a certificate chain error!

After “only excluding” the discovery.dm.microsoft.com from HTTPS decryption, we will notice the enrollment kicks in.

Within this enrollment, we will notice that the EnrollmentType mentions MMPCComplete. …. that is one thing I heard before…

Anyway, just look at the trace mentioning the same…

Once the device is registered for DualEnrollment to MMPC by using the AAD Device credentials it will delete the task that started the MMPC enrollment. Sounds a bit like cleaning up your mess 🙂

I guess besides the EPMAgent enrollment, I just found myself a new hobby….. troubleshooting “Dual Enrollment” With the MMPC enrollment, the real EPM installation kicks in and would start the installation of the Client Side Components / EPMagent.

When troubleshooting the EPM/MMP-C enrollment, you could want to “KickStart” this enrollment on your own turf. I decided to move this part to a dedicated blog post.

10. The MSPAINT Flow

When putting all the pieces together when looking at all the flows/traces and available documentation, we will get this “enrollment flow” (Updated 02-08-2023)

https://call4cloud.nl/wp-content/uploads/2023/08/mmpc-arrival-flow-v3.bmp

11. The New Enrollment Flow

Conclusion

The Microsoft Managed Platform Cloud!!! I didn’t see this one coming! Besides MMPC, what if we combine this with the WCOS check-in? AKA Windows Core OS AKA Windows 10x? In my understanding, that project got ditched some time ago… I guess not!!!

It just feels like a honeypot! MMP-C, LinkedEnrollment, DualEnrollment, WCOS Checkin

Stick with me, and I will try to tell you more about the EPM agent next week!

10 thoughts on “I Killed My Endpoint Privilege Management Enrollment, Hung Her on a Meathook, and Now I Have a Three Picture Deal at MMP-C

  1. Great article… what is the workaround if you want to test EPM and still want to be able to use the Company Portal?

  2. Great article, just opened a support ticket this morning because I was no longer able to open Company Portal on my 3 devices… I was testing EPM this week and yes, it looks like EPM did break Company Portal on my devices. I wonder how long Microsoft takes to find this out…

  3. Ohh, being on the ball I see! Interesting read, especially for the device health tricky bit! 🙂

  4. Hhm, interesting, but what would be the right way to prepare the target devices beforehand via Intune in regard of that Device health issue deployment error? That KB indeed helped me on some older 21h2, but I have a couple of 22h2 cases where even though the version is higher than requested, the error is happening.

  5. Hi Rudy

    Did you get any feedback from MS regarding this issue in EPM? I opened a support ticket with this outcome: “When i tried to repro the issue at my end i couldnot see the issue and was advised by my SMEs that this could not be related to EPM” Your blog is telling me something different… Was this fixed in the meantime so the problem does no longer occur?

    1. Yep… I assume you are talking about mmpc messing up the cp? if so there was a missing “enum”, they indeed fixed it.

  6. Great article, as always Rudy!
    We are testing EPM on a few clients, and just recently it stopped working for two users. When they click Run with elevated access on PowerShell, they get an error popup saying “Not implemented”. Nothing has changed on their computer, except for the regular Windows Updates.

    Just today I got it myself as well.. The change from a couple of days ago is I got Windows update KB5027231 yesterday, and I was testing Security Baselines.

    Logs says the following:

    What might have gone wrong? What can we test?

    Regards
    Patrik

  7. I am seeing this 404 error in the Microsoft-Windows-DeviceManagement-Enterprise-Diagnostics-Provider/Admin logs on a couple Windows 10 22H2 and Windows 11 22H2 devices that have the August 2023 cumulative installed. This error was occurring before the August CU was installed and the devices have always been on the current month’s CU when the issue has been occurring. And ideas what could cause this or how to remediate it? Both devices are missing the ConfigDeviceHealthMonitoringServiceInstance reg key in HKLM\SOFTWARE\Microsoft\PolicyManager\default\DeviceHealthMonitoring\

    MDM ConfigurationManager: Command failure status. Configuration Source ID: (805F5EF6-E05A-42E9-8B20-6E1CB12807A1), Enrollment Name: (MDMFullWithAAD), Provider Name: (Policy), Command Type: (Add: from Replace or Add), CSP URI: (./Vendor/MSFT/Policy/Config/DeviceHealthMonitoring/ConfigDeviceHealthMonitoringServiceInstance), Result: (The system cannot find the file specified.).

Leave a Reply

Your email address will not be published. Required fields are marked *

61  +    =  64

Proudly powered by WordPress | Theme: Wanderz Blog by Crimson Themes.