Call4Cloud

The Incredibly Strange Device Who Stopped Syncing and Became Certificate Zombies!!?

This blog will NOT be about the new Autopatch function! Of course not! This blog will be about multiple weird Intune Device sync errors I was asked to troubleshoot.

I will try to explain all the stuff I looked into and how I started troubleshooting to make sure it was fixed when I was done!

I will divide this blog into multiple parts

  1. Introduction
  2. Troubleshooting Command
  3. 0x80072f99
  4. Troubleshooting MDM certificate issues
  5. Lingering expired MDM certificates
  6. Something is missing?
  7. Certutil
  8. Combining all the steps

1. Introduction

I am going to mention 2 separate issues. One of them was on Reddit and the other one was on the MEM group on LinkedIn. Both of them shared the same issue and error codes but still, they were 2 separate issues

1. Intune: Mobile Device Management (reddit.com)

Looking at the first issue he is mentioning that several computers have “dropped out of Intune”

Text

Description automatically generated

Okay… that’s something you don’t see every day? Let’s look at the second question/issue I got on the MEM group on Linkedin

2. Modern Endpoint Management (MECM | SCCM | Intune | AzureVD | Security | MacOS | iOS ) | Groepen | LinkedIn

Looking at the picture below he is mentioning OMA-DM message failed to be sent. Result: (Unknown Win32 Error code: 0x80072f99

Graphical user interface, text, application, email

Description automatically generated

2. Troubleshooting command

When you need to perform some remote troubleshooting, you could always run the remote diagnostics in Intune but this time I asked them to run a nice PowerShell command on the device itself

wget https://aka.ms/intuneps1 -outfile IntuneODCStandAlone.ps1

powerShell -ExecutionPolicy Bypass -File .\IntuneODCStandAlone.ps1

this wonderful command downloads the required XML file from Github

https://raw.githubusercontent.com/markstan/IntuneOneDataCollector/master/Intune.xml

This XML contains all the locations the PowerShell script needs to pull data from. It’s a need to have when you need to perform some basic troubleshooting

After running this PowerShell command, you will end up with a nice folder with a lot of information in it!! And I mean a lot!

Graphical user interface, text, application, email

Description automatically generated

After I received the zip file with all the diagnostic information in it, I decided to check out the error code I got first as it could tell me in which corner I need to start digging.

3. 0x80072f99

When you need to start troubleshooting, having an error code is always nice to have. If you have an error code and you don’t know what it means, just open the wonderful CMTrace tool and perform the Error Lookup.

As shown below, I looked up the error code: 0x80072f99.

Graphical user interface, text, application

Description automatically generated

Let me translate it if your Dutch isn’t good enough to read it. No credentials were available in the client certificate AKA ERROR_WINHTTP_CLIENT_CERT_NO_PRIVATE_KEY.

The client certificate? That needs to be the Intune device MDM certificate… no question about it! Luckily I already wrote a blog about sync issues and the error code 0x80190190 some time ago. So please read it to get a better understanding of the Intune MDM device certificate

4. Troubleshooting MDM Certificate Issues

Before we continue the troubleshooting part, it’s always best practice to make sure there isn’t anything wrong in the DSREGCMD /status output. Luckily that file is also included in the zip file

It does look fine, does it? Let’s go back to the 0x80072f99 we got! I will guide you through the troubleshooting steps I always take to find the root cause of an issue.

When dealing with Intune Device certificate issues the first thing I always do is check if the GUID matches the enrollment of the Intune Management Extension (IME) log

As shown above, it is mentioning the same enrollment as I noticed in one of the files I got: Microsoft_Enrollments.txt

With this content software\microsoft\enrollments in it!

When taking a good look at the text file and the enrollment information we notice it’s mentioning the DMPCertThumbPrint. You need to be sure if that one matches the information in the Set MdmDeviceCertificate part in the IME LOG

Graphical user interface, text, application

Description automatically generated

As shown above, this time it matches the right certificate.. No issues here, right?

Most of the time when you have sync errors, you are also experiencing issues with the Company Portal. So I decided to also take a look at the log file of the Company Portal! ….

When running this PowerShell script it will fetch this log from the AppData\Local\Packages\Microsoft.CompanyPortal_8wekyb3d8bbwe\LocalState folder as shown below

Graphical user interface, text, application, email

Description automatically generated

After opening this Company Portal log file, I immediately noticed the log is telling me when evaluating if the MDM certificates has a matching local device, there was no matching local device found

Graphical user interface, text, application, email

Description automatically generated

P.S: This log also contains the information about if your device is compliant. Combining this log and first running this command intuneintunemanagementextension://synccompliance could give you the information you need to show users a toast message?

Until now, I didn’t have opened the event log once! Shall we open some of them? just for good old sakes? So I guess it’s time to take a good look at the certificates and the corresponding event log. I really hoped there was something mentioned in the Crypto-NCrypt event log. This event log is also included in the zip file, isn’t that just wonderful?

Graphical user interface, text, application, email

Description automatically generated

As shown above, a lot of errors, so there is definitely something wrong… that’s for sure. It is only showing us the error 0x80090029

That specific error only means that “The requested operation is not supported”. AKA (-2146893783 NTE_NOT_SUPPORTED)

Okay, let’s close the event log as it isn’t telling us anything useful! But luckily we have enough information left from the diagnostic zip file because when running the troubleshooting command it is also exporting the certificate part of the registry

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\SystemCertificates\MY\Certificates\

This export will be placed inside Microsoft_SystemCertificates.txt . I pulled out that part and created a reg file from it and just imported them Graphical user interface, text, application

Description automatically generated

As shown above, this is my own test device… so it also shows my own Microsoft Intune MDM device Ca (49fc1700-x-x-x-x). So please ignore that one. You will notice it has 2 Microsoft Intune Device Certs! Huh did I miss something? I reopened the IME log. And yes as shown below, it’s also mentioning it finds 2 MDM certificates

Graphical user interface, text, application

Description automatically generated

5. Dealing with lingering Expired MDM Certificates

Having 2 MDM certificates, is not the way it should be, because when you have lingering Intune MDM device certificates you could run into the same issue as I was telling you in the introduction

Graphical user interface, text

Description automatically generated with medium confidence

As shown above, when you run into the 0x80072f99 sync error, make sure you only have 1 valid MDM certificate. If one of them is expired, remove that one NOT the still valid one!! Now let’s continue to solve the second issue because by removing the duplicate certificate our second issue was still not resolved.

6. Something is Missing

Shall we take another look at he the certificates I got? Because looking at certificates is always fun!

As shown above, it looks a lot like it is missing an icon which the expired one does have

This icon above will normally show you if you have the private key for this certificate. It’s even mentioned in the System Event log. Normally I am good at ignoring Schannel events, but this one (36869) is also mentioning the fact that the certificate doesn’t have the private key information property attached to it!

Apparently, the System Event log is also a good place to start looking when you have Certificate issues. So I decided to ask him to run this command to find out if the certificate key pair was still on the device.

$a = Get-ChildItem Cert:\LocalMachine\My\ | where{$_.issuer -like "*Microsoft Intune MDM Device*"}
$a = $a.thumbprint
$a = Get-Item Cert:\LocalMachine\My\$a
$a.PrivateKey.CspKeyContainerInfo.UniqueKeyContainerName
Graphical user interface, text

Description automatically generated

As shown above, the output of the PowerShell script is mentioning a “Unique Container Name”. When opening the c:\programdata\microsoft\crypto\keys there needs to be a file with that exact same name! Luckily in this case the key pair still was still available on the device, so what is going on here?

7. Certutil

Back in the good old days, I also stumbled upon this issue when importing a certificate on a server. After importing it was immediately removed. Back then I solved it by running the command certutil -repairstore my “Thumbprint of the certificate”

I decided to automate it a little bit

Write-Host "List certificates without private key: " -NoNewline
$certsWithoutKey = Get-ChildItem Cert:\LocalMachine\My | Where-Object {$_.HasPrivateKey -eq $false}
if($certsWithoutKey) {
    Write-Host "V" -ForegroundColor Green
    $Choice = $certsWithoutKey | Select-Object Subject, Issuer, NotAfter, ThumbPrint | Out-Gridview -Passthru
    if($Choice){
        Write-Host "Search private key for $($Choice.Thumbprint): " -NoNewline
        $Output = certutil -repairstore my "$($Choice.Thumbprint)"
        $Result = [regex]::match($output, "CertUtil: (.*)").Groups[1].Value
        if($Result -eq '-repairstore command completed successfully.') {
            Write-Host "V" -ForegroundColor Green
        } else {
            Write-Host $Result -ForegroundColor Red
        }
    } else {
        Write-Host "No choice was made." -ForegroundColor DarkYellow
    }
} else {
    Write-Host "There were no certificates found without private key." -ForegroundColor DarkYellow
}

8. Combining all the steps

To solve the second issue that had a lingering MDM certificate AND a valid one but without the private key we took these 5 steps

– Deleted the expired certificate

– Ran the Certutil script, chose the certificate without exportable key

– After running the script the Certificate was fixed and the private key was paired again

– Reboot

– Restart Log on with username/ password and NOT by using Windows Hello

Conclusion

Certificates and Intune go together like a horse and carriage and knowing how to troubleshoot these kinds of sync errors is always valuable information to have.

Its Important GIFs | Tenor

Leave a Reply

Your email address will not be published. Required fields are marked *

1  +  5  =