MacOS Big Sur currently has an upgrade issue. In certain situations, the upgrade will fail and then get stuck with a neverending progress bar.
UPDATED 12/13/21
UPDATE! This issue is fixed in both the Monterey 12.1 and Big Sur 11.6.2 Full installer.
I’ll start with a quick summary of the macOS stuck progress bar situation.
If you are upgrading from macOS High Sierra, Mojave, or Catalina to Big Sur your upgrade could fail if it matches a very particular condition.
If folders in multiple directories in the location below
/private/var/folders/*/*/C/com.apple.metadata.mdworker
have more than 20,000 files your upgrade could fail.
In some situations users have reported 500,000 – 2,000,000 files!!!
This is the reason why the problem could never be reproduced.
If you build a fresh system the file counts for spotlight will be very low. You might have 20-400 entries. This would allow the upgrad to finish properly.
Table of contents
- Different situations. The macOS stuck progress bar issue has many different situations.
- How can I check how many mdworker files my system has before an upgrade.
- I have more than 20,000 files. How do I clear them out before the upgrade?
- The files have been deleted! Now what?
- What if you booted your failed Mac into safe mode after having the issue?
- Will Apple fix this problem?
- Final Thoughts
- Thank you!
1. Different Situations
- Why check? Just because your upgrade failed does not mean that this is the exact issue. Sadly upgrades could fail for multiple reasons. This article is for one very specific issue that usually affects Macs that have been in service for a long time.
- Your Mac hasn’t started the upgrade yet. You can prevent the problem from happening by clearing out the files in that folder before upgrading.
- Your upgrade has already Failed – If your upgrade has already failed you can rescue it!
- System that has failed but was started up with Safe Mode – Your upgrade failed and you used Safe Mode to troubleshoot. You will need to recreate the user account.
I’ll go over each situation below.
2. How can I check how many mdworker files my system has before an upgrade.
We have two situations here. If your Mac Upgrade has already failed. The 2nd situation is you are planning an upgrade and want to make sure it does not fail.
1. If you did not start the upgrade yet and want to check to see if your Mac is affected.
Run this command to check how many mdworker files you have.
sudo find /private/var/folders/*/*/C/com.apple.metadata.mdworker -type d | wc -l
The command could take some time to run depending on how many files your system has. Below is an example of a system that is ready for an upgrade.
If you are under 20,000 files, you should be good to go for the upgrade. If the file count is larger then you will need to clear out these files. (See section
2. If your Mac has already failed the upgrade.
You will need to run the command in macOS recovery or through Target Disk Mode with a 2nd Mac.
I’ll start by showing you how to check in macOS recovery.
Boot your Mac to recovery. Power it down then back up while holding down command R
Once you are in recovery, you might be asked to authenticate if your Mac is encrypted with FileVault 2 or Find My Mac. Do so, and then you will be at the main recovery screen.
Click Disk utility. You might have to mount “Macintosh HD – Data”
Both Macintosh HD and Macintosh HD – Data might be mounted. If they are greyed out then you will need to mount them. If they are already mounted, you can continue to the next section.
You can also click “View” from the menu bar and then select “Show all devices”. Click on Macintosh HD – Data and then click the Mount Button.
Now that you have made sure Macintosh HD – Data is mounted, you can quit disk utilities and open up terminal from the utilities in the menu bar at the top.
Once in terminal, you can run this command to check to see how many mdworker files your Mac has.
NOTE ON DIFFERENT LANGUAGES!!! The word DATA is different in your language. For example, in French it is “Macintosh HD – Données” If you are unsure, you can look in disk utility or run this command ls /Volumes
find /Volumes/Macintosh\ HD\ -\ Data/Previous\ System/private/var/folders/*/*/C/com.apple.metadata.mdworker -type d | wc -l
find /Volumes/Macintosh\ HD\ -\ Data/Previous\ System/private/var/folders/*/*/C/com.apple.mdworker.bundle -type d | wc -l
NOTE: the find command might take a LONG time. You can hit it control c to quit the command and it will still give you a number. If the number is large, you know that you have the problem and don’t have to wait for the entire count.
Notice the folder structure has changed. When a upgrade gets stuck in this situation, the /private/var/folder is in the Previous System folder.
Now we know that this system is affected because it has 1,790,715 mdworker spotlight files.
3. I have more than 20,000 files. How do I clear them out before the upgrade?
Now that we know our system is affected we can delete them. We have 2 different situations. The first one is if your Mac has not failed the upgrade and you want to delete the files to prevent a problem. The 2nd one is if your Mac has already failed the upgrade.
- If your Mac has not failed the upgrade yet
UPDATED: 11/09/21
You can run both commands to remove the problematic directories.
rm -rf /private/var/folders/*/*/C/com.apple.mdworker.bundle rm -rf /private/var/folders/*/*/C/com.apple.metadata.mdworker
Or run both find commands
sudo find /private/var/folders/*/*/C/com.apple.mdworker.bundle -mindepth 1 -delete
sudo find /private/var/folders/*/*/C/com.apple.metadata.mdworker -mindepth 1 -delete
You might not need to use sudo . If you get an error just take that part off and run find without sudo
NOTE: Both the rm and the find command might take a LONG time.
When finished, go to the section 4
- Your Mac has already failed the upgrade.
Run both commands from macOS recovery terminal.
rm -rf /Volumes/Macintosh\ HD\ -\ Data/Previous\ System/private/var/folders/*/*/C/com.apple.mdworker.bundle rm -rf /Volumes/Macintosh\ HD\ -\ Data/Previous\ System/private/var/folders/*/*/C/com.apple.metadata.mdworker
Or you can use find to delete them
find /Volumes/Macintosh\ HD\ -\ Data/Previous\ System/private/var/folders/*/*/C/com.apple.mdworker.bundle -mindepth 1 -delete
sudo find /Volumes/Macintosh\ HD\ -\ Data/Previous\ System/private/var/folders/*/*/C/com.apple.metadata.mdworker -mindepth 1 -delete
NOTE: the rm & find command might take a LONG time.
You will know the commands worked when the command prompt appears again. This means the command finished successfully.
When finished go to section 4
4. The files have been deleted! Now what?
You have successfully checked the file count and cleared the excess file count.
If you removed the files before the upgrade you should be ready to go for a safe upgrade. Make sure that you always backup your files before any upgrade.
If your Mac had a failed upgrade, all you need to do is reboot and the upgrade should finish!!!
5. If your Mac failed the upgrade & you booted up in Safe Mode
In this situation, if you booted a failed upgrade with safe mode it would have started the Setup Assistant. If you created a new user, you would find that the previous user on the system was gone. The good news is, the user home folder is still there. All you will need to do is boot to recovery, remove the extra files mentioned above so the upgrade can finish. Once the upgrade finishes, you can add the user back. This is taking into account that you have a service account or admin account on the Mac. If not, you might have to pull the data off the system and reinstall macOS.
6. Will Apple fix this issue?
I think they will.
The fix will require a new full installer. You know by now that I’m a big fan of macOS full installers. I hope that Apple will continue to release them for each update.
I will let you know and update this article when the issue is fixed.
6. Final thoughts
This problem has been plaguing users since at least July of 2021. This means that the issue in the big sur installer could go all the way back to 11.4 or 11.5.
Jeff Johnson wrote about this exact issue back in August of this year.
https://lapcatsoftware.com/articles/macBS.html
At the very end of the article, he says this.
“I still don’t know why my Big Sur update got stuck. I don’t think I’ll ever know.“
Sadly this article did not come in time for you Jeff, but at least now you know what happened. Maybe this article will help save others until Apple fixes this issue.
7. Thank You
This issue was a huge group effort from multiple #MacAdmins.
A few weeks ago, I started putting pieces together that this was not an isolated issue.
I created the #MacAdmins Slack channel #mojave_upgrade_problem. In this channel is where multiple experts came together with information to help solve this issue.
In the end, after giving Apple Engineers a ton of information, they came back with a folder location to check for files. From that info, we figured out that if we removed the excess files the upgrade would finish!
Thank you
- Apple Engineering
- stevemaser
- bradtchapman
- paulfank
- atlauren
- Jeff
- ljcacioppo
- gmarnin
- magnusviri
+ Everyone else that helped with the macos stuck progress bar issue. 👍
Hello,
Just a post to confirm that this solution worked for me too. Yesterday, I tried to upgrade from Mojave to Big Sur and got stuck. I let the Mac on all night but the progress bar was still at the same level this morning.
I was ready to roll back to yesterday’s Time Machine save but thought I could try one last time then found your article.
The faulty folder in my case had 700K elements and after deletion and reinstalling, I finally got back to the login screen with Big Sur installed.
Regarding your article, I would suggest to had a quick warning about languages impacting the find commands in the terminal because I am French and for me it was not “Macintosh HD – Data”, it was “Macintosh HD – Données” so maybe non-English users could be warned of that because nothing was found when I enter “Data”.
Thanks for the help 🙂
Nico, I’m really glad that you were able to rescue your Mac!!!
Thanks for the suggestion, I’ll add that now.
Does anyone know what the root cause is of the actual spotlight problem is that’s leaving all these orphaned files laying around? Lots of log spamming by spotlight stuff as well…
CSS, my guess is that it is a problem in Mojave and High sierra. If you check on catalina or big sur the folder amount is WAY smaller.
Thank you for your research and sharing! I have a lot of Big Sur upgrades to do and besides a test-upgrade with a fresh Mojave, 2 of the first 4 Macs failed.
Here is a discussion and some more code about the “com.apple.metadata.mdworker”:
https://forums.macrumors.com/threads/spotlight-mdworker-shared-file-flooding-system-log-removal-stops-search-index-from-working-properly.2248163/post-30206048
Thank you, this solved my problems!
I use a Mac mini late 2014 and started my upgrade from Mojave to Big Sur on October 20 before midnight and had exactly the described problem: in the morning black screen with apple logo and progress bar at 26%. Since I expected a long update on a spinning HD, I waited until after lunch. I then did several recommended actions: zap parameter RAM, disconnect all USB devices, boot into recovery System and repair disk with disk utility. But after each reboot I ended at the same progress bar at 26%. Then I installed MacOS anew from the recovery System, also unsuccessful.
I was ready to erase the disk and restore from backup, when I found your article around 6 pm MES. Counted the files, I had 435k. Within one and a half hour I was at the Big Sur login screen. 🙂
Some notes:
– since I have a german system, the data volume was called „Macintosh HD – Daten“ (That this second part is language specific surprised me).
I hit other bugs:
– after the upgrade, my user account was no longer an admin user. I had to force a new setup run to create a new user with admin Privileges (i.e. „rm /var/db/.AppleSetupDone“ in recovery System)
– the downloaded podcast files and podcast subscriptions from iTunes did not migrate to the new Podcast App
The worst upgrade experience I have ever had. But would have been worse without your article.
Alfred, thank you very much for sharing your experience. I’m really glad that you are back up an running again!!!
Just curious – is the intent to find files, or directories with that find command? the ‘-type d’ option finds directories. 500K-2M directories? Sheesh! What is Apple doing in there?!
What is the impact on day-to-day use if I delete these files?
I checked, and I have 271,864 files that resulted from the ‘sudo find /private/var/folders/*/*/C/com.apple.metadata.mdworker -type d | wc -l’ command.
Are these some cached files that speed up the system? Does it make sense to clear this out periodically? If so, I wonder what Apple does not clear out this directory on each reboot.
I am still running Mojave on a late 2015 iMac 27″ Retina system. No doubt I need to upgrade to Big Sur soon (worried about losing 32-bit compatibility) for some old apps I still use occasionally.
I am hoping for an answer to this as well. I did a quick find on our systems and have some at over 5,000,000. That find command is brutal on those systems and I feel I should delete them, but want to know what side affects that might cause.
Any clue yet if Apple will address this in the macOS Monterey installer? Or do we have to 🤞 and test on Monday when 12.0.1 goes public and we finally see a full installer?
If we are targeting devices enrolled in Jamf, is there any harm in running this script silently deleting any files in that directory ahead of time?
That would be the idea Bruce, run that script on a policy to clear them out before they upgrade. You can also run it as part of a self service upgrade.
Over the summer, one of my remote users ran into this issue. She took the laptop to a local authorized Apple service provider, and they wiped her drive then installed Big Sur. Ugh! Wish I had know about this issue at that time.
This happened to me but installing Monterey Beta 10 from scratch using OCLP 0.3.0 stable, then again when updating to RC1 21A558 in my Macbook Pro 9,2 (i7). In both cases i had to hard reboot the mac then the installation process did continue without further issues.
I do face this exact problem, but found this article one day after I ran into the problem. Bad luck.
Is it ok to simply delete the “suspect to be faulty” files without checking first how many there are? The find command took way over an hour for me and there is no way how I can see if it still needs more time or got stuck. I found over 617,000 files.
Appreciate that you wanted to get this out as fast as possible, but can I suggest some amendments please: Chapter 4, for failed upgrade, using sudo leads (for me) to an error, leaving out sudo worked.
The last command in chapter 4 appears to have an excess space before com.apple.metadata.mdworker. I left this out and it worked.
Do say somewhere that the actual delete takes as long as the original respective find.
(and yes, you’re right, the command line is not exactly my home turf)
Michael, I will modify those sections. I was trying to get this out there asap. It is ok to remove the entire folder, I did that and the upgrade and system was fine after. The delete will take some time but hopefully not as long as the find.
Thank you sir this, Mr.
Thank you!!!!
The second command in “already failed the update” section contains an extraneous space:
sudo find /Volumes/Macintosh\ HD\ -\ Data/Previous\ System/private/var/folders/*/*/C/ com.apple.metadata.mdworker -mindepth 1 -delete
It should be:
sudo find /Volumes/Macintosh\ HD\ -\ Data/Previous\ System/private/var/folders/*/*/C/com.apple.metadata.mdworker -mindepth 1 -delete
Thank you Alex! I’ve edited that line to fix it 👍
I had issues with a number of bigsur upgrades. Since doing upgrades in safe mode I haven’t had any issues since