Well, that is a huge mistake that you will regret later. His co-workers won’t be around when you are in your seventies and needing money — and can’t work. Please roll it over to Fidelity or Vanguard– and invest conservatively in a balanced fund or equity-income fund.
When you lose your job, any loans become due and payable — or are taxed as ordinary income, plus penalties. You may avoid the penalty because of the Covid provisions.
I’d rather see you sell the second car and put the money toward house payments. Believe me, this crazy time will end, he will get back to work — and your retirement money needs another 20 years to grow. So do everything possible to avoid cashing it in.