First of all, I’m willing to bet that you placed your home in a REVOCABLE LIVING TRUST! Not an IRREVOCABLE trust. In fact, I’ll help you sue the lawyer who did it if this was an irrevocable trust!
And probably the estate planner helped you re-title the house in the name of the RLT. But YOU (and probably you and your wife) are the TRUSTEES of that RLT. And you can refinance at any time. You are likely to find it difficult if not impossible to refinance if you are retired and no longer have income from work.
So if you want to pay down that loan on a 20 year schedule, you can always add an extra amount — the same amount every month — to your monthly payment. Specify that it is to go toward the PRINCIPAL of the loan balance.
BUT WHY WOULD YOU DO THAT? If you placed all your retirement funds into this house (and paid the taxes at that point), surely you need to keep some liquidity on the side for emergencies. The only one who will benefit from you putting more equity in your house is your heirs!
I think you really need a session with a fee-only FIDUCIARY financial planner to review — and explain — your situation. Find one you can trust at www.Wealthramp.com. And please write back and let me know that your home is indeed in a Revocable Living Trust. Ignorance is dangerous.