Why You Don’t Need to Worry about Taxes in Retirement

As a tax accountant, I am programmed to think about taxes. To worry about people paying too much in taxes. To obsess over tactics that allow clients to grind down their federal and state payroll and income tax liabilities.

In short, if there’s a personality type that worries too much about taxes, I’m in that group. But with that said, let me endorse and underscore something that William Bernstein, the Portland physician, financial writer and investment adviser, said in a recent interview with the International Business Times (see here).

Bernstein said people shouldn’t worry about the taxes they’ll pay in retirement because they’ve saved a bunch of money inside their tax-deferred accounts.

Thank you, Dr. Bernstein. Taxes on retirement accounts are not something that many people need to worry about. And because people do worry about them too much, I want to share some additional tax-related thoughts to amplify his remark.

Most People Are in the Great Majority

Let’s get the most significant reason—and the most awkward reason—out here first: Most people don’t save anywhere near enough for retirement to worry about the taxes they’ll owe when they withdraw money from their tax-deferred accounts.

Note: This reality explains why most people should not use a Roth-style account, something we’ve discussed in other posts at this blog, including here.

You can get data from a bunch of different sources that elaborates on this awkward reality. And the data doesn’t all perfectly sync up, but just to start with one reasonable source: Look at the savings data reported by the Employee Benefits Retirement Institute in a fact sheet based on 2014 data. (See, for example, figure 3 on page 2 of this document.) The median retirement savings of someone in the 55-or-older age group—people either on the doorstep of retirement or in retirement—equals about $50,000. That savings level may produce around $2,000 a year of retirement income, but surely that income won’t be taxable.

And the fact is, even retirees who’ve saved a lot more won’t often pay much in income taxes during retirement. Let me dig into this a bit because people often don’t do a thorough job with their accounting.

Remember that Social Security benefits often aren’t taxable. Benefits only get taxed when the sum of any non–Social Security income plus half of any Social Security benefits exceeds $25,000 (or $32,000 for a married couple filing jointly). This means, however, that when you combine typical Social Security benefits with withdrawals from even nicely sized IRAs and 401(k)s, most people just don’t have much tax to worry about.

The accounting gets complicated, but we have mocked up realistic tax returns in our office where a single taxpayer with $25,000 of Social Security benefits who is withdrawing required minimum distributions from a $450,000 IRA pays no income taxes.

We’ve done the same thing for a married couple with $32,000 in Social Security benefits who are withdrawing required minimum distributions from a $600,000 IRA, and again our calculations show this couple pays no income taxes.

It is tricky to accurately calculate the percentile rank of someone with these retirement savings balances. (There’s an online calculator you can use here that estimates the percentile rank of various retirement savings balances based on Federal Reserve data.) But I feel comfortable guessing that these balances put someone in the top 10% or very close to the top 10%.

Note: In another post here we talk about what Federal Reserve and IRS data suggest the top 10%, top 5% and top 1% look like.

The bottom line? Most of us simply don’t save enough for retirement to pay much income tax during retirement.

Why the Top 10% Probably Doesn’t Need to Worry, Part 1

If you or you and your spouse have done a really outstanding job saving for retirement—say you’ve amassed a big six-figure or even a seven-figure nest egg—I still don’t think you need to worry. Or at least not too much.

But first, a digression: Let’s not lose sight of the fact if you’ve amassed a big six-figure or a seven-figure nest egg that you’ve adequately prepared for retirement. Outstanding work. You’ve accomplished the tough part. Sure, you’ll pay some taxes. Somebody has to, right?

But even though you will pay some taxes on retirement account withdrawals, you quite likely won’t pay as much as you might worry.

If you’ve accumulated so much money in your retirement accounts that you’re worried about income taxes, for example, you will probably actually leave substantial amounts of savings, intact and untaxed, inside your IRA and any similar accounts.

In other words, and not to put too fine a point on this, but you or you and your spouse will probably die before you deplete your savings. (Again, remember that the scenario we’re talking about here is where you’ve amassed some rather substantial nest egg, so you do, in fact, need to pay rather substantial amounts of tax on withdrawals.)

Whatever you leave untaxed after you’re gone, however, is irrelevant. You are not, by definition, going to pay that tax. Maybe your heirs will. Maybe. But note that if you split something like an IRA among several beneficiaries (like kids and grandkids), you probably solve most of the theoretical tax problem we’re worrying about that way.

A $2,000,000 balance in a bunch of different tax-deferred accounts (a balance that’s pretty tough to hit if you work out the numbers) becomes five “mere” $400,000 retirement accounts if you bequeath your savings to your three kids, your little sister, and that special-needs grandchild.

The tax on a $400,000 account, as we’ve discussed, is just not that bad if it’s drawn down in retirement.

Why the Top 10% Probably Doesn’t Need to Worry, Part 2

Okay, so what if you or your spouse does live an absurdly long time. Long enough that you guys do seriously begin to erode some gigantic account balances. That does mean you’re drawing down tax-deferred accounts and that you will pay bigger tax bills as you withdraw this money.

Is this where things all get ugly? Well, maybe not—at least in a discussion about taxes.

If you live into your late eighties or nineties or beyond, you will probably begin to load up your tax return both with big required minimum distributions and with significant medical expense itemized deductions for stuff like assisted-living care. But here’s the thing people miss about this scenario: This income and these expenses may net out against each other on your tax return and mean you don’t need to worry about taxes on distributions.

In other words, if you’re paying, say, $100,000 a year for assisted care, that will shelter the income taxes on nearly 90% of $100,000 of retirement account withdrawals. (Medical expense itemized deductions are only deductible to the extent total medical expenses exceed 10% of the taxpayer’s adjusted gross income and only if the taxpayer itemizes deductions. Accordingly, you won’t be able to shelter 100% of a $100,000 withdrawal via $100,000 of medical itemized deductions—only a generous chunk of the withdrawal.)

Only a Small Lucky Cohort Needs to Worry

So who’s left to worry about their taxes on retirement income? People who save a bunch, live a long time, get a lucky streak in their investments and then stay healthy and independent to the very end.

No offense, but I don’t think you want to worry about falling into this group. Gosh, we should be so lucky.

Want More Information?

The Internet is loaded with information, but you can’t believe everything you read online. And making the info fit your specific situation is very difficult without guidance. We are here to help. We have some wonderful ebooks available that are chock full of great advice. You should check them out.

Want advice tailored to you? Give us a call or drop us an email to find out if we’re a good fit to work together to help you with your taxes and answer your questions. And keep in mind that all of our clients are eligible to receive complimentary copies of our ebooks, so contact us before you buy a book. Maybe you will end up with the whole collection for free!