Podcast Summary: Ready For Retirement – Why $5 Million is the Tax "Danger Zone"

Host: James Conole, CFP®
Episode Date: February 8, 2026

Episode Overview

In this episode, host James Conole dives into why reaching a $5 million portfolio changes retirement tax planning fundamentally. Rather than focusing on accumulation, James explains that the new game is all about protecting and strategically withdrawing assets to minimize lifetime tax liabilities. He outlines the "tax danger zone" that comes with this level of wealth and presents actionable tax strategies tailored to high net worth retirees to help navigate complex account structures, avoid substantial tax missteps, and ensure more wealth stays in your pocket throughout retirement.

Key Discussion Points & Insights

1. Why $5 Million is a “Different Game” in Retirement

• Account Complexity Increases:
  • At this level, retirees rarely have just one account type. Most have a combination of brokerage accounts, 401(k)s, Roth IRAs, HSAs, stock options, and more.
  • Managing withdrawals from these various account types becomes the driving force behind your tax bill, not merely your overall net worth.
• Portfolio Itself Can Push You into Higher Brackets:
  • Even without earned income, portfolio income, gains, and required minimum distributions (RMDs) can trigger steep tax brackets, IRMAA surcharges, and other costly tax surprises.
• Shift in Focus:
  • The goal becomes tax minimization over your entire retirement, not just reducing this year's tax bill.
  • Quote:

    “I need you to stop thinking through how do you reduce this year's tax liability and instead think about how do you reduce your lifetime tax liability.” (03:54)

2. Strategic Withdrawals: Controlling Your Tax Bracket (04:20)

• Distribution Planning:
  • The retiree has full control over which accounts to draw from and can strategically "manufacture" their taxable income to fill target tax brackets—unlike during working years.
  • Utilize Roth accounts, HSAs, and brokerage accounts to optimize withdrawals instead of defaulting to traditional pre-tax accounts.
• Lifetime Tax Efficiency:
  • Emphasizes pulling the right amount from the right accounts each year to avoid higher brackets in the future.

3. Intentional Roth Conversions (07:12)

• Not All Roth Conversions Are Equal:
  • “Roth conversions require that you have an understanding of what tax bracket are you in today. But more importantly, what tax bracket will you be in in the future?” (07:49)
  • Avoid blanket advice like "just convert to Roth in retirement." Instead, conversions should be tailored—factor in future RMDs, expected income, and potential tax increases.
• Case Study: Eric and Donna
  • Example with $7.5 million in investments; their income is $1 million/year, currently in the highest bracket.
  • Key Insight:
    • Tax “valley years” (between retirement and required minimum distributions) present an optimal window for tax planning.
    • Mistake: Over-converting and filling higher tax brackets (like the 35% bracket) can cost nearly $1 million in lost wealth.
    • Right approach: Target a sweet spot (e.g., the 22% bracket for Eric and Donna) for conversions, potentially saving a quarter-million in taxes.
  • Quote:

    “These aren't just little mistakes that you can get away with. These are six figure, almost seven figure mistakes.” (12:30)

4. Asset Location (17:15)

• Distinction from Asset Allocation:
  • Asset location is about placing investments in the right account types for tax efficiency.
• Common Mistake:
  • Holding high-yield, tax-inefficient investments (e.g., REITs) in taxable brokerage accounts leads to significant tax drag.
  • Illustration:
    • A $1M REIT with a 6% yield yields $60,000… which could be cut in half after federal and state taxes.
    • Solution: Reposition to tax-advantaged accounts (IRA, Roth IRA).
  • Key Takeaway:
    • After-tax returns become paramount at higher portfolio levels.
    • Quote:

      “It’s no longer your returns that matter, it's your after-tax returns that matter.” (19:14)

5. Capital Gain and Charitable Planning (21:13)

• Capital Gain Harvesting:
  • Take advantage of preferential tax treatment for long-term gains, especially the 0% bracket for married couples filing jointly with taxable income under ~$99,000 in 2026.
  • Realizing gains up to the threshold can allow you to reset cost basis tax-free each year.
• Charitable Giving Strategies:
  • Don’t just give cash—gift highly appreciated stock directly to charities to avoid the tax on gains and receive a full deduction.
  • Use of donor-advised funds to bunch large donations in one year while retaining flexibility to distribute gifts incrementally.
  • Quote:

    “By gifting highly appreciated securities, you are getting a tax benefit. The charity is not paying taxes, and you're still getting the full deduction amount.” (28:50)

  • Consider qualified charitable distributions (QCDs) from IRAs after age 70½ to further minimize RMDs and taxes.

6. Final Reflections: The Real Risk and Power of Planning (34:30)

• Critical Takeaway:
  • The risk isn’t paying taxes, it’s paying them at the wrong time, which can mean paying far more than necessary.
  • Comprehensive retirement planning must include laser-focused tax strategies.
  • Quote:

    “If you have a financial plan that doesn't include taxes in it, you don't really have a financial plan.” (36:20)

Notable Quotes & Memorable Moments

• On the shift in retirement focus:

  “The goal isn't just to keep growing it like it used to be. Now the goal starts to shift to controlling it and protecting it, especially from taxes.” (00:15)

• On tax inefficiency turning into huge costs:

  “Those tax mistakes that used to be little inefficiencies at the edges, those now turn into six figure problems that will compound throughout your retirement.” (00:35)

• On comprehensive planning:

  “Tax planning should be a very, very critical piece of all that. If you're looking for that and not getting that, reach out to us.” (17:10)

Important Timestamps

• 00:15: Setting the stage – Shift from growth to protection above $5M
• 03:54: Importance of lifetime tax minimization
• 07:49: Why intentional Roth conversions beat blanket advice
• 12:30: The costly mistake of over-converting (Case Study: Eric & Donna)
• 17:15: Asset location explained
• 19:14: After-tax returns vs. pre-tax returns
• 21:13: Capital gain and charitable planning strategies
• 28:50: Gifting appreciated securities and donor-advised funds
• 34:30: "The real risk here isn't paying taxes, it's paying them at the wrong time."
• 36:20: “If your financial plan doesn’t include taxes in it, you don’t really have a financial plan.”

Conclusion

James Conole’s episode offers a roadmap for high net worth retirees to move beyond generic financial advice and execute sophisticated tax minimization strategies. By focusing on account withdrawal sequencing, personalized Roth conversions, optimal asset location, and strategic giving, retirees with $5 million+ can meaningfully preserve and grow their wealth over the long term. The episode drives home that proactive, nuanced tax planning isn’t optional at this level—it's essential for “return on life” in retirement.