At least one influential Congressman thinks so...and he's introducing legislation to do just that:
Besides imposing hefty new federal taxes on gasoline, the forthcoming bill would, in Dingell's words, seek to "remove the mortgage interest deduction on McMansions - homes over 3,000 square feet." Dingell said he recognizes that proposals like these will be highly controversial, but he says they are essential to achieving the environmental goal of reducing carbon emissions by 60 to 80 percent by 2050.
While the concept sounds good, I'm afraid that it would simply result in a bunch of still low-density, 2,950 square foot houses. Which, obviously, wouldn't be much of a long-term improvement.
Rather than this approach, I would first introduce legislation eliminating the MID on all second homes. That too is incremental, but seems politically more viable and doesn't use a seemingly arbitrary square footage number.