But now, with the "fiscal cliff" deadline just days away and no deal in sight, more investors are paying attention.
"This is a matter of a few personalities; it isn't something where you can analyze spreadsheets to figure out what's going on," said David Kelly, chief global strategist at JPMorgan Funds. "There are very few investors on one side or the other who have wanted to make a strong bet on this one."
Many investors believe that the higher taxes and lower government spending could push the US back into recession.
"This is not small potatoes," said Hugh Johnson, chairman and chief investment officer of Hugh Johnson Advisors in Albany, New York. "We're not going to miss a recession by much in 2013 as is."
To be sure, plenty of traders think the "fiscal cliff" is overhyped. Even if the government misses the Monday deadline, the higher taxes and lower government spending would take effect only gradually, and Congress could always repeal them.
Still, that doesn't mean they're not worried about the economy in general.
Derrick Irwin, portfolio manager for Wells Fargo Advantage Funds, said he's "not particularly concerned" about the fiscal cliff - but he is concerned about the economy.
"The economy is going to do what the economy is going to do," Irwin said, "it just doesn't look too good."
Trading volume was light, with many investors still on Christmas vacation. About 2.8 billion shares traded hands, compared to an average this year of about 3.6 billion. Light volume can make the market more volatile. When fewer shares are changing hands, relatively small trades can move the overall market.
The yield on the 10-year Treasury note edged down to 1.74 percent from 1.75 percent.
China sends first oceangoing patrol vessel to South China Sea