Perhaps Obama, maybe... The economy took a turn for the worse during the 2008 campaign but Bush was a lame duck anyway. It did get worse during his first term, but was on the upward improvement by 2012 election sequence (if you look at the "Epic Rap Battle of History" done with Obama vs. Mitt Romeny for the election, Romney's line given "took you four years to get unemployment below 8%" was a last minute change as the day prior to the release of the video it was announced that the unemployment rate was below 8% for the first time in his administration). Obama did actually perform worse in the 2012 race than the 2008 race (in 2008, he won something like 10 states more than his opponent, in 2012, he won 2 states more) and many critics felt that the economic growth under him was largely in spite of Obama's policies, not because of them.
To anyone who follows the economy and politics, they are rarely tied to each other strongly, though the President will claim a good economy and push a bad one on his predecessor. Usually, the health of the economy is critical at election time only in the public eye as a good economy will usually get a candidate (or party in the case of a lame duck) elected and a bad economy will get the incumbent party thrown out. It's generally held that the Regan question applies ("are you better off now then you were four years ago?" Which Regan famously asked during his bid against the Carter presidency... and the answer was not one Carter would end up liking). So while Obama did have a down turn ongoing through much of his Presidency, the Economy did look like it was starting to turn good during the 2012 election... even if it was very slow when compared to economic trends prior to it and didn't really speed up until 2017 when Obama was out.
If we go by the president's party instead of the president, then it could be argued that the Democrats in 2016 were ousted despite the economy being in a recovery mode. This also counts for only two of the last three one term presidents, as Bush (H.W.) and Carter were both presidents during bad economic re-election periods, though Bush's problem was his fix broke a campaign promise (read my lips, no new taxes... technically, he didn't break that promise as he raised existing taxes but technically, the voters didn't care about the letter vs. spirit of the promise) so it's debatable if it was strictly over what Clintion famously said was "the economy, stupid".
Carter was very much ousted because of his poor handling of middle east issues leading to the OPEC Embargo and fuel shortages in the US, though he also had the Iran Hostage Crisis that was bogging down his support. Regan focused on the economy first as the crisis was ongoing for sometime and at least one network anchor on the nightly news was reliably giving the count in days as to how long that problem was going on for.
President Johnson is debatable as he served as president for the later half of the JFK presidency and was re-elected in 64, but declined to run for a "second term" in 68 due to his handling of Vietnam. To be certain, there was a lot of problems in the 68 election year alone that made the year rather tumultuous (same with 2016... it was a difficult year before we even discuss that it was also an election year). Democrats didn't secure the White House for a variety of reasons, but the economy isn't the first problem that comes to mind (Vietnam was much more to blame for Johnson specifically and the '68 Democrat party convention was... to put it mildly... more embarrassing theater of war than Vietnam... and that was little to do with the politics of 'Nam.)