Which account balances should consumers pay off first to become debt free?
This is a question that polarizes camps from different sides. If you had $300 extra each month to apply towards debt elimination you would want to follow the process that works the best. There are two popular beliefs:

Pay extra on the highest interest rate first
Pay extra on the lowest balance first

Pay extra on the Highest Interest Rate
This process is most efficient. Interest is charged on the outstanding loan balances, so sending $300 extra to a 19% credit card will charge less interest next month than applying the extra to a 3% car loan. Less interest is a good thing.
Pay extra on the Lowest Balance
This method is not mathematically as efficient as the Highest Interest Rate process. It requires extra payments to be applied towards the debt with the lowest payoff balance, not necessarily a debt that charges interest at a higher percentage. This is more commonly known as the Debt Snowball Method.
They both work in theory, but which one works best?
Two researchers from Northwestern University, David Gal and Blakeley B. McShane, obtained access to a unique data set that gave precise information as to how 6,000 people finally eliminated their credit card debt. Researcher Gal said:

"We found that closing debt accounts, independent of the dollar balances of the closed accounts, predicted successful debt elimination at any point in the debt settlement program"

The survey says...
Reducing balances or paying less interest were not the leading factors of success, it was getting the accounts closed! Accomplishing little goals motivates people to continue in a process that appears to be working, like checking things off a To-Do list. The sad truth is that Americans aren't diligent enough to stick through a process that will cause them to pay less penalties (interest). The good news is the Debt Snowball Method has a better success rate of keeping people in the game - all the way to the finish line!
A visual representation of how it feels

Using an awesome excel spreadsheet from Vertex42 we can compare the differences between the two methods and see the true net effect. This couple has a New Year's Resolution to start the process of paying off $45,035 in debt after the ball drops in Times Square. They have $1,200 a month to put towards their goal, $261.77 of which is used to attack their six debts after the $938.23 in minimum payments.
Doing it by the numbers

Following the traditional advice, this couple would pay their minimums and then apply all their extra money towards their highest interest debt - a 22.99% credit card. Continuing this process every month will give them their first win (paying off the card) in nine months. Once it is gone they can take the minimum payment of $48.69 and the extra $261.77 and apply it to the next debt on their list - their "Chasing" credit card. This debt will be eliminated in another twelve months (a total of 21 months after beginning the process). Keep attacking the debt in this manner and they will have the last debts paid off in July 2016 and have paid $5,228.07 in interest charges.
The math doesn't work!
If your goal is to pay the least amount of interest, then staying out of debt completely is the best way! However, most Americans accept debt as a way of life until they get a wake-up call when they get laid off, someone dies or is born, or they realize retirement is on the horizon. Math alone does not get people out of debt, people sticking to a plan get out of debt.
Staying motivated is the key

Consider this: The couple will hit their first win in only four months by paying extra towards their smallest debt. Whew! Wouldn't that feel good? Now they have the extra cash-flow to attack the next smallest debt and the second one is knocked out in another eight months. They are more likely to continue the process because they feel they are making progress.
Side effects