Second homes aren't only for rich people. They have become mainstream, whether they're used for vacations or for rental income.

More than 9 million dwellings in this country are second or third homes, accounting for about 6 percent of residential sales.

You don't have to have a pile of cash on hand to buy a vacation house. You can use the equity in your primary residence to help pay for a second or third home. Make sure you explain to your lender what you're doing, and you might want to consult with a tax adviser, too.

"There are so many variables, so many moving pieces," says Anthony Hsieh, president of HomeLoanCenter.com.

"But the high-level perspective is you need to see how much equity you do have in your existing home and what are the benefits of pulling out equity in your existing home vs. borrowing. It's always about the cheapest cost of borrowing."

It's not always easy to identify the cheapest cost of borrowing. The lender needs to know which house will be your primary residence and which will be secondary.

In most cases, you will find that the interest rate on an owner-occupied home will be about three-eighths of a percentage point lower than for a nonowner-occupied house.

With today's low rates, that means you might be able to get a loan for 5.5 percent on your primary residence and 5.875 percent on the loan for the second home.

"That argument automatically gives you more motivation to get as large of a loan as possible on your primary residence because it's the cheapest cost of borrowed money," Hsieh says.

Many borrowers resist this line of reasoning because they want to build equity in the home they live in. It seems instinctually like the smart thing to do.

So if you have a lot of equity in your primary residence and want to buy a vacation home, it might make sense to refinance the mortgage on the primary home for more than the current loan balance. That's called a "cash-out refi" because you borrow more than the current balance, pay off the current loan and get the remainder in cash.

You can use the cash from your primary home's equity to make a down payment on your second home.