The New York Times has a really good article on this topic:

Let’s be honest. No one actually knows when and where the housing market will bottom. Experts have been proclaiming the bottom is now — this very moment — since Alan Greenspan notoriously predicted the worst was over way back in 2006.

Different economists and firms have different takes on whether the market is overpriced or underpriced. There are conflicting views about the Bay Area housing market as well. Personally I continue to believe that the market is still overpriced.

Maybe. The prices are still a bit too high but they may come down to reasonable levels in a few months (6-18 months). If you are looking for a detailed explanation, follow along in this long post.

Do house prices appreciate in the long-run?

Historically, house prices show small appreciation in real terms over long periods. Most people find it hard to believe this and cite anecdotal data. The S&P/Case-Shiller Home Price Indices measure the nominal value of the residential real estate market in 20 metropolitan regions in the United States. The following chart shows the national index since 1890. Other than a 40 year dip between 1910 and 1950, the house prices have typically hovered around a very narrow range. The big dip in 1910 started with World War I. The dip continued as construction costs came down due to mass production techniques. The prices didn’t really go up during the great depression either. Finally, World War II helped bring the prices back to the 1910 levels. Since then the prices have stayed fairly stable except for the 70’s and the 80’s booms that lasted a few years. Something strange has happened since 2000 and the house prices have skyrocketed.

Has something changed fundamentally since 2000? No. Prices for houses are set in the same way as any other good - supply and demand.

The demand is affected by:

1. Wealth: This is fairly intuitive - the more wealth people have, the more demand there will be for houses.
2. User cost of capital: This is a fancy way of combining the effects of interest rates, maintenance, depreciation, property taxes, and risk of owning a house.
3. Demographics: The more people that move into an area, the more demand for houses there is.
4. Preferences: A lot of people in the Bay Area want to live in San Francisco making it more expensive than other places in the region.

The supply is influenced by:

1. Construction costs: Construction costs are a combination of material costs, labor costs and equipment costs. Cost of construction has been fairly stable since the 70s.
2. Land prices: The Bay Area has limited land area. The land value appreciates over time as more people move in and want houses.
3. Regulation: Local and state governments do not allow construction on all available land. They are also interested in ensuring that prices don’t collapse suddenly. So while we do see new construction in the Bay Area, it is fairly limited.

It is obviously hard to predict how demand and supply will change. However, looking at historic trends, it is clear that house prices do not appreciate significantly over long periods. The boom since 2000 is a bubble. There have been real estate bubbles in the past (70s and 80s, Japan in the 80s). Real estate bubbles take time to ‘pop’ because real estate market is not as liquid as the stock market and buyers and sellers have imperfect information in the short-run.

Is the Bay Area different from the rest of the country?

Yes, the houses in the Bay Area are more expensive than the national average but they mirror the national trends quite closely.The following chart shows the Bay Area prices and the national prices since 1987. Note that the scale on this chart (including the index values) are different from the previous chart (I could not get data that is consistent).

How can I be sure that this is a bubble?

The best way to predict if there is currently a bubble is to compare house prices to rental prices. Rental markets are typically not subject to bubbles. In rising real estate markets, people want to buy houses because they believe that they can sell the house at a higher price in the future. This is a self-fulfilling prophecy that leads to bubbles in the short-run. There is no such expectation in rental markets.  Price to Rent ratio and  Price to Income ratio are good indicators of a bubble.

The following chart tracks the Price to Rent trend for the Bay Area (The values on the vertical axis are not meaningful in an absolute sense and should only be looked at for relative comparison):

The current ratios are still quite high compared to the baseline in the mid 90s. If the trends hold, the prices may be reasonable in a few months (6-18 months). The Price to Rent ratio should drop by about 33% for the bubble to be over. Check out HousingTracker.net for the latest ratios. Unfortunately, I have used different data sources compared to this site because the sources that this site used did not go back long enough.

Sources:

1. Irrational Exuberance
2. Real Estate Demand and Supply: Are real estate markets efficient?
3. S&P/Case-Shiller Home Price Indices
4. BLS CPI Surveys