The prospect of early elections has intensified fears over the country's financial future, causing investors to sell off the country's stocks.
The coalition government's candidate for the presidential post, the 73-year-old former European commissioner Stavros Dimas, garnered 168 votes from parliament's 300 seats — short of the 180 votes needed to win.
The government now has 10 days to dissolve parliament and declare early elections in a move that some fear could endanger the debt-ridden country's international bailout.
Opinion polls have consistently shown the left-wing main opposition Syriza party ahead. The party opposes the terms of the bailout deals which kept Greece from defaulting on its debts.
Syriza has pledged to roll back some of the reforms implemented in order for the country to qualify for billions of euros in rescue funds from other eurozone countries and the International Monetary Fund — although it has recently somewhat softened its rhetoric about unilaterally pulling out of the bailout deal.
Prime Minister Antonis Samaras had said an election could be "disastrous" while heavily indebted Greece is negotiating with its creditors.
Investors reacted badly to the vote, with the Athens stock exchange's benchmark general index down 10.8 percent in midday trading minutes after the vote.
Dimas said the vote still showed a strong majority in Parliament backed his candidacy.
"I think I expected the result. I remain calm, as ever," he said. His first comments as he left Parliament were to express his wishes for the safe rescue of hundreds of people stranded on a Greek ferry on fire in the channel between Italy and Albania.
The ferry caught fire on Sunday with 478 people on board. One person has died and so far more than 320 have been rescued.
