In his concession speech, Sarkozy said: "I have no bitterness, I have no sadness, and I wish the best for my country'', Reuters reports.
"It's time for me to try a life with more private passions than public ones," he said, thanking his supermodel-turned-singer wife Carla Bruni and his children.
Sarkozy promised to quit politics altogether after Hollande defeated him in May 2012. However, he returned to the fray in September 2014, citing the need to rescue France from what he described as the socialist's catastrophic presidency.
According to Guardian, the divisive former president Sarkozy suffered a humiliating defeat, knocked out of the race after he ran a hard-right campaign on French national identity, targeting Muslims and minorities. His poor score after a campaign in which he suggested banning Muslim headscarves from universities and was forced to protest his innocence faced with several legal investigations into corrupt campaign financing, showed he had become just as much a hate figure on the right as on the left.
Fillon, a socially conservative, free-market reformer who admires Margaret Thatcher and voted against same-sex marriage, came close to winning the nomination straight out, with around 43% of the poll.
He now faces a second-round runoff against more moderate Alain Juppe, the mayor of Bordeaux who was prime minister under Jacques Chirac.
Fillon and Juppe have clashed most forcefully over Fillon's proposals to slash the cost of government, most notably by axing 500,000 public sector jobs over five years.
Behind his still-boyish looks and refined demeanor, the 62-year-old Fillon is as close to a true economic and social conservative as they come in France.
The winner of the Republican primary is likely to make the presidential run-off, where he or she will probably face far-right leader Marine Le Pen.