Shinzo Abe is vowing to stay on as Japan's prime minister despite the crushing defeat suffered by his ruling coalition Sunday in upper house elections.

According to exit polls, Abe's Liberal Democratic Party (LDP) is losing grip on the majority it held along with its coalition partner in the upper house -- while the opposition Democratic Party of Japan made big gains.

It would be considered unusual for a prime minister to resign after an upper-house defeat. But calls for Abe to step down from voters and from those within his party were expected to grow due to a string of government scandals and the bungling of pension records.

Abe, 52, conceded exit poll results are "severe," but he said he was "determined" to carry out promises he made and brushed off questions about whether he should resign.

"We need to restore the people's trust in the country and the government," he told reporters at his party's headquarters.

"I must push ahead with reforms and continue to fulfill my responsibilities as prime minister.

"The responsibility for this utter defeat rests with me."

Abe's top lieutenant, Hidenao Nakagawa, said late Sunday he would step down to take responsibility for the party's setback.

The ruling party has controlled Japan virtually uninterrupted since 1955. It maintains control in the powerful lower chamber, which chooses the prime minister. Abe has dismissed opposition calls for an election for the lower house to test his mandate. But according to Japan's Kyodo news agency, Abe would reshuffle his cabinet, possibly in late August.

"The nation has spoken very clearly," Democratic Party of Japan leader Naoto Kan told reporters. "Naturally, our sights are on the lower house and our final goal is a change in government."

Polls by major TV networks showed the LDP and its junior coalition partner, the New Komei Party, emerging with 104 seats. That represents a 28-seat loss that left it far short of the 122 needed to control the house.

The Democratic Party, meanwhile, appeared set to win 111 seats, up from 83. Official results were not expected until early Monday local time.

Abe promised to build a "beautiful Japan" when he took office in September as the country's youngest-ever prime minister. He was commended for his efforts to repair strained relations with South Korea and China.

But then came a series of government scandals and gaffes, including one involving Administrative Reform Minister Genichiro Sata. He stepped down in December over charges of misusing of political funds. Then in May, Abe's agriculture minister killed himself amid allegations he also misused public money. The new agriculture minister became embroiled in another funds scandal.

Last month, the government came under fire when Defence Minister Fumio Kyuma suggested the 1945 U.S. nuclear bombings of Japan were justified. Kyuma's stepped down after the public outcry that followed.

With a report from The Associated Press