Alberta's Progressive Conservatives lost the seat held by Ralph Klein while he was premier at a byelection in Calgary on Tuesday night, in a defeat that is seen as a symbolic blow to Premier Ed Stelmach and his provincial party.

Liberals won the Calgary-Elbow seat, which the Conservatives had held for more than 35 years, while the Tories held on to the other byelection seat in Drumheller-Stettler.

Liberal candidate Craig Cheffins, a political neophyte, defeated Tory candidate Brian Heninger by just under 800 votes.

"We knew that we were running a good campaign, we were building on previous campaigns, but I'll fully admit it was a bit of a surprise to us," Cheffins said Tuesday night.

During the byelection campaign in Calgary-Elbow, Heninger tried to distance himself from the premier, who has recently found Tory support declining across the province, particularly in Calgary.

But Heninger only had the most gracious of compliments for the premier on Tuesday night.

"I think the premier is a great man, and I think that he is kind-hearted and a hard worker and an honest man and I think he deserves all his support from the PC party," Heninger said of Klein's successor.

"And he certainly has all of my support, he is an amazing man."

Stelmach, a farmer-turned politician, said he was disappointed by the result in Calgary-Elbow.

"Calgarians have sent a message and I have heard that message clearly," Stelmach said in a statement on Tuesday night.

"I will redouble my efforts to communicate with and unite all Albertans so we can work together to reach our full potential in this great province.

"Let me assure you, mine is a government with a clear plan for dealing with the growth pressures in Calgary."

Support for Stelmach's Conservatives has been dropping in recent polls, but they still hold 61 seats of 83 seats.

But it was no small feat winning a former premier's seat, said provincial Liberal Leader Kevin Taft.

It "symbolizes a shift, I think, that is happening in Calgary," Taft told CTV Calgary.

It could mean trouble for the right-wing party, said one political scientist.

"No question it's a coup. This is a riding that's a heartland for the Conservatives," political scientist David Taras said.

"They've won it nine straight times -- this is bedrock. If the Liberals can take this riding, then the world has changed in Calgary."

While the Tories lost the seat in Calgary-Elbow, an affluent riding in the southwestern part of the city, they held on to one in the rural riding of Drumheller-Stettler.

Drumheller-Stettler was recently vacated by former finance minister and deputy premier Shirley McClellan.

Tory candidate Jack Hayden, a Stelmach loyalist defeated Liberal candidate Tom Dooley by 3,178 votes.

Before the byelections on Tuesday, the Tories held 60 of the legislature's 83 seats; the Liberals had 15; the NDP had four and the Alliance had one. There was one Independent and two vacant seats.

With reports from CTV Calgary