MOGADISHU, Somalia - Tens of thousands of famine-stricken Somali refugees were cold and drenched after torrential rains pounded their makeshift structures in the capital, as the UN raised concerns that renewed conflict in the country may jeopardize relief work.

Rains are needed to plant crops and alleviate the drought that has lead to famine in Somalia, but added to the misery of many refugees who live in structures made of sticks, flattened milk cans and pieces of cloth.

Disgruntled refugees in several camps in Mogadishu said Sunday that more aid is needed.

"We are living in plight, we left our homes, lost our animals and farms so we ask everyone to help us to survive," Abdi Muse Abshir said.

Lul Hussein, a mother of five, said her family had a sleepless night after their makeshift home crumbled.

"We are starved and we don't have enough help," she said. "Who's helping us? No one! So we are already between death and bad life."

Aid agencies have limited reach in Somalia where Islamist militants are waging an insurgency against the country's weak U.N.-backed government.

The most dangerous group among the militants ---- the al-Qaida-linked al-Shabab ---- has barred major relief organizations from operating in the territories it controls.

The U.N. fears tens of thousands already have died in Somalia in areas held by the Islamist rebels because food aid could not reach them.

The African Union peacekeeping force anticipates that al-Shabab may try to attack the camps that now house tens of thousands of famine refugees in Mogadishu, disrupting even further the distribution of food aid. The AU force launched a new offensive Thursday to push the militants' front line farther back from the camps.

The drought and the famine it has caused in Somalia have affected more than 11.5 million people in the Horn of Africa and created a triangle of hunger where the borders of Ethiopia, Kenya and Somalia meet.

The World Program says it cannot reach 2.2 million Somalis who live in al-Shabab controlled territory in south-central Somalia.

The U.N. Office for the Coordination of Humanitarian Affairs said, in a report released Sunday, that the military offensive against al-Shabab is raising concerns that conflict will jeopardize humanitarian response efforts during the drought crisis.

"The current conflict will cause more civilian casualties and further displacements as the number of drought IDPs (Internally Displaced Persons) in the capital continues to increase," the report said.

The report said displacement is already occurring in areas close to the districts affected by fighting in Mogadishu.

The AU peacekeeping force has said it will keep aid agencies informed about the offensive to try and minimize its impact on relief work.

The UNOCHA report about the situation in Somalia up to July 29 said that up to 100,000 famine refugees had arrived in Mogadishu in June and July and a total of 160,000 since the crisis began.

The refugees in the camps in Mogadishu complained that camps are cramped with makeshift homes that have little privacy.

"We can't even change our clothes because everyone is here," said Mumino Jimale. "We ask the world to help us in getting better settlements to be safe from cold and rains."

Also Sunday, Pope Benedict XVI urged the world not to forget to the victims of famine in the Horn of Africa during his weekly blessing to pilgrims.

"It is forbidden to be indifferent in the face of the tragedy of the starving," the pope said from his summer residence in Castelgandolfo, south of Rome, Italy.

The pope invited the faithful "to think of the many brothers and sisters who in these days, in the Horn of Africa, are suffering the dramatic consequences of famine, aggravated by war and the absence of solid institutions."

Aid agencies say the region's drought is one of the worst in years in terms of the number of people affected.

Thousands of Somali refugees are flooding camps in Ethiopia and Kenya in search of food after several seasons without rain decimated livestock and killed crops in their homeland.