Military commander Maj. Gen. Lucky Irabor confirmed an accidental bombardment in the northeastern town of Rann, near the border with Cameroon, saying "some" civilians were killed.
It was believed to be the first time Nigeria's military has acknowledged making such a mistake in a region where villagers have in the past reported civilian casualties in the near-daily bombings targeting the Islamic militants.
Nigeria's President Muhammadu Buhari expressed deep sadness and regret at "this regrettable operational mistake."
A Borno state government official, who was helping to coordinate the evacuation of wounded from the remote area by helicopters, said more than 100 refugees and aid workers were among the dead. He spoke on condition of anonymity because he was not authorized to speak to reporters.
Doctors Without Borders said its team based in Rann counted at least 52 bodies and was treating 200 wounded, many in critical condition, and the death toll was expected to rise.
"This large-scale attack on vulnerable people who have already fled from extreme violence is shocking and unacceptable," said Dr. Jean-Clément Cabrol, the aid group's director of operations.
As night fell, the group's team struggled to stabilize the seriously wounded. "We hope that during the night not many more people will die," said the group's head of emergency programs, Hugues Robert, describing a complex evacuation because the area is insecure.
Photographs of the carnage showed a man carrying a wounded child, his clothing stained with blood, as well as bloodied victims being treated on the ground outside a tent clinic overflowing with the wounded. Nearby, corpses lay covered by blankets and prayer mats, alongside mounds of hastily dug graves.
After the attack, the charred remains of makeshift corrugated iron lean-tos and mud homes filled the landscape.
The International Committee for the Red Cross said six workers with the Nigerian Red Cross were among the dead and 13 were wounded. "They were part of a team that had brought in desperately needed food for over 25,000 displaced persons," spokesman Jason Straziuso said in a statement from Nairobi, Kenya.
Two soldiers were also wounded, as well as Nigerians working for Doctors Without Borders, Irabor said, without giving a precise figure.
The general, who is the theater commander for counterinsurgency operations in northeast Nigeria, said he ordered the mission based on information that Boko Haram insurgents were gathering in the area, along with geographic coordinates.
It was too early to say if a tactical error was made, he said, adding that the bombing would be investigated.
Doctors Without Borders spokesman Etienne l'Hermitte in Geneva urged authorities to facilitate cross-border land and air evacuations.
"Our medical and surgical teams in Cameroon and Chad are ready to treat wounded patients. We are in close contact with our teams, who are in shock following the event," he said in a statement.
Villagers have previously reported civilian casualties in airstrikes on Boko Haram positions in northeastern Nigeria.
Some of the schoolgirls kidnapped by the insurgents in 2014 and freed last year have said three of their classmates were killed by air force bombardments, according to the freed girls' parents. Of the nearly 300 schoolgirls who were abducted, 196 remain missing.
The bombings have helped drive Boko Haram out of many towns and villages and, according to Buhari, the insurgents' last stronghold in the Sambisa Forest last month.
Boko Haram's 7-year-old Islamic uprising has killed more than 20,000 people and forced 2.6 million from their homes, creating the continent's worst humanitarian crisis, with the United Nations warning some 5.1 million people face starvation.
(AP)