In spring 2011, Joplin, Missouri, was devastated by an EF5 tornado with estimated winds exceeding 200 mph. The storm caused 161 fatalities, injured over 1,000 people, and damaged and destroyed around 8,000 homes and businesses. The tornado carved a mile-wide path through the densely populated south-central area of the city, leaving behind miles of splintered rubble and causing over $2 billion in damage.

The powerful winds of tornadoes often surpass the design limits of most residential and commercial buildings. Traditional methods of assessing damage after a disaster can take weeks or even months, delaying emergency response, insurance claims and long-term rebuilding efforts.

New research from Texas A&M University might change that. Led by Dr. Maria Koliou, associate professor and Zachry Career Development Professor II in the Zachry Department of Civil and Environmental Engineering at Texas A&M, researchers have developed a new method that combines remote sensing, deep learning and restoration models to speed up building damage assessments and predict recovery times after a tornado. Once post-event images are available, the model can produce damage assessments and recovery forecasts in less than an hour.

The researchers published their model in Sustainable Cities and Society.

“Manual field inspections are labor-intensive and time-consuming, often delaying critical response efforts,” said Abdullah Braik, coauthor and a civil engineering doctoral student at Texas A&M. “Our method uses high-resolution sensing imagery and deep learning algorithms to generate damage assessments within hours, immediately providing first responders and policymakers with actionable intelligence.”

The model does more than assess damage — it also helps predict repair costs and estimate recovery times. Researchers can assess these timelines and costs in different situations by combining deep learning technology, a type of artificial intelligence, with advanced recovery models.

“We aim to provide decision-makers with near-instantaneous damage assessment and probabilistic recovery forecasts, ensuring that resources are allocated efficiently and equitably, particularly for the most vulnerable communities,” Braik said. “This enables proactive decision-making in the aftermath of a disaster.”

How It Works

Researchers combined three tools to create the model: remote sensing, deep learning and restoration modeling.

Remote sensing uses high-resolution satellite or aerial images from sources such as NOAA to show the extent of damage across large areas.

“These images are crucial because they offer a macro-scale view of the affected area, allowing for rapid, large-scale damage detection,” Braik said.

Deep learning automatically analyzes these images to identify the severity of the damage accurately. The AI is trained before disasters by analyzing thousands of images of past events, learning to recognize visible signs of damage such as collapsed roofs, missing walls and scattered debris. The model then classifies each building into categories such as no damage, moderate damage, major damage, or destroyed.

Restoration modeling uses past recovery data, building and infrastructure details and community factors — like income levels or access to resources — to estimate how long it might take for homes and neighborhoods to recover under different funding or policy conditions.

When these three tools are combined, the model can quickly assess the damage and predict short- and long-term recovery timelines for communities affected by disasters.

“Ultimately, this research bridges the gap between rapid disaster assessment and strategic long-term recovery planning, offering a risk-informed yet practical framework for enhancing post-tornado resilience,” Braik said.

Testing The Model

Koliou and Braik used data from the 2011 Joplin tornado to test their model due to its massive size, intensity and availability of high-quality post-disaster information. The tornado destroyed thousands of buildings, creating a diverse dataset that allowed the model to be trained and tested across various levels of structural damage. Detailed ground-level damage assessments provided a reliable benchmark to check how accurately the model could classify the severity of the damage.

“One of the most interesting findings was that, in addition to detecting damage with high accuracy, we could also estimate the tornado’s track,” Braik said. “By analyzing the damage data, we could reconstruct the tornado’s path, which closely matched the historical records, offering valuable information about the event itself.”

Future Directions

Researchers are working on using this model for other types of disasters, such as hurricanes and earthquakes, as long as satellites can detect damage patterns.

“The key to the model’s generalizability lies in training it to use past images from specific hazards, allowing it to learn the unique damage patterns associated with each event,” Braik said. “We have already tested the model on hurricane data, and the results have shown promising potential for adapting to other hazards.”

The research team believes their model could be critical in future disaster response, helping communities recover faster and more efficiently. The team wants to extend the model beyond damage assessment to include real-time updates on recovery progress and tracking recovery over time.

“This will allow for more dynamic and informed decision-making as communities rebuild,” he said. “We aim to create a reliable tool that enhances disaster management efficiency and supports quicker recovery efforts.”

The technology has the potential to transform how emergency officials, insurers and policymakers respond in the crucial hours and days after a storm by delivering near-instant assessments and recovery projections.

Funding for this research was provided by the National Science Foundation.

University of Oklahoma researcher Matthew Potthoff, Ph.D., is the lead author of the study, which provides valuable insight about the mechanism of action of the hormone, which is a target for a new class of highly anticipated drugs that are in Phase 3 clinical trials.

“Fatty liver disease, or MASLD (metabolic dysfunction-associated steatotic liver disease), is a buildup of fat in the liver. It can progress to MASH (metabolic dysfunction-associated steatohepatitis) during which fibrosis and, ultimately, cirrhosis can occur. MASLD is becoming a very big problem in the United States, affecting 40% of people worldwide, and there is currently only one treatment approved by the Food and Drug Administration to treat MASH. A new class of drugs, based on FGF21 signaling, is showing good therapeutic benefits in clinical trials, but until now, the mechanism for how they work has been unclear,” said Potthoff, a professor of biochemistry and physiology at the University of Oklahoma College of Medicine and deputy director of OU Health Harold Hamm Diabetes Center.

The study’s results demonstrated that FGF21 was effective at causing signaling in the model species that changed the liver’s metabolism. In doing so, the liver’s fat was lowered and the fibrosis was reversed. The hormone also sent a separate signal directly to the liver, specifically to lower cholesterol.

“It’s a feedback loop where the hormone sends a signal to the brain, and the brain changes nerve activity to the liver to protect it,” Potthoff said. “The majority of the effect comes from the signal to the brain as opposed to signaling the liver directly, but together, the two signals are powerful in their ability to regulate the different types of lipids in the liver.”

Similar to the family of weight loss drugs known as GLP-1s (glucagon-like peptide 1), which help regulate blood sugar levels and appetite, FGF21 acts on the brain to regulate metabolism. In addition, both are hormones produced from peripheral tissues — GLP-1 from the intestine and FGF21 from the liver — and both work by sending a signal to the brain.

“It is interesting that this metabolic hormone/drug works primarily by signaling to the brain instead of to the liver directly, in this case,” he said. “FGF21 is quite powerful because it not only led to a reduction of fat, but it also mediated the reversal of fibrosis, which is the pathological part of the disease, and it did so while the mice were still eating a diet that would cause the disease. Now, we not only understand how the hormone works, but it may guide us in creating even more targeted therapies in the future.”

But if the model mistakenly identifies reports with both conditions, the most likely diagnosis could be quite different: If a patient has tissue swelling and an enlarged heart, the condition is very likely to be cardiac related, but with no enlarged heart there could be several underlying causes.

In a new study, MIT researchers have found that vision-language models are extremely likely to make such a mistake in real-world situations because they don’t understand negation — words like “no” and “doesn’t” that specify what is false or absent.

“Those negation words can have a very significant impact, and if we are just using these models blindly, we may run into catastrophic consequences,” says Kumail Alhamoud, an MIT graduate student and lead author of this study.

The researchers tested the ability of vision-language models to identify negation in image captions. The models often performed as well as a random guess. Building on those findings, the team created a dataset of images with corresponding captions that include negation words describing missing objects.

They show that retraining a vision-language model with this dataset leads to performance improvements when a model is asked to retrieve images that do not contain certain objects. It also boosts accuracy on multiple choice question answering with negated captions.

But the researchers caution that more work is needed to address the root causes of this problem. They hope their research alerts potential users to a previously unnoticed shortcoming that could have serious implications in high-stakes settings where these models are currently being used, from determining which patients receive certain treatments to identifying product defects in manufacturing plants.

“This is a technical paper, but there are bigger issues to consider. If something as fundamental as negation is broken, we shouldn’t be using large vision/language models in many of the ways we are using them now — without intensive evaluation,” says senior author Marzyeh Ghassemi, an associate professor in the Department of Electrical Engineering and Computer Science (EECS) and a member of the Institute of Medical Engineering Sciences and the Laboratory for Information and Decision Systems.

Ghassemi and Alhamoud are joined on the paper by Shaden Alshammari, an MIT graduate student; Yonglong Tian of OpenAI; Guohao Li, a former postdoc at Oxford University; Philip H.S. Torr, a professor at Oxford; and Yoon Kim, an assistant professor of EECS and a member of the Computer Science and Artificial Intelligence Laboratory (CSAIL) at MIT. The research will be presented at Conference on Computer Vision and Pattern Recognition.

Neglecting negation

Vision-language models (VLM) are trained using huge collections of images and corresponding captions, which they learn to encode as sets of numbers, called vector representations. The models use these vectors to distinguish between different images.

A VLM utilizes two separate encoders, one for text and one for images, and the encoders learn to output similar vectors for an image and its corresponding text caption.

“The captions express what is in the images — they are a positive label. And that is actually the whole problem. No one looks at an image of a dog jumping over a fence and captions it by saying ‘a dog jumping over a fence, with no helicopters,'” Ghassemi says.

Because the image-caption datasets don’t contain examples of negation, VLMs never learn to identify it.

To dig deeper into this problem, the researchers designed two benchmark tasks that test the ability of VLMs to understand negation.

For the first, they used a large language model (LLM) to re-caption images in an existing dataset by asking the LLM to think about related objects not in an image and write them into the caption. Then they tested models by prompting them with negation words to retrieve images that contain certain objects, but not others.

For the second task, they designed multiple choice questions that ask a VLM to select the most appropriate caption from a list of closely related options. These captions differ only by adding a reference to an object that doesn’t appear in the image or negating an object that does appear in the image.

The models often failed at both tasks, with image retrieval performance dropping by nearly 25 percent with negated captions. When it came to answering multiple choice questions, the best models only achieved about 39 percent accuracy, with several models performing at or even below random chance.

One reason for this failure is a shortcut the researchers call affirmation bias — VLMs ignore negation words and focus on objects in the images instead.

“This does not just happen for words like ‘no’ and ‘not.’ Regardless of how you express negation or exclusion, the models will simply ignore it,” Alhamoud says.

This was consistent across every VLM they tested.

“A solvable problem”

Since VLMs aren’t typically trained on image captions with negation, the researchers developed datasets with negation words as a first step toward solving the problem.

Using a dataset with 10 million image-text caption pairs, they prompted an LLM to propose related captions that specify what is excluded from the images, yielding new captions with negation words.

They had to be especially careful that these synthetic captions still read naturally, or it could cause a VLM to fail in the real world when faced with more complex captions written by humans.

They found that finetuning VLMs with their dataset led to performance gains across the board. It improved models’ image retrieval abilities by about 10 percent, while also boosting performance in the multiple-choice question answering task by about 30 percent.

“But our solution is not perfect. We are just recaptioning datasets, a form of data augmentation. We haven’t even touched how these models work, but we hope this is a signal that this is a solvable problem and others can take our solution and improve it,” Alhamoud says.

At the same time, he hopes their work encourages more users to think about the problem they want to use a VLM to solve and design some examples to test it before deployment.

In the future, the researchers could expand upon this work by teaching VLMs to process text and images separately, which may improve their ability to understand negation. In addition, they could develop additional datasets that include image-caption pairs for specific applications, such as health care.