Application Of Multi Criteria Models

Multicriteria models are interesting to me because they intuitively feel as though they have more explanatory power than single critereon models. When models built on one or two variables make the caveat that, of course, they cannot explain all the variation in outcome because there are so many factors at play in the universe, multicriteria models appear to be a magical solution. They can account for lots of potential varibles - but just like their simpler single critereon cousins, multicriteria models are not in fact, omnipotent.

One of the most important questions to ask any model, but especially multicriteria models, is: How closely do the results of my model align with reality? We can throw variables at the computer all day long, but unless they can successfully model the subject at hand, even the most mathematically sophisticated model won’t be worth that much. But the eternal question remains: How do you test the accuracy of a novel metric? If you could already quantify reality, chances are, you probably wouldn’t be trying to model it (unless you are doing a replication study!!).

One approach to model validity is fieldwork; going out to see if the results of the model hold true in a sample section of the real world. This may be more practical in some applications than others. One step removed from this is using outcome data for validation, as Rufat et al. did with Hurricane Sandy and social vulnerability index (SoVI) scores in 2018. Another option, availed by Spielman et al. is to test internal consistency by replicating the model at different scales to see whether the trends hold true regardless of the size of the dataset. Certainly, these are just a few ways to test model validity in a specific context, but they demonstrate that not only is it possible, but it is essential to test the validity of multicriteria models. Just because more of the variables that may contribute to an outcome are included doesn’t mean that the model actually acounts for more variation.