Multidimensional Analysis of Deprivation and Fragility Patterns of Migrants in Lombardy, Using Partially Ordered Sets and Self-Organizing Maps

Abstract

In this paper, we present a multidimensional fuzzy analysis of the levels and the patterns of poverty and social fragility of migrants’ families, in the Italian region of Lombardy, in year 2014. Migrants’ poverty emerges as a complex trait, better described as a stratification of nuanced patterns than in black and white terms; Lombard migrants are in fact affected, to different extents, by “a diffused sharing of deprivation facets” and cannot be trivially split into deprived and non-deprived. The paper employs innovative data analysis tools from the Theory of Partially Ordered Sets; compared to mainstream monetary approaches, this leads to more realistic estimates of poverty diffusion and eliminates some well-known biases of standard evaluation procedures, providing strong support to the use of partial order concepts and tools in social evaluation studies.

Appendices

Appendix 1: Computing Evaluation Functions on Migrants’ Data

In this Appendix, we describe step-by-step the implementation of the posetic evaluation procedure on the migrants’ data and the computation of the evaluation functions which assign poverty and fragility scores to statistical units. Below, we focus on the main logic of the posetic procedure; technical and mathematical details can be found in Fattore (2016) and Fattore and Arcagni (2016).

1. Building the Poverty and Fragility posets To build the evaluation spaces of poverty and fragility, the first step is to cast the data into poset terms. A configuration $\varvec{p}=(p_1,p_2,p_3,p_4)$ of scores on the four poverty attributes considered in the paper is called poverty profile. The set $\varPi$ of $5\times 3\times 3\times 4=180$ poverty profiles generated by taking all the combinations of attributes’ scores comprises all of the possible deprivation configurations, given the input multi-indicator system. Two poverty profiles $\varvec{p}_a$ and $\varvec{p}_b$ can be ordered in terms of deprivation, only if either all the scores of $\varvec{p}_a$ are not higher than those of $\varvec{p}_b$ or viceversa; in this case, $\varvec{p}_a$ and $\varvec{p}_b$ are called comparable. On the contrary, if $\varvec{p}_a$ and $\varvec{p}_b$ comprise so-called conflicting scores (e.g. if $p_{a1}<p_{b1}$ and $p_{b2}<p_{a2}$), then they cannot be ordered and are called incomparable. The set $\varPi$ is thus turned into a partially ordered set $(\varPi ,\unlhd )$, the Poverty poset, where $\unlhd$ is a partial order relation defined by:

$$\begin{aligned} \varvec{p}_a\unlhd \varvec{p}_b \Leftrightarrow p_{ai}\le p_{bi}\quad \forall \, i=1,\ldots ,4. \end{aligned}$$

(2)

The poset $\varPi$ has a bottom, or minimum, profile (1111), representing the worst deprivation configuration, and a top, or maximum, profile (5334), which represents the best situation. The analogous construction on the fragility attributes defines the Fragility poset$\varPhi$, which comprises $6\times 2\times 2\times 2=48$ profiles and has (1111) and (6222) as bottom and top profiles, respectively. The Fragility poset is depicted in Fig. 6, as a Hasse diagram (the Hasse diagram of the bigger Poverty poset is displayed in "Appendix 2").

2. Poverty and fragility thresholds Posets $\varPi$ and $\varPhi$ represent the spaces of poverty and fragility (notice that both of the posets may comprise profiles not realized in the data). To turn them into evaluation devices, poverty and fragility thresholds must be set in them. Explicitly, we identify a subset $\pi _{\ell }$ of mutually incomparable profiles (a so-called antichain), representing achievement configurations “on the edge” of deprivation, so that any profiles in $\pi _{\ell }$ or below an element of $\pi _{\ell }$ (i.e. profiles in the so-called downset of $\pi _{\ell }$) are identified as deprived. Dually, we identify an antichain $\pi _u$ composed of profiles representing configurations “on the edge” of well-being, so that profiles in $\pi _{u}$ or above an element of $\pi _{u}$ (i.e. profiles in the so-called upset of $\pi _{u}$) are identified as completely non-deprived.⁷ Based on the lower and upper thresholds, $\pi _{\ell }$ and $\pi _{u}$, poverty profiles are then classified in three subsets: the subset of deprived profiles (on or below $\pi _{\ell }$), the subset of non-deprived profiles (on or above $\pi _u$) and the subset of ambiguously deprived profiles (configurations not belonging to the two other classes). The proper selection of the thresholds is clearly crucial in view of evaluation. In this study, as stated in previous paragraphs, the lower poverty threshold reduces to just one profile and has been set to $\pi _{\ell }=3221$ (i.e. Living arrangement = “Living at the workplace”and Economic aid = “Kindship’s aid”and Medical treatment renunciation = “Return to the country of origin for medical treatment”and “Income belonging to the first quartile”). The non-deprivation threshold is instead set as $\pi _u=4333$ (i.e Living arrangement = “Tenant alone or with family”and Economic aid = “No aid”and Medical treatment renunciation = “No renunciation”and “Income belonging to the third quartile”).

Lower and upper thresholds $\phi _{\ell }$ and $\phi _u$ are similarly set in the Fragility poset (see Fig. 6), namely as $\phi _{\ell }=(3122, 3221)$ (i.e. Working dynamics = “Run into instability or persistent precariousness” and Legal status = “Illegal”and Dependent family in country of origin = “No”and One-income family = “No”; or Working dynamics = “Run into instability or persistent precariousness” and Legal status = “Legal”and Dependent family in country of origin = “No”and One-income family = “Yes”) and to $\phi _{u}=3222$ (i.e. Working dynamics = “Run into instability or persistent precariousness” and Legal status = “Legal”and Dependent family in country of origin = “No”and One-income family = “No”). The lower fragility threshold comprises two profiles, since alternative score configurations may well represent fragility conditions. The upper threshold, instead, is composed of a single profile which is the minimum profile to be above both the elements of $\phi _{\ell }$. Table 7 reports the selected thresholds for both posets.

Fig. 6

Hasse diagram of the Fragility poset $\varPhi$. Two nodes $\varvec{p}_a$ and $\varvec{p}_b$ are connected by a downward path, if and only if $\varvec{p}_a\lhd \varvec{p}_b$; nodes not connected by a downward path are incomparable. In green, non-deprived profiles (the lowest green profile represents the upper threshold $\phi _u=3222$). In red, profiles representing certainly fragile conditions, the two “highest” red profiles constitute the lower threshold $\phi _{\ell }=(3122, 3221)$. (Color figure online)

Table 7 Lower and upper thresholds of the Poverty and Fragility posets

3. Performing evaluation Lower and upper thresholds inject into the Poverty and the Fragility posets a minimum amount of exogenous information on deprivation that triggers the evaluation procedure, leading to the computation of two evaluation functions, namely the identification function $idn(\cdot )$ and the (so-called, see Fattore 2016) relative severity function $svr(\cdot )$. The first assigns to each poverty or fragility profile $\varvec{p}$ an identification score, to be interpreted in a fuzzy spirit, i.e. as the membership degree of $\varvec{p}$ to poverty or to fragility; it assumes value 0 on profiles on or above the upper threshold, value 1 on profiles on or below the lower threshold and values in (0, 1), on all of the other profiles. When different from 0, the evaluation score can be complemented with a severity score, measuring the “depth” of deprivation or fragility. The identification score is computed based on a fundamental property of finite posets, i.e. that they are “equivalent” (see Fattore 2016 for a precise formulation) to the set of their linear extensions. These are linear orders obtained by resolving the incomparabilities of the input poset, in all possible ways; on each linear extension, profiles are either above or on/below the threshold and so can be classified as deprived (1) or non-deprived (0) in a binary way. Counting the fraction of linear extensions classifying a specified profile as deprived, one gets its deprivation score (see Fig. 7 for a simple example, taken from Fattore (2016)); in formulas, writing $idn_{\ell }(\varvec{p})$ for the identification score of profile $\varvec{p}$ in linear extension $\ell$, $idn(\varvec{p})$ for the final identification score of profile $\varvec{p}$ and $\varOmega$ for the set of linear extensions of the input poset (either the poverty or the fragility one), we have:

$$\begin{aligned} idn(\varvec{p})=\frac{|\{\ell \in \varOmega : idn_{\ell }(\varvec{p})=1\}|}{|\varOmega |}. \end{aligned}$$

(3)

On a linear extension $\ell$, it is also possible to measure the distance between a deprived profile $\varvec{p}$ (in $\ell$) and the non-deprived profile nearest to it (in $\ell$), as the number of edges between them, in the Hasse diagram of $\ell$. By averaging such distances over $\varOmega$, one gets the severity score of $\varvec{p}$, which is then normalized, dividing it by the highest severity score in the poset (which, in this case, is that of the bottom profile). Figure 8 depicts the identification and severity functions for both the Poverty and the Fragility profiles. As it can be seen, the identification functions resemble a logistic shape, although they get to assume both values 0 and 1. Similar comments can be given for the severity functions which are, however, more “linear” since, by construction, they get value 1 only on the bottom profile. It is important to realize that both evaluation functions are computed based on the thresholds and the structure of the partial order relation, with no aggregation of input ordinal scores; evaluation scores, in fact, are extracted directly from the network of comparabilities/incomparablities defining the posets (details can be found in Fattore 2016).

Fig. 7

Hasse diagrams of the product of linear orders $(1-2-3)$ and $(1-2)$ and of its five linear extensions. In gray, completely deprived profiles; bold circles represent threshold profiles. Profiles classified as completely deprived in the poset on the left are those which are classified as deprived in any linear extension on the right, namely profile 31, 21, 12, 11 (which thus have deprivation score equal to 1). Profiles 32, which is the top of the poset, is classified as non-deprived in any linear extension and thus its deprivation score is 0. Profile 22 is classified as deprived by two linear extensions out of five, so its deprivation score is 2 / 5

Fig. 8

Evaluation functions for the Poverty and the Fragility posets. Profiles are listed in increasing order of identification scores

The identification and severity scores of a profile are finally assigned to each statistical unit sharing that achievement configuration. Since the datasets present some missing values, some statistical units cannot be assigned to a specific poverty or fragility profile, but can be only assigned to a subset of the respective poset (Fattore 2016). As a result, these profiles get associated intervals of identification and severity scores, since their true scores are in between those of the bottom and the top of the subposet they are assigned to. Once each statistical unit has been assessed in terms of identification and severity, score distributions and overall indicators can be computed (possibly with intervals due to missing observations), in the spirit of the Head Count Ratio or Poverty Gap (Alkire and Foster 2011), and one can proceed to the socio-economic analysis of the results.

Appendix 2: Methodological Innovation

In this section, we describe how the posetic procedure formalized in Fattore (2016) has been modified, in order to allow for the lower and the upper poverty thresholds to be set independently of each other. The text below is more technical than the rest of the paper and is meant to complete it.

In the original procedure, setting the lower threshold implicitly sets the upper one to the antichain generating the intersection of the upsets of the lower threshold elements, in formulas:

$$\begin{aligned} \pi _{u}=\bigcap _{\varvec{p}\in \pi _{\ell }}\varvec{p}\!\uparrow \end{aligned}$$

(4)

(where $\varvec{p}\!\uparrow$ is the upset of element $\varvec{p}$); in fact, the identification function equals 0 on all and only the profiles which are “greater than” all of the profiles in $\pi _{\ell }$. A profile scored 0 by the identification function is certainly non-deprived, but in general this need not mean it represents a “full well-being” condition. Since in the present study we want the upper threshold to really identify well-being conditions, the set of profiles scored 0 by the identification function must be properly restricted. In practice, we want that the lower and the upper thresholds can be set independently of each other and, at the same time, we want the resulting identification function to be consistent with both of them. To this goal, one must change the structure of the input (from now on, called basic) partial order itself, in such a way that in the new poset the chosen upper threshold will be again the intersection of the elements of the lower threshold (this way, the identification function computed running the procedure described in "Appendix 1" will be necessarily consistent with the thresholds). To be practical, there are two main cases:

• 1. In the basic poset, $\pi _u$ comprises $\bigcap _{\varvec{p}\in \pi _{\ell }}\varvec{p}\!\uparrow$, i.e. some other profiles, in addition to those in $\cap _{\varvec{p}\in \pi _{\ell }}\varvec{p}\!\uparrow$ (in the original poset), are declared to be “non-deprived” by the analyst. In this case, the poset must be modified by inserting an edge between the corresponding nodes in the Hasse diagram, in such a way that each element of $\pi _u$ is made greater than all of the elements of $\pi _{\ell }$ (getting what is technically called an extension of the basic poset).

• 2. In the basic poset, $\pi _u$ is in the upset of $\bigcap _{\varvec{p}\in \pi _{\ell }}\varvec{p}\!\uparrow$, i.e. some of the profiles declared as “non-deprived” in the basic poset, are now considered as partly deprived. This is the case of the present paper and the solution is more subtle than in the previous case. In fact, here no simple changes to the partial order structure solves the problem. The point can be illustrated as follows. If a profile $\varvec{p}$, greater than each element of $\pi _{\ell }$ in the basic poset, is now to be considered as partly deprived, this means that some other “latent attribute”(i.e. some other implicit ordering criterion) is involved in the definition of the evaluation space. Informally, the profiles “lack” some attributes that would make incomparable some profiles that, in the basic poset, are actually comparable. Clearly, it is not possible to explicit such attributes, so the only way to modify the poset, based on the previous assumption, is to introduce exogenously an additional profile, to set it as a part of $\pi _{\ell }$ and to make it incomparable with the profiles which must be made partly deprived. This way, $\pi _u=\cap _{\varvec{p}\in \pi _{\ell }}\varvec{p}\!\uparrow$ comprises only the desired profiles and the resulting identification function is consistent with the thresholds. Notice that inserting the exogenous profile is the “minimum modification to the basic poset” achieving the desired result; form this point of view, no arbitrary modifications are made to the input poset, since the introduction of the new profile is the “essential” implication of the setting of $\pi _u$. This is an important point: in the posetic procedure, all of the exogenous information on evaluation criteria (e.g. the social value system assumed by the analyst or by the policy maker) must be turned into modifications of the input poset (addition of edges, insertion of profiles and setting of the thresholds). As a result, the poset representing the evaluation space of migrants’ deprivation, in the present paper, has the shape depicted in Fig. 9. This is the new input to the evaluation procedure described in "Appendix 1", producing the results discussed in the main text.

Fig. 9

Migrants’ deprivation poset, modified as explained in "Appendix 2". Notice that the added profile (LT) is part of the lower threshold and that only the minimum number (i.e. here 1) of edges between it and other profiles have been inserted, so as that $\pi _u=\cap _{\varvec{p}\in \pi _{\ell }}\varvec{p}\!\uparrow$ (in red, profiles on or below $\pi _{\ell }$; in green, profiles on or above $\pi _u$). (Color figure online)