A Holistic Model of Hatred: Development and Validation of the Abuelaish Hatred Scale (AHS) Volume 8 - Issue 2

Izzeldin Abuelaish¹*, Steven Stein²*, Jonathan Bridekirk^2,3, Sydney Mann² and Susan Yousufzai¹

• ¹Dalla Lana School of Public Health, University of Toronto, Toronto ON, Canada
• ²Multi-Health Systems Inc., Toronto ON, Canada
• ³Department of Psychology, York University, Toronto ON, Canada

Received: April 24, 2024;   Published: May 02, 2024

Corresponding author: Izzeldin Abuelaish, MD, MPH, Dalla Lana School of Public Health, University of Toronto,155 College Street, Toronto, ON M5T 3M7. Steven Stein, PhD, Multi-Health Systems Inc., 3770 Victoria Park Avenue, Toronto, ON M2H 3M6.

DOI: 10.32474/SJPBS.2024.08.000281

Abstract PDF

Abstract

The concept of hatred has been studied for centuries by philosophers and theologians, and only more recently in the social sciences. Although there is no consensus on a scientific or comprehensive theory, definition, or model of hatred, it is clear that hatred is a multidimensional construct expressed both internally and externally. In this paper, we describe the development and psychometric qualities of a holistic, public health model of hatred, called the Abuelaish Hatred Scale (AHS). Six constructs of hatred were developed: 1) Physical/Somatic Symptoms, (2) Mistrust, (3) Intolerance, (4) Ingroup Superiority, 5) Activism, (6) Retribution. The AHS was tested using factor analysis, confirmatory factor analysis, and convergent validity using two combined samples of participants (n=1,314). A 35-item scale was developed and confirmatory factor analysis demonstrated a 6-factor solution showing best fit to the data. Convergent validity was determined against conceptually relevant theoretical constructs from the Anger Disorder Scale (ADS), and Emotional Quotient Inventory 2.0 (EQ-i 2.0). Cliff’s delta demonstrated small to medium effect size differences between each sample for all 35-items. Based on the results, we believe the AHS could provide clinical utility and be used as part of a remediation program for people consumed with excessive hatred of others.

Keywords: Hate, Hatred, Assessment, Psychometrics, Scale Development

Public Significance Statement: Hatred can easily be justified as a subject within psychology. While there are no agreed upon definitions of hatred in the literature, we reviewed a number of studies that looked at conceptualizing and identifying features of hatred. By identifying components of hatred and measuring them reliably, this new scale enables more research in the area, as well as targeting remedial programs relevant to people with extreme levels of hatred.

Introduction

Hatred has been studied for centuries by philosophers and theologians, and only more recently by social psychologists, anthropologists, and evolutionary scientists. However, there is still no consensus on a scientific or comprehensive theory, definition, or model of hatred [1,2]. The prevalence of hatred is a pressing public safety and health issue that has serious effects on the public, health care community, governments, political leaders, and academic and non-academic institutions [3]. Although consensus may not be reached, definitions have been proposed in the literature. Hatred has been defined as a “negative emotion that motivates and may lead to negative behaviours with severe consequences” [4]. Hatred is part synonymous with extreme dislike, aversion, resentment, disgust, anger, and rage, and includes many other emotional traits and states of being [1]. Some definitions of hatred include an intense and chronic feeling, a judgment of its target as “bad, immoral, dangerous” [5], and a tendency, desire, or intention to be violent, often to the extreme of destroying its target. Most alarmingly, hatred is reported to involve the dehumanisation of the other [4,6], which serves as a gateway through which moral barriers can be removed and violence can be perpetrated.

Theoretical Models of Hatred

Theoretical models of hatred have been published in the literature, but few replicable models with actionable strategies that could be used to reduce or eliminate hatred have been provided [7-9]. One of the more widely cited theoretical models of hatred has been the Duplex Theory of Hate [10,11]. This theory has two aspects. The first is structural and involves the Negation of Intimacy (which includes repulsion and disgust), Passion in Hate (intense anger or fear), and Decision-Commitment (devaluation and diminution through contempt). The second aspect includes the narratives about the target group that people are taught (often as children) in order to justify the feelings of hatred towards that group. The Duplex Theory of Hate describes the basic cognitive processing that leads to experiencing hatred towards individuals and is described as the same process as experiencing hatred towards groups of people [12]. Looking at the three components of the structural aspect more closely, Negation of Intimacy refers to the seeking of distance from another person or group (the target) being dehumanized. Passion in Hate is expressed as intense anger or fear. Through passion, the individual becomes aroused toward fight or flight—either to strike at the hated target or to run away from it. It is not just how one perceives the target of hatred, but also what action is taken (fight or flight). This feature has also been described as the superimposing of the psychosomatic aspects of the flight or fight response [13]. The third component, Decision-Commitment, is characterized by “cognitions of devaluation and diminution through contempt for the targeted group” [14]. Those experiencing hatred feel contempt towards the target group, viewing them as barely human or subhuman. The use of propaganda, repeated stories, or “education,” enables this component to thrive. In order to measure the model and its components, Sternberg created the Triangular Hate Scale [11]. The THS includes 29 items rated on a 9-point Likert-type ordered categorical rating scale of agreement, ranging from 1 = ‘not at all’ to 9 = ‘extremely’. The factor structure has been explored using both exploratory and confirmatory factor analysis, and determined to have three components (intimacy, passion, and commitment; Sternberg & Sternberg [11]). The THS has demonstrated acceptable internal consistency across a variety of samples [15], with coefficient alpha for the subscales exceeding 80. [16].

Although the validity of the THS has been supported using a multitrait-multimethod approach, the specifics surrounding this approach lack external validity for the study of hatred [17]. Specifically, the THS was developed using undergraduate university students, where participants were asked to respond to the questionnaire after reading hypothetical scenarios designed to evoke hatred [11]. Researchers have questioned the use of these hypothetical scenarios as there is little evidence in their ability to manipulate and evoke hatred [18, 19]. In addition, research has shown undergraduate students commonly report they have never experienced hatred before, therefore serving as a flawed sample in which to validate this measure [1]. Therefore, whether the results from the THS can be generalized to people experiencing real hatred in their lives is unknown.

A Holistic Public Health Model of Hatred

In a comprehensive literature review by Fischer et al. [1], it was demonstrated that hatred has been conceptualized in a variety of ways beyond Sternberg’s Duplex Theory of Hate. Hatred has been conceptualized to be: (1) a syndrome that manifests itself within the hater [3]; (2) an emotional attitude and sentiment towards others [20]; (3) a normative judgment of others [19]; (4) a generalized evaluation of others based on one’s ingroup [21]; (5) a motive to devalue others politically [22]; and (6) a form of generalized anger directed towards the hated group [23]. These six constructs, which are described in Figure 1, capture a comprehensive understanding of hatred as a contagious disease, involving cognitive schemas, mindsets, and emotions. It has been theorized that hatred exhibits as physiological/somatic symptoms with negative health consequences for the hater [1, 24]. Hatred has a negative physical and emotional impact that disturbs the homeostasis and emotional equilibrium of an individual, leading to physical, psychological, social, and behavioral changes [3]. According to this view, excessive hatred can cause psychosomatic symptoms through its stress-like qualities. Hatred has been directly associated with several health problems; for example, hateful speech has been demonstrated to increase clinical anxiety levels and has been observed to be correlated with disrupting immune systems and enhancing growth of cancer and chronic inflammatory disease [25-28]. Because of the physical toll hatred can have on a person, any assessment of hatred should include a component measuring physical symptoms, such as feeling dizzy, muscle tension, headaches, and a racing heart [29].

Another component that has been associated with hatred is the emotional attitude and sentiment of mistrust for the “other” group [1]. Research has shown partisan groups overestimated their perceived disagreement and dissimilarity, further exacerbating their hatred for one another [30]. In two studies by Chambers and Melnyk [20]. it was demonstrated the more partisan groups (e.g., prochoice vs. pro-life, or Democrat vs. Republican) believed that the other group would disagree or oppose their core values and beliefs, the less that group could be trusted. As a result, participants attributed more negative traits to members of other group and more positive traits to members of their own group. According to Chambers and Melnyk [20], people do not merely hate because of differences in opinion, but because they mistrust people who oppose the things they believe in most. This motivational narrative leads to mistrust and, as a result, falsely believing they have negative views on other, unrelated important, as well as trivial, issues, contributing to one’s hatred. Another contributing factor of hatred is the normative judgment leading to intolerance for other groups. Research has shown individuals will demonstrate intolerance for an outgroup when they believe that outgroup represents a set of stable characteristics that are anger-evoking and will never change [4]. A series of studies have looked at the relationship between intolerance and group-based hatred among Israelis during various times of relative peace and conflict [19, 31]. In a set of studies by Halperin et al. [19], it was demonstrated that hatred was stronger than other established predictors of intolerance, such as democratic norms, psychological authoritarianism, and threat perceptions. In addition, it was also demonstrated that participants with higher levels of hatred toward the outgroup were more likely to see the other group as incapable of changing, and therefore, were less willing to change their own views on that group [31]. According to Halperin et al. [31], for those experiencing long-term hatred, intolerance is an important dimension and common expression of hatred. Feelings and beliefs of ingroup superiority can be an important facet of hatred, and perhaps stem from intolerant attitudes toward the outgroup. Those experiencing feelings of hatred do not believe in the possibility of improving group relations [21]. The very notion of White supremacist groups such as the KKK and Nazi political parties supports this. In a review of the literature on white supremacy, it was found that “white supremacists need the constant reassurance that they are better than anyone else” [21].

As stated by Deaux and colleagues, “the groups to which we belong are an integral part of our self-concept. Thus, discrimination between groups results from individual group members’ motivations to enhance or maintain self-esteem” [32]. Intergroup superiority requires a clear distinction between one’s ingroup versus the outgroup, along with a perception that the outgroup represents a homogeneous entity [1]. It is expected, then, that people who harbour hatred believe the ingroup they identify with are high in beliefs and feelings of superiority. One of the characteristics of hatred that has been discussed in the literature is the motive to devalue others through political activism [1]. Research has shown political activism to be one of the most common actions taken in groupbased hatred [4]. According to Wright and colleagues [22], political activism can occur as either normative or nonnormative action. Normative action is defined as a group act that is intended to affect public policy and conforms to the norms of the dominant social system, such as laws and regulations. This type of action includes peaceful protesting and organizing demonstrations. Nonnormative acts violate these norms and rules, extending beyond what is regarded as acceptable and legal [22]. This type of action can include looting, causing physical harm to others, or destruction of property.

In a study by Shuman and colleagues [33], normative and nonnormative actions were observed among Palestinian protesters. When measuring a range of emotions, they found that anger predicted normative behaviors and hatred predicted nonnormative actions. For many experiencing group-based hatred for a long period of time, violent alternatives may seem reasonable and a successful way of protest, and perhaps a manifestation of a group’s hatred. Finally, hatred can also be examined as a generalized form of anger where individuals seek revenge and retribution towards those they hate [34]. Studies have pointed out the desire of individuals to punish, or even eliminate, the target of hatred [23]. In a study by Pearson et al. [35], White participants read about a provoked or unprovoked violent assault perpetrated by a Black assailant on a White victim. Findings demonstrated that, regardless of whether the attack was provoked or unprovoked, White participants that expressed higher levels of hatred towards the Black assailant were more likely to recommend more punitive punishments, such as longer prison sentences, or even the death penalty. As theorized by Fischer et al. [1], the motivational goal of hatred is not to just hurt the target, but to eliminate the target, either mentally or physically. In many cases, the act of seeking revenge or retribution is a matter of restoring equity in suffering between the individual and the target of hatred [24]. While the Duplex theory outlines some important aspects of hatred, additional factors have been identified through other hate-related studies identified above. Sternberg [36] has even expanded his conceptualization of hatred to having a “disease- like” quality that is contagious and transmissible to others. Individuals or groups can spread and inspire hatred in various ways as hatred itself can take on various forms [36]. The more forms of hatred that are expressed, the greater the hatred can grow. Recent evidence has emerged to support the idea that hatred can manifest in various forms both at the interpersonal and intergroup level [2], although there is no multidimensional measure of hatred that captures these various forms. Based on the six constructs of hatred identified above, the goal of this study was to first 1) develop a comprehensive holistic model and scale to measure hatred and to evaluate hatred as a contagious disease and public health issue [3,36], and second 2) to conduct a robust validation of the measure against conceptually relevant external criterion measures, which are theoretically related constructs. The tool developed should apply in areas of the world that are at high risk for hate-related violence and associated health issues.

Materials and Methods Part 1

Scale Development

Item Pool Development: An initial pool of 62 items were created for the Abuelaish Hatred Scale (AHS). Items were generated by subject-matter experts and test-development experts to capture the six constructs of hatred outlined above. A thorough examination of existing hatred literature and self-report measures [11] was conducted to inform the selection and creation of possible items. In addition, all items were designed to focus on group-level hatred as opposed to individual-level hatred. This narrowed scope simplified the measure conceptually and allowed it to be tailored to address hatred against specific groups. The items were written to include both positively and negatively worded items to avoid positive and negative response style biases. Eleven items were written as reverse- scored items (e.g., “It’s important to hear the views of groups other than my own”). Five judges with an expertise in the hatred literature were asked to indicate which, if any, of the six elements of hatred were measured for each of the 62 items listed. Those that were classified in the same category by 3 out of the 5 judges were considered to be adequate to demonstrate reasonable consensual validation for the content validity of that item as representing the category. A total of 45 out of the 62 items were judged to have “consensus.” The remaining 17 items were rewritten or edited as per judges’ recommendations. These items did not undergo further review by judges but were included to determine if they could empirically load on specific factors.

Participants: Two samples of participants were collected between April and December 2018 for this study. All participants were recruited and compensated online through Cloud Research’s Prime Panels.

Normative Sample: This sample of 1,015 individuals aged 18 years or older from the U.S. were collected to reflect the proportions of the 2010 U.S. Census [37] with respect to gender, ethnicity, educational level, and geographical region. In addition, information was also collected on participants’ political affiliation, employment status, religion, and income range. See Table 1 for a breakdown of the Normative sample characteristics.

Hatred Sample: This sample of 299 individuals aged 18 years or older were collected to represent the target population for the development of the AHS [38]. The target population for this tool was people who openly express feelings of hatred or dislike towards another group of people. To identify this group, we chose to use a short measure of national identity and attitudes towards multiculturalism [39]. This scale was chosen, not because it necessarily equates with hatred, but because it demonstrates the participant’s willingness to endorse items that demonstrate extreme negative feelings towards a group of other individuals. A participant was included in this group if they responded to the Collective Threat questions in a manner suggestive of harboring negative attitudes towards immigration. These participants were selected to complete the AHS because they indicated that they were less tolerant towards immigrants and multiculturalism and were comfortable endorsing ideas that are not necessarily socially desirable. There were 2,974 potential participants directed to the AHS, and 10.6% of these participants passed this selection criteria; however, 16 participants were removed from the sample for failing attention checks, excessive speed, and/or patterned responding. See Table 1 for a breakdown of the Hatred sample characteristics.

Both the Normative sample and Hatred sample were combined and randomly split into two equal samples, with proportions of 77.3% Normative and 22.7% Hatred in each sample. Sample A of 657 participants was used for scale development, and Sample B of 657 participants was used for cross-validation. The two samples did not differ in gender, χ2(1) = 0.34, p > .05, or age, t(1312) = 0.32, p > .05.

Procedure

Participants for both the Normative sample and Hatred sample were recruited and compensated through an online survey panel vendor, Cloud Research. Participants were brought to the assessment link through the vendor and were presented with an informed consent form to read. If they did not agree to participate, they were directed out of the study and were not compensated. Participants who consented were sent to the demographic section of the survey. Only participants who reported their age as 18 years or older were eligible to participate. The protocol of this study was approved by the University of Toronto’s Research Ethics Board (protocol ID: 37953). For the Normative sample, participants completed the demographic questions and then responded to the 62 items on the AHS. For the Hatred sample, participants completed the demographic questions and then responded to the Collective Threat items, in order to screen participants for the Hatred sample, and finally, they responded to the 62 items on the AHS. No demographic information was used to disqualify participants.

Measures

Demographic Questions

A series of demographic questions were created to help determine any group differences and describe the samples. These questions included: gender, age, ethnicity, religion, political affiliation (range from liberal to conservative), area of residence, employment status, highest level of education achieved, employment status, and income range.

Collective Threat

A short measure of national identity and attitudes towards multiculturalism [39] was used as a screening tool to quickly assess if a participant would express strong dislike for a group of people. This scale consisted of 4 items: the first question had two response options, whereas the other three were rated on a 5-point Likert-type scale (1 = ‘strongly disagree’ to 5 = ‘strongly agree’). This scale includes items such as “Immigrants increase crime rates” and “Immigrants improve our society by bringing in new ideas and cultures” (reverse coded). These items were only used to qualify respondents to the Hatred sample who would be open to expressing a dislike for a group of people.

Abuelaish Hatred Scale (AHS)

The AHS version used in this sample was a 62-item self-report scale measuring six constructs of hatred: Physical/Somatic Symptoms, Mistrust, Intolerance, Ingroup Superiority, Activism, and Retribution. All items were rated on a 5-point Likert-type scale from (1 = ‘strongly disagree’ to 5 = ‘strongly agree’). Examples of the test items include “My skin itches when I think of certain groups of people” (Physical/Somatic Symptoms), “I feel that there are certain groups of people I cannot trust at all” (Mistrust), “The more I learn about different groups of people the more I understand them” (Intolerance), “Belonging to the right group affects your afterlife” (Ingroup Superiority), “I would join a movement that is aimed at fighting certain groups of people” (Activism), and “Sacrificing myself to hurt others can bring me rewards in life and afterlife” (Retribution) 1.

Statistical Analysis Plan

Exploring the Factor Structure

The first half of the combined subsample (Sample A; n = 657) was used for Exploratory Factor Analysis (EFA) using R Studio [40] based on polychoric correlations. Parallel analysis [41] and Velicer’s Minimum Average Partial analysis [42] were run to determine the number of factors to retain. EFA with Principal Axis Factoring (PAF) using direct oblimin rotation was used to test different factor solutions. Model fit indices were explored using the Tucker-Lewis incremental fit index (TLI; > .90 acceptable fit; [43]), the root means square error of approximation (RMSEA; < .08, acceptable fit; e.g., [44]), and the percentage of variance explained (>50% acceptable; e.g., [45]).

Foot Note

¹Data and material used in this study will not be made available.

In addition to the EFA, effect sizes were also examined for all 62 items using Cliff’s delta (Cliff’s d; [46]) using R Studio (rcompanion package; Mangiafico, [47]). Since the Hatred sample was identified as openly expressing feelings of hatred or dislike towards another group of people, it was important the effect size differences between the Normative sample and Hatred sample were not negligible. Therefore, in order to maximize factor structure parsimony, items were eliminated in the following situations: (1) their factor loading on a specific factor was < .40; (2) or they cross-loaded on another factor with a difference > .20 [48]; (3) they demonstrated low inter-item correlations (r < .30); or (4) Cliff’s d demonstrated negligible effect sizes between the Normative sample and Hatred sample. Confirming the Factor Structure. The second half of the combined subsample (Sample B; n = 657) was used for Confirmatory Factor Analysis (CFA) using Weighted Least Square Mean and Variance Adjusted (WLSMV) estimators due to ordinal data with multivariate non-normality [49] in R studio (lavaan package; [50]). Robust model fit indices for the CFA included: robust chi square/ degree of freedom (ratio ≤ 3 indicates good fit; Cole, [51]); TLI [43] and confirmatory fit index (CFI; Bentler, [52]), with values approaching 1 implying a good model fit (relying on criteria of > .95 for good fit); and RMSEA (Steiger et al., [44]) and standardized root mean square residual (SRMR), with values approaching 0 indicating a good model fit (relying on criteria of < .08 for acceptable fit; e.g., Hooper et al., [53]). Hierarchical bifactor models were also run to establish if the AHS was better explained by a general factor (representing a broad construct of hatred) with specific subfactors (representing narrower sub constructs of hatred). Bifactor models assume the general and specific subfactors are orthogonal, and therefore, the model covariances between the general and specific factors were constrained to be orthogonal [54]. Internal consistency. Internal consistency reliability for items in the full AHS scale and subscales was conducted using coefficients alpha [55] and omega, based on polychoric correlations in R studio (psych package; [40]).

Results Part 1

Exploring the Factor Structure with Sample A

The factor structure of the original 62 item questionnaire was explored using data from Sample A of 657 participants. Participants used all response options for every item (i.e., non-zero selections of all scale values ranging from strongly disagree to strongly agree). Furthermore, all items had a standard deviation less than 0.50, and only 4 items demonstrated significant skewness or kurtosis (skewness > 2, or kurotsis > 7; e.g., [56]). The Kaiser-Meyer-Olkin (KMO) measure verified the sampling adequacy for the analysis (.96) and Bartlett’s Test of Sphericity was significant (p < .001), indicating the appropriateness of using this combined subsample for factor analysis [48].

Initial Exploration of the Factor Structure and Item Reduction

To determine the number of factors to retain, Horn’s [41] parallel analysis and Velicer’s Minimum Average Partial (MAP; Velicer et al., [42]) analysis were run, both suggesting an 8-factor solution. The scree plot of actual and simulated eigenvalues displayed 6 factors with eigenvalues > 1, but 2 factors with eigenvalues < 1 and sitting on the line for simulated data. Principal Axis Factoring with oblimin rotation was run with 8- and 6-factor solutions explored. The 8-factor solution, in line with results from the parallel analysis and MAP analysis, resulted in multiple items with low loadings or cross-loadings on multiple factors. In addition, 2 of the 8 factors had fewer than 4 items met criteria for factor loadings for the hypothesized factor. Next, in line with the scree plot of actual and estimated eigenvalues, and our theoretical underpinnings outline above, a 6-factor solution was explored. Almost all items loaded > .40 on to a factor with fewer items cross-loading. In addition, the 6-factor solution produced a theoretically salient solution with factors representing identified in the introduction: 1) Physical/Somatic Symptoms, (2) Mistrust, (3) Intolerance, (4) Ingroup Superiority, 5) Activism, 6) Retribution. Therefore, the 6-factor solution was selected to facilitate item removal within dimensional categories.

Upon further inspection of the 6-factor solution, items were reduced from 62 to 35 based on extreme skewness/kurtosis, low communalities, poor factor loadings, and negligible Cliff’s d effect size differences between the Normative sample and Hatred sample. In addition, items were also removed if they did not represent the dimensional categories or did not represent important aspect of the conceptual content. After item removal, 35 items were retained for further exploration of the six hypothesized scales: (1) 6 Physical/ Somatic Symptoms items, (2) 8 Mistrust items, (3) 5 Intolerance items, (4) 5 Ingroup Superiority items, (5) 6 Activism items, and (6) 5 Retribution items².

Foot Notes

²Table 2 presents item stems for all 35 items but not the full item description.

Further Evaluation of the Factor Structure

Horn’s [41] parallel analysis and Velicer and colleagues’s [42] MAP analysis were re-run, for the reduced 35-items testing 6-, 5-, 4-, 3-, 2-, 1-factor solutions, respectively. All six potential factor solutions were re-examined using PAF with the 6-factor solution resulting in the most theoretically meaningful constructs of hatred. For the 6-factor solution all items loaded > .40 onto a principal factor and did not cross-load onto a subsequent factor (no difference > .20; see Table 2). For solutions with 1 through 5 factors, not all items met criteria (e.g., item loadings fell below 0.40 and cross-loadings were common). In addition, these factor solutions provided little interpretative meaning in terms of theoretical salience. Fit indices for the EFAs of the 1- through 6-factor solutions were compared (see Table 3). Overall, the solutions with 1 through 5 factors demonstrated poor fit in terms of both the RMSEA and TLI fit indices. Although the fit indices gradually improved from the 1-factor solution up to the 6-factor solution, the 6-factor solution is just below our established limits of acceptable model fit. Regardless, the 6-factor solution provided the best interpretation and theoretical salience, explaining 68% of the variance of hatred: (1) Physical/Somatic Symptoms (13%), (2) Mistrust (14%), (3) Intolerance (10%), (4) Ingroup Superiority (9%), (5) Activism (12%), (6) Retribution (9%), with small to moderate correlations between factors (r = -.17 to .37). Based on the EFA results, the next step was to replicate the 6-factor structure with the second half of our total sample using CFA.

Confirming the Factor Structure with Sample B

The factor structure identified in Sample A was confirmed using data from 657 participants in Sample B using CFA (WLSMV) with robust model fit indices reported. The KMO (.96) and Bartlett’s Test of Sphericity (p < .001) indicated that the sample was adequate for factor analysis.

Fit Indices for 1-6 Factor Solutions

The six factor models identified in the EFA were confirmed, with the 6-factor solution providing a very good fit to the data, RM_{SEA robust} = .06 [90% CI (.06, .06); p < .001], CFI_robust = .96, TLI_robust = .95, and SRMR = .07 (see Table 3 for other model fit indices). The remaining models tested produced poor to acceptable fit to the data, indicating that a more parsimonious solution was not an equal or superior fit to the data, and these alternative models were therefore rejected.

Higher-Order CFA

Next, we tested a bifactor CFA model with the 6-factor solution, in which items load onto both a general factor, and specific sub-factors. The bifactor CFA model was not identified and produced poor fit to the data. Lastly, a second-order CFA was run on the 6-factor solution in which items were indicators of hatred sub-factors, and these sub-factors were indicators of an overall factor of hatred. The second-order model produced a good fit to the data similar to the original 6-factor solution (see Table 3). Both analyses indicate that when using the 6-factor solution, the AHS can be interpreted as measure of one underlying construct, as well as a multidimensional measure.

Internal Consistency

The total 35-item scale and its individual factors were internally consistent based on ordinal alpha (α ranged from .79 – .94) and omega (ω ranged from .80 – .94) reliability statistics (see Table 2).

Normative Scores and Group Differences

Using the final 35-items from the AHS, the Normative sample from both Sample A and Sample B were combined (N = 1,015) and used to create T-scores for the total score and all six subscales. In order to create the T-scores, raw scores were first calculated for the total score and each subscale and using the frequency distributions of the raw scores for each, were then converted to empirical percentiles, and then into T-scores. The purpose of creating T-scores for the total score and subscales was to make the AHS more user-friendly for researchers and professionals by providing standardized means and standard deviations (M = 50, SD = 10). All group differences reported below are based on T-scores. Using the Normative sample, gender and age differences were assessed to determine normative groups for standardization. Results demonstrated that women scored significantly lower on the total score and all 6 subscales compared to men, with effect sizes ranging from small (d = -0.14) to small-medium (d = -0.48; see Table 4 for a breakdown for both groups). When examining age as a continuous variable as it relates to raw scale scores, results demonstrated the total score and most subscales, except for Mistrust, were negatively correlated with age. The following subscales were significantly correlated with age: Physical/Somatic Symptoms, r(1013) = -0.08, p = .014, Mistrust, r(1013) = 0.12. p < .001, Ingroup Superiority, r(1013) = -0.09, p = .003, Retribution, r(1013) = -0.19. p < .001, and Activism subscales, r(1013) = -0.06, p = .049. On the other hand, when examining age as a categorical variable, Tukey’s HSD post-hoc analyses demonstrated that most significant differences being observed were taking place between the youngest and oldest participants within the Normative sample (Partial η2 = .04; see Table 5 for a breakdown across age categories). Although some differences were observed, the differences were small and may reflect important trends established in the literature, so a single normative group was established, rather than subdividing by age or gender.

Finally, when comparing the Normative Sample to the Hatred Sample, results demonstrated the Hatred sample scored significantly higher on the total score and all six subscales, with effect sizes ranging from small (d = 0.3) to medium (d = 0.71; see Table 6 for a breakdown for both samples).

Materials and Methods Part 2

Validation of the AHS Measures Against Conceptually Relevant External Criterion Measures

Objective 2 examined the convergent validity of the 6 constructs of hatred that make up the AHS, consisting of (1) Physical/Somatic Symptoms, (2) Mistrust, (3) Intolerance (4) Ingroup Superiority (5) Activism, and (6) Retribution, with the subscales that make up the Anger Disorder Scale (ADS), as well as the Emotional Quotient Inventory 2.0 (EQ-i 2.0).

Participants

The original study was created with 1,015 people from the general population and 299 people who were more extreme in their negative attitudes toward others (e.g., viewed as threats). For those 1,015 people, after they completed the AHS, they were randomly sent to one of the follow-up studies to complete one additional test. The other tests included the ADS, and the EQ-i 2.0. A series of demographic questions noted above were also used to characterize any group differences and describe the samples that completed the ADS and EQ-i 2.0 (see Table 7).

Measures

The ADS version used for this study was a 74-item self-rated inventory that helps identify clinically dysfunctional anger in individuals ages 18 and older [57]. Specifically, this scale measures various constructs of anger, including the Reactivity/Expression higher order factor, which is a composite of the following subscales: Scope of Anger Provocations, Physiological Arousal, Duration of Anger Problems, Rumination, Impulsivity, Coercion, and Verbal Expression. Other subscales of the ADS include Suspiciousness, Resentment, Tension Reduction, Brooding, Revenge, Physical Aggression<, Relational Aggression, Passive Aggression, and Indirect Aggression. Examples of the test items include, “in general, I get angry about”, rated on a 5-point Likert-type scale (1= ‘almost nothing’ to 5 = almost everything in my life’). Other items were also measured on a 5-point Likert-type scale based on duration of angry, such as “when I get angry it usually lasts” (1 = ‘only a few minutes’ to 5 = ‘a month or more’), or “when I get angry, I avoid talking to people about the problem” (1 = ‘never’ to 5 = ‘always’). The EQ-i 2.0 asks about the frequency with which people performed behaviors related to emotional intelligence in the past and uses people’s reports of their past behaviors to identify areas of development and predict future behavior [58, 59]. Specifically, the EQ-i 2.0 used in this study was a 132-item self-report scale measuring six constructs of emotional intelligence: (1) Self Perception, (2) Self Expression, (3) Interpersonal, (4) Decision Making, (5) Stress Management, and (6) Happiness. Subscale composite scores were created as follows: Self-Perception included, self regard, self-actualization, and emotional self-awareness. Self-Expression included emotional expression, assertiveness, and independence. Interpersonal, included questions that made up the subscales for interpersonal relationships, empathy, and social responsibility. Decision Making, included questions regarding problem solving, reality testing, and impulse control. Stress management included flexibility, stress tolerance, and optimism. The subscale for Happiness consisted of 8 questions. All items were rated on a 5-point Likert-type scale (1 = ‘Never/Rarely’ to 5 = ‘Always/ Almost Always’). Examples of each of the subscale items include: “looking at both my good and bad points, I feel good about myself” (Self Perception); “I pay attention to how I’m feeling” (Emotional/ Self Expression); “it’s easy for me to make friends” (Interpersonal); “when I’m really upset, I can’t decide what to do” (Decision Making); “I keep calm in difficult situations” (Stress Management); and “I am satisfied with my life” (Happiness).]

Statistical Analysis Plan

T-scores were created for the total score of each scale, and all subscales of the AHS, ADS and EQ-i 2.0. In order to create the T-scores, raw scores were first calculated for the total score and each subscale. Using the frequency distributions of the raw scores for each, distributions were then converted to empirical percentiles, and then into T-scores. SPSS version 28 was used to conduct Pearson correlations between the T-scores for each of the 6-subscales of the AHS and the composite T-scores that make up the subscales of the EQ-i 2.0 and ADS. We hypothesized small negative correlations between the subscales of the AHS and the EQ-i 2.0 (Hypothesis 2a). On the other hand, we hypothesized to expect moderate, positive correlations between the subscales of the AHS and ADS (Hypothesis 2b).

Results Part 2

The EQ-i 2.0 scale was completed by 258 participants and consisted of more male (51%) than female participants. The age of the participants ranged from 18-54, with most participants in the 50- 54 age range (41%), and the least in the 18-24 age range (7%). The study sample that completed the ADS, included 246 participants, who mostly identified as females (56%). The age of the participants ranged from 18-60+, with most participants in the 60+ age range (30%), and the least in the 18-24 age range (7%). Overall, the two independent samples resemble each other in relation to demographic characteristics, although participants from the ADS sample were mostly females (see Table 7).

Convergent Validity Correlations

EQ-i 2.0 Sample

As hypothesized, the subscales of the AHS and EQ-i 2.0 were negatively correlated, with small correlations (see Table 8). A significant negative correlation was found for each of the AHS subscales and the total EQ-i 2.0 score. Specifically, the Physical/Somatic Symptoms subscale was negatively correlated with each subscale of the EQ-i 2.0, as follows: self-perception r (256) = -.28, p < .01, emotional self-awareness, r (256) = -.19, p < .01, interpersonal r (256) = -.22, p <.01, decision making r (256) = -.41, p < .01, stress management r (256) = -.29, p < .01, and happiness r (256) = -.24, p <.01. Correlations between Mistrust were significant for all dimensions of the EQ-i 2.0, except for emotional self-awareness. For the correlations, r’s varied between -.14 and -.25 (p’s <.05 and .01, respectively). The results show that Intolerance was significantly correlated with each subscale of the EQ-i 2.0. All correlations were negative and varied between -.26 and -.46 (p < .01). Ingroup T-scores of the AHS was negatively correlated with decision making r (256) = -.23, p < .01, and stress management r (256) = -.14, p < .05. The correlations varied between -.15 and -.36 for Retribution and the 6 subscales of the EQ-i 2.0 (p’s <.05 and .01, respectively). In relation to the Activism subscale of the AHS, all dimensions of the EQ-i 2.0 was significantly correlated, except for Emotional Self Awareness. Correlations were negative and varied between r = -.19 and -.32 (p < .01).

ADS Sample

Participants that did not answer all questions of the ADS or AHS were excluded from the analysis (n=5). Regarding hypothesis 2b, the results of the correlations between the AHS and ADS subscales, show moderate, positive correlations for most subscales, thereby supporting the hypothesis (see Table 9). Specifically, the Physical/ Somatic Symptoms subscale of the AHS was significantly correlated with all subscales of the ADS, except for Duration of Anger, with r’s varying between .16 and .27 (p <.05 and p <.01, respectively). Mistrust correlated significantly positive with suspiciousness, brooding, the various aggression subscales including, physical, passive, and indirect aggression, as well as scope of provocation, physiological arousal, duration of anger, rumination, impulsivity, coercion, and hurt social rejection (r’s varied between .13 and .35, p < .05 and p < .01, respectively). Intolerance correlated significantly positive with, suspiciousness, resentment, revenge, physical and relational aggression, scope of provocation, physiological arousal, rumination, impulsivity, coercon, verbal expression, hurt social rejection and episode length (r’s varied between .13 and .35, p < .05 and p < .01, respectively). The Ingroup subscale showed significant positive correlations with revenge, physical and relational aggression, scope of provocation, coercion, verbal expression, and hurt social rejection (r’s varied between .13 and .19, p < .05 and p < .01, respectively). Retribution was significantly corelated with suspiciousness, resentment, tension reduction, brooding revenge, physical, relational, and indirect aggression, scope of provocation, ruminaton, impulsivity, coercion, verbal expression, hurt social rejection, and episode length (r’s varied between .13 and .31, p < .05 and p < .01, respectively). Results in relation to the Activism subscale of the AHS, showed significantly positive correlations with all subscales of the ADS, with correlations varying between .21 and .41 (p < .01). However, we found that the Duration of Anger subscale of the ADS, negatively correlated with, the following AHS constructs: Intolerance, Ingroup, and Retribution.

Table 9: Pearson Correlations between the T- scores of the Anger Disorder Scale constructs and Abuelaish Hatred Scale (AHS), constructs (n=241).

Discussion

Go to

• Abstract
• Introduction
• Theoretical Models of Hatred
• A Holistic Public Health Model of Hatred
• Materials and Methods Part 1
• Procedure
• Measures
• Statistical Analysis Plan
• Results Part 1
• Materials and Methods Part 2
• Results Part 2
• Convergent Validity Correlations
• Discussion
• Limitations & Future Directions
• Acknowledgement
• Discussion
• References

This study was designed to develop a holistic measure of hatred towards others. Based on our literature review, an initial pool of 62 items was developed to capture these six constructs of hatred: (1) Physical/Somatic Symptoms, (2) Mistrust, (3) Intolerance, (4) Ingroup Superiority, (5) Activism, (6) Retribution. Using five expert judges, 45 out of the 62 items reached consensus and the remaining 17 items were rewritten according to judges’ recommendations. All items were designed to focus on group-level hatred as opposed to individual-level hatred. The AHS was explored using factory analysis and confirmatory factory analysis with two unique samples of participants: A Normative sample representing recent American population demographics [37], and a Hatred sample comprising individuals who exhibited hatred towards immigrants and scored high on collective threat. Exploratory factor analysis resulted in a 35-item scale. CFA demonstrated a 6-factor solution had the best fit to the data, although a higher-order model also had acceptable model fit. Other psychometric properties of the AHS were acceptable, with high internal consistency, and Cliff’s d demonstrating hypothesized small to medium effect size differences between the Normative sample and Hatred sample for all 35-items. In addition, we examined the correlations between the AHS, the ADS and EQ-i 2.0 subscales, with two subsamples of participants representing the general population, who were randomly selected to complete one test. In general, the AHS subscales correlate negatively with the EQ-i 2.0 and show a positive correlation with the ADS. However, not in line with hypothesis 2b, we found that the duration of anger subscale of the ADS showed negative correlations with 3 constructs of the AHS, including Intolerance, Ingroup and Retribution (which was expected to be positive), although these were not significant. Overall, the validity of the AHS is further supported by the results showing the expected relationships with validated scales, namely the ADS and EQ-i 2.0. In contrast to previous hatred scales, the AHS provides a more holistic and health-related measure of hatred based on current findings.

Aside from the Duplex Theory of Hate [10,11], there is a lack of empirically derived models of hatred in the psychological literature. For example, the Sternberg Triangular Hate Scale (THS) is one of the most widely used measures of hatred. On the other hand, recent literature and findings point to hatred as more multidimensional than what Sternberg initially proposed [1]. Although the THS has demonstrated some conceptual and empirical strength, there have been concerns about this instrument with respect to both its conceptual basis and the methodology used in prior studies. Conceptually, it does not contain all of the elements of hatred that have been summarized in the recent literature [14]. Methodologically, although its validity has been demonstrated using a multitrait-multimethod approach [16], using undergraduate university student participants in order to measure hatred has been called into question by other researchers [4,18]. Since undergraduate students do not represent the external validity needed for the study of hatred, it was important for the development of the AHS, that a collection of American participants that exhibited hatred were used. By having a Normative sample and Hatred sample to compare, we are better able at providing more external validity for the six constructs within the AHS. As demonstrated by the results, across all six subscales and the total score, the Hatred sample scored higher than the Normative sample, lending validity evidence through meaningful differences being groups, as hypothesized, when using this instrument.

Although it is important to demonstrate that the AHS can measure someone experiencing hatred versus someone who does not, there are some gender and age differences. Across the measure, men scored higher than women on the total AHS and all 6 subscales. Research has demonstrated that men are more outwardly expressive of their hatred and more likely to support direct action with hated members of the other group [60]. This finding would be most apparent in the AHS Activism and Retribution subscales. On the other hand, there is not enough empirical work to support why these differences might exist for each of the 6 subscales, particularly Physical/Somatic Symptoms. Although some research has theorized women might be more perceptive of the expression of hatred compared to men, research on this matter remains inconclusive [61]. In addition, a lot of empirical studies on hatred do not report gender differences as part of their overall results or theorize why such differences might exist [35]. In addition, it was demonstrated across the AHS subscales that hatred seemed to decrease with age except for Mistrust. Post-hoc analyses demonstrated that most of the age differences were observed at the extreme ends of our data, between the youngest (18 - 24yrs) and oldest (60+yrs) participants. But, based on post-hoc analysis, only the Mistrust and Retribution subscales continued to show apparent age differences. For the Mistrust scale, younger participants scored higher in trustful of others than the older participants. These finding parallels other research supporting increased trust of others by early adults (and younger) compared with older age groups [62]. We start off life trusting others, but as we age, we learn to be more cautious and less trusting of others, in particular strangers or those who may appear “different” from us.

Figure 1: Holistic public health model of hatred.

The Retribution scale, on the other hand, while also significantly differentiated by age, found that younger people were higher in retribution than older adults. It may be that the younger group, while being more trusting of others, was more likely to want to retaliate, perhaps when that trust was broken. In general, younger adults are more likely to retaliate against others they have conflict with or are likely to hate. According to reported police data, the majority of hate crimes are carried out by younger people. In the United States, it is estimated that nearly 50% of all hate crimes are committed by individuals younger than 20 years of age, compared to 25% for nonhate crime offences. One of the advantages of developing a model of hatred is the identification of specific correlates of hatred (see Figure 1). By looking at the specific pattern of scales in individuals high in hatred towards others we can develop targeted interventions potentially reducing some of the violence attributed to hatred. This supports Sternberg’s [10] premise on dealing with hate:

I believe that the best ways to combat hate are to understand it, to recognize it in oneself and to reject it. Moreover, I believe that wisdom ultimately may be the best cure. Wise people do not hate because they understand things from other people’s points of view, including those of people with whom they may have strong disagreements. Teaching people to think wisely, therefore, may be the best way to teach them to reject hate. (p. 5)

In order to teach people “wise thinking”, it is important to be more specific about where their thinking may be “unwise.” For example, if their hatred is based on mistrust and fear of others, then targeted interventions could include increasing exposure to other groups, learning more about other groups, and better understanding the challenges and history of different groups. If hatred is based on Intolerance, then a different set of interventions could be applied. These can include learning about the contributions of other groups to society and role playing where they play a member of the other group. If the target is afflicted with Ingroup Superiority, the intervention could involve having the target first share the perceived strengths of their own group, then explore any in-group challenges. They can then explore the strengths and challenges of other groups as comparisons. If they score high in Retribution various aspects of assertiveness, negotiation skills, and other psychological skills could be taught. People high in Activism can be taught the differences between constructive and destructive protest. In other words, a series of interventions could be targeted based on the hatred profile of the individual.

Limitations & Future Directions

Go to

• Abstract
• Introduction
• Theoretical Models of Hatred
• A Holistic Public Health Model of Hatred
• Materials and Methods Part 1
• Procedure
• Measures
• Statistical Analysis Plan
• Results Part 1
• Materials and Methods Part 2
• Results Part 2
• Convergent Validity Correlations
• Discussion
• Limitations & Future Directions
• Acknowledgement
• Discussion
• References

One of the common criticisms of measuring traits that are perceived as negative (or positive) is that they may be underreported by participants and therefore easily manipulated [63]. Measuring hatred directly through self-report can be challenging in terms of peoples’ potential to present an overly positive impression. Also, due to the level of introspection required, participants may lack self-awareness into the various form of hatred being described in the AHS. While the present study did not correlate the AHS scores with scores from measures of social desirability, this investigation is a likely next step to consider. In addition, since the AHS measures hatred more holistically, it may be important to assess the “impairment of the normal state” among participants, which was not considered in this study. For example, if hatred is to be viewed as a disease that manifests itself in certain signs and symptoms, then a distinct clinical picture may be able to be obtained [3]. By measuring other psychological, social, physical and spiritual aspects of participants in relationship to their hatred scores, results may provide more clarity into the health consequences of hatred and how different types of hatred are formed. Some assessments worth considering include well-being, anxiety, depression, stress, PTSD, self-esteem, and other physiological measures. Although this study draws comparisons to the Sternberg Triangular Hate Scale (THS), it is unclear what the convergent validity might be between the AHS and the THS. Recently Sternberg [14] has expanded his theory of hatred to include more constructs similar to some subscales measured in the AHS. Yet without any hatred measure developed to capture Sternberg’s FLOTSAM theory [14], it might be worth examining the extent to which the AHS’s subscales correlated with the subscales from the THS. Finally, future research will have to explore the theoretical underpinnings of our holistic public health model of hatred in order to determine whether different exposures lead to different types of hatred (see Figure 1).

Acknowledgement

Go to

• Abstract
• Introduction
• Theoretical Models of Hatred
• A Holistic Public Health Model of Hatred
• Materials and Methods Part 1
• Procedure
• Measures
• Statistical Analysis Plan
• Results Part 1
• Materials and Methods Part 2
• Results Part 2
• Convergent Validity Correlations
• Discussion
• Limitations & Future Directions
• Acknowledgement
• Discussion
• References

We would like to offer our special thanks to Joanna Solomon for her valuable and constructive feedback.

Declaration of Interest

Go to

• Abstract
• Introduction
• Theoretical Models of Hatred
• A Holistic Public Health Model of Hatred
• Materials and Methods Part 1
• Procedure
• Measures
• Statistical Analysis Plan
• Results Part 1
• Materials and Methods Part 2
• Results Part 2
• Convergent Validity Correlations
• Discussion
• Limitations & Future Directions
• Acknowledgement
• Discussion
• References

The authors declare that they have no conflict of interest to report.

References

Go to

• Abstract
• Introduction
• Theoretical Models of Hatred
• A Holistic Public Health Model of Hatred
• Materials and Methods Part 1
• Procedure
• Measures
• Statistical Analysis Plan
• Results Part 1
• Materials and Methods Part 2
• Results Part 2
• Convergent Validity Correlations
• Discussion
• Limitations & Future Directions
• Acknowledgement
• Discussion
• References

1. Fischer A, Halperin E, Canetti D, Jasini A (2018) Why we hate. Emotion Review 10(4): 309-320.
2. Martínez CA, van Prooijen JW, Van Lange PAM (2022) Hate: Toward understanding its distinctive features across interpersonal and intergroup targets. Emotion 22(1): 46-63.
3. Abuelaish I, Arya N (2017) Hatred-a public health issue. Medicine Conflict and Survival 33(2): 125-130.
4. Halpern E (2008) Group-based hatred in intractable conflict in Israel. Journal of Conflict Resolution 52(5): 713-736.
5. Navarro JI, Marchena E, Menacho I (2013) The psychology of hatred. The Open Criminology Journal 6: 10-17.
6. Steinberg RJ (2005) Understanding and combating hate. In R. Sternberg (Ed.) The psychology of hate (pp:37-50) Washington DC: APA.
7. Klein M, Riviere J (1937) Love hate and reparation. Hogarth.
8. Papiasvili ED (2019a) Hate (its vicissitudes and its relations) revisited: Part I-individual. International Forum of Psychoanalysis 28(2): 71-82.
9. Papiasvili E (2019b) Hate (its vicissitudes and its relations) revisited: Part II-groups and culture. International Forum of Psychoanalysis 28(2): 83-94.
10. Sternberg RJ (2003) A Duplex Theory of Hate: Development and application to terrorism, massacres, and genocide. Review of General Psychology 7(3): 299-328.
11. Sternberg R, Sternberg K (2008) The nature of hate. Cambridge University Press.
12. Hamilton DL, Sherman SJ (1996) Perceiving persons and groups. Psychological Review 103: 336-355.
13. Galdston R (1987) The longest pleasure: A psychoanalytic study of hatred. International Journal of Psychoanalysis 68: 371-378.
14. Sternberg R (2005) The psychology of hate. APA. Sternberg RJ (2020) FLOTSAM: A theory of the development and transmission of hate In R.
15. Shabahang R, Rezaei S, Madison TP, Aruguete MS, Bagheri Sheykhangafshe F et al., (2020) Celebrity hate: Credibility and belief in a just world in prediction of celebrity hate. Psychology of Popular Media 10(4): 500-504.
16. Weis K (2006) Explorations of the duplex theory of hate. Berlin: Logos Verlag.
17. Toussaint LL, BarryM, Enomoto M, Anians W, Rodamaker K et al., (2020) Hateful Emotional Responses Scale (HatERS): Development and initial evaluation. Journal of Hate Studies 16(1): 48-54.
18. Aumer K, Bahn ACK, Harris S (2015) Through the looking glass, darkly: Perceptions of hate in interpersonal relationships. Journal of Relationships Research 6(e3): 1-7.
19. Halperin E, Canetti-Nisim D, Hirsch-Hoefler S (2009) The central role of group‐based hatred as an emotional antecedent of political intolerance: Evidence from Israel. Political Psychology 30(1): 93-123.
20. Chambers J, Melnyk D (2006) Why do I hate thee? Conflict misperceptions and intergroup mistrust. Personality and Social Psychology Bulletin 32(10):1295-1311.
21. Arena MP, Arrigo BA (2000) White supremacist behavior: toward an integrated social psychological model. Deviant Behavior 21(3): 213-244.
22. Wright SC, Taylor DM, Moghaddam FM (1990) Responding to membership in a disadvantaged group: From acceptance to collective protest. Journal of Personality and Social Psychology 58(6): 994-1003.
23. Seip EC, Rotteveel M, van Dillen LF, van Dijk WW (2014) Schadenfreude and the desire for vengeance. In WW van Dijk, JW Ouwerkerk (Eds.) Schadenfreude: Understanding pleasure at the misfortune of others (pp. 227-241). Cambridge University Press.
24. Solomon RC (1977) The rationality of the emotions. The Southwestern Journal of Philosophy 8(2): 105-114.
25. DeRijk R, Michelson D, Karp B, Petrides J, Galliven E et al., (1997) Exercise and circadian rhythm-induced variations in plasma cortisol differentially regulate interleukin-1 beta (IL-1 beta), IL-6, and tumor necrosis factor-alpha (TNF alpha) pro-duction in humans: High sensitivity of TNF alpha and resistance of IL-6. Journal of Clinical Endocrinology and Metabolism 82(7): 2182-2191.
26. De Rijk R, Sternberg EM (1997) Corticosteroid resistance and disease. Annals of Medicine 29(1): 79-82.
27. Elenkov IJ, Chrousos GP (1999a) Stress, cytokine patterns and susceptibility to disease. Best Practice & Research Clinical Endocrinology & Metabolism, 13(4): 583-595.
28. Elenkov IJ, Chrousos GP (1999b) Stress hormones, Th1/Th2 patterns, pro/anti-inflammatory cytokines and susceptibility to disease. Trends in Endocrinology and Metabolism 10(9): 359-368.
29. Painuly N, Sharan P, Mattoo SK (2005) Relationship of anger and anger attacks with depression. European archives of psychiatry and clinical neuroscience 255(4): 215-222.
30. Chambers JR, Baron RS, Inman ML (2006) Misperceptions in intergroup conflict: Disagreeing about what we disagree about. Psychological Science 17(1): 38-45.
31. Halperin E, Russell AG, Dweck CS, Gross JJ (2011) Anger, hatred, and the quest for peace: Anger can be constructive in the absence of hatred. Journal of Conflict Resolution 55(2): 274-291.
32. Deaux K, Dane FC, Wrightsman LS (1993) Social Psychology in the ’90s (6th ed.) Pacific Grove California: California: Brooks & Cole Publishing.
33. Shuman E, Cohen-Chen S, Hirsch-Hoefler S, Halperin E (2016) Explaining normative versus nonnormative action: The role of implicit theories. Political Psychology 37(6): 835-852.
34. Jasini A, Fischer AH (2018) Characteristics and social determinants of intergroup hate [Unpublished manuscript].
35. Pearson AR, Dovidio JF, Pratto F (2007) Racial prejudice, intergroup hate, and blatant and subtle bias of whites toward blacks in legal decision making in the United States. International Journal of Psychology & Psychological Therapy 7(2): 145-158.
36. J Sternberg (2020) Perspectives on hate: How it originates, develops, manifests, and spreads (pp: 3-24) American Psychological Association.
37. (2011) United States Census. Bureau2010 Census.
38. Clark LA, Watson D (2019) Constructing validity: New developments in creating objective measuring instruments. Psychological Assessment 31(12): 1412-1427.
39. Sumino T (2017) National identity and public attitudes toward multiculturalism in Canada: Testing the indirect effect via perceived collective threat. Canadian Journal of Behavioural Science / Revue canadienne des sciences du comportement 49(3): 183-194.
40. Revelle W (2021) psych: Procedures for Personality and Psychological Research. Northwestern University Evanston Illinois USA.
41. Horn JL (1965) A rationale and test for the number of factors in factor analysis. Psychometrika 30(2): 179-185.
42. Velicer WF, Eaton CA, Fava JL (2000) Construct explication through factor or component analysis: A review and evaluation of alternative procedures for determining the number of factors or components. In RD Goffin & E. Helmes (Eds.) Problems and solutions in human assessment (pp. 41-71) Kluwer Academic/Plenum Publishers.
43. Tucker LR, Lewis C (1973) A reliability coefficient for maximum likelihood factor analysis. Psychometrika 38: 1-10.
44. Steiger JH, Shapiro A, Browne MW (1985) On the multivariate asymptotic distribution of sequential chi-square statistics. Psychometrika 50(3): 253-263.
45. Mokkink LB, de Vet HCW, Prinsen CAC, Patrick DL, Alonso J et al., (2018) COSMIN Risk of bias checklist for systematic reviews of patient-reported outcome measures. Quality of Life Research 27(5): 1171-1179.
46. Macbeth G, Razumiejczyk E, Ledesma RD (2011) Cliff's Delta calculator: A non-parametric effect size program for two groups of observations. Universitas Psychologica 10(2): 545-555.
47. Mangiafico S (2022) rcompanion: Functions to support extension education program evaluation. R package version 2(4): 13.
48. Tabachnick BG, Fidell LS (2014) Using multivariate statistics (7th ed.) Pearson Education.
49. Flora DB, Curran PJ (2004) An empirical evaluation of alternative methods of estimation for Confirmatory Factor Analysis with ordinal data. Psychological Methods 9(4): 466-491.
50. Rosseel Y (2012) lavaan: An R package for Structural Equation Modeling. Journal of Statistical Software 48(2): 1-36.
51. Cole DA (1987) Utility of confirmatory factor analysis in test validation research. Journal of Consulting and Clinical Psychology 55(4): 584-594.
52. Bentler PM (1990) Comparative fit indexes in structural models. Psychological Bulletin 107(2): 238-246.
53. Hooper D, Coughlan J, Mullen M (2008) Evaluating model fit: a synthesis of the structural equation modelling literature. In 7th European Conference on research methodology for business and management studies.
54. Reise SP (2012) The rediscovery of bifactor measurement models. Multivariate Behavioral Research 47(5): 667-696.
55. Cronbach LJ (1951) Coefficient alpha and the internal structure of tests. Psychometrika 16(3): 297-334.
56. Kim HY (2013) Statistical notes for clinical researchers: assessing normal distribution (2) using skewness and kurtosis. Restorative Dentistry & Endodontics 38(1): 52-54.
57. DiGiuseppe R, Tafrate RC (2004) Anger Disorders Scale (ADS). Toronto: Multi-Health Systems.
58. Multi-Health Systems (2011) EQ-i 2.0® Emotional Quotient-Inventory 2.0® Multi-Health Systems. Retrieved November 1” 2022.
59. Van Zyl CJ (2014) The psychometric properties of the emotional quotient inventory 2.0 in South Africa. SA Journal of Industrial Psychology 40(1): 1-8.
60. Janhevich DE (2001) Hate crime in Canada: An overview of issues and data sources. Statistics Canada.
61. Wojatzki M, Horsmann T, Gold D, Zesch T (2018) Do women perceive hate differently: Examining the relationship between hate speech, gender, and agreement judgments. 14th Conference on Natural Language Processing Vienna Austria.
62. Sutter M, Kocher MG (2007) Trust and trustworthiness across different age groups. Games and Economic Behavior 59(2): 364-382.
63. Smith DB, Robie C (2004) In personality and organizations. Psychology Press.
64. Beck A (1972) Depression: Causes and Treatment. Philadelphia: University of Pennsylvania Press.