Development and Psychometric Evaluation of an Object Relations Measure for the Rotter Incomplete Sentences Blank

John T. Rucker III & Radhika Krishnamurthy

        Research in psychoanalytic personality assessment has involved the development and validation of affect-laden representations of self and others, i.e., object relations, which are substrates of interpersonal variables assessed by self-report and performance-based personality measures. With both self-report and performance-based elements, sentence completion measures such as the Rotter Incomplete Sentences Blank (RISB) offer an alternative method of assessing personality (mal) adjustment by eliciting and scoring completions of a respondent’s feelings to short sentence stems (Rotter & Rafferty, 1950; Rotter et al., 1992). The current study sought to develop and evaluate an object relations (OR) measure for the RISB. RISB sentence stems related to self and others were selected to develop the OR scale (15 stems) and self- and other-representation subscales (9 stems and 6 stems, respectively). Several test score reliability and validity analyses were undertaken following contemporary psychometric guidelines. Inter-rater reliability coefficients for outpatient mental health clients (N = 50) were r = .80 (Self subscale), r = .88 (Other subscale), and r = .88 (Total scale). Internal consistency reliabilities for outpatients (N = 123) demonstrated medium-to-large alpha (α = .77; α = .54; α = .77) and omega (ω = .79; ω = .55; ω = .78) coefficients. Test-retest reliabilities (r = .71; r = .65; r = .64) using a college student sample (N = 20) demonstrated moderate temporal stability of the scales. External validity evaluations examined correlations between RISB OR scale scores and relevant Rorschach (N = 84) and MMPI-2 (N = 111) scale scores of outpatient clients. Convergent validity correlations were in the small-to-large effect size range with several MMPI-2 interpersonal scales but were minimal in relation to Rorschach variables. Discriminant validity was most clearly shown with Rorschach variables but not MMPI-2 scales. In addition, results showed significant differences in mean RISB OR scale scores of outpatients and college students (N = 50 each), but not between personality disordered and non-personality disordered outpatients (N = 45 each). The findings of this study suggest adequate psychometric standing of the RISB OR scales and subscales as measures of object relations. Implications and future directions are discussed.

Introduction:

        The assessment of relatedness in the psychoanalytic tradition involves appraisal of a person’s internal object relations. These intrapsychic relational structures shape the quality and tenor of interpersonal functioning. Maladaptive object relations are found among persons with dependent personalities (Bornstein, 2003), eating disorders (Heesacker et al., 1990), schizophrenia (Spear & Sugarman, 1984), borderline personality disorder (Yeomans et al., 2015), and narcissistic personality disorder (Diamond et al., 2022).

Broad-band self-report and performance-based clinical personality tests typically contain variables assessing facets of interpersonal functioning, albeit not necessarily assessing object relations directly. Sentence Completion tests (SCTs) call for completion of short sentence stems and contain both performance-based and self-report elements. SCTs represent an alternative technique for assessing personality dynamics, including representations of self and others that can be considered to constitute object relations.

        One well-known SCT is the Rotter Incomplete Sentences Blank (RISB; Rotter & Rafferty, 1950; Rotter et al., 1992). The RISB has been the most widely used SCT (Haak, 2003; Hojnoski et al., 2006; Holaday et al., 2000), and has standardized norms and scoring procedures. It has been applied to the assessment of psychosocial functioning (Weis et al., 2008) including the detection of dependency patterns (Dunlap, 1951, as cited in Rotter et al., 1992; Fitzgerald, 1958, as cited in Rotter et al., 1992), the ability to predict social behavior (Chance, 1958), social conduct based on trust and popularity (Jessor et al., 1963), anxiety, inhibited interpersonal styles, and hostility (Geller, 1966, as cited in Rotter et al., 1992; Geller, 1968, as cited in Rotter et al., 1992; Renner et al., 1962; Torstrick et al., 2015), social introversion (Vernallis et al., 1970), (un)conscious relational needs in borderline personality pathology (Schiltz & Schiltz, 2016), and perceptions of self and others (Janda, 1998). Some researchers have devised special scoring systemsfor the RISB including measures assessing attitudes toward the law and the criminal justice system (Hulin & Maher, 1959, as cited in Rotter et al., 1992), self-acceptance, self-description, and acceptance of others (Shepard, 1979, as cited in Rotter et al., 1992), perceptual accuracy of healthcare staff among inpatients (Watts, 1973, as cited in Rotter et al., 1992), and a cognition rating form (Lehnert et al., 1996).

        Social cognitive elements revealed by the RISB render this hybrid method of direct and indirect responding useful in assessing interpersonal functioning. However, unlike commonly used performance-based methods such as the Rorschach and Thematic Apperception Test, the RISB does not contain a specific index related to interpersonal functioning. The current study focused on the development and initial psychometric evaluation of an object relations measure for the RISB. In this endeavor, we drew from a contemporary understanding of object relations as “units composed of a representation of the self and a representation of another, linked by an affect related to or representing a drive” (Yeomans et al., 2015, pp. 3-4). Test score reliability evaluation included inter-rater, internal consistency, and test-retest reliability analyses. Construct validity evaluation consisted of convergent and discriminant validity with relevant scales and indices from the Rorschach (Exner, 2001; Meyer et al., 2011) and Minnesota Multiphasic Personality Inventory-2nd edition (MMPI-2; Butcher et al., 1989).

Methods:

Participants:

        Samples of outpatient clients and college students were used in this study. Participants in the outpatient sample were N = 123 adult clients (n = 70 women, 56.9%, n = 53 men, 43.1%) who had previously received psychological services from a university-affiliated community mental health clinic in Central Florida. Inclusion was based on the following criteria: (a) age 18 years or older; (b) completion of a RISB; (c) provision of a mental health diagnosis; and (d) having received psychotherapy services. The sample was predominantly Caucasian (85.5%), followed by Hispanic (8%), Asian American (4%), and African American (2.4%) participants. The mean age was 37.8 (SD = 14.36; range = 18-69). Relationship status was as follows: single = 78 (63.4%), married = 15 (12.2%), divorced and/or separated = 21 (17%), in a relationship = 7 (5.7%) or widowed = 2 (1.6%). Primary diagnoses were mood disorders (48%), substance use disorders (13.8%), bipolar disorders (8.0%), schizophrenic disorders (2.4%) and personality disorders (36.6%), with some co-occurring diagnoses.

Participants in the college sample consisted of N = 50 students (n = 26 women, 52%; n = 24 men, 48%) who were: (a) 18 years of age or older; and (b) consented to complete the RISB. The sample was predominantly Caucasian (60.0%), with smaller frequencies of African American (18.0%), Asian American (16.0%), and Hispanic (6.0%) students. The mean age of the college participants was 22.8 (SD = 7.4; range = 18-54). Relational status consisted of single = 35 (70%), married = 3 (6%), divorced = 2 (4%) and in a relationship = 10 (20%). Eight (16%) participants had prior mental health diagnoses and eleven (22%) were receiving psychotherapy at the time of this study. Additional data for the college sample included number of close friends (M = 6.3; SD = 4.8), subjective appraisal on level of adjustment to college on a scale of one to five (M = 4.0; SD

= 0.8) and level of satisfaction with their relationships on a scale of one to five (M = 3.9; SD = 0.9).

Instruments:

        RISB. The primary instrument of interest in this study was the RISB, adult form. Following the RISB manual’s scoring instructions, each item was scored on a 7-point scale with higher scores representing psychological maladjustment (6 = severe conflict) and lower scores representing psychological adjustment (0 = strong positive). Prior studies involving the entire RISB protocol have found strong inter-rater reliabilities with correlation coefficients ranging from .72 to .99 (Rotter et al., 1992). With reference to internal consistency reliability, Rotter et al. (1992) reported that RISB items “are not designed to be equivalent” and as such, “assessing the internal consistency is not strictly applicable” (p. 41). Nevertheless, studies assessing the internal consistency of the entire RISB protocol showed coefficients ranging from .69 to .86 (Catanzaro 1988, as cited in Rotter et al., 1992; Rotter et al., 1949; Rotter et al., 1954; Torstrick et al., 2015). The RISB manual reports temporal stability for this instrument is “sensitive to personality adjustment,” and “falls somewhere in the middle of the state-trait continuum – reflecting change but also maintaining moderate stability over time” (Rotter et al., 1992, p. 43). Construct validity evaluations with the RISB demonstrated significant correlations with severe anxiety, chronic psychosis, personality disorders, and severe substance use (Rotter et al., 1992). In addition, Lah (1989) found significant correlations between maladjustment, as measured by the Overall Adjustment Score (OAS), and sociometric indices of unhappiness, non-self-acceptance, nervousness, and humorlessness.

        Rorschach. Rorschach variables selected for convergent validity analyses of the RISB included the Coping Deficit Index (CDI), Good Human /Poor Human Representational responses (GHR/PHR), Oral Dependent Language (ODL), and Mutuality of Autonomy/Mutuality of Pathology (MAH/MAP). The information processing efficiency index (Zd) and the illogical/incoherent thinking index (WSum6) were used to evaluate discriminant validity. The CDI measures “social ineptness as well as an unrewarding or chaotic interpersonal history,” and can indicate vulnerabilities in coping with social stressors (Krishnamurthy et al., 2011, p. 304), executive function errors among adult inpatients (r = .30; Ilonen et al., 2016), and an inability to manage one’s emotions (r = -.49; Vishwakarma et al., 2016). The GHR/PHR variables are measures of interpersonal involvement that correlate with histories of healthy or conflictual interactions with others. More frequent PHR responses are indicative of a proneness to respond to others in ill-advised or undesirable ways (Exner, 2000, pp. 322-326, as cited in Weiner, 2003). Soenning et al. (2010) found higher frequencies of PHR among rejected, aggressive children, whereas “popular” children had higher frequencies of GHR. The ODL index has been found to correlate with help seeking behaviors (r = .85), dependency (r = .58), and insecure attachment (r =

.19) (Bornstein, 1996). Lastly, the MAH/MAP indices have gained support as measures of object relational interplay (Bombel et al., 2009; Graceffo et al., 2014; Monroe et al., 2013).

MMPI-2. MMPI-2 scales related to self/interpersonal functioning included Low Self-Esteem (LSE), Social Discomfort (SOD), Self-Alienation (Pd5), and Social Alienation (Pd4) for convergent validity analyses. Schizophrenia (Sc) and Type A (TPA) scales were used for discriminant validity analyses. The LSE scale has correlated inversely with measures of global self-worth (r = -.54) and sociability (r = -.49) among undergraduate students (Brems & Lloyd, 1995). SOD has correlated inversely with measures of sociability (r =

-.25; Hathaway & McKinley, 1989) and paranoia/mistrust when interacting with others (r = .35; Sellbom et al., 2005). Self-Alienation (Pd5) has been found to have an inverse correlation with a measure of social potency (r

= -.38; Lilienfeld, 1999) and positive correlation with a lack of self-integration (r = .29; Wrobel, 1991), and reflects feeling misunderstood, guilty, unhappy, and self-reproaching (Nichols, 2011, as cited in Friedman et al., 2015). The Social Alienation (Pd4) subscale of the MMPI-2 has positive correlations with measures of feeling isolated or ungratified in social relations (r = .18; Wrobel, 1991), a lack of belongingness (r = .28; Wrobel, 1991), and externalization of blame (r = .60; Lilienfeld, 1999).

Procedures:

        Archival data from the records of outpatient clinic clients evaluated between 2013 and 2020 were extracted to yield a N = 123 outpatient sample. In addition, with provision of informed consent, a sample of N = 50 college students enrolled at two private universities in the southern United States was collected, who were tested with the RISB between November 2020 and September 2021.

        External validity analyses involved indices and scales from N = 84 available Rorschach protocols and N = 111 MMPI-2 profiles from the outpatient sample. To assess if the RISB OR scales could differentiate between diagnostic categories, the outpatient sample was divided into n = 45 personality disordered versus n = 45 randomly selected non-personality disordered participants. Similarly, the RISB OR scales’ ability to differentiate scores of outpatients and controls was assessed using n = 50 randomly selected outpatients and N = 50 college students. Lastly, two supplemental analyses compared randomly selected subsamples of n = 42 outpatients with n = 42 college students who had never received a mental health diagnosis, as well as n =39 outpatients with n =39 college students who were not receiving psychotherapy at the time of RISB testing.

Scale Development:

        RISB OR scale items were chosen based on their social cognitive stems that elicit a representation of self and others. Fifteen stems were selected that elicited self- and other representations, thus constituting the total OR scale (see Table 1). Nine of the fifteen stems were allocated to the Self-representation subscale due to their elicitation of associations related to images of the self (e.g., “My mind,” “I am very,” “I need,” “I wish”); six were allocated to the Other-representation subscale due to their elicitation of generalizations and/or personalized images of others (e.g., “Men,” “Other people,” “My father,” “A mother”). The RISB’s emphasis on expressing “your real feelings” steeps sentence completions in affect, thus prompting the RISB OR measure to correspond with the contemporary definition of object relations.

Table 1. RISB Object Relations Subscale(s)

Self-representation subscale (Item #) Other-representation subscale (Item #) I feel (12) Men (7)

I can’t (15) People (10)

I suffer (20) A mother (11)

I failed (21) Other people (19)

My mind (23) My father (35)

I need (25) Most women (40)

I am very (32) I wish (34)

I (37)

Results:

Preliminary analyses yielded the following means and standard deviations for the RISB OR scales among N = 50 college student participants: Total scale M = 46.6 (SD = 9.7), Self-subscale M = 29.2 (SD = 6.8), and Other-subscale M = 17.4 (SD = 5.1). Likewise, the means and standard deviations for N = 123 outpatient participants were: Total scale M = 60.1 (SD = 11.2), Self-subscale M = 37.5 (SD = 7.8), and Other- subscale M = 22.6 (SD = 5.4). The OR scales were significantly intercorrelated and achieved medium-to-large intercorrelations in both samples, as shown in Table 2.

Table 2. Pearson Correlations between RISB OR Scales

Inter-rater reliability. Inter-rater reliability analyses of the RISB OR scales were derived from ratings by the first author and a trained clinical psychology graduate student following the test’s manual’s item-scoring guidelines, using a n = 50 randomly selected outpatient subsample. Results showed the following ICCs for the OR scales: Total Scale r = .88, Self-subscale r = .80, and Other -subscale r = .88, demonstrating strong inter-rater reliability and aligning with prior ICCs observed for the entire RISB protocol (Rotter et al., 1992).

Internal consistency reliability. Internal consistency reliability using N = 123 outpatients were measured using Cronbach’s alpha (α) as well as MacDonald’s Omega (ω) as the subscales measure different facets of the OR construct: self- and other representations. The internal consistency reliabilities of the OR Total scale (α =.77; ω

= .78) and the Self subscale (α = .77; ω = .79) were strong, representing large effect sizes, and was at an acceptable, medium-effect size level for the other subscale (α = .54; ω = .55). Additional post-hoc analyses were conducted to determine if alternative item combinations would improve the internal consistency reliability of the other subscale, but none achieved an alpha or omega greater than .54.

Test-retest reliability. Test-retest reliability analysis used a n = 20 college student subsample retested over a 2–3-week interval, with blind scoring by the first author. Results demonstrated moderate coefficients for the Total scale (r = .64; p< .05), Self-subscale (r = .71; p< .01), and other- subscale (r = .65; p< .05). Results from all reliability analyses of the RISB OR scales are summarized in Table 3.

Table 3. Reliability Analyses of RISB OR Scales

*p< .05 **p< .01

Construct Validity. Convergent validity analyses with N = 84 Rorschach protocols achieved one small significant correlation (r = .23; p< .05) between the Self subscale and MAP. Discriminant validity was most clearly demonstrated for Zd, with correlations under .10 across all three of the OR scales. Given the limited evidence for convergent validity with Rorschach indices based on the variables selected, the reach was extended in a post-hoc analysis to Rorschach variables that are one-step removed from the self- and other-representation constructs. Rorschach variables selected for post-hoc convergent validity analysis included the Isolation Index, SumT, SumV, SumY, morbid (MOR), aggressive movement (AG), fictional and part human representations ((H)+Hd+(Hd)), depression index (DEPI), coping ability (EA), current stressors (es), and distorted human movement (M-). Populars (P) and Intellectualization Index (Intel) were selected for discriminant validity analysis. Only one of the convergent validity variables achieved statistically significant, meaningful correlations with OR scale scores: Morbid (MOR) with the Total scale (r = .24) and Other subscale (r = .22), whereas fictionalized/part human representations correlated inversely with the Total scale (r = -.23) and Self subscale (r = -.23). Discriminant validity was demonstrated with Populars (P) and Intellectualization Index (Intel) as both achieved effect sizes below .10 with all three RISB OR scales. All results related to the Rorschach are shown in Table 4.

Table 4. Pearson Correlations of RISB OR Scales Scores and Rorschach Indices (N = 84)

*p< .05

        Convergent validity analyses using N = 111 MMPI-2 profiles achieved the strongest and most consistent convergence with OR scale scores. The Self subscale and Total scale achieved correlations with all four MMPI-2 scales chosen for initial validity analyses. The Other subscale achieved no correlations with these four MMPI-2 scales. Acceptable discriminant validity levels below .10 were not demonstrated with Sc and TPA. In uniformity with procedures used with Rorschach indices, the reach was extended in a post-hoc analysis to MMPI-2 scales that are one-step removed from self- and other-representation constructs. MMPI-2 scales Sc1 (Social Alienation), Sc3 (Lack of Ego Mastery, Cognitive), DEP3 (Self-Deprecation), Subjective Depression (D1), Familial Discord (Pd1), Authority Problems (Pd2), Family Problems (FAM), Welsh’s A, Psychasthenia (Pt), and Social Introversion (Si) were selected for convergent validity analysis. MMPI-2 scales Sc6 (Bizarre Sensory Experiences) and Obsessiveness (OBS) were chosen for discriminant validity analysis. Several of these post-hoc MMPI-2 scales reached statistical significance with small-to-medium effect size correlations with respective RISB OR scales. Discriminant validity was most clearly demonstrated with Sc6 in relation to the Self subscale, with a correlation under .10. The full set of results are shown in Table 5.

Table 5. Pearson Correlations of RISB OR Scales Scores and MMPI-2 Scale Scores (N = 111)

MMPI-2 Scale Self Subscale Other Subscale Total Scale

*p < .05 **p < .01

Group comparisons. Four analyses of variance (ANOVAs) were conducted to compare RISB OR scores between diagnostic categories as well as outpatients versus controls. The first ANOVA between n = 45 personality disordered and n = 45 randomly selected non-personality disordered outpatients displayed nonsignificant results. In contrast, the second ANOVA between n = 50 randomly selected outpatients and N = 50 college students achieved significant results for all three RISB OR scales: Total scale, F(1, 49) = 14.1, p<

.001; Self subscale, F(1, 49) = 13.4, p< .001; Other subscale, F(1, 49) = 17.2, p< .001, with outpatients scoring significantly higher than college students. The third ANOVA assessed the mean differences in scores between n

= 42 college student participants who had never received a mental health diagnosis and n = 42 randomly selected outpatient participants with diagnoses at the time of RISB completion. Results were significant for all three scales: Total scale, F(1, 83) = 32.3, p< .001; Self subscale, F(1, 83) = 27.5, p< .001; Other subscale, F(1, 83) = 17.8, p< .001, with higher mean scores for outpatient participants. The final ANOVA was conducted with n = 39 college student participants who were not receiving psychotherapy at the time of this study and n = 39 randomly selected outpatient participants who were receiving psychotherapy at the time of RISB completion. The results demonstrated significant differences for all three scales: Total scale, F(1, 77) = 30.7, p< .001; Self subscale, F(1, 77) = 20.6, p< .001; Other subscale, F(1, 77) = 22.4, p< .001, with higher mean scores for outpatient participants.

Discussion:

        Object relations theorists’ position self- and other-representations linked by affects as the foundations of relating that persist throughout the lifespan and occur with minimal conscious amendment (Diamond et al., 2022; Yeomans et al., 2015). As such, these affect-laden templates for relationships are of particular interest in the assessment of interpersonal (mal)adjustment. Performance-based and self-report measures assess features of interpersonal functioning in indirect and direct ways, respectively. The RISB operates as a hybrid method with direct responding in terms of what the examinee chooses to write, and thus, in what is revealed. This SCT method, however, is unique in that across stems it retains an indirect injection of interpersonal nuance evidenced in themes, complexity of the completions, and affective associations to particular stems. Such indirect features in RISB completions include themes of social orientation, identification, psychological adjustment, perceptual attitudes, and social schemas (Lomax & Lam, 2011; Wollitzer et al., 1973). The current study capitalized on the current trends in object relations (OR) assessment by developing and evaluating an OR measure for the RISB. Of the 40 RISB stems, 15 were selected for the RISB OR Total scale. These scales achieved medium-to-large effect size intercorrelations ranging from .31 to .90 across outpatient and college student participants, suggesting reasonable coherence in assessing dysfunction across self- and other-representations.

        The first reliability analysis of this study focused on evaluating the inter-rater reliability of the RISB OR scales. Each scale achieved ICCs at r >.80, demonstrating acceptable inter-rater agreement. Examination of the psychometric adequacy of the OR scales also rested on demonstrating additional evidence of score reliability. Internal consistency reliabilities were strong for the Total scale and Self subscale, but moderate for the Other subscale. The lower alpha and omega values for the Other subscale may be accounted for by a couple of factors. First, the Other subscale has the fewest items, thus requiring each item’s scores to carry a greater weight. Second, the manner by which representations are elicited by key Other subscale items (“A mother,” “My father,” “People,” “Other people,” “Most women,” “Men”) is unique. Two stems prompt respondents to shift from a subjective to an objective characterization of the figures (“A mother” vs. “My father;” “People” vs. “Other people”). This shift to objectivity is associated with greater reflective capacity and higher ego functioning (Pronin et al., 2004), and thus, elicits responses modified by a cognitive-perceptual buffer. Despite these considerations, the content of these stems is relevant to object relations and could still prove insightful when the contents of completions are examined. The final component of reliability analysis involved test-retest reliability, which demonstrated moderate stability of OR scores. These coefficients were smaller than those found in prior studies using a two-week time interval (Arnold & Walter, 1957, as cited in Rotter et al., 1992; Richardson & Soucar, 1971, as cited in Rotter et al., 1992). On the other hand, all three OR scales achieved higher coefficients than a previous test-retest study using intervals greater than two weeks (Churchill & Crandall, 1955). Overall, our results suggest the RISB OR scales have adequate temporal stability based on similarities to stability coefficients obtained in other test-retest studies using the entire protocol (Rotter et al., 1992).

        The second half of the study focused on evaluating the external validity of the RISB OR scales. In the RISB and Rorschach analyses, the Self subscale achieved a small, significant correlation with the Rorschach MAP variable, indicating the scale’s correlation with a harmful or destructive visions of relationships. Of the Rorschach variables selected in this study for post-hoc analyses, only two correlated significantly with OR scales. The Rorschach Morbid (MOR) variable had a small yet significant positive correlation with the Total scale and other subscale. This finding suggests a sensitivity of these RISB OR scales to detect maladjusted, if not damaged or flawed, self-representations. Furthermore, the Rorschach fictionalized human representation variable achieved small negative correlations with the Total scale and Self subscale, suggesting the scales may reflect the degree to which the examinee bases his/her other-representations on real life people. Discriminant validity analyses with the Rorschach demonstrated effect sizes under .10 for the Populars (P) and Intellectualization Index variables, thus demonstrating the OR scales do not align with unrelated constructs of perceptual conventionality and reasoning devoid of affective influence.

        In the RISB and MMPI-2 analyses, the initial four MMPI-2 scales achieved statistically significant medium effect size correlations with the Total scale. In relation to the Self subscale, Pd4 and Pd5 reached statistically significant small-to-medium effect sizes, whereas LSE and SOD achieved statistically significant medium effect sizes. The significant correlations with the Self subscale and Total scale indicate the scales’ detection of self- and other representations as they relate to the participants’ appraisal of, and relationship with, themselves and others. A post-hoc method similar to that with the Rorschach was extended to MMPI-2 scales to further assess self/other constructs of the RISB OR scales; the results were promising. Ten MMPI-2 scales selected for the post-hoc convergent analyses involved direct appraisal of self and others, and seven correlated significantly with the Self subscale. The numerous significant correlations paralleled prior findings demonstrating that RISB scores correlate positively with relevant MMPI scale scores (Rotter et al., 1992). Furthermore, the effect size magnitudes for the RISB Self and Other subscales in relation to MMPI-2 scales supported their differentiated constructs. Post-hoc discriminant validity analyses with targeted effect sizes below. 10 was only achieved with the Self subscale and Sc6. Therefore, discriminant validity was not substantially demonstrated.

        The results from the current study appeared to replicate findings from Schlicht et al. (1969) in obtaining minimal convergent correlations with the Rorschach, but discriminant validity findings were more promising. On the other hand, as noted earlier, our results showed favorable convergent validity with several MMPI-2 scales but limited evidence of discriminant validity with this self-report measure. These findings appear to demonstrate that a SCT measure such as the RISB is truly a hybrid of self-report and performance-based approaches of personality assessment, particularly in evaluating maladjusted self- and other-representations.

        Overall, the reliability and validity of the RISB OR scales achieved psychometric results commensurate with prior RISB studies involving the entire protocol. Despite these promising results, the temporal stability of the OR scales needs further validation with a larger sample, and the internal consistency of the Other subscale may also need enhancement. The RISB OR scales do not convincingly correlate with Rorschach variables that measure interpersonal features. Nonetheless, small-to-medium correlations with noteworthy variables (MOR and MAP) are suggestive of the scales’ ability to detect pessimistic or pathological social cognitive components of object relations representations in a manner like the Social Cognition and Object Relations Scale (SCORS) for the TAT (Stein et al., 2011). Furthermore, a multitude of correlations with MMPI-2 scales of self/interpersonal functioning indicate the RISB OR scales elicit features of object relations (i.e., self-esteem, social alienation, interpersonal wariness, and self-regard). As such, the construct validity of the RISB OR scales can be most accurately conceived of as measures of (mal)adjustment in self- and other representations.

        Prior studies of convergence analyses between self-report and performance-based tasks reveal low convergences between corresponding constructs across measures (Archer & Krishnamurthy, 1993a, 1993b; Meyer, 1997). However, Meyer (1996) asserted that such cross-method disagreement is not emblematic of test score invalidity and may, in fact, enhance interpretation. Sentence completion methods are hybrid measures as they have more structure than the Rorschach but less than self-report measures such as the MMPI-2 (Watson, 1978). Since the RISB involves the completion of moderately ambiguous sentence stems to express personal feelings, it may be considered to have unique method variance compared to the highly ambiguous Rorschach stimuli and less ambiguous, verbally mediated MMPI-2 items.

        This study also compared mean scores for the OR scales between two broad diagnostic subgroups within the outpatient sample. Significant differences with the OR scales were not observed between outpatients with and without personality disorders. Nevertheless, all the score floors for the OR scales among the personality disordered sample were higher than the non-personality disordered sample, suggesting a greater likelihood for personality disordered persons to trend towards maladjustment. Statistically significant differences in RISB OR scale scores were, however, found in other comparisons. Specifically, mean scores from all RISB OR scales differed significantly between college students and outpatient participants in the expected direction. In addition, statistically significant score differences were observed between college students not receiving mental health treatment and outpatients actively receiving mental health treatment at the time of RISB completion. Similarly, statistically significant differences were found between student participants who had never received a prior mental health diagnosis andoutpatient clients who had a diagnosis at the time of RISB testing. These results indicate the RISB OR scale scores were successful in differentiating clinical from non-clinical samples.

Limitations:

        The limitations of this study are noted. First, the internal consistency reliabilities for the Other-representation subscale did not achieve coefficient levels similar to prior RISB studies and were notably lower than the Self-representation subscale and Total scale. Therefore, continued refinement of the Other-representation subscale’s internal consistency is warranted. Second, the sample size of the control group was limited. Due to this relatively small sample size, as well as their demographics as college students, replication studies with more diverse participants are needed. Replication studies might extend the validity and reliability analyses of the RISB OR scales with samples of different ages, socioeconomic and cultural backgrounds. A third limitation of this study concerns the Rorschach variables used for external validity comparisons. These variables were coded by graduate students in supervised practicum training. As such, errors in coding may have occurred, which may have impacted the strength of correlations. Future research extending the construct validity of object relations within the RISB OR scales might involve correlations with the SCORS and/or Bell Object Relation and Reality Testing Inventory (BORRTI; Bell, 1995).

Contributions:

        This study served to develop a measure for the RISB that detects object relations, and it produced promising findings. To the best of our knowledge, a measure for the RISB has not been developed for nearly 26 years, since Lehnert et al. (1996), thus making this work an advancement in accordance with contemporary psychometric standards. In addition, this study marks the first time in nearly 45 years the temporal stability of the RISB was evaluated, since Rabinowitz and Shouval (1977). The internal reliability for the RISB OR scales is adequate, and the construct validity findings suggest the scales are sufficient measures of (mal)adjusted self-and other-representations constituting object relations. Moreover, the internal validity results for the OR scales generally corresponded to those of prior RISB studies involving an entire protocol, suggesting these results may operate as a partial replication. Our study also examined correlations between selected RISB items, MMPI-2 scales, and Rorschach variables. Such comparisons have not been studied in over 50 years (cf. Schlicht et al., 1969; Snow, 1972).

        SCTs have been popular assessment tools in clinical practice for nearly eight decades. Although interest in their use has declined recently (Piotrowski, 2018), the capacity of the RISB to elicit aspects relevant to object relations advocates for the relevancy of an object relations scale for this measure. As such, this set of RISB OR scales serve as potentially useful measures of object relations in clinical personality assessments and may serve to rejuvenate interest in clinical and research applications of SCTs.

Acknowledgments:

We offer our thanks to the Society for Personality Assessment for providing dissertation grant support for the first author; Karina Guerra-Guzman, graduate research assistant for the second author for her scoring assistance; and Scott A. Gustafson, Ph.D. for permitting data extraction from the psychology clinic archives.

References:

Archer, R. P., & Krishnamurthy, R. (1993a). A review of MMPI and Rorschach interrelationships in adult samples. Journal of Personality Assessment, 61(2), 277-293. https://doi.org/10.1207/s15327752jpa6102_9

Archer, R. P., & Krishnamurthy, R. (1993b). Combining the Rorschach and the MMPI in the assessment of adolescents. Journal of Personality Assessment, 60(1), 132-140. https://doi.org/10.1207/s15327752jpa6001_10

Bell, M. D. (1995). Bell Object Relations and Reality Testing Inventory: (BORRTI): Manual. Los Angeles, CA: Western Psychological Services.

Bombel, G., Mihura, J. L., & Meyer, G. J. (2009). An examination of the construct validity of the Rorschach Mutuality of Autonomy (MOA) Scale. Journal of Personality Assessment, 91(3), 227-237. https://doi.org/10.1080/00223890902794267

Bornstein, R. F. (1996). Construct validity of the Rorschach Oral Dependency Scale: 1967-1995. Psychological Assessment, 8(2), 200-205. https://doi.org/10.1037/1040-3590.8.2.200

Bornstein, R. F. (2003). Projective assessment of interpersonal dependency. In D. L. Segal & M. J. Hilsenroth (Eds.), Comprehensive handbook of psychological assessment (pp. 476-484). New York, NY: John Wiley & Sons, Inc.

Brems, C., & Lloyd, P. (1995). Validation of the MMPI-2 Low Self-Esteem content scale. Journal of Personality Assessment, 65(3), 550-

556. https://doi.org/10.1207/s15327752jpa6503_13

Butcher, J. N., Dahlstrom, W. G., Graham, J. R., Tellegen, A., & Kaemmer, B. (1989). Minnesota Multiphasic Personality Inventory-2 (MMPI-2): Manual for administration and scoring. Minneapolis, MN: University of Minnesota Press.

Chance, J. E. (1958). Adjustment and prediction of others’ behavior. Journal of Consulting Psychology, 22(3), 191-194. https://doi.org/10.1037/h0046256

Churchill, R., & Crandall, V. J. (1955). The reliability and validity of the Rotter Incomplete Sentences Test. Journal of Consulting Psychology, 19(5),345-350. https://doi.org/10.1037/h0048744

Diamond, D., Yeomans, F. E., Stern, B. L., & Kernberg, O. F. (2022). Treating pathological narcissism with transference-focused psychotherapy. New York, NY: Guilford Press.

Exner, J. E., Jr. (2001). A Rorschach workbook for the Comprehensive System (5th ed.). Asheville, NC: Rorschach Workshops. Friedman, A. F., Bolinskey, P. K., Levak, R. W., & Nichols, D. S. (2015). Psychological assessment with the MMPI-2/MMPI-2-RF. New

York, NY: Routledge.

Graceffo, R. A., Mihura, J. L., & Meyer, G. J. (2014). A meta-analysis of an implicit measure of personality functioning: The Mutuality of Autonomy Scale. Journal of Personality Assessment, 96(6), 581-595. https://doi.org/10.1080/00223891.2014.919299

Haak, R. A. (2003). The sentence completion as a tool for assessing emotional disturbance. In C.R. Reynolds & R. W. Kamphaus (Eds.), Handbook of psychological and educational assessment of children: Personality, behavior, and context (pp. 159-181). New York, NY: Guilford Press.

Hathaway, S. R., & McKinley, J. C. (1989). Minnesota Multiphasic Personality Inventory-2 (MMPI-2): Manual for administration and scoring. Minneapolis, MN: University of Minnesota Press.

Object Relations Measure for RISB: 29

Heesacker, R. S., & Neimeyer, G. J. (1990). Assessing object relations and social cognitive correlates of eating disorder. Journal of Counseling Psychology, 37(4), 419-426. https://doi.org/10.1037/0022-0167.37.4.419

Hojnoski, R. L., Morrison, R., Brown, M., & Matthews, W. J. (2006). Projective test use among school psychologists: A survey and critique. Journal of Psychoeducational Assessment, 24(2), 145-159. https://doi.org/10.1177/0734282906287828

Holaday, M., Smith, D. A., & Sherry, A. (2000). Sentence completion tests: A review of the literature and results of a survey of members of the Society for Personality Assessment. Journal of Personality Assessment, 74(3), 371-383. https://doi.org/10.1207/S15327752JPA7403_3

Ilonen, T., & Salokangas, R. K. R. (2016). The Rorschach Coping Deficit Index as an indicator of neurocognitive dysfunction. Rorschachiana, 37(1), 28-40. https://doi.org/10.1027/11925604/a000075

Janda, L. H. (1998). Psychological testing: Theory and applications. Needham Heights, MA: Allyn & Bacon.

Jessor, R., Liverant, S., & Opochinsky, S. (1963). Imbalance in need structure and maladjustment. Journal of Abnormal and Social Psychology, 66(3), 271-275. https://doi.org/10.1037/h0046146

Krishnamurthy, R., Archer, R. P., & Groth-Marnat, G. (2011). The Rorschach and performance based assessment. In T. M. Harwood, L. E. Beutler, & G. Groth-Marnat (Eds.), Integrative assessment of adult personality (3rd ed., pp. 276-328). New York, NY: Guilford Press.

Lah, M. I. (1989). New validity, normative, and scoring data for the Rotter Incomplete Sentences Blank. Journal of Personality Assessment, 53(3), 607-620. https://doi.org/10.1207/s15327752jpa5303_17

Lehnert, K. L., Overholser, J. C., & Adams, D. M. (1996). The Cognition Rating Form: A new approach to assessing self-generated cognitions in adolescent sentence completions. Psychological Assessment, 8(2), 172-181. https://doi.org/10.1037/1040-3590.8.2.172

Lilienfeld, S. O. (1999). The relation of the MMPI-2 Pd Harris-Lingoes subscales to psychopathy, psychopathy facets, and antisocial behavior: Implications for clinical practice. Journal of Clinical Psychology, 55(2), 241-255. https://doi.org/10.1002/(SICI)1097-4679(199902)55:2

Lomax, C. L., & Lam, D. (2011). Investigation into activation of dysfunctional schemas in euthymic bipolar disorder following positive mood induction. British Journal of Clinical Psychology, 50(2), 115-126. https://doi.org/10.1348/014466510X497841

Meyer, G. J. (1996). The Rorschach and MMPI: Toward a more scientifically differentiated understanding of cross-method assessment.

Journal of Personality Assessment, 67(3), 558-578. https://doi.org/10.1207/s15327752jpa6703_11

Meyer, G. J. (1997). On the integration of personality assessment methods: The Rorschach and MMPI. Journal of Personality Assessment, 68(2), 297-330. https://doi.org/10.1207/s15327752jpa6802_5

Meyer, G. J., Erard, R. E., Erdberg, P., Mihura, J. L., & Viglione, D. J. (2011). Rorschach Performance Assessment System: Administration, coding, interpretation, and technical manual. Toledo, OH: Rorschach Performance Assessment Systems LLC.

Monroe, J. M., Diener, M. J., Fowler, C. J., Sexton, J. E., & Hilsenroth, M. J. (2013). Criterion validity of the Rorschach Mutuality of Autonomy (MOA) scale: A meta-analytic review. Psychoanalytic Psychology, 30(4), 535-566. https://doi.org/10.1037/a0033290

Piotrowski, C. (2018). Sentence completion methods: Summary review of 70 survey-based studies of training and professional settings. SIS Journal of Projective Psychology and Mental Health, 25, 60-75.

Pronin, E., Gilovich, T., & Ross, L. (2004). Objectivity in the eye of the beholder: Perceptions of bias in self versus others. Psychological Review, 111(3), 781-799. https://doi.org/10.1037/0033-295X.111.3.781

Rabinowitz, A., & Shouval, R. (1977). Fantasy as a medium for the reduction of trait versus state aggression. Journal of Research in Personality, 11(2), 180-190. https://doi.org/10.1016/00926566(77)90015-0

Renner, K. E., Maher, B. A., & Campbell, D. T. (1962). The validity of a method for scoring sentence completion responses for anxiety, dependency, and hostility. Journal of Applied Psychology, 46(4), 285-290. https://doi.org/10.1037/h0042841

Rotter, J. B., Lah, M. I., & Rafferty, J. E. (1992). Rotter Incomplete Sentences Blank – 2nd ed.. New York, NY: Psychological Corporation.

30: Rucker & Krishnamurthy

Rotter, J. B., & Rafferty, J. E. (1950). Manual: The Rotter Incomplete Sentences Blank: College form. New York, NY: Psychological Corporation.

Rotter, J. B., Rafferty, J. E., & Lotsof, A. B. (1954). The validity of the Rotter Incomplete Sentences Blank: High school form. Journal of Consulting Psychology, 18(2), 105-11. https://doi.org/10.1037/h0062560

Rotter, J. B., Rafferty, J. E., & Schachtitz, E. (1949). Validation of the Rotter Incomplete Sentences Blank for college screening. Journal of Consulting Psychology, 13(5), 348-356. https://doi.org/10.1037/h0062499

Schiltz, L., & Schiltz, J. (2016). Borderline functioning and life trauma: A structural approach. Archives of Psychiatry and Psychotherapy, 18(2), 12-21. https://doi.org/10.12740/APP/63358

Schlicht, W. J., Jr., Carlson, H. J., Skeen, D. R., & Skurdal, M. A. (1969). Self-images without awareness and projective methods of personality assessment. Journal of Projective Techniques & Personality Assessment, 33(5), 419-423. https://doi.org/10.1080/0091651X.1969.10120610

Sellbom, M., Graham J. R. & Schenk, P. W. (2005). Symptom correlates of MMPI-2 scales and code types in a private-practice setting.

Journal of Personality Assessment, 84(2), 163-171. https://doi.org/10.1207/s15327752jpa8402_06

Snow, S. T. (1972). Factor analysis of Rotter’s Incomplete Sentences Blank (ProQuest document ID: 615841399) [Doctoral dissertation, Rutgers State University]. Dissertation Abstracts International.

Soenning, S. A., Fireman, G. D., & Clopton, J. R. (2010). The measurement of social competence in children using the Rorschach Inkblot Test: A validation study. Psychology, 1(4), 261-272. https://doi.org/10.4236/psych.2010.14035

Spear, W. E., & Sugarman, A. (1984). Dimensions of internalized object relations in borderline and schizophrenic patients. Psychoanalytic Psychology, 1(2), 113-129. http:// doi.org /10.1037/0736-9735.1.2.113

Stein, M. B., Hilsenroth, M., Slavin-Mulford, J., & Pinsker, J. (2011). Social Cognition and Object Relations Scale: Global rating method (SCORS-G). (4th ed.). Boston, MA: Massachusetts General Hospital and Harvard Medical School.

Torstrick, A., McDermut, W., Gokberk, A., Bivona, T., & Walton, K. E. (2015). Associations between the Rotter Incomplete Sentences Blank and measures of personality and psychopathology. Journal of Personality Assessment, 97(5), 494-505. https://doi.org/10.1080/00223891.2015.1015679

Vernallis, F. F., Shipper, J. C., Butler, D. C., & Tomlinson, T. M. (1970). Saturation group psychotherapy in a weekend clinic: An outcome study. Psychotherapy: Theory, Research, and Practice, 7(3), 144-152. https://doi.org/10.1037/h0086575

Vishwakarma, P., Dwivedi, S., & Kumar, R. (2016). Emotional intelligence, Exner’s special indices on Rorschach in schizophrenia. SIS Journal of Projective Psychology and Mental Health, 23, 66-72.

Watson, R. I. (1978). The sentence completion method. In B. B. Wolman (Ed.), Clinical diagnosis of mental disorders (pp. 255-279). New York, NY: Plenum Press.

Weiner, I. B. (2003). Principles of Rorschach interpretation: Basic considerations in Rorschach testing. Mahwah, NJ: Lawrence Erlbaum Associates.

Weis, R., Toolis, E. E., & Cerankosky, B. C. (2008). Construct validity of the Rotter Incomplete Sentences Blank with clinic-referred and non-referred adolescents. Journal of Personality Assessment, 90(6), 564-573. https://doi.org/10.1080/00223890802388491

Wollitzer, A. O., Pulos, S. M., & Vitale, J. H. (1973). A scoring method for the Michigan Sentence Completion Test. Newsletter for Research in Mental Health & Behavioral Sciences, 15(2), 26-28.

Wrobel, T. A. (1992). Validity of Harris and Lingoes MMPI subscale descriptors in an outpatient sample. Journal of Personality Assessment, 59(1), 14-21. http://dx.doi.org/10.1207/s15327752jpa5901_2

Yeomans, F. E., Clarkin, J. F., & Kernberg, O. F. (2015). Transference focused psychotherapy for borderline personality disorder: A clinical guide. Arlington, VA: American Psychiatric Publishing.