REVIEW OF DISABILITY STUDIES: AN INTERNATIONAL JOURNAL
	Volume 12 Issue 2&3
	

Research Articles
Securing Personal Input from Individuals Aging with Intellectual Disability: Do Differing Methodologies Produce Equivalent Information?
        Stuart Wark, Ph.D. (1), Miranda Cannon-Vanry, M.A. (1), Marie Knox, Ph.D. (1, 2), Marie Parmenter (2), Rafat Hussain, Ph.D. (1, 4), Matthew Janicki, Ph.D. (1, 5), Chez Leggatt-Cook, Ph.D. (6), Meaghan Edwards, M.A. (2), Trevor Parmenter, Ph.D. (1, 2)
(1)   School of Rural Medicine, University of New England, Australia
(2)   Centre for Disability Studies, Sydney Medical School, University of Sydney, Australia
(3)   Australian National University, Australia
(4)   ANU Medical School & Research School of Population Health, Centre for Ageing, Health and Wellbeing, Australian National University, Australia
(5)   Department of Disability and Human Development, University of Illinois at Chicago, Chicago, USA
(6)   Uniting Care Community, Australia
Abstract: Research is limited on whether differing methodologies for facilitating personal contributions from individuals aging with intellectual disability produce equivalent knowledge outcomes. Two matched purpose-developed tools examined five quality-of-life domains. Results showed substantial variance between qualitative interview responses and Likert-scale data, and indicate validity concerns for using either methodology in isolation.
Keywords: aging, intellectual disability , likert-scales, qualitative interviews
Introduction 
The life expectancy of people with intellectual disabilities has undergone a remarkable Determining Quality of Life and Life Satisfaction increase in recent decades (Coppus, 2013). However, it has long been recognized that the mere extension of life does not necessarily result in a high quality of life as people age (Brown, 1993). This understanding has resulted in an increased focus on supporting active, positive, or successful aging (Boudiny, 2013). One of the current issues facing both the disability and aged care sectors is the relative scarcity of research data regarding the ongoing desires and expectations of individuals with intellectual disability about their lives and gathered directly from them (Bigby et al., 2014; Nind, 2011), and particularly as they age (Parmenter et al., 2013; Wark et al, 2013a). This lack of knowledge in turn results in difficulties for family members, service providers and public sector administrators in identifying societal or other structural impediments (Wark et al, 2014a), and understanding how to best meet the needs of this cohort of people (Cummins, 1997; Wark et al., 2013b, 2014b). 
There remains a significant question concerning the best mechanisms that can be used to gain the direct and accurate input of people with intellectual disabilities. This situation is particularly relevant for individuals with more severe intellectual impairment and / or significant communication difficulties (Boland et al., 2008; Walmsley, 2001). It is highly pertinent in light of Australia’s current national disability reforms which specifically nominate the need to ensure the contribution of all individuals with the disability in any decisions about their life (Bonyhady, 2014).
Determining Quality of Life and Life Satisfaction
One of the main methodologies for assessing an individual’s level of satisfaction with their life has been to use a quality of life (QoL) measure, with a number of validated scales focusing on specific population sub-groups (e.g., Perales et al., 2013; Makai et al., 2014). QoL has become one of the major measures of disability organisations with the framework used for evaluating the appropriateness of services and their delivery mechanisms (Kober & Eggleton, 2009). The research literature would appear to support the concept of QoL measurement having to occur through a variety of different qualitative and quantitative indicators, but it is also recognized that these dimensions will vary both across time and individuals (Chun Yu et al., 1996; Emerson et al., 2013; Felce & Perry, 1997). 
Quantitative indicators are useful in providing a summary of general achievements against pre-determined goals, with Likert-type scales often utilized to quantify an individual’s levels of satisfaction (e.g. Cummins & Lau, 2010). However, adequate inclusion of appropriate qualitative indicators is also recommended as this mechanism directly involves people with disabilities in the assessment process (Cummins, 1997). Alternative options, including seeking the input from proxy respondents such as carers rather than the actual person, have been shown to be potentially unreliable (Stancliffe, 1995). So while it is acknowledged that not all people with an intellectual disability may be able to self-report (Finlay & Lyons, 2001), facilitating the direct involvement of the individual is desirable whenever possible. The issue of how to best assess perceptions of individuals with an intellectual disability as they age with respect to their lifestyle still remains, as there is little agreement on a precise definition of QoL for this cohort, how to measure it, or even what factors are most appropriate to assess (Wark, 2011).
Comparing Outcome Data for People with Intellectual Disability
Concerns about the reliability of information gained from a proxy-respondent, when compared to self-reports, support the wider movement towards greater involvement of individuals with intellectual disability in making decisions about their life (Emerson et al., 2013). Two of the main approaches for gaining a direct contribution from people with intellectual disabilities are semi-structured qualitative methodologies utilizing interviews and structured quantitative methodologies using Likert-type rating scales (Beail & Williams, 2014; Brown et al., 2009; Sigstad, 2014). Likert-type scales, often modified to include pictorial aids, such as smiley face emoticons (Schmidt et al., 2010), are commonly used with people with intellectual disabilities, and particularly among those with communication difficulties. While Likert-type scales can provide a complementary tool to support input from people with intellectual disabilities, in some scenarios they are used in isolation as an alternative for individuals who may not be otherwise able to verbally contribute their opinion in an interview situation (Hartley & MacLean, 2006). In Australia the Commonwealth Government specifically recommends the use of alternative communication strategies and methods in order to facilitate the direct involvement of individuals with intellectual disability (Department of Social Services, 2014a); however, this approach contains an inherent assumption that the information from all such methods provide equivalent knowledge. If the goal of using alternative communication tools, such as Likert-type scales, is to facilitate the involvement of people with intellectual disabilities in making decisions about their lives (Ottmann & Crosbie, 2012), it remains unclear whether the information gathered from a qualitative interview and a quantitative Likert-scale is directly comparable.
This issue is highly relevant in Australia currently due to major legislative changes, including the National Disability Insurance Scheme (Commonwealth Government of Australia, 2013; Parmenter, 2014), individualized funding, and person-centered approaches (Ageing Disability and Home Care, 2013). A key aspect of these reforms is a strong focus on gaining the direct views of all individuals, particularly the person with a disability, through the use of any relevant tools necessarily to gain this input (National Disability Insurance Agency, 2014). This approach involves carers, health professionals, and advocates determining life aspirations and assessing quality of life in collaboration with individuals with intellectual disability. In practice, it may result in a situation where some people are interviewed using open-ended qualitative methodologies about their thoughts or opinions, while others participate through completion of a pre-determined quantitative tool using a Likert-type scale. This presents a major equity problem; if there is no level of equivalency of information gained through responses from either a Likert-type tool or a personal interview, one cohort may be disadvantaged as their opinions and desires may be identified incompletely or incorrectly.
Current Study
The goal of the current pilot project was to use two matched and purpose-developed tools, a semi-structured qualitative interview and a structured Likert-type quantitative scale, to examine the perspectives of individuals aging with an intellectual disability regarding their quality of life. This paper examines part of the cross-sectional data gained from a cohort of participants who completed both tools during the one session. The results from 20 participants are compared to determine if there were similarities in the response patterns and if any conclusions can be drawn about the equivalency of the two methodologies.
Methodology
Recruitment
A request for expressions of interest and an information package regarding the study was sent to disability and aged care agencies in major metropolitan (Sydney, Wollongong, Brisbane and the Sunshine Coast) and rural (Toowoomba and Parkes) locations across New South Wales and Queensland in Australia. The study specified that participants needed to be older adults with an intellectual disability; no precise definition of ‘older’ was provided due to the inherent variability in the sample, but a nominal minimum age of 55 was suggested. 
On-site information sessions for potential participants, carers, support staff, families and advocates/guardians (where applicable) were held which explained the goals of the project. Participants then self-nominated to the researchers, with both written and verbal informed consent being gained from each individual prior to commencement. Individuals had to be capable of participating both in a face-to-face interview and to respond to survey questions, although assistance was provided to overcome any issues with literacy. Formal ethics approval was provided by Institutional Ethics Committee (institutional name and ethics approval number to be provided after blind review).
Participants
This paper reports on 20 sequential qualitative interviews and 20 quantitative Likert-type scale scores from the same individuals. Each participant in the study had to be able to give informed consent to participate in the research. Consent was determined initially by the participant in line with the protocol of the New South Wales Government (Attorney General’s Department, 2008), but also verified by the support organisation and/or person responsible. One individual indicated a strong desire to participate, however, it was agreed by the organisation and interviewer that they were not capable of providing informed consent. The decision was made to still ‘interview’ the person as a number of their peers were involved and they did not wish to be ‘left out’, however the person’s data was not recorded or included.
The 20 participants were split evenly between New South Wales and Queensland (10 from each state); however there were more males than females (12 to 8) and slightly more rural than metropolitan area residents (11 to 9). The average age of all participants was not able to be definitively collated, as it was apparent in some cases that the age provided by the participants was not chronologically feasible (i.e., four interviewees volunteered ages that were inconsistent or impossible, and/or they could not provide a precise date of birth). Attempts were made to confirm all ages with family or carers, but accurate data was simply not available for two participants. Of the ages that could be verified, the range was from 52 to 79 years with a mean of 60 years.
Tools
The participants were interviewed using a semi-structured format and were also asked to complete a hard copy of a five-point Likert-type scale featuring both written and pictorial (smiley faces) prompts. The Likert-style scales used a simple 1=very happy through to 5=very unhappy delineation. A sample of the Likert-scale with its associated pictorial aid is included as Figure 1.
 Figure 1 – Likert-Scale - five-point Likert-type scale featuring both written and pictorial (smiley faces) prompts. The Likert-style scale used 1=very happy, 2=happy, .=not happy or unhappy and 5=very unhappy delineation. 

Figure 1 – Likert-Scale
The interview featured semi-structured questions relating to key domains of ‘residence’, ‘support’, ‘health’, ‘changes in health associated with aging’, and ‘life satisfaction’. The interview data were captured through audio-taping with independent transcription. Each participant was also asked to use a five-point Likert-style scale to rate five questions that related to the key domains explored in the interviews:
* “How happy are you living where you live?” (Residence),
* “How happy/satisfied are you that the help you receive?” (Support), 
* “How is your health?” (Health),
* “How has your health changed as you’ve gotten older?” (Changes in Health), and
* “How happy/satisfied are you with your life today?” (Life Satisfaction).
While it is recommended to assess acquiescent responding of participants in Likert-scales through the inclusion of specific neutral or negative questions (e.g. Cuskelly, Moni, Lloyd & Jobling, 2013), this was deliberately not included in this project for two reasons. Firstly, the purpose of the current research was to examine the similarity of data gained from individuals aging with intellectual disability through differing methodologies, and how these results relate to real world practice. It was not considered likely that all studies would follow the recommended acquiescence protocol, and therefore it was not appropriate to do so in this project in order to best replicate potential real-world scenarios. This issue was felt to be particularly relevant for disability or aged-care organizations seeking the personal contribution of service-users, rather than more highly structured academic research projects. It was also believed that, due to the short nature of the tool and the very specific alignment of five questions to the five quality of life domains, the inclusion of any additional items could be potentially confusing to participants. 
Coding of Interview Data
All 20 interview transcripts were individually coded by two members of the team according to whether each separate statement was positive or negative in terms of the individual’s experience, and then categorized into the nominated five domains. Statements could be coded multiple times, with both positive and negative components within the one statement (e.g. “My health problems are much worse now [coded as negative], but my support staff are excellent” [coded as positive]). Statements that were neutral, ambiguous, or unrelated to the interview question (“e.g. “look at that dog over there”) were not included in the analysis. Any disagreements were resolved through discussion. The first three interviews were separately coded by an independent third person to ensure the overall ratios of positive and negative statements were the same.
Initial consideration was made for also coding the ‘intensity’ of the statements, however this proved to be impractical. Attempting to objectively quantify the underlying positive or negative strength of each statement within a cohort of geographically dispersed individuals aging with intellectual disability was not considered feasible. A high level of familiarity with each participant was deemed necessary to accurately distinguish emotional engagement with the content from other personal and environmental factors. As an example, a number of individuals spoke with a uniform neutral tone throughout the interview, and any attempt to categorize the intensity of their different responses was considered to be highly subjective and liable to substantial inter-rater variance.   
Results
Interviews


There was an average of 63.9 statements per interview that were coded as either positive or negative, with a range of 11 to 193. There were an average 38.1 positive statements and 25.8 negative statements per interview, which represented a ratio of approximately 1.48 times as many positive statements as negative. 
An estimate of the overall ‘positivity’ of each interview was calculated by comparing the number of positive statements against the number of negative statements. A number below 1 indicated that the interview was skewed towards the negative, while a number above 1 indicated a positive result. Table 1 presents the breakdown of the positive versus negative statements within each interview, and the overall positivity score for each individual’s interview.
Table 1 - Overall Positivity of Interviews
Interview Number
	Interviews
	

	Positive
Statements
	Negative Statements
	Overall Positivity
	1
	30
	33
	0.91
	2
	35
	30
	1.17
	3
	108
	85
	1.27
	4
	30
	18
	1.67
	5
	45
	31
	1.45
	6
	6
	5
	1.20
	7
	30
	16
	1.88
	8
	80
	27
	2.96
	9
	65
	72
	0.90
	10
	39
	23
	1.70
	11
	36
	13
	2.77
	12
	17
	5
	3.40
	13
	29
	20
	1.45
	14
	29
	27
	1.07
	15
	26
	17
	1.53
	16
	25
	3
	8.33
	17
	29
	33
	0.88
	18
	23
	23
	1.00
	19
	56
	14
	4.00
	20
	24
	20
	1.20
	Likert Scores
Of the 100 distinct Likert scores (20 different participant scores across the five scales), only six were rated as being lower than the neutral position. There were 10 neutral scores, with the remaining 84 scores all either being positive (28) or very positive (56). This shows a very strong trend towards the ‘very positive’ end of the scale (mean score 1.6). These data are represented in Figure 2 below.
 Figure 2 Chart - Likert score distribution - bar graph

x-axis = likert scores, ranging from 1 to 5
y-axis = distribution of scores, ranging from 0 to 60

likert scale is based on the previous table 

Figure 2 Distribution of Likert Scale Scores
Alternative text description – The image depicts a bar chart of the y-axis is labeled Distribution of Scores ranging from 0 to 60 and the x-axis is labeled Likert Scores with 1, 2, 3, 4, and 5. The results of the likert score for 1 has a distribution score is 54; likert score for 2 has a distribution score is 28; likert score for 3 has a distribution score of ~8; likert score for 4 has a distribution score of ~2; and the likert score for 5 has a distribution score of ~less than 0.
Comparisons Across the Five Domains
An analysis of the positivity of each of the five separate key issue categories was conducted. These five categories were: Residence (“How happy are you living where you live?”); Support (“How happy/satisfied are you with the help you receive?”); Health (“How is your health?”); Change in Health (“How has your health changed as you’ve gotten older?”) and Life Satisfaction (“How happy/satisfied are you with your life today?”).
For the first domain, Residence, there was a ratio of 0.99 positive to negative statements. This is very close to a completely neutral position, with approximately as many negative comments as positive ones. However, the mean Likert-type scale score for this domain was very high at 1.45, with 14 of the 20 participants rating their happiness as “very happy”. Overall, this result seems to be contradictory as a Likert mean score closer to 3 (neutral) would have been more expected. 
The second domain related to the Support received by each individual. The ratio of 2.62 positive to negative statements indicated that the study participants were generally very happy with the support provided to them. However, the overall Likert-type scale average for this question was very similar to the first domain (1.4). The Likert result is closer to what the interviews would predict, with a high ratio of positive statements corresponding to a high Likert score for this domain. The distribution pattern of Likert scores for Residence and Support were both similar to the overall distribution, as shown in Figure 3 below.
 Figure 3 Chart - Likert score distribution chart with support and residence data

y-axis is the distribution of scores, ranging from 0 to 90
x-axis is the likert scores, ranging from 1 to 5

The line graph depicts a three line chart of the distribution of scores in relations to a five point Likert scores each line measures the total, residence or support as described in the previous paragraph. 

Supports' line graph begins at 10 in 1 likert score and ends at 0 in 5 likert score

Residence's line graph begins at ~25 in 1 likert score and ends at 0 in 5 likert score

Total's line graph begins at ~82 in 1 likert score and ends at ~2 in 5 likert score.  
Figure 3 Total Likert Scores Versus Likert Scores for Support and Residence
Alternative text description – The image depicts a three line chart of the distribution of scores from 0-90 on the y-axis in relations to a five point Likert scores on the x-axis with the three lines measuring the support, residence and total as described in the previous paragraph. The line chart for supports likert score 1 has a distribution score of ~12, likert score 2 has a distribution score of ~7, likert score 3 has a distribution score of ~0, likert score 4 has a distribution score of ~0, and likert score 5 has a distribution score of ~0. The line chart for residence likert score 1 has a distribution score of ~27, likert score 2 has a distribution score of ~10, likert score 3 has a distribution score of ~0, likert score 4 has a distribution score of ~0, and likert score 5 has a distribution score of ~0. The line chart for total likert score 1 has a distribution score of ~81, likert score 2 has a distribution score of ~31, likert score 3 has a distribution score of ~12, likert score 4 has a distribution score of ~8, and likert score 5 has a distribution score of ~2.
The third domain, Health, showed a neutral ratio of 1.1 positive to negative statements. The Likert-type scale mean for this question was 2.2, which is indicative of a generally positive perspective rather than a neutral one. The fourth domain, Changes in Health, revealed a very strong negative bias in the statements, with a ratio of 0.26 positive statements for every negative one. However, the mean Likert score was 1.95, which indicated a comparable, or in fact a slightly more positive position, to that of the Health domain. Figure 4 shows the similarity in the Likert score distribution for the Health and Changes in Health domains.
 Figure4Chart.jpg

Figure 4 Chart - Likert score distribution chart with support and health data

y-axis is the distribution of scores, ranging from 0 to 60
x-axis is the likert scores, ranging from 1 to 5

The line graph depicts a three line chart of the distribution of scores in relations to a five point Likert scores each line measures the total, health or support as described in the previous paragraph. 

health line graph starts at ~-5 in 1 likert score and ends at ~-5 in 5 likert score
changes in line graph begins at ~-1 in 1 likert score and ends at 0 in ~-5 likert score
Total's line graph begins at ~57 in 1 likert score and ends at ~-5 in 5 likert score.  

Figure 4 Total Likert Scores Against Likert Scores for Domains 3 and 4
Alternative text description – The image depicts a three line chart of the distribution of scores from 0-60 on the y-axis in relations to a five point Likert scores on the x-axis with the three lines measuring the Total Likert, health or changes in health as described in the previous paragraph. The line chart for total likert score 1 has a distribution score of ~58, likert score 2 has a distribution score of ~28, likert score 3 has a distribution score of ~10, likert score 4 has a distribution score of ~4, and likert score 5 has a distribution score of ~2. The line chart for health likert score 1 has a distribution score of ~5, likert score 2 has a distribution score of ~6, likert score 3 has a distribution score of ~4, likert score 4 has a distribution score of ~0, and likert score 5 has a distribution score of ~2. The line chart for changes in health score 1 has a distribution score of ~8, likert score 2 has a distribution score of ~5, likert score 3 has a distribution score of ~4, likert score 4 has a distribution score of ~1, and likert score 5 has a distribution score of ~0.
The final domain, Life Satisfaction, covered a wide range of issues with respect to the interviews. The overall ratio of positive to negative statements was 2.0 which indicated two positive statements for every negative one. The Likert-type scale mean score of 1.35 is again very positive, and the 2:1 ratio would appear to support this Likert score.
Figure 5 shows a scatter plot map of the interview mean positivity scores compared to the relevant domain mean Likert score. Lines indicating neutral positions (3 on the Likert axis and 1 on the positivity axis) have been included. While there visually appears to be a general tendency across all five domains for high Likert scores, this trend is not matched by the positivity scores which are distributed widely and it is difficult to discern any pattern.
 Figure 5 Chart - Scatter plot of Positivity Ratio versus Likert Score for each Domain

y-axis is the likert scores, ranging from 1 to 5
x-axis is the ratio of positivity, ranging from 0 to 3

There are 5 scattered points residence, support, health, changes in health, life satisfaction

residence scattered plot is at ~1.3 likert scale with a 1 ratio of positivity
changes in health scattered plot is at ~2 likert scale with ~.4 ratio of positivity
Support scattered plot is at ~ 1.3 likert scale with a ~2.6 ratio of positivity
health scattered plot is at ~2.2 likert scale with a ~1.3 ratio of positivity
life satisfaction scattered plot is at ~1.3 likert scale with a ~2 ratio of positivity 

Figure 5 Scatter Plot of Positivity Ratio Versus Likert Score for Each Domain
Alternative text description – The image depicts a XY chart that compares the ratio of positivity (0, 0.5, 1, 1.5, 2, 2.5 and 3) on the x-axis to a five point Likert score on the y-axis. The measurements were based on residence, support, health, changes in health, and life satisfaction.  As stated in the previous paragraph neutral positions were indicated with a solid line on the 3 on the Likert axis and 1 on the positivity axis. The changes in health has a Ratio of Positivity ~0.2 and likert score of ~less than 2; Residence has a Ratio of Positivity ~1 and likert score of ~1.2; Health has a Ratio of Positivity ~1.3 and likert score of ~2.2; Life Satisfaction has a Ratio of Positivity ~2 and likert score of ~1.2; and Support has a Ratio of Positivity ~2.6 and likert score of ~1.3;   
While the overall responses to interviews were generally skewed towards a positive score, the ratio of 1.48 positive statements to negative statements does not appear particularly strong. The responses on the Likert-type scales are far more positive than what would be predicted by the interview data, both overall and specifically in the domains. On the basis of these preliminary analyses, it would appear that there is a discrepancy between what study participants were telling the interviewer verbally, and what they are indicating on the Likert-type scales. 
An example of the contradictory nature of the results can be seen when comparing the third domain Health to the fourth domain Changes in Health. There was a very strong negative bias in the statements for Changes in Health, with a ratio of 0.26 positive statements for every negative one. However, the mean Likert score for this domain was 1.95 which represents a similar position to that seen in Health. With the sample being composed of older individuals who are likely to be experiencing increasing health problems associated with aging, the overall negativity of the interview responses in relation to changing health needs was to be expected. However, the strong positivity of the Likert-type scale for the domain does not seem consistent with the views expressed in the interviews.
Discussion
In the field of intellectual disability, Likert-type scales are often used as an alternative mechanism to seek the opinion of individuals who may not otherwise be able to verbalize their thoughts effectively. In this scenario, there is an implicit but largely untested belief that the two methods will yield equivalent results. The current study used two purpose-developed tools that specifically examined the same key domains. If the Likert-type scores from the current study were presented to a government funding body in isolation, it would appear that adults aging with an intellectual disability are generally very happy with all aspects of their lives and the support they receive. However, when the interview data is also included, the current project indicated little relationship between the Likert-type scale data and the overall positivity of the interviews, with a wide range of scores across the five domains.
The inconsistencies are demonstrated by the fact that the Likert-type scale data indicated that the perceived health of this cohort is actually improving as they get older. Direct comparison of the Likert-type scores versus the matched personal interview shows a number of individuals who quantitatively on the Likert-type scale indicated a perception that their health had improved as they aged, but qualitatively outlined a multitude of health conditions that had both emerged in recent years and had caused significant personal hardship.
It is speculated that there are a number of reasons for the discrepancy between the two measurements. One of the themes anecdotally evident in many of the interviews was an enhanced sense of self-esteem that emerged simply from being asked for their contribution. A number of participants informally commented that this was the first time they had ever been directly interviewed by ‘researchers’, and where their opinion was both sought and was seen to be valued. This sense of importance may have contributed to an over-estimation of scores within the Likert-type scales. 
Likert tools by their very nature may tend to capture a ‘snapshot’ of the participant at that particular point in time, whereas interviews are able to explore issues based on a series of statements encapsulating longer-term views. In terms of the health example, the individual may have felt ‘good’ on the day of the interview as a result of the external focus and attention being specifically paid to them, which led to higher reported scores on the Likert-type scale. However, in the course of the qualitative interview the individual and interviewer were able to discuss and consider other aspects of their health over a period of time. The individuals were able to more clearly articulate their actual health problems, and the summary ‘picture’ of their health then became far less positive. These findings concur with the work by Owen & Wilson that the abstract concept of ‘time’ presents difficulties for many people with intellectual disability; and a simple snapshot such as a Likert-type scale can further magnify this problem when considering changes that have occurred over a period of months or years (Owen & Wilson, 2006). 
The other important issue that needs to be considered is whether people with intellectual disability are conditioned to pick the ‘smiley face’ option. The concept of acquiescence bias in verbal interviews is well known (Rapley & Antaki, 1996), and it is entirely possible that participants are showing a similar trend of acquiescence towards picking the smiley face on a Likert-type scale even when it completely contradicts the information they have just verbally offered. The use of pictorial communication systems by individuals with intellectual disability and/or communication impairments has been commonplace both during and post-school for many decades (e.g. Mirenda & Locke, 1989; Stephenson & Linfoot, 1996). In fact, using pictorials such as smiley faces to supplement Likert-type scales has been recommended to overcome acquiescence bias (Heal & Sigelman, 1995). However, it is problematic if the smiley face itself is subject to potential issues of acquiescence. Concerns regarding acquiescence and relying purely on one data source have been previously identified as a serious issue for people with cognitive impairments (Beadle-Brown et al., 2012; Cuskelly, Moni, Lloyd & Jobling, 2013) and the results of the current investigation tend to support this position, however the small size of this pilot means that further research is required to better examine this possibility.
Limitations 
This was a pilot study, there are a number of acknowledged limitations, and any results need to be considered with caution. The study only looked at a total of 20 individuals, spread across two states of Australia. While this sample size is not necessarily small for qualitative interview data analysis, it is recognized to be lower than desirable when analyzing Likert-type scales responses. It is possible that the results seen in the current pilot may be diminished if the sample size was increased; however, the overall consistent positivity of the Likert responses would indicate that it is unlikely to disappear completely. 
As noted earlier, it is also acknowledged that the Likert-type scale used did not contain any ‘acquiescence’ questions to ascertain if questions were being answered illogically. This was deliberate, as many support organization surveys and workplace tools do not use such methodology to check the consistency of participant responses. In any subsequent research, it is recommended to include such questions as it may assist to begin differentiating between various cohort responses within the sample.
Implications
As acknowledged above, this is a pilot study and the findings therefore need to be considered with caution. However, the results of this research have potential implications for services that use Likert-type tools as an approximation of other methodologies, such as personal interview, in seeking contribution from individuals with an intellectual disability. This is a situation that may increasingly arise in Australia with the current movement towards a National Disability Insurance Scheme. As more people receive individualized funding packages and have a greater level of personal control, their direct participation in making decisions about their life should be vital. However, if the person is unable to clearly articulate their thoughts verbally, the question that arises is how to best support the individual to contribute. 
The current study provides preliminary data to indicate that Likert-type scales do not necessarily provide an equivalent result to an in-depth interview in a cohort of people able to respond in either format. It is also recognized that responding to an interviewer may have its own set of constraints, including acquiescence to an authority figure, capacity to read facial expressions and non-verbal cues, and so forth. Therefore, it is arguable that solely using any one tool to gain the input of people with intellectual disability or communication impairments is going to accurately represent their thoughts and desires. 
A practical example of the dilemma can be seen with voting on an enterprise agreement within an employment environment for people with intellectual disability. Likert-type tools are often used to legally gauge the level of support for a new workplace agreement. For example, Australian Disability Enterprises (ADE) provides workplace support for people with disabilities (Department of Social Services, 2014a). Many of these services have operated under a formal enterprise bargaining agreement and wage assessment system, and voting for these enterprise agreements can include both verbal and written responses. This approach is designed to facilitate the direct involvement of all affected individuals, which in the context of a specialist disability employment support service, can include persons with more severe intellectual disability and communication impairments. Tools that are used to assess understanding and agreement to an enterprise agreement can include simple Likert-type scales and / or short interview. All participants in the voting process are independently supported to ensure, as best as possible, that they understand the agreement under debate (Department of Social Services, 2014b). The results of the current study raise a question as to whether the responses solely gathered through Likert-type questionnaires can be considered to be equivalent to those individuals who verbally indicated their support, and therefore there could arguably be a legal question regarding the validity of the voting process.


The findings indicate a need to re-consider how participation from individuals with intellectual disability is supported, and how information is gathered. Current legislation and guidelines in Australia refers to the need to optimize communication to facilitate the personal participation of individuals with intellectual disability (e.g. Families & Community Services, 2014), but there are no details on exactly how this is to occur, or whether different approaches can be considered to provide equivalent outcomes. Whilst further research is required to better establish whether there is a significant discrepancy between information gathered via personal interview and other methodologies, the preliminary implications indicate a need to review government legislation to ensure all people have an equal opportunity to have their views accurately represented.   
Conclusions
With the recent introduction of wide-ranging changes to the way disability support is provided in Australia, there has been, and will continue to be, a very necessary and desirable attempt to increase the direct contribution of individuals with an intellectual disability in the development of their own programs and life goals. However, there is little research that examines whether differing methodologies of seeking this input produce equivalent outcomes. The current pilot study with 20 older adults with an intellectual disability found a significant difference between what was verbally indicated through interview and what was reported on a Likert-type scale. This discrepancy indicates that there may be problems with using either methodology as equivalent mechanisms for seeking the contribution of individuals with intellectual disability who may have communication impairments. It is acknowledged that this is a pilot study, and it is therefore recommended that further research be conducted with a larger sample size to see if these results are replicated.
Acknowledgements
This research was supported through funding from Illawarra Retirement Trust and the Disability Policy Research Working Group. The authors further acknowledge the contribution made by the Collaborative Research Network on Mental Health and Well-being in Rural Communities, supported by the Department of Industry, Innovation, Science, Research and Tertiary Education, Commonwealth Government of Australia.
Stuart Wark (PhD) is the Second Year Clinical Coordinator with the School of Rural Medicine at the University of New England in Armidale, Australia. He has a two decade working history in the human services fields, and retains strong relationships with both rural and metropolitan community-based Non-Government Organisations (NGO) and health providers. 
Miranda Cannon-Vanry is a PhD candidate with the School of Rural Medicine at the University of New England. She has a substantial work history within the fields of community-based health services, with a specific focus on the provision of appropriate support for individuals with lifelong intellectual disability.
Marie Knox (PhD) is a Senior Research Fellow with the Centre for Disability Studies at the University of Sydney, and also holds adjunct positions with the Faculty of Health at Queensland University of Technology and with the School of Rural Medicine at the University of New England. Marie has an extensive background in research in the disability field in both Queensland and NSW. In particular, she has undertaken collaborative projects with human service organisations and with the service users of human services. She has a sound knowledge of and expertise in notions of inclusion and disability, and has skills in pragmatic approaches to developing and implementing organisational policy initiatives to reflect these notions.
Marie Parmenter, until her retirement in 2009, was a Research Fellow at the Centre for Disability Studies, University of Sydney. Prior positions include Research Fellow in the Unit for Community Integration Studies at Macquarie University and District Officer with the New South Wales Department of Community Services. Marie continues part time research work in the areas of ageing and community inclusion.  Volunteer work includes assisting people with intellectual disabilities who are involved with the criminal justice system.
Rafat Hussain (PhD) is an Associate Professor with the ANU Medical School Centre and the Centre for Research on Ageing, Health & Wellbeing, Research School of Population Health at the Australia National University. She also holds an adjunct professorial position with the School of Rural Medicine at the University of New England. Rafat's considerable professional and academic experience spans clinical medicine, public health, and health services management. She has specific expertise across a range of disciplinary areas, and both teaches and researches in public and population health, health services research, health promotion and research methodologies.
Matthew P. Janicki (PhD) is an Associate Research Professor at the Department of Disability and Human Development in the College of Applied Health Sciences at the University of Illinois at Chicago, and also holds an adjunct professorial position with the School of Rural Medicine at the University of New England. Currently, he is leading a study of specialised group homes designed for dementia related care of adult with intellectual disabilities, serves as the co-chair of the US National Task Group on Intellectual Disabilities and Dementia Practices, and is the founding editor of the Journal of Policy and Practice in Intellectual Disabilities.
Chez Leggatt-Cook (PhD) is the Principal Researcher at UnitingCare Community in Queensland, Australia. She has expertise in disability, homelessness, social exclusion, human and social services and non-profit organisations. Her work is driven by a central focus on social justice, with a specific interest in using research to inform and drive best practice within organisations.
Meaghan Edwards is a PhD candidate at the Faculty of Medicine at the University of Sydney and a research fellow at the Centre for Disability Studies. In this role she conducts research and tutoring in the areas of person-centred planning for people with intellectual/developmental disability, quality of life, family quality of life, and inclusion. She also does work in the fields of community and social support.
Trevor Parmenter (PhD) held the Foundation Chair of Developmental Disability in the Sydney Medical School, University of Sydney, from its inception in 1997 and Director of the Centre for Disability Studies until his retirement in 2009. He is currently Professor Emeritus in the Sydney Medical School and Honorary Professor in the Faculties of Education and Social Work and Health Sciences at the University of Sydney; and Adjunct Professor in the School of Rural Medicine, University of New England. Former positions include Professorial Fellow and Director of the Unit for Community Integration Studies at Macquarie University. He is a Past President of the International Association for the Scientific Study of Intellectual and Developmental Disabilities (IASSIDD), and is the Honorary Chair of the 15th IASSIDD World Congress.
 




References
Ageing Disability and Home Care. (2013). Personal centred approaches - Living life my way.  Retrieved April 15, 2013 from www.adhc.nsw.gov.au/pca
Attorney General's Department of NSW. (2008). Capacity Toolkit. Sydney: New South Wales Government.
Beadle-Brown, J., Ryan, S., Windle, K., Holder, J., Turnpenny, A., Smith, N., et al. (2012). Engagement of people with long-term conditions in health and social care research: Barriers and facilitators to capturing the views of seldom heard populations.  QORU working paper 2849. Retrieved 26 June, 2014 from http://www.pssru.ac.uk/publication-details.php?id=4390
Beail, N., & Williams, K. (2014). Using qualitative methods in research with people who have intellectual disabilities. Journal of Applied Research in Intellectual Disabilities, 27(2), 85-96. 
Bigby, C., Frawley, P., & Ramcharan, P. (2014). Conceptualizing inclusive research with people with intellectual disability. Journal of Applied Research in Intellectual Disabilities, 27(1), 3-12. 
Boland, M., Daly, L., & Staines, A. (2008). Methodological issues in inclusive intellectual disability research: A health promotion needs assessment of people attending Irish disability services. Journal of Applied Research in Intellectual Disabilities, 21(3), 199-209. 
Bonyhady, B. (2014). Tides of change: The NDIS and its journey to transform disability support. The Australian Journal on Psychosocial Rehabilitation, 7(1), 7-9. 
Boudiny, K. (2013). ‘Active ageing’: From empty rhetoric to effective policy tool. Ageing & Society, 33(06), 1077-1098. 
Brown, R. (1993). Quality of life issues in ageing and intellectual disability. Australian and New Zealand Journal of Developmental Disabilities, 18(4), 219-228. 
Brown, R., Schalock, R. L., & Brown, I. (2009). Quality of life: Its application to persons with intellectual disabilities and their families - introduction and overview. Journal of Policy and Practice in Intellectual Disabilities, 6(1), 2-6. 
Burnett, P. (1989). Assessing satisfaction in people with an intellectual disability living in community-based residential facilities. Australian Disability Review, 1, 14-19. 
Chun Yu, A., Jupp, J., & Taylor, A. (1996). The discriminant validity of the Lifestyle Satisfaction Survey (LSS) for the assessment of Australian adults with intellectual disabilities. Journal of Developmental and Intellectual Disabilities, 21(1), 3-15. 
Commonwealth Government of Australia. (2013). National Disability Insurance Scheme.  Retrieved 11 June, 2013 from http://www.ndis.gov.au/
Conroy, J., & Feinstein, C. (1990). Measuring quality of life: Where have we been, where are we going? In R. Schalock, & Begab, M., (Ed.), Quality of life: Perspectives and issues (pp. 227-234). Washington, D.C.: American Association on Mental Retardation.
Coppus, A. M. (2013). People with intellectual disability: What do we know about adulthood and life expectancy? Developmental Disabilities Research Reviews, 18, 6-16. 
Cummins, R. A. (1997). Self-rated quality of life scales for people with an intellectual disability: A review. Journal of Applied Research in Intellectual Disabilities, 10(3), 199-216. 
Cummins, R. A., & Lau, A. L. (2010). Personal Wellbeing Index – Intellectual Disability (English), 3rd Edn. Melbourne: Australian Centre on Quality of Life: Deakin University.
Cuskelly, M., Moni, K., Lloyd, J., & Jobling, A. (2013). Reliability of a method for establishing the capacity of individuals with an intellectual disability to respond to Likert scales. Journal of Intellectual and Developmental Disability, 38(4), 318-324. 
Department of Social Services. (2014a). Australian Disability Enterprises. Retrieved 24 June, 2014 from http://www.dss.gov.au/our-responsibilities/disability-and-carers/program-services/for-service-providers/australian-disability-enterprises
Department of Social Services. (2014b). Wage assessments in Australian Disability Enterprises. Retrieved 24 June, 2014 from http://www.dss.gov.au/our-responsibilities/disability-and-carers/program-services/for-people-with-disability/wage-assessments-in-australian-disability-enterprises
Emerson, E., Felce, D., & Stancliffe, R. J. (2013). Issues concerning self-report data and population-based data sets involving people with intellectual disabilities. Intellectual and Developmental Disabilities, 51(5), 333-348. 
Felce, D., & Perry, J. (1997). Quality of life: The scope of the term and its breadth of measurement. In R. Brown (Ed.), Quality of life for people with disabilities: Models, research and practice (pp. 56-71). London: Stanley Thornes.
Finlay, W. M., & Lyons, E. (2001). Methodological issues in interviewing and using self-report questionnaires with people with mental retardation. Psychological Assessment, 13(3), 319-335. 
Finlay, W. M. L., & Lyons, E. (2002). Acquiescence in interviews with people who have mental retardation Mental Retardation, 40(1), 14-29. 
Hartley, S. L., & MacLean, W. E. (2006). A review of the reliability and validity of Likert-type scales for people with intellectual disability. Journal of Intellectual Disability Research, 50(11), 813-827. 
Heal, L. W., & Sigelman, C. K. (1995). Response biases in interviews of individuals with limited mental ability. Journal of Intellectual Disability Research, 39(4), 331-340. 
Kober, R., & Eggleton, I. R. C. (2009). Using quality of life to evaluate outcomes and measure effectiveness. Journal of Policy and Practice in Intellectual Disabilities, 6(1), 40-51. 
Makai, P., Brouwer, W., Koopmanschap, M., Stolk, E., & Nieboer, A. P. (2014). Quality of life instruments for economic evaluations in health and social care for older people: A systematic review. Social Science & Medicine, 102, 83-93. 
Mirenda, P., & Locke, P. (1989). A comparison of symbol transparency in nonspeaking persons with intellectual disabilities. Journal of Speech and Hearing Disorders, 54, 131-140. 
National Disability Insurance Agency. (2014). Planning and Assessment - Facilitating the Participant's Statements of Goals and Aspirations. Retrieved 24 June, 2014 from http://www.ndis.gov.au/document/325
Nind, M. (2011). Participatory data analysis: A step too far? Qualitative Research, 11(4), 349-363. 
Owen, A. L., & Wilson, R. R. (2006). Unlocking the riddle of time in learning disability. Journal of Intellectual Disabilities, 10(1), 9-17. 
Parmenter, T. (2014). NDIS and the aged care system: Assessing the capability of the aged care sector to support people with disability. Paper presented at the 5th Annual National Disability Summit 2014, Melbourne, 14 March 2014. http://www.slideshare.net/informaoz/trevor-parmenter
Parmenter, T., Hussain, R., Janicki, M., Knox, M., Leggatt-Cook, C., Wark, S., et al. (2013). Enablers and impediments to successful ageing-in-place. Paper presented at the 48th Australasian Society for Intellectual Disability Conference, Sydney, 6 November 2013.
Perales, J., Cosco, T. D., Stephan, B. C. M., Haro, J. M., & Brayne, C. (2013). Health-related quality-of-life instruments for Alzheimer's disease and mixed dementia. International Psychogeriatrics, 25(5), 691-706. 
Rapley, M., & Antaki, C. (1996). A conversation analysis of the 'acquiescence' of people with learning disabilities. Journal of Community and Applied Social Psychology, 6(3), 207-227. 
Schmidt, S., Power, M., Green, A., Lucas-Carrasco, R., Eser, E., Dragomirecka, E., et al. (2010). Self and proxy rating of quality of life in adults with intellectual disabilities: Results from the DISQOL study. Research in Developmental Disabilities, 31(5), 1015-1026. 
Sigstad, H. M. H. (2014). Characteristic interviews, different strategies: Methodological challenges in qualitative interviewing among respondents with mild intellectual disabilities. Journal of Intellectual Disabilities, 18(2), 188-202. 
Stancliffe, R. J. (1995). Assessing opportunities for choice-making: A comparison of self- and staff reports. American Journal on Mental Retardation, 99(4), 418-429. 
Stephenson, J., & Linfoot, K. (1996). Pictures as communication symbols for students with severe intellectual disability. Augmentative and Alternative Communication, 12(4), 244-256. 
Walmsley, J. (2001). Normalisation, emancipatory research and inclusive research in learning disability. Disability & Society, 16(2), 187-205. doi: 10.1080/09687590120035807
Wark, S. (2011). Implication of quality of life measurement on ageing with intellectual disability. Intellectual Disability Australasia, 32(4), 3-7. 
Wark, S., Hussain, R., & Edwards, H. (2013a). Ageing with an intellectual disability: Support issues in rural localities. Paper presented at the 12th National Rural Health Conference, Adelaide, 10 April 2013. http://nrha.org.au/12nrhc/wp-content/uploads/2013/06/Wark-Stuart_ppr.pdf
Wark, S., Hussain, R., & Edwards, H. (2013b). Rural and remote area service provision for people aging with intellectual disability. Journal of Policy and Practice in Intellectual Disabilities, 10(1), 62-70.
Wark, S., Hussain, R., & Edwards, H. (2014a). Impediments to community-based care for people ageing with intellectual disability in rural New South Wales. Health & Social Care in the Community, 22(6): 623-633.
Wark, S., Hussain, R., & Edwards, H. (2014b). The training needs of staff supporting individuals ageing with intellectual disability. Journal of Applied Research in Intellectual Disabilities, 27(3), 273-288. 






	Page