Intraclass Correlation Coefficient

Please check the following documents.

It can run on SPSS, but data cannot be interpreted about kappa, weighted kappa, ICC, consistency, and agreement.

FIM_TOTAL1 and FIM_TOTAL1R

Correlations
		FIM_TOTAL1	FIM_TOTAL1R
FIM_TOTAL1	Pearson Correlation	1	.912^**
	Sig. (2-tailed)		.000
	N	209	209
FIM_TOTAL1R	Pearson Correlation	.912^**	1
	Sig. (2-tailed)	.000
	N	209	209
**. Correlation is significant at the 0.01 level (2-tailed).

There is a significant strong positive correlation between FIM_TOTAL1and FIM_TOTAL1R

RAI_TOTAL1 and RAI_TOTAL1R

Correlations
		RAI_TOTAL1	RAI_TOTAL1R
RAI_TOTAL1	Pearson Correlation	1	.935^**
	Sig. (2-tailed)		.000
	N	209	209
RAI_TOTAL1R	Pearson Correlation	.935^**	1
	Sig. (2-tailed)	.000
	N	209	209
**. Correlation is significant at the 0.01 level (2-tailed).

There is a significant strong positive correlation between RAI_TOTAL1and RAI_TOTAL1R

SAMP WEEK 6

Calculate Intraclass Correlation Coefficient (ICC) and Kappa and Weighted Kappa:

Calculate the ICC for the following pairs of measures (In SPSS, use: Analyze > Scale > Reliability Analysis):

FIM_TOTAL1 and FIM_TOTAL1R
RAI_TOTAL1 and RAI_TOTAL1R
SRH1 and SRH1R
QoL1 and QoL1R
In these calculations, consider whether you are interested in “consistency” or “agreement” as discussed in the course text (Streiner, Norman and Cairney, p. 167-169), and whether you are considering the times of the ratings (time 1 and time 2) as fixed or random factors.
Compare the ICC values with the Pearson’s correlations obtained in Week 1; discuss how would you interpret the differences in the coefficients.

Calculate Kappa:

In SPSS, for Kappa, use: Analyze > Crosstabs
Take the admission and discharge scores for two individual items from the RAI or FIM scales (e.g., RAI_WALK1 and RAI_WALK2) and calculate kappa.
Calculate kappa for SRH1 and SRH1R
Use quadratic weights for weighted kappa. Briefly discuss differences in results between weighted and unweighted kappa.

Calculate Weighted Kappa:

Calculate weighted kappa for the same items used above (RAI and/or FIM items, SRH)
In SPSS, use: Analyze > Crosstabs, to create a table
Using the values from the table, and using a quadratic weighting scheme, calculate weighted kappa using the utility in the VassarStats website.

Part 1 calculations

FIM_TOTAL1 and FIM_TOTAL1R

Correlations
		FIM_TOTAL1	FIM_TOTAL1R
FIM_TOTAL1	Pearson Correlation	1	.912^**
	Sig. (2-tailed)		.000
	N	209	209
FIM_TOTAL1R	Pearson Correlation	.912^**	1
	Sig. (2-tailed)	.000
	N	209	209
**. Correlation is significant at the 0.01 level (2-tailed).

There is a significant strong positive correlation between FIM_TOTAL1and FIM_TOTAL1R

RAI_TOTAL1 and RAI_TOTAL1R

Correlations
		RAI_TOTAL1	RAI_TOTAL1R
RAI_TOTAL1	Pearson Correlation	1	.935^**
	Sig. (2-tailed)		.000
	N	209	209
RAI_TOTAL1R	Pearson Correlation	.935^**	1
	Sig. (2-tailed)	.000
	N	209	209
**. Correlation is significant at the 0.01 level (2-tailed).

There is a significant strong positive correlation between RAI_TOTAL1and RAI_TOTAL1R

Solution

SAMP WEEK 6

Part 1 calculations

FIM_TOTAL1 and FIM_TOTAL1R

Correlations
		FIM_TOTAL1	FIM_TOTAL1R
FIM_TOTAL1	Pearson Correlation	1	.912^**
	Sig. (2-tailed)		.000
	N	209	209
FIM_TOTAL1R	Pearson Correlation	.912^**	1
	Sig. (2-tailed)	.000
	N	209	209
**. Correlation is significant at the 0.01 level (2-tailed).

There is a significant strong positive correlation between FIM_TOTAL1and FIM_TOTAL1R

Intraclass Correlation Coefficient
	Intraclass Correlation^b	95% Confidence Interval		F Test with True Value 0
	Intraclass Correlation^b	Lower Bound	Upper Bound	Value	df1	df2	Sig
Single Measures	,909^a	,882	,930	21,145	208	208	,000
Average Measures	,952^c	,938	,964	21,145	208	208	,000
Two-way mixed effects model where people effects are random and measures effects are fixed.
a. The estimator is the same, whether the interaction effect is present or not.
b. Type A intraclass correlation coefficients using an absolute agreement definition.
c. This estimate is computed assuming the interaction effect is absent, because it is not estimable otherwise.

Two-way mixed model as observations are random, but raters are not.

Type is absolute agreement, as systematic differences are relevant.

The Pearson’s correlation coefficient is pretty close to single measure ICC, which is significant and excellent as its far above 0.75.

RAI_TOTAL1 and RAI_TOTAL1R

Correlations
		RAI_TOTAL1	RAI_TOTAL1R
RAI_TOTAL1	Pearson Correlation	1	.935^**
	Sig. (2-tailed)		.000
	N	209	209
RAI_TOTAL1R	Pearson Correlation	.935^**	1
	Sig. (2-tailed)	.000
	N	209	209
**. Correlation is significant at the 0.01 level (2-tailed).

There is a significant strong positive correlation between RAI_TOTAL1and RAI_TOTAL1R

Intraclass Correlation Coefficient
	Intraclass Correlation^b	95% Confidence Interval		F Test with True Value 0
	Intraclass Correlation^b	Lower Bound	Upper Bound	Value	df1	df2	Sig
Single Measures	,930^a	,903	,949	29,764	208	208	,000
Average Measures	,964^c	,949	,974	29,764	208	208	,000
Two-way mixed effects model where people effects are random and measures effects are fixed.
a. The estimator is the same, whether the interaction effect is present or not.
b. Type A intraclass correlation coefficients using an absolute agreement definition.
c. This estimate is computed assuming the interaction effect is absent, because it is not estimable otherwise.

Two-way mixed model as observations are random, but raters are not.

Type is absolute agreement, as systematic differences are relevant.

The Pearson’s correlation coefficient is pretty close to single measure ICC, which is significant and excellent as its far above 0.75.

(single measure is for single rater, average measure is higher and is for reliability of different raters averaged together)

Correlations
		SRH1	SRH1R
SRH1	Pearson Correlation	1	,837^**
	Sig. (2-tailed)		,000
	N	209	209
SRH1R	Pearson Correlation	,837^**	1
	Sig. (2-tailed)	,000
	N	209	209
**. Correlation is significant at the 0.01 level (2-tailed).

Intraclass Correlation Coefficient
	Intraclass Correlation^b	95% Confidence Interval		F Test with True Value 0
	Intraclass Correlation^b	Lower Bound	Upper Bound	Value	df1	df2	Sig
Single Measures	,834^a	,788	,871	11,147	208	208	,000
Average Measures	,910^c	,881	,931	11,147	208	208	,000
Two-way mixed effects model where people effects are random and measures effects are fixed.
a. The estimator is the same, whether the interaction effect is present or not.
b. Type A intraclass correlation coefficients using an absolute agreement definition.
c. This estimate is computed assuming the interaction effect is absent, because it is not estimable otherwise.

Two-way mixed model as observations are random, but raters are not.

Type is absolute agreement, as systematic differences are relevant.

The Pearson’s correlation coefficient is pretty close to single measure ICC, which is significant and excellent as its far above 0.75.

Correlations
		QoL1	QoL1R
QoL1	Pearson Correlation	1	,905^**
	Sig. (2-tailed)		,000
	N	209	209
QoL1R	Pearson Correlation	,905^**	1
	Sig. (2-tailed)	,000
	N	209	209
**. Correlation is significant at the 0.01 level (2-tailed).

Intraclass Correlation Coefficient
	Intraclass Correlation^b	95% Confidence Interval		F Test with True Value 0
	Intraclass Correlation^b	Lower Bound	Upper Bound	Value	df1	df2	Sig
Single Measures	,890^a	,847	,919	18,495	208	208	,000
Average Measures	,942^c	,917	,958	18,495	208	208	,000
Two-way mixed effects model where people effects are random and measures effects are fixed.
a. The estimator is the same, whether the interaction effect is present or not.
b. Type A intraclass correlation coefficients using an absolute agreement definition.
c. This estimate is computed assuming the interaction effect is absent, because it is not estimable otherwise.

Two-way mixed model as observations are random, but raters are not.

Type is absolute agreement, as systematic differences are relevant.

The Pearson’s correlation coefficient is pretty close to single measure ICC, which is significant and excellent as its far above 0.75.

*RAI_WALK1 RAI_WALK2 Crosstabulation**
Count
		RAI_WALK2							Total
		0	1	2	3	4	5	6	Total
RAI_WALK1	0	25	0	0	1	0	0	0	26
	1	6	1	0	0	0	0	0	7
	2	67	9	7	0	1	0	1	85
	3	8	5	2	0	0	0	0	15
	4	23	4	4	0	1	0	1	33
	5	9	5	6	2	0	3	0	25
	6	7	2	4	1	2	0	2	18
Total		145	26	23	4	4	3	4	209

Symmetric Measures
		Value	Asymptotic Standardized Error^a	Approximate T^b	Approximate Significance
Measure of Agreement	Kappa	,051	,022	2,418	,016
N of Valid Cases		209
a. Not assuming the null hypothesis.
b. Using the asymptotic standard error assuming the null hypothesis.

Unweighted Kappa is poor, but significant 0.051.

Kappa with Quadratic Weighting based on atatched webpage is 0.1852.

Weighted kappa are meaningful only if the categories are ordinal and if the weightings ascribed to the categories faithfully reflect the reality of the situation.

Whereas unweighted kappa does not distinguish among degrees of disagreement, weighted kappa incorporates the magnitude of each disagreement and provides partial credit for disagreements when agreement is not complete.

*SRH1 SRH1R Crosstabulation**
Count
		SRH1R					Total
		1	2	3	4	5	Total
SRH1	1	8	11	2	0	0	21
	2	3	40	9	0	0	52
	3	1	8	33	15	1	58
	4	0	1	8	31	9	49
	5	0	0	0	12	17	29
Total		12	60	52	58	27	209

Symmetric Measures
		Value	Asymptotic Standardized Error^a	Approximate T^b	Approximate Significance
Measure of Agreement	Kappa	,503	,043	13,679	,000
N of Valid Cases		209
a. Not assuming the null hypothesis.
b. Using the asymptotic standard error assuming the null hypothesis.

Unweighted Kappa is moderate, and strongly significant.

Kappa with Quadratic Weighting is 0.8338, which is very good.

Value of K	Strength of agreement
< 0.20	Poor
0.21 – 0.40	Fair
0.41 – 0.60	Moderate
0.61 – 0.80	Good
0.81 – 1.00	Very good