PSYC 2002: Linear Regression

Contents

1. 1. Formula for Linear Regression (a.k.a, Line of Best Fit)
1. 1.1. Unexplained and Explained Variation around the Regression Line
2. 1.2. Coefficient of Determination
3. 1.3. Coefficient of Non-Determination
4. 1.4. Standard of Error of Estimate
5. 1.5. Estimated Population Error Variance
6. 1.6. Linear Regression with Z-Scores

1. Linear Correlation

Linear regression uses past information about the relationship between variables to make predictions. If we find that two varaibles are linearly related, we can use the slope equation to predict one varaible (the criterion variable) from the other (the predictor variable).

The line of best fit (a.k.a., least-squares regression line) minimizes the squared distance from each ($x, y$) coordinate to your line. If you remember back to elementary school, the slope equation of a line is calculated as:

$y = mx + b$

The regression equation is similarly calculated as:

$Y^\prime = b_{Y}X + a_{Y}$

Where

• $Y^\prime$ is the predicted $Y$
• $b_{Y}$ is the slope (the change in $y$ per change in $x$)
• $X$ is the predictor variable
• $a_{Y}$ is the y-intercept (where the regression line intercepts with the y-axis)

Often, we'lll be asked to figure out the regression line equation:

1. Calculate slope (use 1 of 2 ways):
   1. Based on the correlation coefficient and the standard deviation of $x$ and $y$:

      $\displaystyle b_{Y} = r_{xy} (\frac{S_{y}}{S_{x}}) $

   2. Based on the quotient of the cross product divided by the $x$ sum of squares:

      $\displaystyle b_{Y} = \frac{SS_{xy}}{SS_{x}} $

      $\displaystyle b_{Y} = \frac{\sum (X - \bar X)(Y - \bar Y)}{\sum (X - \bar X)^2} $

2. Calculate the y-intercept by rearranging the original regression line formula, using the means of $x$ and $y$:

   $\displaystyle a_{Y} = \bar Y - b_{Y} \bar X$

1.1. Unexplained and Explained Variation around the Regression Line

Once we have our regression line equation, we can determine how awesome our predictions are by calculating the total variation: explained variation plus unexplained variation.

total variation = explained variation + unexplained variation

$SS_{tot} = SS_{reg} + SS_{error}$

$\displaystyle \sum (Y - \bar Y)^2 = \sum (Y^\prime - \bar Y)^2 + \sum (Y - Y^\prime)^2$

To make this clear, you have three points when you calculate the regression equation: actual $x$, actual $y$ and predicted $y$.

• The total error is equal to the sum of squares for the actual $y$ values minus the mean of $y$.
• The regression error (explained variation) is equal to the sum of squares for the predicted $y$ minus the mean of $y$.
• The unexplained error (unexplained variation) is equal to the sum of squares for the actual $y$ values minus the predicted $y$ values.

1.2. Coefficient of Determination

If we need to know the coefficient of determination, or the proportion of variance in one variable that can be explained by another variable, we simple need to divide the explained error in $y$ by the total error:

$\displaystyle r^2 = \frac{SS_{reg}}{SS_{tot}} = \frac{\sum (Y^\prime - \bar Y)^2}{\sum (Y - \bar Y)^2} $

1.3. Coefficient of Non-Determination

If we need to know the coefficient of non-determination (a.k.a., alienation), or the proportion of variance in one variable that cannot be explained by another variable, we simply need to divide the unexplained error in $y$ by the total error:

$\displaystyle 1 - r^2 = \frac{SS_{err}}{SS_{tot}} = \frac{\sum (Y - Y^\prime)^2}{\sum (Y - \bar Y)^2} $

1.4. Standard of Error of Estimate

Because the regression line minimizes the sum of squared errors of prediction, we can use the square root of the average squared error of prediction to determine the accuracy of the prediction. This gives us the standard deviation of the points around the regression line. This is known as the standard error of estimate and is calculated as:

$\displaystyle S_{Y.X} = \sqrt{\frac{SS_{err}}{N}} = \sqrt{\frac{\sum (Y - Y^\prime)^2}{N}} $

If we just want to know the variance of the points around the line, we would calculate it without the square root:

$\displaystyle S_{Y.X}^2 = \frac{SS_{err}}{N} = \frac{\sum (Y - Y^\prime)^2}{N} $

1.5. Estimated Population Error Variance

We can calculate the estimated population error variance by dividing the unexplained variance by degrees of freedom (instead of $N$) (Note that $S$ actually have a hat. This means "esimate".):

$\displaystyle \hat S_{Y.X} = \sqrt{\frac{\sum (Y - Y^\prime)^2}{N - 2}} $

1.6. Linear Regression with Z-Scores

If we are working with standardized z-scores, we can simplify our equation a little. Because of the y-intercept with z-scores is always 0, we multiply the z-score covariance by the predictor variable $Z_{x}$:

$\displaystyle Z_{Y}^\prime = r_{xy}Z_{X} $

This expands to:

$\displaystyle Z_{Y}^\prime = (\frac{\sum Z_{x}Z{y}}{N})Z_{X} $

PSYC 2002: Correlation

Contents

1. 1. Linear Correlation
1. 1.1. Covariance and Linear Correlation - Formulas Explained!
2. 1.2. Understanding r

1. Linear Correlation

Correlational designs allow us to measure each participant on each variable and find a relationship between two variables (which is calculated using a correlational coefficient and a scatterplot). This design looks for a relationship between two or more variables. Remember, you cannot infer causation.

The correlational coefficient, $r$, is a statistic used to quantitatively express the extent to which two variables are related, and the direction of that relationship (know when to use these):

• (Nominal - same) Phi coefficient uses two dichotomous, nominal variables.
• (Nominal - not same) Bi-serial $r$ uses nominal variables (they don't have to be dichotomous).
• (Ordinal - ranks) Spearman $r$ uses ordinal variables and must be expressed as ranks.
• (Ordinal - allows ties) Kendall's Tau uses ordinal data when the data allows you to have ties.
• (Interval-ratio) Pearson's $r$ is for interval-ratio data (Pearson product moment correlation coefficient).

1.1. Covariance and Linear Correlation - Formulas Explained!

The cross product sum of squares (or "sum of cross product" or "cross product") is equal to the sum of the product of the $x$ and $y$ deviation scores. This indicates the degree to which two random variables ($x$ and $y$) covary. This is represented as:

$\displaystyle {SS}_{xy} = \sum{(X - \bar{X})(Y - \bar{Y})}$

Because the cross product varies depending on the scale (e.g., you would get a different result if you used a 1-10 scale versus a 1-100 scale), you can adjust the cross product by dividing it by the sample population, $N$. Dividing by the sample population normalizes the result.

This gives you the unstandardized covariance of $x$ and $y$:

$\displaystyle S_{xy} = \frac{\sum{(X - \bar{X})(Y - \bar{Y})}}{N}$

A linear correlation is the standardized version of covariance. This value allows us to compare covariances for different $x$ and $y$ pairs (much like how z-scores allow us to compare raw scores from different sized samples). It should be obvious that just like covariance, the linear correlation indicates a linear relationship between two variables - the extent to which a change in one variable ($x$, for example) is linearly related to a change in another variable ($y$, for example).

$\displaystyle r_{xy} = \frac{S_{xy}}{S_x S_y}$

Which, when expanded, looks like this:

$\displaystyle r_{xy} = \frac{\dfrac{\sum (X - \bar{X})(Y - \bar{Y})}{N}}{\sqrt{\dfrac{(X - \bar{X})^2}{N}} \sqrt{\dfrac{(Y - \bar{Y})^2}{N}}}$

OR in other words:

$\displaystyle \text{linear correlation} = \frac{\text{covariance of X and Y}}{\sqrt{\text{X variance}} \sqrt{\text{Y variance}}}$

We can further factor out the N (same shit, faster to calculate) and we get the linear correlation as:

$\displaystyle r_{xy} = \frac{\sum{(X - \bar{X})(Y - \bar{Y})}}{\sqrt{{(X - \bar{X})}^2{(Y - \bar{Y})}^2}}$

And (even better!) if you happen to have the Z-scores already calculated, you can compute the covariance between the z-scores using a simplified formula:

$\displaystyle r_{xy} = \sum{\frac{Z_x Z_y}{N}}$

1.2. Understanding $r$

Once we have calculated the correlation coefficient, $r$, we have to interpret it:

1. The direction can be upward ($+r$), downward ($-r$), or non-existent ($r = 0$).
2. The nature (form) of the relationship (linear) (Pearson's $r$) or curved (Spearman's $rho$).
3. The consistency or strength. The closer $r$ is to 1, the more consistent the relationship (i.e., the closer the data points appear on the scatter plot).
   • Even if $r = 0$, there may be a relationship - it may just be curvilinear.

Be careful! There are 3 factors that can affect $r_{xy}$:

1. Nonlinearity: if the relationship between two variables is curvilinear, $r$ may be low. Statistical computation is not a substitute for actually looking at the graph.
2. Outliers: single outliers can have a major impact on the size of the correlation (just like outliers can have a major impact on your mean and standard deviation, it should be no surprise that it would affect a statistic that uses standard deviation and mean in it's formula.)
3. Range Restriction: if the range of the $x$ or $y$ values are restricted, the 'true' correlation may be larger than what you computed.

We can use the coefficient of determination ($r^2$ or $r_{xy}^2$) to determine the strength of the relationship between two variables. This is the proportion of variance in one variables that can be explained by the variance in another variable. Naturally, any squared value must be positive, so $0 \leq r_{xy}^2 \leq 1$.

• E.g., if we have a correlation coefficient $r = 0.80$, we calculate $r_{xy}^2 = 0.64$ and finally $1 - r^2 = 1 - 0.64 = 0.34$.
  This says that 34% of the variability in one variable cannot be explained by the variability in another.

In social sciences, we normally use this as a guide (but it's a loose guide! Some populations, like criminals, even a 0.30 correlation is viewed as a major find - so check with your peers and their findings first).

• Small correlation: $0 \leq r \leq 0.30$
• Medium correlation: $0.30 \leq r \leq 0.50$
• Large correlation: $0.50 \leq r \leq 1$

PSYC 2002: Descriptive Statistics

Contents

1. 1. Measures of Central Tendency
2. 2. Measures of Variability (Dispersion or Spread)
3. 3. Transformations
   1. 3.1. Linear Transformations
   2. 3.2. Non-Linear Transformations
4. 4. Standard Normal Distribution
   1. 4.1. Standard Scores (Z-Scores)
   2. 4.2. Percentile Rank

1. Measures of Central Tendency

Measures of central tendency describe the most typical value in the distribution. There are three measures:

1. Mode: indicates the most frequently occurring category or the category that contains the most data values. Used primarily for nominal data but can be used with others.
   • Unimodal: 1 mode.
   • Bimodal: 2 modes.
   • Multimodal: many models.
   • Rectangle/Uniform: no modes (all the scores have the same frequency).
2. Median: indicates the data point that falls in the middle of a ranked data set (i.e., 50^th percentile, $X_2$ or the second quartile, $Q_2$). Used primarily for ordinal data. Can be useful with skewed interval and ratio data.
   • Order the numbers from the smallest to largest and find the middle number. If there are two middle numbers, calculate the middle of the two.
   • Equation:
     
     $\displaystyle \text{median location} = \frac{N+1}{2}, \text{where N is the number of your sample}$
3. Median: indicates the average value for a data set. Used for interval and ratio data.
   • Equation: where x-bar is equal to the sum of each value ($X$), divided by the number of values.
     $\displaystyle \bar{X} = \frac{\sum X}{N}$
     • Summed Deviations: the summed deviations (the distance from any point to the mean) will always equal zero $\sum(X - \bar{X}) = 0$.
     • The Sum of the Squared Deviations (a.k.a., concept of least squares) gives you the unadjusted measure of dispersion. $\sum(X - \bar{X}^2 )$ (This is ${SS}_x$... you'll use this in the Linear Regression post.)
   • Weighted Mean: calculated as the overall mean for two or more different samples. The weight is calculated as the data set with more scores. X-bar is equal to the sum of each value ($i$) multiplied by the weight ($a$), divided by the sum of the weights.
     
     $\displaystyle \bar{X} = \frac{\sum_{i=1}^k a_i X_i}{\sum_i^k a_i}$
   • Mean, median and mode relationships: in single-peaked, symmetrical distributions, the mean, median and mode are all int he same place (the middle). In skewed distributions, the mode is at the peak, the median is in the middle (duhh), and the mean is pulled towards the skew (i.e., the tail).

2. Measures of Variability (Dispersion or Spread)

There are four measures of variability:

1. Range (R): the difference between the highest and lowest scores.
   • Issues: sensitive to extreme scores and doesn't provide much information.
2. Interquartile Range (IQR or IR): describes the variance (spread) of the data set by removing the outliers. It is calculated as the difference between the 75^th ($Q_3$) and the 25^th ($Q_1$) scores. $Q_2$ is the median (or 50^th) percentile.
   • $IQR = Q_3 - Q_1$
     $\text{Where } Q_3 = \text{values of } X_.75 = (N + 1)(.75)$
     $\text{And } Q_1 = \text{value of } X_.25 = (N + 1)(.25)$
   • Issue: removes the extreme scores, which may be important.
3. Variance Estimate: because $SS_x$ is affected by $N$, we can correct for this difference by calculating the variance.
   
   $\displaystyle S_{x}^2 = \frac{SS_x}{N} = \frac{\sum (X - \bar{X})^2}{N}$
   • Unbiased Variance Estimate: provides a sample of the population variance (i.e., systematically underestimates the value of the population variance). In order to obtain an unbiased estimate of the population variance (variation estimate), divide $SS_x$ by the degrees of freedom instead of N.
     
     $\displaystyle \hat{S}_x^2 = \frac{SS_x}{df} = \frac{SS_x}{N-1} = \frac{\sum (X - \bar{X})^2}{N-1}$
4. Standard Deviation ($S_X$): calculate the standard deviation by squaring the variance.
   
   $\displaystyle \text{Standard Deviation} = \sqrt{\text{Variance}}$
   
   
   $\displaystyle S_x = \sqrt{S_x^2} = \sqrt{\frac{{SS}_x}{N}} = \sqrt{\frac{\sum (X - \bar{X})^2}{N}}$
   • Deviation Score, or distance of a score from the mean, is equal to $X - \bar{X}$; the sum of these scores is always zero.
   • Sum of Squares is equal to ${SS}_x = \sum (X - \bar{X})^2$

These are the roles to follow in order to select a measure of variability:

1. When mean is used as the measure of central tendency, standard deviation should be used as the measure of variability.
2. When median is used as the measure of central tendency, interquartile range should be used as the measure of variability.

3. Transformations

3.1. Linear Transformations

Linear transformations use a constant, which changes the mean, standard deviation, but not the shape. There are two ways to manipulate a dataset linearly:

• Add/Subtract: calculate the sum, $b$, to all the scores.
  • The mean changes by the same constant:
    if $X + b \text{, then } \bar{X} + b$
  • The standard deviation does not change.
• Multiply/Divide: calculate the product or quotient, $m$ of all the scores.
  • The mean changes by the same factor:
    if $X \times m \text{, then } \bar{X} \times m$
  • The standard deviation changes by the same factor:
    if $X \times m \text{, then } S_x = S_x \times m$

3.2. Non-Linear Transformations

Non-linear transformations use a square root or a $log$, which changes the mean, standard deviation, and the shape. The scores are affected non-uniformly.

• Useful to reduce the skew of the data, which allows you to manipulate and interpret your data more. However, some argue that it distorts the scale.

4. Standard Normal Distribution

The Standard Normal Distribution is a mesokurtic curve made up of z-scores. It has a mean of 0 and a standard deviation of 1. It allows us to calculate the proportion of area below specific sections of the curve (namely, $Q_1$ to $Q_4$).

• We know that 50% of the scores are above and 50% of the scores are below the mean.
• -1 to +1 standard deviations from the mean is 68.26% of the scores (31.13% above and below).
• -2 to +2 standard deviations from the mean is 95.44% of the scores (the above values plus 13.59% above and below).
• -3 to +3 standard deviations from the mean is 99.74% of the scores (the above values plus 21.15% above and below).
• The rest makes up the last 0.26% (0.13% above and below).

4.1. Standard Scores (Z-Scores)

Z-Score: describes the number of standard deviations any score is above or below the mean. It is calculated by dividing the deviation score (score minus mean) by the standard deviation.
E.g., A Z-score of -1.1 means that the score is 1.1 standard deviations below the mean.

$\displaystyle \text{Sample: } z = \frac{X - \bar{X}}{S_x}$

$\displaystyle \text{Population: } z = \frac{X - \mu_x}{\sigma_x}$

Z-scores are useful for two reasons:

• Allow you to describe the exact location of a score within a single distribution.
• Allow you to compare z-scores in one distribution to z-scores in another distribution. It gives you a way to compare across different scales.

There are three characteristics of z-scores:

• The mean z-scores will always equal zero.
• The variance and standard deviation of z-scores will equal to 1.
• Converting the observed scores (X scores) to Z scores does not change the shape of the distribution.

4.2. Percentile Rank

The percentile rank tells us the percentage of scores that fall below a certain point. The standard normal curve allows us to find the percentages because we know the percent of area that falls under the standard normal curve.

There are a few z-score tables online. The important thing to understand is what's meant by "area beyond z" and "area between mean and z". With simple rearranging, we can calculate pretty much anything by referring to the graph.

• Note 1: Column 3 + Column 5 = .5000
• Note 2: The area under the entire curve = 1
• Note 3: Because the standard normal distribution curve is symmetrical, the values for the above table are the same for their negative counterparts.

PSYC 2002: Math!

Contents

1. 1. Review of Basic Math
   1. 1.1. Real Limits
   2. 1.2. Order of Operations
   3. 1.3. Summary of Notation
   4. 1.4. Rounding
2. 2. Exploratory Data Analyses
3. 3. Graphs

The wonderful world of Math! If you are confident with the material below, feel free to skip this post. I'm not great at math so I find quick reviews helpful at priming my brain to be in math-mode.

1. Review of Basic Math

1.1 Real Limits

The precision of measurement depends on the unit of measurement, which is typically selected by the researcher.

Real Limits are the numbers that establish the upper and lower limits within which the true valure is contained. To determine the real limit, divide the unit of measurement in half, and add AND subtract it from $X$ to get the limit for $X$.

$X = X \pm\frac{1}{2} \text{Unit of Measurement}$

1.2 Order of Operations

The order of operations must be completed in PEDMAS:

• Parentheses;
• Exponents, square roots;
• Division, multiplication; and
• Addition, subtraction.

1.3 Summary of Notation

Here's a quick summary of common notation:

• $x$ or $y$: variables used to represent a set of observations or data.
• $N$ = number of scores or observation in a population data set.
• $n$ = number of people in each group.
• $X_i$ = score on $X$ for person $i$.
• $Y_i$ = score on $Y$ for person $i$.
• $\sum$ (sum): where you add everything together that are represented by a variable
  • E.g. $\sum X$ means "add all $X$ values together."
  • $\sum_{i=\text{here is where you start}}^{\text{here is where you end}} \text{here are the values you calculate}$

Here are rules of (in)equality, using constants, where $N$ is the sample size and $C$ is a constant.

1. When you are calculating the sum of a constant, multiply the sample size $N$ by the constant.
   
   $\displaystyle \sum_{i=1}^N C = NC$
2. If you multiply the sum of $X$ by a constant, you can remove the constant from the summation and multiply the result by the constant.
   
   $\displaystyle \sum_{i=1}^N CX_i = C \sum_{i=1}^N X_i$
3. If you are calculating the sum of $X$ plus the sum of $Y$, you can calculate the sum separately and add the results together.
   
   $\displaystyle \sum_{i=1}^N (X_i + Y_i) = \sum_{i=1}^N X_i + \sum_{i=1}^N Y_i$
4. However, this only works with additions and subtractions, not multiplications! For example,
   
   $\displaystyle \sum_{i=1}^N X_i Y_i \neq \sum_{i=1}^N X_i \sum_{i=1}^N Y_i$

1.4 Rounding

Round at the last step only (i.e., only round your result). Round calculations to TWO more digits than the unit of measurement. Here are two rules for rounding:

1. If the remainder is less than 5, drop the remainder. For example, $8.33333 = 8.33$
2. If the remainder is greater than or equal to 5, increase the last digit by 1. For example, $5.66666 = 5.67$

2. Exploratory Data Analyses

Using exploratory data analyses, we can determine the best number that represents our data, the variance (spread) of the data set, how individual values compares to the entire set, whether there exists a systematic relationship between the variables being studied, and the shape of the distributions.

1. Skew: the relative symmetry of a distribution of scores.
2. Kurtosis: the degree to which values cluster versus being distributed across or throughout the range of values.

We can use stem and leaf graphs to display the original values in our data set. The stem (left of the line) represents the interval whereas the leaf (right of the line) represents the last digit values within the interval; the frequency of each scores is represented by the repeated values.

Stem	Leaf
9	8	5	5	3	2	1
8	9	6	5	5	3	1
7	7	6	6	5	4	1
6	9	8	8	6
5	8
4	6
3	8

We can use frequency distributions to arrange scores in order of magnitude. The distribution shows the number of times each scores occurs (i.e., the frequency of the scores). It is composed of two elements: a class and a frequency.

• Class: categorizing a grouping of values that are similar to each other.
• Frequency: the number of times a score, $X$, occurs in a data set. $f(X)$

A frequency distribution table is created by listing all the $X$ scores from the highest to lowest and translating the frequencies into a table.

Stem	Leaf
11	11	11	11	11	11	11
10	10	10	10	10	10	10	10	10	10	10
9	9	9	9	9	9	9	9	9
8	8	8	8	8	8	8
7	7	7	7	7	7
6	6	6	6

$\uparrow$turns into$\downarrow$

Stem	Leaf
11	6
10	10
9	8
8	6
7	5
6	3

Grouped frequency distributions are used when the scores are spread out, especially when there are a small amount of frequencies. For example, salary ranges are normally displayed in grouped frequency distribution tables.

The cumulative frequency is the total frequency (or total number of scores) at or below that score. The total frequency of scores below the upper true limit of the class interval. The cumulative frequency distribution is where the scores are arranged in order of magnitude and the distribution shows the total frequency at or below each score.

Exam Score ($X$)	$f(X)$	Cumulative $f(X)$
30-39	4	4
40-49	6	10
50-59	10	20
60-69	160	180
70-79	10	190
80-89	6	196
90-100	4	200

Relative Frequency (a.k.a. proportion) is where we divide the frequency of a score or class by $N$ (the total number of scores).

$\displaystyle \text{Relative Frequency} = \frac{f(X)}{N}$

The relative frequency distribution is where the scores are arranged in order of magnitude and the distribution shows the relative frequency for each score.

The cumulative relative frequency is the relative frequency of scores at or below that point. The cumulative relative frequency distribution is where the scores are arranged in order of magnitude and the distribution shows the relative frequency at or below each score.

Exam Score ($X$)	$f(X)$	Cumulative $f(X)$	Relative $f(X)$	Cumulative Relative $f(X)$
30-39	4	4	0.02	0.02
40-49	6	10	0.03	0.05
50-59	10	20	0.05	0.1
60-69	160	180	0.8	0.9
70-79	10	190	0.05	0.95
80-89	6	196	0.03	0.98
90-100	4	200	0.02	1

Percent is where you multiply the relative frequency by 100.

$\displaystyle \text{Percent} = \frac{f(X)}{N} \times 100$

The percent distribution displays the scores arranged in order of magnitude and the distribution shows the percent for each score.

The cumulative percentage is the percentage of score at or below that point, or the percentage of scores below the upper true limit of the class interval. The cumulative percent distribution is where the scores are arranged in order of magnitude and the distribution shows the total percent at or below each score.

Exam Score $(X)$	$f(X)$	Cumulative $f(X)$	Relative $f(X)$	Cumulative Relative $f(X)$	% $(X)$	Cumulative X $(X)$
30-39	4	4	0.02	0.02	2	2
40-49	6	10	0.03	0.05	3	5
50-59	10	20	0.05	0.1	5	10
60-69	160	180	0.8	0.9	80	90
70-79	10	190	0.05	0.95	5	95
80-89	6	196	0.03	0.98	3	98
90-100	4	200	0.02	1	2	100

Converting frequencies to proportions or percentages allows us to compare two or more groups when the size of the groups differs.

3. Graphs

There are five fundamental characteristics of a graph:

1. Two axes are drawn at a right angle.
2. The horizontal axis is the x-axis (abscissa) and the vertical axis is the y-axis (ordinate).
3. The IV is plotted along the x-axis and the DV is plotted along the y-axis.
4. The variables must be clearly labelled on both the x- and y-axes.
5. The graph should contain all of the information needed to understand the data and nothing more.

Bar graphs have spaces between the bars because they represent discrete, qualitative (nominal) data. The categories are plotted along the x-axis.

In contrast, histograms have no spaces between them. They are used to represent ordinal, interval, or ratio data. Quantitative values are plotted along the x-axis.

• For continuous variables, a class interval is a category of numbers with specified limits. Each number on the x-axis represents the midpoint.
  • Height corresponds to the frequency for the category.
  • Each number on the x-axis represents the midpoint of the class interval.
• For interval and ratio data, the width of the bar extends to the real limits for the category. E.g., for a value of 3 that falls equally between 2 and 4, the real limits would be 2.5 to 3.5.

Polygons and histograms are similar. If you add a dot at the top of each category's midpoint and connect the lines, you get a polygon. Make sure to join the outer limits of the polygon line to the x-axis at zero (to indicate frequency = 0).

PSYC 2002: Introduction to Statistics in Psychology

Contents

1. 1. Statistical Terminology
2. 2. Scientific Research Methods
3. 3. Scales of Measurement
4. 4. Experimental Designs
   1. 4.1. Experimental Design (True Experiments)
   2. 4.2. Quasi-Experimental Research

1. Statistical Terminology

Statistics: the process of collecting data in a systematic way and making decisions based on probability. They allow us to make educated, mathematical decisions based on probabilities. There are two ways statistics are used:

• Descriptive Statistics: used to describe a group or particular data sample.
• Inferential Statistics: used to make inferences about population parameters based on sample statistics (i.e., where we generalize from a sample to a population)

Let's review some important definitions:

• Population: the entire group of individuals you could possibly measure on a variable.
  • A parameter is a summary value that describes a population (e.g., the average score for a population).
  • Population number ($N$).
  • Standard deviation ($\sigma$) (sigma).
  • Mean ($\mu$) (mu).
  Population uses uppercase alphabet and greek letters.
• Sample: the group of individuals or scores that you select from a population and measure in your experiment. They have the same characteristics as the population you're interested in studying.
  • A statistic is a summary value that describes a sample (e.g., the average score for a sample). These are used to estimate population parameters but they're not perfect because of sampling errors.
  • Sample number ($n$).
  • Standard deviation ($s$).
  • Mean ($\bar{x}$) (x-bar)
  Samples use lowercase alphabet and greek letters.
• Random Sample: a sample where each individual in the population has an equal chance of being selected.
• Random Assignment: where each individual has an equal chance to be placed in each treatment condition.
• Sampling Error: a naturally occurring difference between a sample statistic and its corresponding population parameter.

2. Scientific Research Methods

Research Design: the systematic process of collecting data in order to answer specific questions. There are two major classifications:

1. Experimental: tests your hypotheses on the causal effects of the Independent Variable (IV) on the Dependent Variable (DV). The IV is actively manipulated and there are two levels. Your participants must be randomly assigned to conditions and you need to specific your procedures for testing your hypothesis. Also, you have to control for major threats to internal validity.
2. Non-Experimental:
   1. Descriptive Research: describes a group using numerical scores on variables.
      1. Use: often a starting point in research.
      2. Statistics: mean, median, and standard deviation.
   2. Correlation Research: where we look for relationships between two or more vairables. We measure each participant on each variable but there is no causal inferences.
      1. Use: for testing theories and help form predictions; often as a secondary analysis.
      2. Statistics: often requires a linear relationship which is evaluated on strength and direction.
   3. Quasi-Experimental Research: where we test our hypotheses about relationships between the IV and DV but we have limited ability to actively manipulate the IV or randomly assign participants to conditions. We must still include specific procedures for testing the hypotheses and still control for the major threats to internal validity. We cannot make true causal inferences.

Variables: a characteristic or condition that changes or has different values for different individuals; any measure or characteristic that we use in research.

There are three types of experimental variables:

1. Independent Variables (IV): what we manipulate.
   1. Experimental: where the experiment manipulates the variable.
   2. Subject: where the variable is predefined (e.g., gender, smoker versus non-smoker).
2. Dependent Variables (DV): what we analyze or measure.
3. Extraneous Variables (EV): any other variable (confound or not) that is not an IV or a DV.

There are two ways to describe variables:

1. Qualitative (categorical): descriptive qualities that describe something (e.g., male/female, University attended).
   • Discontinuous (discrete): values that can only be whole numbers (no number in between). Measurements are exact (e.g., you have a whole person, not half of a person).
2. Quantitative (numerical): quantifiable qualities that describe something (e.g., age, height)
   • Continuous (infinite): where they can be any number of values between two numbers.
   • Discontinuous (discrete).

In other words, qualitative variables are always discontinuous whereas quantitative variables can be either continuous or discontinuous.

3. Scales of Measurement

The level of measurement determines the types of questions we can ask, the types of analyses we can perform, and the conclusions we can draw. There are four ways to measure variables (NOIR):

1. Nominal: categorizing variables in no particular order.
   • E.g., by gender (male/female/other), or religion (Jewish, Muslim, Hindu)
   • Statistics: chi-square; proportions, percentages, and mode.
2. Ordinal: categorizing variables unequally along a continuum. (We can talk about magnitude.)
   • E.g., by satisfaction (1 through 7).
   • Statistics: Mann-Witney U; proportions, percentages, mode, and median.
3. Interval: categorizing variables equally along a continuum. There is no true zero.
   • E.g., (temperature) it can be 10°C today and 20°C tomorrow but that doesn’t mean that tomorrow will be twice as hot as today.
   • E.g., (IQ) if you have an IQ of zero, you don’t have an absence of intelligence.
   • Statistics: t-test or ANOVA; median, mean and standard deviation.
4. Ratio: categorizing variables equally along a continuum with a true zero.
   • E.g., length, height, and reaction times can all have zero values. A 10 second reaction time is twice as fast as a 5 second reaction time.
   • Statistics: t-test or ANOVA; median, mean and standard deviation.

In Psychology, we normally use interval scales in tests and measurements.

4. Experimental Designs

4.1. Experimental Design (True Experiments)

As summarized above, you are testing your hypothesis on the causal effects of the IV on the DV. Your goal is to determine if there exists a cause-and-effect relationship.

The general process is to randomly select individuals from the population; randomly assign them to different treatment conditions; expose subjects to different levels of the IV; and compare DV differences between conditions.

We must control our confounding variables only when:

• A variable changes systematically with the IV; or
• A variable influences the DV.

4.2. Quasi-Experimental Research

4.2.1. Descriptive Research

You are trying to obtain an accurate description of a population with numerical values. You cannot draw a conclusion - only describe!

The general process is to collect data on variables of interest and use descriptive statistics to summarize the data that you collect.

4.2.2. Correlational Research

As summarized above, you are trying to determine if there exists a relationship between two or more variables. You cannot determine causality or directionality.

The general process is to randomly select individuals from the population (using something like simple random sampling); collect information on the variables of interest; and calculate a correlation coefficient ($r$).

4.2.3. Comparing Intact Groups

This is where we are trying to determine if there exists a relationship between a grouping variable and some other variable. You can find significant differences between groups but you still cannot determine causality and direction.

The general process is to randomly select individuals from the population; place individuals into a group based on a person-variable (e.g., smoker versus non-smoker); and compare the group differences. (Think: non-equivalent group, pre-post group, and developmental designs.)

PSYC 2001: Research Designs

Contents

1. 1. Qualitative Research
2. 2. Quantitative Research
3. 3. Experimental Designs
   1. 3.1. Control Groups
   2. 3.2. Between-Subject Designs
   3. 3.3. Within-Subject Designs
   4. 3.4. Comparing Within- and Between-Subject Designs
   5. 3.5. Single-Subject Research Design
4. 4. Quasi- and Non-Experimental
5. 5. Factorial Designs
6. 6. Correlational Designs

1. Qualitative Research

Other than quantitative research, there are four major approaches to qualitative research:

• Naturalistic Observation: describes  behaviour in real life.
  • Strengths: gives insight into contexts, relationships, behaviours (more real life) and flexible.
  • Weaknesses: time-consuming, hard to document, observe – don’t interpret, access may be hard to get, informed consent may be difficult to get, and may not be representative.
• Questionnaires.
• Conversation Analysis.
• Single-Case Study: an in-depth analysis of a single person or small group. If there is no treatment, it’s called a case history.

We can use indirect measures (e.g., observing people) or content analysis (e.g., looking at patterns and themes in written text or media).                

• Program Evaluation: in order of increasing usefulness, we can use reactions and feelings, learning, changes in skills, and effectiveness to evaluate a program.
  • Issue: Participants may not volunteer or may feel coerced to participate.

There are 6 ways to evaluate a program:

1. Questionnaires: used to quickly/easily access information from a large sample in a non-threatening way.
   • Pros: anonymous, cheap, easy to compare and analyze, already exist.
   • Cons: getting careful feedback is tough, wording can bias responses, representative sample may not be available, limited to questions on the sheet.
2. Interviews: used to fully understand someone’s answers or expand on questionnaire responses.
   • Pros: full range and depth of information, develops relationship with client.
   • Cons: time consuming, hard to analyze and compare, expensive, interviewer can bias response.
3. Documentation Review: used when we want an impression of how a program operates without interrupting the program (where we review applications, finances, memos, etc).
   • Pros: historical information is available, doesn’t interrupt the client, few biases in information.
   • Cons: information may be incomplete, no flexibility (you have what exists and nothing more), need to be clear about what you’re looking for (so much data).
4. Observation: used to gather information about how a program operates (particularly about specific processes).
   • Pros: views operations of a program as they are actually occurring, can adapt to events as they occur.
   • Cons: difficult to interpret observations, hard to categorize observations, can influence behaviour of participants.
5. Focus Groups: used to explore a topic in depth with a discussion (useful in evaluations and marketing).
   • Pros: quickly and reliably get common impressions, fast and efficient way to get a wide range of information, can convey key information about programs.
   • Cons: hard to analyze responses, need a good facilitator, hard to schedule people together.
6. Case Studies: used to fully understand a client’s experience in a program and conduct comprehensive examination through cross comparison of cases.
   • Pros: fully depicts client’s experience, powerful way to portray program to outsiders.
   • Cons: time consuming, provides depth but not breadth.

Developing Surveys and Questionnaires

Surveys evaluate attitudes, feelings, experience, and knowledge, whereas questionnaires evaluate aptitudes, abilities, and characteristics.

There are two characteristics of tests:

• Reliability: inter-rater reliability and evaluation (split-half – is performance in one section relate to performance in another?; and test-retest – are scores consistent over time?)
• Validity: concurrent (do results concur with other tests?) and criterion (does it predict behaviour? Is it meaningful? Is it useful?)

Though we can use existing measures, they can be expensive, intended for special populations, have and restricted access.

There are five types of questions:

1. Yes-No: easy to mark. Measure with frequency/percentage.
2. Forced-Alternative: easy to score but nothing ever fits anyone perfectly.
3. Multiple-Choice: easy to mark.
4. Likert Scales: people tend not to choose the extremes – if you use even numbers, people have to at least agree/disagree. Measure with a mean/modal response.
5. Open-Ended: allows a wide variety of responses. Measure by coding first.

Remove questions that  have low variability, were omitted or commented on by participants, or do an item analysis (if too many people get it wrong, remove the question).

There are three ways to distribute surveys (in addition to online and interviews):

	Pros	Cons
Mail	No need to be present Potential for larger sample	No idea who completed it Questions may not be answered in order Low sample rate (biased return, 20-30%) Incomplete questions Unanswered questions Answers are not representative
Phone	No need to be present Random sample possible Questions in correct order Survey completed Clarification possible	Random number dialing may call same person twice Hang-ups, blocked calls, screening No visuals in survey Harder to establish a rapport Lower response rate (between mail and in-person)
In Person	High return rate (90%) Answer classification possible Ensures surveys are completed Questions answered in correct order	Time Money Training Interviewer Bias Harder to access (in-home)

2. Quantitative Research

There are five major approaches to quantitative research:

1. Descriptive Statistics: describes a group using numerical scores on variables.
   • Use: often a starting point in research.
   • Statistics: mean, median, and standard deviation.
2. Correlational: where we look for relationships between two or more variables. We measure each participant on each variable but there is no causal inferences.
   • Use: for testing theories and help form predictions; often as a secondary analysis.
   • Statistics: often requires a linear relationship which is evaluated on strength and direction.
3. Experimental: tests your hypotheses on the causal effects of the IV on the DV. The IV is actively manipulated and there are two levels. Your participants must be randomly assigned to conditions and you need to specify your procedures for testing your hypothesis. Also, you have to control for major threats to internal validity.
4. Quasi-Experimental: where we test our hypotheses about relationships between the IV and DV but we have limited ability to actively manipulate the IV or randomly assign participants to conditions. We must still include specific procedures for testing the hypotheses and still control for the major threats to internal validity. We cannot make true causal inferences.
5. Non-Experimental: see book!

3. Experimental Designs

There are four characteristics of an experimental design:

1. Two levels of the IV (by manipulating the IV).
2. Include a measurement variable (DV) to compare the participants in each condition.
3. Look for consistent differences between groups.
4. Include control for the major threats to internal validity.

You should also test your hypothesis about the causal effects of the IV, randomly assign participants to conditions, and limit the influence of extraneous variables (so they don’t become confounds).

Causation and Directionality: when we find a relationship between two variables, we have to ensure that we manipulate and control our variables to ensure that we can make a causal and directional conclusion.

• Some variables may be hard to manipulate (ethically or logistically).

3.1. Control Groups

A good experiment will always have a control group, otherwise we can’t be sure if the treatment caused the change or whether it was due to some other variable. The control group must receive the exact same experience as the treatment group (with the exception of the actual treatment).

There are two types of control groups:

1. No-Treatment Group: where the group receives no treatment. This provides a baseline to compare the treatment group with.
2. Placebo Group: where the group receives a placebo treatment. It helps rule out the possibility that the difference between the treatment and no-treatment group are due to participant expectations and regression to the mean (i.e., people will ‘get better’ without treatment so we have to know if it’s a result of the treatment or time).
   • There are three types of placebos:
     • Inert substance – a sugar pill.
     • Sham surgery.
     • False information – telling participants they’re getting the ‘better’ treatment.

Manipulation Check: tests whether the manipulation of the IV actually worked as intended.

3.2. Between-Subject Designs

Between-Subject Design: where we compare scores from separate group (different treatment conditions) to determine which treatment is more effective. Scores are independent.

Groups should be equivalent. The process in which they are created should be the same, they should be treated equally, and they should be composed of equivalent individuals.

• Advantages: each score is independent of other scores.
  • Not influenced by experience from other treatments (i.e., practice effects).
  • Not influenced by fatigue or boredom.
  • Not influenced by contrast effects.
• Disadvantages:
  • Requires a large number of participants (can be difficult with special populations).
  • Individual Differences: characteristics that differ from one participant to another; these can become confounding variables and can produce high variability in the scores.
    • Assignment bias (e.g. older versus younger group)
    • Confounding environmental variables

Variance Between and Within Groups

We want variance within groups but we want to control unwanted sources of variance. There are two forms of variance:

1. Systematic Between-Group Variance: experimental variance due to the IV, or extraneous variance due to confounding variables.
2. Non-systematic Within-Group Variance: due to chance or individual differences.

To determine if there is a systematic difference between groups, we must compare between-group and within-group variance. If between-group variance is greater than within-group variance, there is a significant, systematic difference, meaning the null hypothesis should likely be rejected.

Using a t-test or ANOVA, we can calculate the difference between groups divided by the difference expected by chance. t = (difference between groups)/(difference expected by chance [error]) or F = (difference between groups)/(difference expected by chance [error])

To determine the variability due to chance, we calculate the signal to noise ratio. The signal to noise ratio must be large in order to conclude that the difference is due to something other than chance.

• Noise: the standard deviation.
• Signal:
• 10:3 would be large whereas 8:7 is small
• Signal : Noise, where signal > noise.

There are three ways we can control the variance:

• Maximize experimental variance: make sure that the differences between groups are real by using a manipulation check or maximizing the effect size.
• Control extraneous variance: rule out confounds by making sure the groups are as similar as possible.
• Minimize error variance: control your study with careful measurements.

Threats to Internal Validity

There are two major threats to internal validity with this design:

• Differential Attrition: if too many participants drop out, your groups may be uneven, creating problems.
• Communication between groups:
  • Diffusion: where participants from the no-treatment group start using techniques from the treatment group.
  • Compensatory Equalization: where the no-treatment group wants the treatment.
  • Compensatory Rivalry: where the no-treatment group tries harder than the treatment group to prove themselves.
  • Resentful Demoralization: where the no-treatment resents the treated groups and performs poorly.

Applications

Two-Group Mean Difference: where we compare only two groups of participants and the research manipulates only one independent variable with only two levels (single-factor two-group design).

• Often used when we compare treatment and control group.
• Advantage: simple and provides best opportunity to maximize variance between groups.
• Disadvantage: provides little data for comparison (only 2 data sets).

Comparing Means for More than Two Groups: where we compare multiple groups and the research manipulates the independent variables on several levels (single-factor multiple group design).

• Advantage: provides stronger evidence for real cause-and-effect relationship.
• Disadvantage: minimizes the differences between groups so we have to ensure we manipulate the independent variables sufficiently.

Comparing Proportions for Two or More Groups: when you use nominal or ordinal scales, you can’t compute a mean but you can compare proportions using a chi-square test.

3.3. Within-Subject Designs

Within-Subject Design: where we use a single group of participants and test/observe each individual in all of the different treatments being compared. (Aka. repeated-measures design.) Scores are dependent.

• Advantages: requires few participants and eliminates all issues relating to individual differences.
  • Possible to measure differences between treatments without involving individual differences.
  • Possible to measure differences between individuals – these individual differences can be measured and removed.
  • Easier to detect a treatment effect with this design.
• Disadvantages:
  • Time variables (fatigue, weather) can influence participants and threaten internal validity.
  • Participant attrition – counteract this by getting more participants at the outset than you think you’ll need.

There are two major threats to internal validity:

• Confounding environmental variables (e.g., morning versus afternoon).
• Confounding time variables (history, maturation, instrumentation, testing effects, and regression).

There are two type of order effects:

• Carryover Effect: (treatment-specific experience) participant develops new skills/techniques that carries over to the new study
• Progressive Error: (general experience)
  • Practice Effects: progressive improvement in performance as a participant gains experience.
  • Fatigue: progressive decline in performance as a participant gains experience.

Three ways to control time-related threats and order effects:

1. Controlling Time: shortening the time between experiments can reduce time-related threats to internal validity but increases order effects.
2. Switch to a Between-Subject Design: whenever you have reason to suspect that order effects will play a substantial role, switch to between-subject design
3. Counterbalancing – Matching Treatments with Respect to Time: where we change the order in which treatments are given – the goal is to use every possible order of treatments with an equal amount of participants in each sequence
   • Note that this method doesn’t eliminate order effects but distributes the effects evenly across treatment groups
   • Limitations:
     • Counterbalancing and Variance: counterbalancing increases the variance within the treatment groups.
     • Asymmetrical Order Effects: counterbalancing doesn’t take into account that some treatments may be more demanding (have an unbalanced amount of order effects).
     • Counterbalancing and the Number of Treatments: n! (n factorial) is the number of sequences needed to get a complete counterbalance study; partial counterbalancing can be used (Latin square or randomly assigning treatment group orders).

Applications

Two-Treatment Designs: where there are only two treatment conditions.

• Advantages: easy to conduct and results are easy to understand, easy to counterbalance, easy to get a statistical difference.
• Disadvantages: doesn’t demonstrate any functional relationship between the two data sets – we can’t know how gradual changes in the independent variable would affect the dependent variable.
• Statistical Analysis: t (repeated-measures) or analysis of variance can be used (interval or ratio). For ordinal scales, the Wilcox on test can be used.

Multiple-Treatment Designs:

• Advantage: demonstrates stronger cause-and-effect relationship.
• Disadvantage: too many treatments may not result in statistically significant variation and increases the amount of time to complete the study which increases participant attrition.

3.4. Comparing Within- and Between-Subject Designs

There are three ways in which these design differ significantly:

When you suspect…	Good for…	Bad for…
Individual Differences	Within-subject	Between-subject
Time-Related Factors and Order Effects	Between-subject	Within-subject
Fewer Participants	Within-subject	Between-subject

3.5. Single-Subject Research Design

Single-subject research designs allow researchers to manipulate IVs and measure the DV with a single person (or small group) and infer a causal relationship. There is no control group.

• Useful for studying unusual disorders and unique experiences, or when we’re concerned that group variation will hide individual effects. (Often used in clinical psychology.)
• Flexible and you only need one (or a small group of) participant(s).

Classical Research Design: [baseline] – [treatment] – [post-treatment]

ABAB Reversal Design: [baseline] – [treatment] – [baseline] – [treatment]

• In the treatment conditions, we’re looking for a full reversal of behaviour so we can conclusively so that the treatment is effecting the behaviour.
• The baseline conditions act as the ‘control’.
• This can’t be used for treatments that are expected to have long-lasting or permanent effects.

There are three Multiple-Baseline Designs:

• Multiple-Baseline Across Subjects: where a baseline is established for multiple participants and the treatment is staggered across (only once) each participant.
• Multiple-Baseline Across Behaviours: where baseline behaviours are established for a single participants and treatments are staggered to determine whether the treatment had an effect.
• Multiple-Baseline Across Situations: where baseline behaviours are established in two different situations and the treatments are staggered.
• All of the multiple-baseline designs follow this pattern:
  Observation 1: [---baseline---]           [---treatment---]
  Observation 2: [---baseline---]                            [---treatment---]
• Strengths: good for treatments with long-lasting effects.
• Weaknesses: hard to discern between individual behaviours; treatments may produce a general effect.

There are three other types of single-subject designs:

1. Dismantling (Component-Analysis) Design: where each phase adds or subtracts one component of a complex treatment to determine how each component contributes to the overall treatment.
2. Changing Criterion Design: where each phase is defined by a specific criterion that determines a target level of behaviour (e.g., 20 cigarettes a day for the first week, 10 for the next). To ensure there is a correlation between the treatment and the behaviour, researchers often include backward phases (“now have 30 cigarettes!”) to make sure the behaviour is causally related to the treatment.
3. Alternating-Treatment Design: where two or more treatment conditions are randomly alternated from one observation to the next.

Single-subject designs have low external validity but high internal validity. We can generalize our results using systematic, clinical, and direct replication.

• Systematic replication is where we test the effectiveness of a given treatment across a variety of behaviours (e.g., change the participant, setting, or behaviour).
• Clinical replication is where we test the effectiveness of a combination of treatments for various behaviours, participants, and settings.
• Direct replication is where we test the effectiveness of a given treatment for a single behaviour, multiple times. ABAB design uses this approach.

4. Quasi- and Non-Experimental

Quasi-experimental makes some attempt to reduce threats to internal validity whereas non-experimental makes none. In both cases, the conditions are not changed by manipulating an IV but are defined by pre-existing conditions (e.g., gender, age) or in terms of time (e.g., before and after). These designs look for differences between groups.

There are two categories of designs that can be classified as quasi- or non-experimental:

• Between-subject designs, or non-equivalent group designs; and
• Within-subject designs, or pre-post designs.

Non-equivalent group (between-subject) designs are so named because the researcher cannot control the assignment of individuals to a group.

• All three design subtypes suffer from assignment bias.
• The three design subtypes:
• Differential Design (non-experimental): where participants are assigned to groups on a particular participant character (such as gender or race). The DV is compared between the groups to see if there is a consistent difference. Similar to correlational research.
• Non-Equivalent Group Design (non-experimental): where the control and treatment groups are pre-existing and not randomly assigned.
• Posttest-only Non-equivalent Control Group Design: where two non-equivalent groups are compared – they are measured at the same time but only one group receives the treatment.
  X O (treatment group)
      O (control group)
• Pretest-Posttest Non-Equivalent Group Design (quasi): where two non-equivalent groups (treatment and control ) are compared – the treatment group is measured before AND after the treatment whereas the control group is measured twice (before and after) but with no treatment in between.
  O X O (treatment group)
  O     O (control group)
  
  This design reduces assignment bias, limits threats from time-related  factors and provides evidence for a cause-and-effect relationship.
  
• History effects that affect groups differently are called differential threats. Time-related threats may occur with all of the above as well.

Pre-post group designs (within-subject) measures the pretest and posttest scores of a group before and after treatment to determine if there’s an effect. There is no control group with this design.

There are three types of designs:

• One-Group Pretest-Posttest Design (non-experimental): only one group is used
  O X O
• Time-Series Design (quasi-experimental): multiple observations are made before and after the treatment. This is often used with predictable events (e.g., a change in law).
  O O O X O O O

These designs suffer from time-related threats to internal validity (history, instrumentation, testing effects, maturation, and statistical regression).

Developmental Research Designs: experiments that examine changes in behaviour that are related to age. There are three types:

1. Cross-Sectional: (between-subject) different cohorts are measured at one point in time and then compared.
   • Cohorts: individuals born roughly at the time same and growing up in similar conditions.
   • Cohort Effects: differences between age groups caused by unique experiences other than age. (Be careful! Developmental differences may just be cohort effects.)
2. Longitudinal: examines development by observing or measuring one group of cohorts over time.
3. Cross-Sectional Longitudinal Designs: compare results obtained from separate samples that were obtained at different times (i.e., many cohorts, many times).

	Cross-Sectional	Longitudinal
Strengths	Time efficient No long-term cooperation required Easier to generalize	No cohort or generation effects Assess individual behaviour effects
Weaknesses	Individual changes not assessed Cohort or generation effects	Time consuming Participant dropout may create bias Potential for practice effects Harder to generalize results

5. Factorial Designs

Experimental factorial designs are studies that contain more than one IV (or more than one factor) and allow us to examine the interactive effects of IVs. This is good for external validity.

We use ANOVA for factorial designs. It’s written to express the number of levels of each IV.

For example, if there are two IVs and the first IV has 2 levels and the second IV has 5 levels, it would be a 2 × 5 factorial design.

A Main Effects describes the individual effect of each IV on the DV.

• On a graph, the x variable has an effect if the mean line drawn is not parallel.
• On a graph, the y variable has an effect if the mean points are not in the same place.

An Interaction Effect describes the combined effect of two or more IVs on the DV.

• On a graph, there is an interaction if the lines are not parallel.

6. Correlational Designs

Correlational designs allow us to measure each participant on each variable and find a relationship between two variables (which is calculated using a correlational coefficient and a scatter plot). This design looks for a relationship between two or more variables.

There are three characteristics of a correlation:

1. The direction can be upward (+r), downward (−r), or non-existent (r = 0).
2. The nature of the relationship: linear (Pearson’s r) or curved (Spearman’s rho, ρ).
3. The consistency or strength. The closer r is to 1, the more consistent the relationship (i.e., the closer the data points appear on the scatter plot).

To interpret a correlation:

• The strength of the relationship is indicated by the coefficient of determination (r²). This tells us the variability of one variable that is predicted by another.
• For example, if the correlation coefficient r = .80, then r² = .64 which means that 64% of the differences are predictable.
• Small (r = .10, medium (r = .30), large (r = .64)
  Small (r² = .01, medium (r² = .09), large (r² = .25)
• The significance of the relationship is indicated by the statistical significance of the correlation, which means that the result is unlikely to have occurred by random chance.

There are three applications of correlational designs:

1. Prediction: if x and y are correlated, we can use the x value to predict the y value! We can use a regression line to plot the prediction.
2. Test-Retest Reliability: where we can see if our first and second tests are positively correlated.
3. Concurrent Validity: where we can see if our test correlates positively with a well-tested test.

There are strengths and weaknesses of correlational designs:

Strengths:

• Describes relationships between variables
• Nonintrusive – allows natural behaviour to emerge
• High external validity

Weaknesses:

• Cannot assess causality
• Third-variable problem
• Directionality problem
• Low internal validity

PSYC 2001: Operational Definitions, Scales of Measurement, and Evaluating Measures

Contents

1. 1. Operational Definitions
2. 2. Scales of Measurement
3. 3. Evaluating Measures
   1. 3.1. Reliability
   2. 3.2. Effective Range
   3. 3.3. Validity
   4. 3.4. Confounding Variables

Hypothesis: a sentence that clearly states the relationship we expect to see between our IV(s) and DV(s). Variables are stated in operational definition terms.

1. Operational Definitions

Operational Definition (OD): describes your variables in terms of the way you will measure them.

• Importance: ODs are necessary in order to replicate studies.
  Look at past research to see how variables are operationally defined.

Example of an OD: A nacho can be defined as: the result of covering a corn-chip with cut vegetables (olives, red peppers, mushrooms, and onions) and shredded cheese, and baking it in the oven at 350°C for approximately 15 minutes.

Constructs: hypothetical mechanisms or attributes to explain behaviour (e.g., motivation, self-esteem).

Variables: a characteristic or condition that changes or has different values for different individuals; any measure or characteristic that we use in research

Examples of Typical Psychological Variables:
Cognitive: IQ, memory, reading language
Perceptual: vision, audition, tactile
Social: personality, self-esteem, attachment

There are three types of variables:

• Independent Variables (IV): what we manipulate.
• Dependent Variables (DV): what we analyze or measure.
• Extraneous Variables (EV): any other variable (confound or not) that is not an IV or a DV.

There are two ways to categorize variables:

• Categorical (qualitative): descriptive qualities that describe something (e.g., male/female, University attended)
• Continuous (quantitative): quantifiable qualities that describe something (e.g., age, height)
  
  Note: we can often convert between the two. For example, I may be 22 (continuous) but I am also an adult (categorical). How you define your variables will depend on your study.

We can have any number and combination of IVs and DVs.

We can also measure variables directly (e.g., IQ can be measured using an IQ test) but others have to be measured using more creative techniques; unfortunately, when we can’t measure something directly, we can’t conclusively get an accurate measurement. We can use constructs to help us measure behaviour.

2. Scales of Measurement

There are four ways to measure variables (NOIR):

1. Nominal: categorizing variables in no particular order.
   • E.g., by gender (male/female/other), or religion (Jewish, Muslim, Hindu)
   • Statistics: chi-square; proportions, percentages, and mode.
2. Ordinal: categorizing variables unequally along a continuum. (We can talk about magnitude.)
   • E.g., by satisfaction (1 through 7).
   • Statistics: Mann-Witney U; proportions, percentages, mode, and median.
3. Interval: categorizing variables equally along a continuum. There is no true zero.
   • E.g., (temperature) it can be 10°C today and 20°C tomorrow but that doesn’t mean that tomorrow will be twice as hot as today.
   • E.g., (IQ) if you have an IQ of zero, you don’t have an absence of intelligence.
   • Statistics: t-test or ANOVA; median, mean and standard deviation.
4. Ratio: categorizing variables equally along a continuum with a true zero.
   • E.g., length, height, and reaction times can all have zero values. A 10 second reaction time is twice as fast as a 5 second reaction time.
   • Statistics: t-test or ANOVA; median, mean and standard deviation.

In Psychology, we normally use interval scales in tests and measurement.

3. Evaluating Measures

There are three ways to evaluate your measures: the reliability, effective range, and validity.

3.1. Reliability

Reliability: the consistency of a measurement. There are three  ways to measure how reliable your study is:

1. Inter-rater: the degree of agreement between two independent raters.
2. Test-retest: the degree of consistency (in scores) over time.
3. Internal Consistency: the degree to which all items of a test measure the same thing. This is often measured using the split-half reliability test, where the scores taken from a random sample of questions match the scores taken from another sample of questions.

There are three factors that influence the reliability of your study:

1. Clarity: Clarity of your operation definition;
2. Follow-Through: The extent to which your operational definition is followed; and
3. Observations: The number of observations the overall score is based on (more is better).

3.2. Effective Range

Effective Range: the range of scores we need to get a good idea of the performance on our dependent variable(s). We have to consider:

• Characteristics of the Sample: formulate your study to match the characteristics of your sample. E.g., don’t ask  blind people to watch a movie.

Scale Attenuation Effects: where the range is restricted – people above or below the effective range aren’t accurately measured; this distorts the scores, reducing reliability and validity. This can occur in two ways:

1. Ceiling Effects: there’s an insufficient range at the top of the scale – most high scores are bunched together, preventing us from increasing scores.
2. Floor Effects: there’s an insufficient range at the bottom of the scale – most low scores are bunched together, preventing us from decreasing scores.

3.3. Validity

There are four major types of validity (ESCI) and five others:

1. External: the degree to which your study applies to the real world (i.e., generalizability).
   • Influenced by: quality of sample selection (e.g., random selection, representativeness).
2. Statistical: the degree to which your decision about the null hypothesis is reasonable.
   • Influenced by: reliability of your measures, ability to meet assumptions of statistical tests used.
3. Construct: the degree to which underlying theories fit with your results (and vice versa).
   • Influenced by: quality of rationalistic reasoning, consideration of possible theories.
4. Internal: the degree to which your study accurately demonstrates that measured changes were a result of manipulating the IV.
   • Influenced by: the degree to which you controlled extraneous confound variables.
   • We want to ensure:
     • One Explanation: there should be only one explanation for the results.
     •  Groups Differ: the treatment and non-treatment groups differ.
     • Other: we have to consider that other variables may have an effect (e.g., environment, groups, and time variables).
5. Face Validity: a measure superficially appears to measure what it claims to measure.
6. Concurrent Validity: scores obtained from a new measure relate to scores obtained from an old, established measure. (Note: this always compares new and old.)
7. Predictive Validity: scores obtained from a measure accurately predict behaviour according to a theory.
8. Convergent Validity: strong relationship between scores obtained from two different methods of measuring the same construct.
9. Divergent Validity: demonstrated by using two different methods to measure two different constructs. Then convergent validity must be shown for each of the two constructs. Finally, there should be little or no relationship between scores obtained for the two different constructs when they are measured by the same method. This shows that we’re measuring one construct, not two.  (Note: this always compares new methods.)

3.4. Confounding Variables

We must control our confounding variables only when:

• A variable changes systematically with the IV; or
• A variable influences the DV.

There are three types of confounding variables (or many threats to internal validity):

1. Participant: each person has different demographics.
   • Subject Effects: people tend to be suspicious when they are in an experiment.
     • Demand Characteristics: where people interpret what the purpose of the experiment is and change their behaviour unconsciously.
     • Hawthorne Effect: people try harder because they’re in an experiment.
     • Social Desirability Bias: tendency to respond in a way that will be viewed favourably by others.
     • Control by using deception, placebos, and single/double-blind designs.
   • Selection Effects: use random assignment or match groups to ensure participants are the same before the treatment.
   • Attrition Effects: where participants drop-out or die before completing the experiment.
2. Environmental: experiments may be conducted in different environments.
   • Experimenter Effects: experimenters may introduce bias by interpreting the data in a way that supports their hypothesis.
     • Clever Hans: the horse that could understand cues from audiences.
     • Greenspoon Effect: the guy that emphasized the syllables in nouns to demonstrate the experimenter effect.
     • Control by using single/double-blind designs, standardized instructions, and automated and objective measurements or multiple observers.
   • Diffusion of Treatment: where treatment effects spread from the treatment to the control group.
3. Time: over time, time variables may influence the study.
   • Sequence/Order Effects: experience with other conditions influences performance in each condition.
     • Control by counterbalancing the order of conditions across participants.
   • Practice Effects: experience with a study can influence the performance.
     • Control with counterbalancing.
   • Maturation: participants get older and mature during the experiment.
   • History: events outside the study that influence a participant’s behaviour.
   • Instrumentation: where the instrument used to assess, measure, or evaluate the participant changes over the course of the study.
   • Regression to the Mean: where the scores tend to regress to the mean after practice or time.

There are three ways to control confounds:

1. Holding the variable constant: where we make sure the confound is the same for everyone.
   • Con: limits external validity.
2. Matching: where we match the level of the confound across groups.
   • Participants: match by number/proportion or by average. (E.g., each group has 50% males and 50% females, or the average IQ of each group is 120.)
   • Environment: (e.g., have each experimenter work is 50% of participants from each group.)
   • Time: counterbalance the order of conditions, the time between groups, etc.
   • Con: can be intrusive to participants and time consuming (you have to measure the confound ).
3. Random Assignment: randomly assign participants to treatment and non-treatment groups.
   • Advantages: allows us to control many variable simultaneously.
   • Disadvantages: only works with large groups – make sure you do a check to compare groups on potential confounds because it may not be ‘random’.