Questions about material down the homestretch (PSY 365)
As a case of Spring- and senioritis sets in, now is an excellent opportunity to voice any questions or requests for clarifications you have about the material thus far the second half of the term (viz. factorial ANOVAs, repeated measures ANOVAs, mixed repeated measures ANOVAs). Questions about post hoc tests, degrees of freedom, data interpretation, SPSS, or interactions? Please use the blog and post here!
Sean and I, as well as your fellow classmates, want to help. Your homework is to post a question or request for clarification on this blog post—ideally before Wednesday at 11:00pm. Questions after that are clearly not guaranteed to be answered before class the following morning. Class starts promptly at 9:30am on Thursday and we will meet in our regular classroom.
Bring on the questions!
-Sean McCormick TA PSY 365
I’m having trouble reporting the means and standard deviations of independent variables across each other and independent of each other. How do I do each one correctly?
hi, antonio. when you select Options, before you about to run your factorial ANOVA, select all variables (including OVERALL) from the list and move them into the window “Display Means for”. Then make sure to hit Descriptive Statistics.
This might answer the question. If not post a REPLY and we’ll try again.
I am confused as to when we perform contrasts versus post hoc tests (like the Tukey). I believe we cannot perform contrasts in a factorial design because we are only looking at between-subjects factors and not within-subjects factors. Also, we cannot perform post hoc tests in repeated measured/mixed repeated measures because we are looking at within-groups factors. So is it correct to say that we perform post hoc tests like the Tukey in only factorial and contrasts in only repeated measures/mixed repeated measures?
Thanks!
Ashley Bathgate, PSY 365
Ashley,
The main deciding factor in deciding whether or not to run post hocs or contrasts is whether or not you planned to run them.
Contrasts (sometimes called “Planned Contrasts”) you have planned to conduct even before you ran the omnibus test. For example, you may have a clear and specific hypothesis as to which group will benefit most from a drug. You would still want to know about overall differences between all the groups, but you would choose to run a Contrast to test that specific hypothesis.
Post hoc tests are called post hoc tests because you run them to test some hypothesis after the fact. They are more stringent, and help reduce the likelihood of making Type I error (no fishing for significant results please). They are more commonly used when there’s something going on in the data that you did not expect.
Dr. Ramey, feel free to add something here, as I know I somewhat hedged around Ashley’s question.
-Sean McCormick TA PSY 365
the language of SPSS suggests (for those things we have covered in class) that contrasts (polynomial, simple, repeated) are for within-subjects factors (e.g., time). post hoc tests like Tukey are for following up statistically significant omnibus between-subject effects (e.g., 3 levels of a drug dosage).
Sean is right to say that contrasts are also considered “planned contrasts” in a different context. one can perform these more powerful 1:1 tests if one has certain hypotheses. (i prefer them. they are calculated differently than the standard tests taught in undergraduate stat courses.)
as for factorial designs, we are generally looking for main effects (or simple main effects) and interaction effects, which is why we designed it with two independent variables in the first place. a planned contrast is more straightforward and more limited in that we are only contrasting two levels of a single independent variable. hope that helps.
I missed a class, so this question may already have been answered. When reporting degrees of freedom for Factorial ANOVAs, where exactly do the degrees of freedom come from for the error?
Liz Schickling, PSY 365
Liz,
You can find the error degrees of freedom right below the results of the F test. In a factorial ANOVA it is just below the interaction output, in the same cell in fact.
For the repeated measures ANOVAs the cells on the left have the header “Source” underneath that you will see “Time” or whatever you labeled your within subject factor and that includes the output for the F test. There will be cell below that with the source “Error”, look there. Just make sure that your degrees of freedom come from the same test. For example, if you are using the results from the Greenhouse-Geisser, that you report the Error degrees of freedom from the Greenhouse-Geisser.
Check your outputs, and if you still can’t find it maybe we can post an image.
-Sean McCormick TA PSY 365
Sorry, wasn’t clear. I know where to find them, but where do the actual numbers come from? How are they figured out?
hi, elizabeth. for the error degrees of freedom, you can calculate it as N (the total number of participants) minus the product of the number of levels of each independent variable. N-(levels of IV1)(levels of IV2).
in our factorial ANOVA with 3 levels of depression and 2 levels of drug dosage, we had 30 participants total.
30 – (3*2) = 30 – 6 = 24
is that what you were looking for?
another way to think of it is as there are 5 observations in each cell (5 people who are drug/no-drug and one of the levels of depression). you can calculate and say n-1 is 5-1 = 4. multiply this 4 by 3(levels of depression)*2(levels of drug) and you get 4 * (3*2), or 4 * 6 = 24.
Thanks!
Similar to Ashley’s question, when is it appropriate to run independent t-tests compared to a tukey test? This past homework confused me. I don’t quite understand why we could not do a tukey test for example. How can I know if a post hoc test would be more or less appropriate than t-tests?
Melanie Levitt Psy 365
a Tukey post hoc test can tell you if for a given independent variable there are differences among the levels on the dependent variable. you would run this after a statistically significant omnibus test. this is more stringent than running a bunch of t-tests all at once (in terms of compounding Type I errors).
now, if i am reading into your question correctly, we are most concerned with why we ran t-tests after our 3 x 2 factorial ANOVA. the reason is because we found a statistically significant interaction with our data. this interaction meant that one variable was affecting our dependent variable differently for the different levels of the second independent variable. thus, we knew that drugs (2 levels) were affecting GPA (dependent variable) differently for the 3 levels of depression. so, we needed 3 separate 1:1 tests. we needed to test drug vs. non-drug for each of the levels of depression separately. these are separate independent t-tests. because t-tests are not as stringent as Tukey post hocs, we need a Bonferroni correction to our alpha criterion. three t-tests means our alpha (.05) becomes .05/3.
Tukey post hoc tests can tell us after statistically significant omnibus tests whether there are differences among the levels of an independent variable (greater than 2 levels).
with our data, we were not looking at post hoc tests among the levels of depression (3) because this main effect was not significant. only drug (2 levels, no post hocs required) and interaction effects were significant.
does that help?
Completely, thank you.
In the past homework that was returned, I am confused about the amount of type I errors we conclude for a mixed measures ANOVA. Wouldn’t a type I error be to reject the null across all variables and say that an effect exists when it actually does not as a whole? Why would we break it down into separate type I errors, when the purpose of the study is to make a significant conclusion overall?
Bruce Bickford psy365
Bruce,
The conclusions we can/must draw from a Mixed Repeated Measures ANOVA stem from the number of independent variables we have as well as the interaction of those variables.
Remember that the mixed RMANOVA includes both within-subjects and between-subjects indepedent variables. We are interested in whether or not each of the independent variables has a signficant effect on the dependent variable (main effects). Thus, we should discuss the significance and meaning of the results of those tests.
We are also interested in whether or not the within-subjects and between-subjects variables interact (interaction effect). This also needs to be addressed in terms of significance and meaning.
So, in the case of Gender and Time affecting desire to kill Dr. Ramey, we want to know:
1. Does Gender effect desire to kill? (That is important in it’s own right, is it not?)
2. Does Time effect desire to kill? (also important, right?)
3. Do Gender and Time interact to effect desire to kill? (also, important. And if there is an interaction, it might be hiding something, thus we need post hocs).
Address the sig. and meaning of all three tests (all the main effects and the interaction effects) and you’re set up to have a more complete picture.
-Sean McCormick TA PSY 365
Looking at the results I geet very confused because there are a million (slight exaggeration) different outputs. Do we always look at the Greenhouse one? Also, as far as contasts go, how do we know when to perform them? Is it only when we conduct a post hoc test, or is it regardless?
Dominique Pratel PSYCH 365
Dominique,
So far we have only looked at the results from the Greenhouse-Geisser tests.
We have done that because our data violated an assumption that needs to be met in order to look at “Sphericity Assumed” test. (basically, this is occuring because we are using small data sets to practice with, although, this assumption is often violated with real research data).
In all the HWs thus far, we know that we have violated the assumption of sphericity because (Mauchley’s Test of Sphericity came out significant, which is BAD for when testing assumptions). Thus we use the Greenhouse-Geisser, which uses a different degrees of freem in order to account for the violation of this assumption.
So, is it completely cut and dry, no matter what use the Greenhouse-Geisser for the rest of your research career? No.
However, I would say that it is very likely the case that any data we use in this class will have violated the assumption of sphericity.
Regarding your question about the contrasts and post hocs, check out the question and response above. If that doesn’t help, let us know.
-Sean McCormick TA PSY 365
How do we know when we are to use a univariate or repeated measures anova? I understand how to use spss to get an analysis but I am confused on how we would know which ones to use.
Tom Oakes
Psy365
Tom,
Use the repeated measures when you want to compare multiple time points. It’s like a dependent t-test, the has, for example pre and post measures, or week 1 and week 2. The RM ANOVA could have pre, 1 week, 2 weeks, ….n weeks, (more than 2 times points).
-Sean McCormick TA PSY 365
the repeated measures anova involves a single within-subjects factor with multiple levels. it could be time, as we have done in class, but need not. the issue, it needs to be the same group (e.g.) measured multiple times.
In relation to AJ’s question – I know how to program SPSS to give me the means and standard deviations, but the questions on the homework always asks to find the standard deviation regardless of one variable. The standard deviation given is in relation to both variables interacting with one another. Could you clarify further how to find standard deviation and mean?
Nadia Fernand, PSY 365
Nadia,
If you selected all the correct descriptives in Options as discussed above then you just need to look at the rows that say “Total”.
-Sean McCormick TA PSY 365
What’s the significance of performing a simple and repeated contrast vs. the Polynomial contrast? What can we find in the simple and repeated contrast that we can in the polynomial?
Rick McEwan, Psyc 365
What’s the significance of performing a simple and repeated contrast vs. the Polynomial contrast? What can we find in the simple and repeated contrast that we cannot** find in the polynomial?
Rick McEwan, Psyc 365
polynomial contrasts, rick, are used with within-subjects factors (e.g., time) to tell what the shape of the trend of the data is. is it linear (like a line) or perhaps quadratic (like a U).
Rick,
Sometimes specific comparisons are left out depending on which contrast you choose. Sometimes they go in order, or they might compare the 1st to all.
For example, a Simple Contrast will get you contrasts between 1 & 4, 2&4, 3&4. If you wanted to check for sig. difference between 1&2 or 2&3, for example, you need to run an additional contrast (in that case a Repeated Contrast).
Reminder: Don’t forget to click “Change” once you’ve chosen a different Contrast!
-Sean McCormick TA PSY 365
I find it confusing reporting the f statistic and df for the main effects and interaction because I don’t always know what tables to look at in the Output file. Also, I find it difficult to make conclusions based on the data, regardless of significance, mostly because of the different variables and different types of ANOVAs.
Jessica St.John PSY 365
i think it’s best if you talk with sean or me in person, as visual analysis of SPSS Output will help.
My question is the same as Tom’s question. How do we know when we are to use a univariate or repeated measures anova? I am always confused on this?
univariate in your question refers to factorial ANOVA. factorial ANOVAs (we have done in class) involve two independent variables affecting one dependent variable.
repeated measures ANOVA involve one within-subjects factor (e.g., time) affecting one dependent variable.
After doing a Factorial ANOVA, if we notice on the graph that there seems to be an interaction between the two variables on the dependent variable(like on the homework we just did with Drugs and Depression on GPA) do we have to do an independent t-test for all subsets of each variable, or just the one we noticed from the plot?
Stevannie, PSY 365
graphs are good visual aids. that’s all.
statistical significance is probability issue. how unlikely is it that we are making a mistake when we say we have found some result, for example. the lower the p-value, the more unlikely are results are due to chance. so, we always need to perform a statistical test to make ANY conclusion. graphs can point you in the right direction, but can be deceptive. interpreting them, you could miss things and also get your hopes up.
the independent samples t-tests you mention in your factorial ANOVA were performed because we had a statistically significant interaction and one IV had 2 levels and one IV had 3. so, we needed to follow up where the differences (indicated by the interaction) were.
When we have two independent variables how do we run the contrasts, the simple, repeated and polynomial? Or is it even possible? What do the contrasts tell us anyhow? The last homework was really confusing..
Gabriela Marginean Psy260
Gabi,
Careful, it’s not the number of variables, but the number of levels/groups in the variable. If we are looking for differences in say Gender (male or female) we could not run a contrast because we know that the significant difference must have been between males and females. However, if we were looking at a variable with 3 or more levels, such as Time (Week 1, Week 2, Week 3, Week 4, for example), the Repeated Measures ANOVA would only tell that there is a difference somewhere in these variables, but not which ones specifically. We run contrasts and/or post hocs to determine which specific differences are meaningful. For example, maybe the desire to kill between Week 1 and Week 2 are very different, but the desire to kill between Week 3 & 4 is not very big. That would be nice to know, right?! :)
In other words, contrasts and post hocs allow us to make more specific comparisons than the omnibus test.
-Sean McCormick TA PSY365
polynomial contrasts, recall, are like linear or quadratic or cubic. these are trends in your data (e.g., over time). how you change over time could look like a line going up or down.
simple and repeated contrasts are contrasting two different time points with each other. for example, we could contrast your performance in basketball on day one of a camp and data 4. this is a 1:1 contrast. are they different? we could also look at your change over time from day 1 to 2 to 3 to 4. if it goes up overall over that time, it could look like a line. a polynomial contrast, if performed, could tell you whether it really does look like a line.
One thing that’s confusing for me is when reporting our statistics for repeated measures or factorial ANOVA’s why do we use the “error” choice instead of any of the others. At times it seems that the correct df is 30 based on our different groups sizes, yet the “error” choice is something like 24.
could you give a specific example of different error terms from which you are choosing? that is, what are you looking at where you are choosing between 30 and 24, where you think the former is better?
I have another question from the last homework… I think I understand why and how to do the three t-tests, but I am not sure where the Bonferroni correction comes in. In other words, I know that it is necessary and why, but is there something in SPSS that we need to do, or do we just use an appropriately smaller alpha that we calculate on our own?
Melissa, PSY 365
i like to keep the Bonferroni in human hands. just divide your alpha by the number of tests you ran. test your obtained p-value against that new criterion. (if you think about it, that’s what you do anyway. spss gives you your p and then you compare it to your pre-determined alpha.)
When running any type of ANOVA, is the test statistic always the F ratio? If not, how do we determine what is?
the test statistic for an ANOVA is the F value. and yes, the F value is the ration of two variances (between over within).