Question

What is the difference between t-tests and ANOVA versus regression (one big difference)? Explain it as if you were teaching someone.

How many groups are you allowed to have in an independent samples t-test? How many groups are you allowed to have in an one-way ANOVA?

What are some of the advantages of Multiple Regression?

How is regression with a single variable the same and different from a correlation?

Answer 1

1. t- tests and ANOVA are two different statistical tests which are used in hypothesis testing. The two are different from each other in that while t tests are used for comparing two groups, ANOVA is used for comparing several groups. Thus a t test can be said to be a special case of ANOVA. ANOVA yields F tests as the statistic. ANOVA will usually test factor variables and whether or not between group variance is significant. F-test is a squared t-test (F= 2 t).

Regression is used as a statistical tool to test the impact of some predictor variables on a response variable. Seen in this way, ANOVA can be said to be a special case of multiple regression.

2. In independent t tests, the researcher can have two categorical, independent groups to measure the independent variable. For example, in a study on the effect of gender on experience of stress, gender with the two groups: male and female would be the two independent groups for running the t test.

However, when more than two variable scores are involved, then we use ANOVA or F tests. For ANOVA would involve three or more categorical, independent groups, but it can be used for just two groups also (although it should be mentioned that independent-samples t-test is more commonly used for two groups). Example independent variables in ANOVA include ethnic groups: Hispanic, African American, Asian, Caucasian, etc. in a study measuring the influence of ethnicity on occupational success.

3. Multiple Regression is used to examine the relationship between two or more independent variables and a dependent variable. The biggest advantage of multiple regression analysis is that they allow us draw a comparison between several predictor variables or independent variables on our criterion value( dependent variable). Thus, for instance, advertisers can use multiple regression to study how far does each of the three independent variables- food packaging, jingles, nutritional information influence positive consumer behaviour and accordingly use the data to design a successful advertising campaign.

Another major advantage of Multiple Regression is that It help to to identify outliers, or anomalies. For instance, in determining the allocation of grades in an exam, the school assessment board could find that the number of hours worked, the class size and the teaching material available, all had a strong correlation to the student’s test scores, while student’s own intellectual ability did not. Alternatively, it could be that all of the listed predictor values were correlated to each of the classes being examined, except for one class who was being consistently assigned a higher grade compared to the others.

4. Regression with a single variable is the same as Correlation coefficient as the standardised regression coefficient or the line of best fit is the same as Pearson's correlation coefficient r.

Moreover, Neither simple linear regression nor correlation answer questions of causality directly. That is, they both don’t make any inference that XX causes YY.

The two however are different because correlation typically refers to the linear relationship, but it can refer to other forms of dependence, such as polynomial or truly nonlinear relationships. Linear Regression only pertains to linear relationship between variables along the line of best fit.

The regression equation (i.e., a+bXa+bX) can be used to make predictions on Y axis based on values of X axis, we get the same answer correlation X with Y or vice versa, but the regression of Y on X is different to that of X on Y.