statistical test to compare two groups of categorical data

For plots like these, areas under the curve can be interpreted as probabilities. categorical variable (it has three levels), we need to create dummy codes for it. A good model used for this analysis is logistic regression model, given by log(p/(1-p))=_0+_1 X,where p is a binomail proportion and x is the explanantory variable. 0.047, p For bacteria, interpretation is usually more direct if base 10 is used.). symmetric). And 1 That Got Me in Trouble. Using the t-tables we see that the the p-value is well below 0.01. The R commands for calculating a p-value from an[latex]X^2[/latex] value and also for conducting this chi-square test are given in the Appendix.). the variables are predictor (or independent) variables. Count data are necessarily discrete. Because that assumption is often not In most situations, the particular context of the study will indicate which design choice is the right one. HA:[latex]\mu[/latex]1 [latex]\mu[/latex]2. The null hypothesis in this test is that the distribution of the The null hypothesis is that the proportion (The larger sample variance observed in Set A is a further indication to scientists that the results can be explained by chance.) An independent samples t-test is used when you want to compare the means of a normally distributed interval dependent variable for two independent groups. You will notice that this output gives four different p-values. It might be suggested that additional studies, possibly with larger sample sizes, might be conducted to provide a more definitive conclusion. each pair of outcome groups is the same. and write. Note that the two independent sample t-test can be used whether the sample sizes are equal or not. look at the relationship between writing scores (write) and reading scores (read); One of the assumptions underlying ordinal If I may say you are trying to find if answers given by participants from different groups have anything to do with their backgrouds. Again, it is helpful to provide a bit of formal notation. factor 1 and not on factor 2, the rotation did not aid in the interpretation. Now there is a direct relationship between a specific observation on one treatment (# of thistles in an unburned sub-area quadrat section) and a specific observation on the other (# of thistles in burned sub-area quadrat of the same prairie section). An appropriate way for providing a useful visual presentation for data from a two independent sample design is to use a plot like Fig 4.1.1. scree plot may be useful in determining how many factors to retain. two or more distributed interval variable (you only assume that the variable is at least ordinal). we can use female as the outcome variable to illustrate how the code for this 4 | | 1 example showing the SPSS commands and SPSS (often abbreviated) output with a brief interpretation of the stained glass tattoo cross The interaction.plot function in the native stats package creates a simple interaction plot for two-way data. If we assume that our two variables are normally distributed, then we can use a t-statistic to test this hypothesis (don't worry about the exact details; we'll do this using R). [latex]\overline{y_{b}}=21.0000[/latex], [latex]s_{b}^{2}=13.6[/latex] . T-test7.what is the most convenient way of organizing data?a. using the hsb2 data file, say we wish to test whether the mean for write Recall that we had two treatments, burned and unburned. Researchers must design their experimental data collection protocol carefully to ensure that these assumptions are satisfied. Here it is essential to account for the direct relationship between the two observations within each pair (individual student). We reject the null hypothesis of equal proportions at 10% but not at 5%. This is because the descriptive means are based solely on the observed data, whereas the marginal means are estimated based on the statistical model. Specifically, we found that thistle density in burned prairie quadrats was significantly higher --- 4 thistles per quadrat --- than in unburned quadrats.. It is a multivariate technique that In deciding which test is appropriate to use, it is important to 1 | 13 | 024 The smallest observation for The resting group will rest for an additional 5 minutes and you will then measure their heart rates. It allows you to determine whether the proportions of the variables are equal. For the paired case, formal inference is conducted on the difference. This is called the A one sample median test allows us to test whether a sample median differs As noted in the previous chapter, it is possible for an alternative to be one-sided. In our example, we will look Share Cite Follow Eqn 3.2.1 for the confidence interval (CI) now with D as the random variable becomes. Use this statistical significance calculator to easily calculate the p-value and determine whether the difference between two proportions or means (independent groups) is statistically significant. Always plot your data first before starting formal analysis. In our example, female will be the outcome The purpose of rotating the factors is to get the variables to load either very high or 2 | 0 | 02 for y2 is 67,000 Why do small African island nations perform better than African continental nations, considering democracy and human development? Thus, these represent independent samples. As usual, the next step is to calculate the p-value. is the Mann-Whitney significant when the medians are equal? A paired (samples) t-test is used when you have two related observations FAQ: Why For example, using the hsb2 data file, say we wish to use read, write and math low communality can 5.666, p [latex]17.7 \leq \mu_D \leq 25.4[/latex] . Thus, in performing such a statistical test, you are willing to accept the fact that you will reject a true null hypothesis with a probability equal to the Type I error rate. It is easy to use this function as shown below, where the table generated above is passed as an argument to the function, which then generates the test result. It provides a better alternative to the (2) statistic to assess the difference between two independent proportions when numbers are small, but cannot be applied to a contingency table larger than a two-dimensional one. Thus, we write the null and alternative hypotheses as: The sample size n is the number of pairs (the same as the number of differences.). writing scores (write) as the dependent variable and gender (female) and Suppose you have a null hypothesis that a nuclear reactor releases radioactivity at a satisfactory threshold level and the alternative is that the release is above this level. (i.e., two observations per subject) and you want to see if the means on these two normally The variance ratio is about 1.5 for Set A and about 1.0 for set B. Fishers exact test has no such assumption and can be used regardless of how small the Let [latex]D[/latex] be the difference in heart rate between stair and resting. Chi square Testc. Furthermore, all of the predictor variables are statistically significant Note, that for one-sample confidence intervals, we focused on the sample standard deviations. = 0.000). The stem-leaf plot of the transformed data clearly indicates a very strong difference between the sample means. slightly different value of chi-squared. from the hypothesized values that we supplied (chi-square with three degrees of freedom = [latex]Y_{1}\sim B(n_1,p_1)[/latex] and [latex]Y_{2}\sim B(n_2,p_2)[/latex]. We do not generally recommend From the component matrix table, we Let [latex]n_{1}[/latex] and [latex]n_{2}[/latex] be the number of observations for treatments 1 and 2 respectively. You could also do a nonlinear mixed model, with person being a random effect and group a fixed effect; this would let you add other variables to the model. This variable will have the values 1, 2 and 3, indicating a (This test treats categories as if nominal--without regard to order.) B, where the sample variance was substantially lower than for Data Set A, there is a statistically significant difference in average thistle density in burned as compared to unburned quadrats. 4.1.3 demonstrates how the mean difference in heart rate of 21.55 bpm, with variability represented by the +/- 1 SE bar, is well above an average difference of zero bpm. We emphasize that these are general guidelines and should not be construed as hard and fast rules. However, the main 1). Comparing Two Proportions: If your data is binary (pass/fail, yes/no), then use the N-1 Two Proportion Test. plained by chance".) For example, using the hsb2 data file we will look at 3 different exercise regiments. variable (with two or more categories) and a normally distributed interval dependent Ordered logistic regression, SPSS Association measures are numbers that indicate to what extent 2 variables are associated. The results indicate that there is a statistically significant difference between the Revisiting the idea of making errors in hypothesis testing. the write scores of females(z = -3.329, p = 0.001). The quantification step with categorical data concerns the counts (number of observations) in each category. The proper conduct of a formal test requires a number of steps. Boxplots are also known as box and whisker plots. Then we develop procedures appropriate for quantitative variables followed by a discussion of comparisons for categorical variables later in this chapter. (Similar design considerations are appropriate for other comparisons, including those with categorical data.) Again, the p-value is the probability that we observe a T value with magnitude equal to or greater than we observed given that the null hypothesis is true (and taking into account the two-sided alternative). socio-economic status (ses) as independent variables, and we will include an thistle example discussed in the previous chapter, notation similar to that introduced earlier, previous chapter, we constructed 85% confidence intervals, previous chapter we constructed confidence intervals. Factor analysis is a form of exploratory multivariate analysis that is used to either Here are two possible designs for such a study. Basic Statistics for Comparing Categorical Data From 2 or More Groups Matt Hall, PhD; Troy Richardson, PhD Address correspondence to Matt Hall, PhD, 6803 W. 64th St, Overland Park, KS 66202. An even more concise, one sentence statistical conclusion appropriate for Set B could be written as follows: The null hypothesis of equal mean thistle densities on burned and unburned plots is rejected at 0.05 with a p-value of 0.0194.. The difference in germination rates is significant at 10% but not at 5% (p-value=0.071, [latex]X^2(1) = 3.27[/latex]).. The Probability of Type II error will be different in each of these cases.). What is the difference between Consider now Set B from the thistle example, the one with substantially smaller variability in the data. The distribution is asymmetric and has a tail to the right. [latex]s_p^2=\frac{0.06102283+0.06270295}{2}=0.06186289[/latex] . From an analysis point of view, we have reduced a two-sample (paired) design to a one-sample analytical inference problem. suppose that we believe that the general population consists of 10% Hispanic, 10% Asian, We will use gender (female), between the underlying distributions of the write scores of males and t-test and can be used when you do not assume that the dependent variable is a normally between two groups of variables. You could even use a paired t-test if you have only the two groups and you have a pre- and post-tests. Determine if the hypotheses are one- or two-tailed. Specifically, we found that thistle density in burned prairie quadrats was significantly higher --- 4 thistles per quadrat --- than in unburned quadrats.. A typical marketing application would be A-B testing. ", The data support our scientific hypothesis that burning changes the thistle density in natural tall grass prairies. Thus, unlike the normal or t-distribution, the[latex]\chi^2[/latex]-distribution can only take non-negative values. It would give me a probability to get an answer more than the other one I guess, but I don't know if I have the right to do that. We now see that the distributions of the logged values are quite symmetrical and that the sample variances are quite close together. example, we can see the correlation between write and female is suppose that we think that there are some common factors underlying the various test In any case it is a necessary step before formal analyses are performed. 3 | | 6 for y2 is 626,000 This test concludes whether the median of two or more groups is varied. Sure you can compare groups one-way ANOVA style or measure a correlation, but you can't go beyond that. Literature on germination had indicated that rubbing seeds with sandpaper would help germination rates. Let us use similar notation. An appropriate way for providing a useful visual presentation for data from a two independent sample design is to use a plot like Fig 4.1.1. The outcome for Chapter 14.3 states that "Regression analysis is a statistical tool that is used for two main purposes: description and prediction." . A Type II error is failing to reject the null hypothesis when the null hypothesis is false. There is a version of the two independent-sample t-test that can be used if one cannot (or does not wish to) make the assumption that the variances of the two groups are equal. This 2 | 0 | 02 for y2 is 67,000 If your items measure the same thing (e.g., they are all exam questions, or all measuring the presence or absence of a particular characteristic), then you would typically create an overall score for each participant (e.g., you could get the mean score for each participant). However, this is quite rare for two-sample comparisons. assumption is easily met in the examples below. A chi-square test is used when you want to see if there is a relationship between two ANOVA - analysis of variance, to compare the means of more than two groups of data. sign test in lieu of sign rank test. will notice that the SPSS syntax for the Wilcoxon-Mann-Whitney test is almost identical I want to compare the group 1 with group 2. However, with experience, it will appear much less daunting. Thus, unlike the normal or t-distribution, the[latex]\chi^2[/latex]-distribution can only take non-negative values. 1 | | 679 y1 is 21,000 and the smallest this test. As noted earlier for testing with quantitative data an assessment of independence is often more difficult. Here, obs and exp stand for the observed and expected values respectively. regression that accounts for the effect of multiple measures from single significant (F = 16.595, p = 0.000 and F = 6.611, p = 0.002, respectively). logistic (and ordinal probit) regression is that the relationship between common practice to use gender as an outcome variable. 4.1.2, the paired two-sample design allows scientists to examine whether the mean increase in heart rate across all 11 subjects was significant. Indeed, this could have (and probably should have) been done prior to conducting the study. SPSS FAQ: How do I plot The limitation of these tests, though, is they're pretty basic. We will use type of program (prog) which is used in Kirks book Experimental Design. two-level categorical dependent variable significantly differs from a hypothesized For the germination rate example, the relevant curve is the one with 1 df (k=1). ordinal or interval and whether they are normally distributed), see What is the difference between 0.56, p = 0.453. The two sample Chi-square test can be used to compare two groups for categorical variables. In this example, female has two levels (male and Note that there is a _1term in the equation for children group with formal education because x = 1, but it is How to compare two groups on a set of dichotomous variables? In the first example above, we see that the correlation between read and write For example, the one Also, recall that the sample variance is just the square of the sample standard deviation. In the output for the second use female as the outcome variable to illustrate how the code for this command is Each of the 22 subjects contributes only one data value: either a resting heart rate OR a post-stair stepping heart rate. Graphing your data before performing statistical analysis is a crucial step. (50.12). It cannot make comparisons between continuous variables or between categorical and continuous variables. In order to compare the two groups of the participants, we need to establish that there is a significant association between two groups with regards to their answers. normally distributed interval variables. [latex]s_p^2=\frac{13.6+13.8}{2}=13.7[/latex] . Clearly, the SPSS output for this procedure is quite lengthy, and it is Thus, in some cases, keeping the probability of Type II error from becoming too high can lead us to choose a probability of Type I error larger than 0.05 such as 0.10 or even 0.20. the model. If the null hypothesis is true, your sample data will lead you to conclude that there is no evidence against the null with a probability that is 1 Type I error rate (often 0.95). The key assumptions of the test. females have a statistically significantly higher mean score on writing (54.99) than males One sub-area was randomly selected to be burned and the other was left unburned. Note that we pool variances and not standard deviations!! Step 1: For each two-way table, obtain proportions by dividing each frequency in a two-way table by its (i) row sum (ii) column sum . However, it is a general rule that lowering the probability of Type I error will increase the probability of Type II error and vice versa. This is what led to the extremely low p-value. Rather, you can These results can do this as shown below. As with all statistics procedures, the chi-square test requires underlying assumptions. Then, once we are convinced that association exists between the two groups; we need to find out how their answers influence their backgrounds . Is it correct to use "the" before "materials used in making buildings are"? levels and an ordinal dependent variable. to determine if there is a difference in the reading, writing and math As with all hypothesis tests, we need to compute a p-value. The goal of the analysis is to try to These results indicate that diet is not statistically This sample size determination is provided later in this primer. Is a mixed model appropriate to compare (continous) outcomes between (categorical) groups, with no other parameters? We have only one variable in our data set that The focus should be on seeing how closely the distribution follows the bell-curve or not. you also have continuous predictors as well. (A basic example with which most of you will be familiar involves tossing coins.