Missing Data and Outliers Analysis

Missing Data and Outliers Analysis
1. Describe missing data, provide summary of missing data, similar to the analysis in the Chapter 2 (table 3): Count of missing data/percent per variable, type of missing data (NA, null), total percent of missingness per dataset [ – 20pts]
· If you find that there is no missing data, you still have to report your findings. Your reader does not know your data and you have to show how reliable your data is
2. Plot visualization of missing data pattern [ – 20pts]
· If you do not have missing data, you still need to plot it to show it to your reader
3. Describe what type of missing data you have observed: MCAR, MAR, MNAR or no missing data [- 10pts]
· If you do not have missing data, simply state that you did not observe any missing values
4. Select Imputation method that you will be performing and explain why [20pts]:
· list-wise/pair-wise deletions, mean imputation, regression imputation etc. [ – 10pts]
· perform imputation and provide data statistics. For example, if you perform list-wise deletion, how many observations will you use for consequent analysis. If you perform regression imputation, provide statistic summary [-10pts]
· If you do not have any missing data: instead of imputation, perform data normalization, describe which methods you will use, why and perform the normalization. Provide statistical summary of data after the normalization
5. Outlier analysis:
Option 1.
· Continuous Y:
· You will perform outlier detection using Z-Score, a parametric outlier detection method in one dimensional space. [30pts]
· Z-Score Formula:
· If the data points > |3|, these are extreme values (outliers):
· You can use z from the library (outliers) or build your own function:
Measure – mean(Measure)) / sd(Measure)
· You can test for outliers only for your dependent variable Y (continuous). How many outliers do you have?
Option 2.
· See Chapter 13.3 Checking for Outliers. http://daviddalpiaz.github.io/appliedstats/model-diagnostics.html