Skip to content

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

5 Commits
 
 
 
 
 
 
 
 

Repository files navigation

Getting-And-Cleaning-Data-Project

This project demonstrates my ability to get and clean data from a data source. The goal is to prepare tidy data that can be used for later analysis. This activity is based on Human Activity Recognition Using Smartphones Data Set ( Ref : http://archive.ics.uci.edu/ml/datasets/Human+Activity+Recognition+Using+Smartphones) Data Set Information:

The experiments have been carried out with a group of 30 volunteers within an age bracket of 19-48 years. Each person performed six activities (WALKING, WALKING_UPSTAIRS, WALKING_DOWNSTAIRS, SITTING, STANDING, LAYING) wearing a smartphone (Samsung Galaxy S II) on the waist. Using its embedded accelerometer and gyroscope, we captured 3-axial linear acceleration and 3-axial angular velocity at a constant rate of 50Hz. The experiments have been video-recorded to label the data manually. The obtained dataset has been randomly partitioned into two sets, where 70% of the volunteers was selected for generating the training data and 30% the test data.

The sensor signals (accelerometer and gyroscope) were pre-processed by applying noise filters and then sampled in fixed-width sliding windows of 2.56 sec and 50% overlap (128 readings/window). The sensor acceleration signal, which has gravitational and body motion components, was separated using a Butterworth low-pass filter into body acceleration and gravity. The gravitational force is assumed to have only low frequency components, therefore a filter with 0.3 Hz cutoff frequency was used. From each window, a vector of features was obtained by calculating variables from the time and frequency domain.

Attribute Information:

For each record in the dataset it is provided:

  • Triaxial acceleration from the accelerometer (total acceleration) and the estimated body acceleration.
  • Triaxial Angular velocity from the gyroscope.
  • A 561-feature vector with time and frequency domain variables.
  • Its activity label.
  • An identifier of the subject who carried out the experiment.

Feature Selection

The features selected for this database come from the accelerometer and gyroscope 3-axial raw signals tAcc-XYZ and tGyro-XYZ. These time domain signals (prefix 't' to denote time) were captured at a constant rate of 50 Hz. Then they were filtered using a median filter and a 3rd order low pass Butterworth filter with a corner frequency of 20 Hz to remove noise. Similarly, the acceleration signal was then separated into body and gravity acceleration signals (tBodyAcc-XYZ and tGravityAcc-XYZ) using another low pass Butterworth filter with a corner frequency of 0.3 Hz.

Subsequently, the body linear acceleration and angular velocity were derived in time to obtain Jerk signals (tBodyAccJerk-XYZ and tBodyGyroJerk-XYZ). Also the magnitude of these three-dimensional signals were calculated using the Euclidean norm (tBodyAccMag, tGravityAccMag, tBodyAccJerkMag, tBodyGyroMag, tBodyGyroJerkMag).

Finally a Fast Fourier Transform (FFT) was applied to some of these signals producing fBodyAcc-XYZ, fBodyAccJerk-XYZ, fBodyGyro-XYZ, fBodyAccJerkMag, fBodyGyroMag, fBodyGyroJerkMag. (Note the 'f' to indicate frequency domain signals).

These signals were used to estimate variables of the feature vector for each pattern:
'-XYZ' is used to denote 3-axial signals in the X, Y and Z directions.

The set of variables that were estimated from these signals are:

mean(): Mean value std(): Standard deviation

Additional vectors obtained by averaging the signals in a signal window sample. These are used on the angle() variable:

gravityMean tBodyAccMean tBodyAccJerkMean tBodyGyroMean tBodyGyroJerkMean

The complete list of variables of each feature vector is available in 'features.txt'

Script Functionality

This section explains how the data was obtained and cleaned and processed programmatically using R SCRIPTS

  • Download the Zip file and unzip its contents. Using download.file function, the zipped file was downloaded from the given URL. These files were unzipped into the current working directory.
  • Read the files provided in the unzipped directory into R the required text files as data frames. All the required text files like label files, feature_info files, activity_labels file and train set ( x & Y ) and test set t (X & Y ) files were read into R using the fread function.
  • Rename the columns to appropriate variable names. The column names in test and train set were labelled using values of features files
  • Combine the Train and the Test set
  • Correct the variable names. Variable names for subjects column was renamed to subject.
  • Add the activity data and rename the column. The dataframe was combined with the activity table.
  • Resulting Data Frame. The resulting dataframe was formed.
  • Select the variable names with mean and std value only from the result data frame.
  • Provide descriptive activity names. The activity names table was referenced and used to change the values in the result data frame.
  • Provide descriptive variable names. The variable names was corrected to drop the special characters and formed a meaning full variable name.
  • Groupby subject and activity and print the mean of each variable from the result data frame.
  • View provides a very tidy small data frame out of the collected data.

About

This project demonstrates my ability to get and clean data from a data source

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages