project_3

Human Action Recognition (HAR)

Topic - Machine Learning techniques in sensor based human activity recognition and understanding human actions using still images. Folder for sensor based HAR -"sensor based HAR"

Folder for image classification - "Image Classification"

Sensor-Based HAR

Jupyter notebooks - "DNN_model.ipynb" "LSTM_model.ipynb" "exploratory1.ipynb" Resources folder - "WISDM_ar_latest" Data source for sensor based HAR - WISDM dataset

http://www.cis.fordham.edu/wisdm/dataset.php

Citation - "Activity Recognition using cell phone Accelerometers" - Jennifer R Kwapisz,Gary M.Weiss,Samuel A.Moore

The source consist of raw time series data and transformed dataset.

The raw timeseries data consist of - Raw time series data
Number of examples 1,098,207
Number of attributes – 6
Class Distribution
Walking: 424,400 (38.6%)
Jogging: 342,177 (31.2%)
Upstairs: 122,869 (11.2%)
Downstairs: 100,427 (9.1%)
Sitting: 59,939 (5.5%)
Standing: 48,395 (4.4%)
Contains all x,y,z acceleration values

The reformatted dataset is formed by statistical measures taking 10 second window with 46 transformed attributes in it as given below-

XAVG, YAVG, ZAVG XPEAK, YPEAK, ZPEAK XABSOLDEV, YABSOLDEV, ZABSOLDEV XSTANDDEV, YSTANDDEV, ZSTANDDEV RESULTANT is the average of the square roots of the sum of the values of each axis squared √(xi^2 + yi^2 + zi^2). X0,X1X2…….Z8,Z9

Using the raw data we can plot the timeseries data showing the x,y,z acceleration of different users.For example While jogging the y acceleration would be maximum with less difference between the peaks.While standing the y accelration would be minimum.We can check the profiles for every activity.

The above figure shows the acceleration values for jogging.

The above figure shows the accelration values for walking.

Two techniques were used for sensor based HAR 1.Deep Neural Network(DNN) 2.Long short-term Memory(LSTM)

1.Deep neural network(DNN) was used on the reformated data
There are two models using DNN
Model 1 with 3 layers gives an accuracy of 87.4%

Model 2 with extra hidden layers gives an accuracy of 82%

2.LSTM model was used on raw time series data There are two models using the LSTM technique
Model 1 using 200 steps with Sliding window of 50 ((16692,200,3)) gives an accuracy of 86.1%

Model 2 using 200 steps with Non sliding window((4173,200,3)) gives an accuracy of 87.7%

The above image shows the confusuin matrix for the best model (LSTM with no sliding window- 87.7% accuracy)

Web App for sensor based HAR

We designed a web app based on sensor based HAR.It gives the predicted class and original label on clicking the submit button

Image-Based HAR

Transfer learning techniques were used to build an image classification model that recognizes various classes of human activity. The model was trained on images from the Stanford 40 dataset. You can find out more information about the data here.

We used a only a portion of the full dataset, training on images from the following ten action classes.

Classes

• Applauding
• Climbing
• Cooking
• Feeding a horse
• Holding an umbrella
• Jumping
• Playing guitar
• Riding a bike
• Riding a horse
• Walking the dog

The resulting model achieved a classification accuracy of 96%.