All Projects → ajay-sreeram → Music2dance

ajay-sreeram / Music2dance

Licence: mit
Generating Dance steps for given music with deep learning

Projects that are alternatives of or similar to Music2dance

Sentimentanalysis
文本情感分析
Stars: ✭ 421 (+2376.47%)
Mutual labels:  jupyter-notebook, lstm
Multilabel Timeseries Classification With Lstm
Tensorflow implementation of paper: Learning to Diagnose with LSTM Recurrent Neural Networks.
Stars: ✭ 519 (+2952.94%)
Mutual labels:  jupyter-notebook, lstm
Tensorflow Lstm Regression
Sequence prediction using recurrent neural networks(LSTM) with TensorFlow
Stars: ✭ 433 (+2447.06%)
Mutual labels:  jupyter-notebook, lstm
Stock Prediction Models
Gathers machine learning and deep learning models for Stock forecasting including trading bots and simulations
Stars: ✭ 4,660 (+27311.76%)
Mutual labels:  jupyter-notebook, lstm
Tensorflow Tutorial
Some interesting TensorFlow tutorials for beginners.
Stars: ✭ 893 (+5152.94%)
Mutual labels:  jupyter-notebook, lstm
Easy Deep Learning With Keras
Keras tutorial for beginners (using TF backend)
Stars: ✭ 367 (+2058.82%)
Mutual labels:  jupyter-notebook, lstm
Cryptocurrencyprediction
Predict Cryptocurrency Price with Deep Learning
Stars: ✭ 453 (+2564.71%)
Mutual labels:  jupyter-notebook, lstm
Deeplearning.ai Assignments
Stars: ✭ 268 (+1476.47%)
Mutual labels:  jupyter-notebook, lstm
Telemanom
A framework for using LSTMs to detect anomalies in multivariate time series data. Includes spacecraft anomaly data and experiments from the Mars Science Laboratory and SMAP missions.
Stars: ✭ 589 (+3364.71%)
Mutual labels:  jupyter-notebook, lstm
Video Classification
Tutorial for video classification/ action recognition using 3D CNN/ CNN+RNN on UCF101
Stars: ✭ 543 (+3094.12%)
Mutual labels:  jupyter-notebook, lstm
Thesemicolon
This repository contains Ipython notebooks and datasets for the data analytics youtube tutorials on The Semicolon.
Stars: ✭ 345 (+1929.41%)
Mutual labels:  jupyter-notebook, lstm
Getting Things Done With Pytorch
Jupyter Notebook tutorials on solving real-world problems with Machine Learning & Deep Learning using PyTorch. Topics: Face detection with Detectron 2, Time Series anomaly detection with LSTM Autoencoders, Object Detection with YOLO v5, Build your first Neural Network, Time Series forecasting for Coronavirus daily cases, Sentiment Analysis with BERT.
Stars: ✭ 738 (+4241.18%)
Mutual labels:  jupyter-notebook, lstm
Image Captioning
Image Captioning using InceptionV3 and beam search
Stars: ✭ 290 (+1605.88%)
Mutual labels:  jupyter-notebook, lstm
Zhihu Text Classification
[2017知乎看山杯 多标签 文本分类] ye组(第六名) 解题方案
Stars: ✭ 392 (+2205.88%)
Mutual labels:  jupyter-notebook, lstm
Cryptocurrency Price Prediction
Cryptocurrency Price Prediction Using LSTM neural network
Stars: ✭ 271 (+1494.12%)
Mutual labels:  jupyter-notebook, lstm
Pytorch Ntm
Neural Turing Machines (NTM) - PyTorch Implementation
Stars: ✭ 453 (+2564.71%)
Mutual labels:  jupyter-notebook, lstm
Pytorch Seq2seq
Tutorials on implementing a few sequence-to-sequence (seq2seq) models with PyTorch and TorchText.
Stars: ✭ 3,418 (+20005.88%)
Mutual labels:  jupyter-notebook, lstm
Lstm Human Activity Recognition
Human Activity Recognition example using TensorFlow on smartphone sensors dataset and an LSTM RNN. Classifying the type of movement amongst six activity categories - Guillaume Chevalier
Stars: ✭ 2,943 (+17211.76%)
Mutual labels:  jupyter-notebook, lstm
Attention Networks For Classification
Hierarchical Attention Networks for Document Classification in PyTorch
Stars: ✭ 540 (+3076.47%)
Mutual labels:  jupyter-notebook, lstm
Stockpriceprediction
Stock Price Prediction using Machine Learning Techniques
Stars: ✭ 700 (+4017.65%)
Mutual labels:  jupyter-notebook, lstm

👯 Music2Dance 💃

Generating Dance steps for given music (CNN + LSTM + MDN)

Execution Steps:

1. Installing
    * sudo pip3 install torch torchvision tensorboardX tensorflow numpy scipy imageio opencv-python

2. Creating dataset from videos (Preprocessing videos)
    * python PoseNet/converter/tfjs2python.py
    * python index2_video_shot_processing.py
    * python index3_create_Only_in_out_dataset.py

3. Training
    * python index4_train_mdn_simple_rnn_bnc.py

4. Testing (works directly as pretrained weights included in repo)
    * run jupyter notebook in root directory
    * open index5_test_trained_model_simplernn_for_bnc.ipynb and run all cells
    * use some software(may be windows 10 photos app) to combine generated video and input audio

Results:

Classical Western Mixed
Music2Dance Classical Music2Dance Western Music2Dance Mixed

Music2Dance Cheat Sheet

Directory Structure:

Music2Dance(project root)
|
│   index2_video_shot_processing.py            - Process mp4 video files present in data directory
│   index3_create_Only_in_out_dataset.py       - Creating input/output bundles for training the model
│   index4_train_mdn_simple_rnn.py             - Process and train with training data
│   index4_train_mdn_simple_rnn_for_bnc.py     - Same as above but if training data has two types of dances (like western and classical)
│   index5_test_trained_model_simplernn.ipynb  - Testing trained model with random audio file
│   index5_test_trained_model_simplernn_for_bnc.ipynb
│   ReadMe.md
│
├───custom_utils                  - Utilities
│       datastft.py
│       mobilenet.py
│       video_shot_processing_utils.py
│
├───data
│   ├───test_audio
│   |       test_audio.wav
|   ├───audio_wav_indexed
|   |       audio (1).wav
|   └───video_wav_indexed
|           video (1).mp4
│
├───Object_detection              - Object Detection model to detect human bounding box before finding the pose 
│   │   mscoco_label_map.pbtxt
│   │   string_int_label_map.proto
│   │   visualization_utils.py
│   │
│   └───ssd_mobilenet_v1_coco_2018_01_28
│       │   checkpoint
│       │   frozen_inference_graph.pb
│       │   model.ckpt.data-00000-of-00001
│       │   model.ckpt.index
│       │   model.ckpt.meta
│       │   pipeline.config
│       │
│       └───saved_model
│           │   saved_model.pb
│           │
│           └───variables
├───output
│   ├───motiondance_simplernn     - Saved weights
│   │   └───checkpoints
│   │           epoch_{num}.pth.tar
│   │
│   ├───motiondance_simplernn_bnc
│   │   └───checkpoints
│   │           epoch_bnc_{num}.pth.tar
│   │
│   ├───Result                    - Test result video files are saved here
│   ├───Result_Audio
│   └───Result_Video
└───PoseNet
    └───converter
        │   config.yaml
        │   manifest.json
        │   tfjs2python.py        - Converting TensorflowJs model to python tensorflow understandable model
        │   wget.py
        │
        └───checkpoints
                checkpoint
                model.ckpt.data-00000-of-00001
                model.ckpt.index
                model.ckpt.meta        

References:


Copyright © 2018, Ajay Sreeram

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].