All Projects → freedomofkeima → Transfer Learning Anime

freedomofkeima / Transfer Learning Anime

Transfer Learning for Anime Characters Recognition

Programming Languages

python
139335 projects - #7 most used programming language

Projects that are alternatives of or similar to Transfer Learning Anime

MoeFlow
Repository for anime characters recognition website, powered by TensorFlow
Stars: ✭ 113 (-27.1%)
Mutual labels:  anime, classification, transfer-learning
Ner Bert
BERT-NER (nert-bert) with google bert https://github.com/google-research.
Stars: ✭ 339 (+118.71%)
Mutual labels:  classification, transfer-learning
Face Tracking With Anime Characters
Hello! I have made a Python project where YURI from the game doki doki literature club accesses the webcam and stares directly into the players soul. Hope you enjoy!
Stars: ✭ 320 (+106.45%)
Mutual labels:  opencv, anime
Face Mask Detection
Face masks are crucial in minimizing the propagation of Covid-19, and are highly recommended or even obligatory in many situations. In this project, we develop a pipeline to detect unmasked faces in images. This can, for example, be used to alert people that do not wear a mask when entering a building.
Stars: ✭ 37 (-76.13%)
Mutual labels:  classification, opencv
Caffe2 Ios
Caffe2 on iOS Real-time Demo. Test with Your Own Model and Photos.
Stars: ✭ 221 (+42.58%)
Mutual labels:  classification, opencv
neuralBlack
A Multi-Class Brain Tumor Classifier using Convolutional Neural Network with 99% Accuracy achieved by applying the method of Transfer Learning using Python and Pytorch Deep Learning Framework
Stars: ✭ 36 (-76.77%)
Mutual labels:  classification, transfer-learning
Skin Cancer Image Classification
Skin cancer classification using Inceptionv3
Stars: ✭ 16 (-89.68%)
Mutual labels:  classification, transfer-learning
Anime Inpainting
An application tool of edge-connect, which can do anime inpainting and drawing. 动漫人物图片自动修复,去马赛克,填补,去瑕疵
Stars: ✭ 761 (+390.97%)
Mutual labels:  opencv, anime
Phormatics
Using A.I. and computer vision to build a virtual personal fitness trainer. (Most Startup-Viable Hack - HackNYU2018)
Stars: ✭ 79 (-49.03%)
Mutual labels:  classification, opencv
Captchouli
booru-backed procedurally-generated anime image captcha library and server
Stars: ✭ 104 (-32.9%)
Mutual labels:  opencv, anime
Selfdrivingcar
A collection of all projects pertaining to different layers in the SDC software stack
Stars: ✭ 107 (-30.97%)
Mutual labels:  classification, opencv
Transfer Learning Suite
Transfer Learning Suite in Keras. Perform transfer learning using any built-in Keras image classification model easily!
Stars: ✭ 212 (+36.77%)
Mutual labels:  classification, transfer-learning
Keras transfer cifar10
Object classification with CIFAR-10 using transfer learning
Stars: ✭ 120 (-22.58%)
Mutual labels:  classification, transfer-learning
Trace.moe
Anime Scene Search by Image
Stars: ✭ 3,231 (+1984.52%)
Mutual labels:  opencv, anime
Deeppicar
Deep Learning Autonomous Car based on Raspberry Pi, SunFounder PiCar-V Kit, TensorFlow, and Google's EdgeTPU Co-Processor
Stars: ✭ 242 (+56.13%)
Mutual labels:  opencv, transfer-learning
Sru Deeplearning Workshop
دوره 12 ساعته یادگیری عمیق با چارچوب Keras
Stars: ✭ 66 (-57.42%)
Mutual labels:  classification, transfer-learning
Autoannotationtool
A label tool aim to reduce semantic segmentation label time, rectangle and polygon annotation is supported
Stars: ✭ 113 (-27.1%)
Mutual labels:  classification, opencv
Robovision
AI and machine leaning-based computer vision for a robot
Stars: ✭ 126 (-18.71%)
Mutual labels:  classification, opencv
Stb Tester
Automated Testing for Set-Top Boxes and Smart TVs
Stars: ✭ 148 (-4.52%)
Mutual labels:  opencv
Sltk
An OpenCV-based structured light processing toolkit.
Stars: ✭ 151 (-2.58%)
Mutual labels:  opencv

Transfer Learning for Anime Characters

Warning: This repository size is quite big (approx. 100 MB) since it includes training and test images.

Introduction

This repository is the continuation of Flag #15 - Image Recognition for Anime Characters.

In Flag #15, we can see that Transfer Learning works really well with 3 different anime characters: Nishikino Maki, Kotori Minami, and Ayase Eli.

In this experiment, we will try to push Transfer Learning further, by using 3 different anime characters which have hair color similarity: Nishikino Maki, Takimoto Hifumi, and Sakurauchi Riko.

This experiment has 3 main steps:

  1. Utilize lbpcascade_animeface to recognize character face from each images
  2. Resize each images to 96 x 96 pixels
  3. Split images into training & test before creating the final model

raw directory contains 36 images for each characters (JPG & PNG format). The first 30 images are used for training while the last 6 images are used for test.

As an example, we got the following result after applying Step 1 (cropped directory is shown at the right side):

lbpcascade_animeface can detect character faces with an accuracy of around 83%. Failed images are stored in raw (unrecognized) for future improvements.

Since we have 3 characters and 6 test images for each which are not part of training, resized_for_test contains 18 images in total. Surprisingly, almost all characters are detected properly!

Update (Nov 13, 2017): See animeface-2009 section below, which push face detection accuracy to 93%.

Requirements

Steps

  1. The following command is used to populate cropped directory.
$ python bulk_convert.py raw/[character_name] cropped
  1. The following command is used to populate resized_for_training & resized_for_test directory.
$ python bulk_resize.py cropped/[character_name] resized

After running the step above, you can decide how many images will be used in resized_for_training and how many images will be used in resized_for_test.

  1. Re-train the Inception model by using transfer learning:
$ bazel-bin/tensorflow/examples/image_retraining/retrain --image_dir ~/transfer-learning-anime/resized_for_traning/
$ bazel build tensorflow/examples/image_retraining:label_image
  1. At this point, the model is ready to use. We can run the following command to get the classification result:
$ bazel-bin/tensorflow/examples/image_retraining/label_image --graph=/tmp/output_graph.pb --labels=/tmp/output_labels.txt --output_layer=final_result:0 --image=$HOME/transfer-learning-anime/resized_for_test/[character name]/[image name]

If everything works properly, you will get the classification result. See TensorFlow Documentation for more options.

Optionally, sample model can be downloaded by running download_model.sh script inside models (example) directory.

Result Analysis

Initially, we run the experiment with 2 characters: Nishikino Maki and Takimoto Hifumi.

INFO:tensorflow:2017-11-10 08:50:36.151387: Step 3999: Train accuracy = 100.0%
INFO:tensorflow:2017-11-10 08:50:36.151592: Step 3999: Cross entropy = 0.002191
INFO:tensorflow:2017-11-10 08:50:36.210147: Step 3999: Validation accuracy = 100.0% (N=100)
INFO:tensorflow:Final test accuracy = 92.9% (N=14)

The result is as the following:

Image Classification OK/NG
nishikino maki (score = 0.99874)
takimoto hifumi (score = 0.00126)
OK
nishikino maki (score = 0.75519)
takimoto hifumi (score = 0.24481)
OK
nishikino maki (score = 0.99513)
takimoto hifumi (score = 0.00487)
OK
nishikino maki (score = 0.98629)
takimoto hifumi (score = 0.01371)
OK
nishikino maki (score = 0.99723)
takimoto hifumi (score = 0.00277)
OK
nishikino maki (score = 0.99695)
takimoto hifumi (score = 0.00305)
OK
Image Classification OK/NG
takimoto hifumi (score = 0.63084)
nishikino maki (score = 0.36916)
OK
takimoto hifumi (score = 0.99728)
nishikino maki (score = 0.00272)
OK
takimoto hifumi (score = 0.99972)
nishikino maki (score = 0.00028)
OK
takimoto hifumi (score = 0.98852)
nishikino maki (score = 0.01148)
OK
takimoto hifumi (score = 0.99456)
nishikino maki (score = 0.00544)
OK
takimoto hifumi (score = 0.96630)
nishikino maki (score = 0.03370)
OK

From the result above, 10 out of 12 have threshold > 0.95, while the lowest threshold is 0.63.

At this point, I decided to add Sakurauchi Riko, which is known for its similarity to Nishikino Maki.

INFO:tensorflow:2017-11-10 13:13:59.270717: Step 3999: Train accuracy = 100.0%
INFO:tensorflow:2017-11-10 13:13:59.270912: Step 3999: Cross entropy = 0.005526
INFO:tensorflow:2017-11-10 13:13:59.328139: Step 3999: Validation accuracy = 100.0% (N=100)
INFO:tensorflow:Final test accuracy = 80.0% (N=15)

With 3 similar characters, the result is as the following:

Image Classification OK/NG
nishikino maki (score = 0.99352)
sakurauchi riko (score = 0.00612)
takimoto hifumi (score = 0.00036)
OK
nishikino maki (score = 0.47391)
sakurauchi riko (score = 0.37913)
takimoto hifumi (score = 0.14696)
OK
nishikino maki (score = 0.95976)
sakurauchi riko (score = 0.02797)
takimoto hifumi (score = 0.01227)
OK
nishikino maki (score = 0.88851)
sakurauchi riko (score = 0.07526)
takimoto hifumi (score = 0.03623)
OK
nishikino maki (score = 0.99025)
sakurauchi riko (score = 0.00766)
takimoto hifumi (score = 0.00209)
OK
nishikino maki (score = 0.96782)
sakurauchi riko (score = 0.02783)
takimoto hifumi (score = 0.00435)
OK

As you can see above, the similarity between Nishikino Maki and Sakurauchi Miko starts to lower down the confidence level of the resulted model. Nevertheless, all classifications are still correct, where 4 out of 6 maintain the threshold of > 0.95.

Image Classification OK/NG
takimoto hifumi (score = 0.86266)
nishikino maki (score = 0.13632)
sakurauchi riko (score = 0.00102)
OK
takimoto hifumi (score = 0.87614)
sakurauchi riko (score = 0.12334)
nishikino maki (score = 0.00051)
OK
takimoto hifumi (score = 0.99964)
sakurauchi riko (score = 0.00023)
nishikino maki (score = 0.00013)
OK
takimoto hifumi (score = 0.99417)
nishikino maki (score = 0.00472)
sakurauchi riko (score = 0.00110)
OK
takimoto hifumi (score = 0.94923)
sakurauchi riko (score = 0.04842)
nishikino maki (score = 0.00235)
OK
takimoto hifumi (score = 0.96029)
sakurauchi riko (score = 0.02822)
nishikino maki (score = 0.01150)
OK

Interestingly, the addition of 3rd character increases the confidence level of several Takimoto Hifumi testcases (see 1st and 4th result). Overall, this character can be easily differentiated compared to the other two.

Image Classification OK/NG
sakurauchi riko (score = 0.98747)
takimoto hifumi (score = 0.01054)
nishikino maki (score = 0.00199)
OK
sakurauchi riko (score = 0.96840)
takimoto hifumi (score = 0.02895)
nishikino maki (score = 0.00265)
OK
sakurauchi riko (score = 0.97713)
nishikino maki (score = 0.02167)
takimoto hifumi (score = 0.00119)
OK
sakurauchi riko (score = 0.90159)
nishikino maki (score = 0.06989)
takimoto hifumi (score = 0.02852)
OK
sakurauchi riko (score = 0.99713)
takimoto hifumi (score = 0.00184)
nishikino maki (score = 0.00103)
OK
sakurauchi riko (score = 0.79957)
nishikino maki (score = 0.19310)
takimoto hifumi (score = 0.00733)
OK

From this experiment, it seems that the current bottleneck is located at Step 1 (face detection), which have the overall accuracy of 83% in face detection.

animeface-2009

nagadomi/animeface-2009 provides another method of face detection. 13 out of 21 unrecognized images are now recognized in cropped (unrecognized) directory.

Current found limitations: it seems the script requires more memory and slower to run compared to lbpcascade_animeface.xml.

Image Classification OK/NG
nishikino maki (score = 0.99296)
sakurauchi riko (score = 0.00694)
takimoto hifumi (score = 0.00010)
OK
nishikino maki (score = 0.93702)
sakurauchi riko (score = 0.04017)
takimoto hifumi (score = 0.02281)
OK
nishikino maki (score = 0.99406)
sakurauchi riko (score = 0.00565)
takimoto hifumi (score = 0.00030)
OK
Image Classification OK/NG
takimoto hifumi (score = 0.99242)
nishikino maki (score = 0.00431)
sakurauchi riko (score = 0.00327)
OK
takimoto hifumi (score = 0.99596)
sakurauchi riko (score = 0.00403)
nishikino maki (score = 0.00001)
OK
takimoto hifumi (score = 0.98369)
sakurauchi riko (score = 0.01498)
nishikino maki (score = 0.00133)
OK
takimoto hifumi (score = 0.99796)
sakurauchi riko (score = 0.00189)
nishikino maki (score = 0.00015)
OK
takimoto hifumi (score = 0.99601)
nishikino maki (score = 0.00335)
sakurauchi riko (score = 0.00064)
OK
takimoto hifumi (score = 0.99960)
sakurauchi riko (score = 0.00029)
nishikino maki (score = 0.00011)
OK
takimoto hifumi (score = 0.99995)
nishikino maki (score = 0.00004)
sakurauchi riko (score = 0.00001)
OK
Image Classification OK/NG
sakurauchi riko (score = 0.84480)
nishikino maki (score = 0.12101)
takimoto hifumi (score = 0.03419)
OK
sakurauchi riko (score = 0.94310)
nishikino maki (score = 0.04296)
takimoto hifumi (score = 0.01393)
OK
sakurauchi riko (score = 0.96176)
takimoto hifumi (score = 0.03217)
nishikino maki (score = 0.00607)
OK

Since this method gives better result in detecting anime character face and classification still works with almost the same result, the overall face detection accuracy is now around 93%.

License

lbpcascade_animeface.xml is created by nagadomi/lbpcascade_animeface.

Copyright for all images are owned by their respective creators.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].