CoreML and Vision object detection with a pre-trained deep learning SSD model

This project shows how to use CoreML and Vision with a pre-trained deep learning SSD (Single Shot MultiBox Detector) model. There are many variations of SSD. The one we’re going to use is MobileNetV2 as the backbone this model also has separable convolutions for the SSD layers, also known as SSDLite. This app can find the locations of several different types of objects in the image. The detections are described by bounding boxes, and for each bounding box, the model also predicts a class.

Side Note

• Currently, I do not have an iPhone, so I'm unable to test the app on a physical device. I apologize in advance for maybe possible bugs.

  Kind regards,

  Ilija 🖖 😄

Requirements

• Physical device! Because the simulator does not have a camera
• Swift 4.2+
• Xcode 9.2+
• iOS 11.0+

Getting the files

• Use GitHub to clone the repository locally, or download the .zip file of the repository and extract the files.

Auto genereted model helper class

• Once the model is imported the compiler generates a model helper class on build path automatically. Then we can access the model through model helper class by creating an instance.

Model Metadata

• Here we can see the inputs the model aspects and the outputs it generates, as well as auto genereted model helper class.

Camera Usage Description

• Add permission in info.plist for device's camera access.

• Or you can open the info.plist file as raw XML and add the following code:

<key>NSCameraUsageDescription</key>
      <string>Camera Needed For Object Detection And Classification</string>

Import COCO Dataset Labels

License

MIT License

Copyright (c) 2019 Ilija Mihajlovic

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.