llSourcell / Tensorflow.js_explained
Programming Languages
⚡️ Fast In-Browser Object Detection 👀
Detect objects in images right in your browser using Tensorflow.js! Currently takes ~800ms to analyze each frame on Chrome MBP 13" mid-2014.
Supports Tiny YOLO, as of right now,
tfjs does not have
support to run any full YOLO models (and your user's computers probably
can't handle it either).
Demo
(You can only get so far with 1 FPS)
Install
Yarn
yarn add tfjs-yolo-tiny
Or NPM
npm install tfjs-yolo-tiny
Usage Example
import yolo, { downloadModel } from 'tfjs-yolo-tiny';
const model = await downloadModel();
const inputImage = webcam.capture();
const boxes = await yolo(inputImage, model);
// Display detected boxes
boxes.forEach(box => {
const {
top, left, bottom, right, classProb, className,
} = box;
drawRect(left, top, right-left, bottom-top, `${className} ${classProb}`)
});
API Docs
yolo(input, model, classProbThreshold, iouThreshold, filterBoxesThreshold)
Args
| Param | Type | Default | Description |
|---|---|---|---|
| input | tf.Tensor | - | Expected shape (1, 416, 416, 3) Tensor representing input image (RGB 416x416) |
| model | tf.Model | - | Tiny YOLO tf.Model |
| classProbThreshold | Number | 0.4 | Don't return detections below a certain class probability. |
| iouThreshold | Number | 0.4 | Don't return boxes that have more intersection over union than a more likely box. See non max suppression. |
| filterBoxesThreshold | Number | 0.01 | Don't return boxes that have a box_prob * class_prob of less than this threshold. |
Returns
Returns an array of objects.
| Property | Type | Description |
|---|---|---|
| top | Number | Pixels from top of image where bounding box starts |
| left | Number | Pixels from left of image where bounding box starts |
| bottom | Number | Pixels from top of image where box ends. |
| right | Number | Pixels from left of image where box ends. |
| classProb | Number | Probability of the class in the bounding box. |
| className | String | Human name of the class. |
downloadModel(url)
Args
| Param | Type | Default | Description |
|---|---|---|---|
| url | string | See DEFAULT_MODEL_LOCATION | Tiny YOLO Model config path. See tf.loadModel |
Returns
Returns a Promise that can resolve to a tf.Model.
Contributing
PR's are more than welcome! Perf improvement or better test coverage are probably the two biggest areas of immediate need. If you have thoughts on extensibility as well, feel free to open an issue!
Install Dependencies
yarn install
Run Tests
If you're running tests, make sure to yarn add @tensorflow/[email protected]
so that you you don't get tfjs package not found errors. If you're developing,
make sure to remove tfjs as a dependency, as it'll start using the
local version of tfjs intead of the peer version.
Note: Test coverage is poor, definitely don't rely on them to catch your errors.
yarn test
Build
yarn build
Or during development, use watch mode, you can use the demo app to test out changes.
yarn watch