jina-ai / Jina Hub

Jina's one-stop registry for hosting Pods and enabling easy neural search on the cloud

Contribute • Jobs • Website • Slack

Use Hub Image in Flow API

# my_image_tag is the built image name

from jina import Flow
with Flow().add(uses='docker://' + my_image_tag):
    pass

Create a new Hub Pod/App

Prerequisites

• Install Docker.
• pip install "jina[hub]"

📦 Create a new Executor

jina hub new --type pod

It will start a wizard in CLI to guide you create your first executor. The resulted file structure should look like the following:

MyAwesomeExecutor/
├── Dockerfile
├── manifest.yml
├── config.yml
├── README.md
├── requirements.txt
├── __init__.py
└── tests/
    ├── test_MyAwesomeExecutor.py
    └── __init__.py

Link to the documentation for creating a containerized pod is here.

Create a new App

jina hub new --type app

It will start a wizard in CLI to guide you create your app.

Upload your Image to Jina Hub

Build your Docker image using our naming conventions:

jina hub build -t jinahub/type.kind.jina-image-name:image_version-jina_version <your_folder>

Push to Jina Hub:

jina hub login, then copy/paste the token into GitHub to verify your account

jina hub push jinahub/type.kind.jina-image-name:image-jina_version 

Use py_modules to import multiple files

By default, jina hub new --pod creates a Python module structure and guides you to write MyAwesomeExecutor class into __init__.py. If you have some other files that need to be imported for MyAwesomeExecutor, say helper.py, you can change metas.pymodules in config.yml to import those files. Note, you have to write the dependencies in reverse order. That is, if __init__.py depends on A.py, which again depends on B.py, then you need to write:

!MyAwesomeExecutor
with:
  ...
metas:
  py_modules:
    - B.py
    - A.py
    - __init__.py

Legacy Hub Pod structure

This is also a valid structure, it works but not recommended:

- MyAwesomeExecutor/
    |
    |- Dockerfile
    |- manifest.yml
    |- README.md
    |- requirements.txt
    |- MyAwesomeExecutor.py
    |- helper.py
    |- config.yml
        |- metas.py_modules
            |- helper.py
            |- MyAwesomeExecutor.py

Please note that:

• Here MyAwesomeExecutor directory is not a python module, as it lacks __init__.py under the root;
• to import foo.py, you must to use from jinahub.foo import bar, where jinahub is the common namespace for all external modules;
• in config.yml:metas.py_modules, helper.py needs to be put before MyAwesomeExecutor.py.

Test a Pod/App locally

jina hub build /MyAwesomeExecutor/

More Hub CLI usage can be found via jina hub build --help

Work with your own repository

1. Create a new repository
2. pip install "jina[hub]" && jina hub new --type pod

git checkout -b feat-new-executor
git add .
git commit -m "feat: add new executor"
git push

4. Add the Hub Updater and Hub Builder Github Actions to the Github workflow of your hub-type repo.
5. Make a Pull Request on feat-new-executor -> master

Reference

manifest.yml Schema

manifest.yml must exist if you want to publish your Pod image to Jina Hub.

manifest.yml annotates your image so that it can be managed by Jina Hub. To get better appealing on Jina Hub, you should carefully set manifest.yml to the correct values.

Naming conventions

All apps and executors should follow the naming convention:

jinahub/type.kind.jina_image_name:image_version-jina_version

For example:

jinahub/pod.encoder.transformertfencoder:0.0.16-0.9.33

From Jina 0.4.10 onwards, Jina Hub is referred as a Git Submodule in jina-ai/jina.

Contributing

We welcome all kinds of contributions from the open-source community, individuals and partners. Without your active involvement, Jina won't be successful.

Please first read the contributing guidelines before the submission.

Contributors ✨

Community

• Code of conduct - play nicely with the Jina community
• Slack workspace - join #general on our Slack to meet the team and ask questions
• YouTube channel - subscribe to the latest video tutorials, release demos, webinars and presentations.
• LinkedIn - get to know Jina AI as a company and find job opportunities
• - follow and interact with us using hashtag #JinaSearch
• Company - know more about our company and how we are fully committed to open-source.

Open Governance

As part of our open governance model, we host Jina's Engineering All Hands in public. This Zoom meeting recurs monthly on the second Tuesday of each month, at 14:00-15:30 (CET). Everyone can join in via the following calendar invite.

• Add to Google Calendar
• Download .ics

The meeting will also be live-streamed and later published to our YouTube channel.

Join us

Jina is an open-source project. We are hiring full-stack developers, evangelists, and PMs to build the next neural search ecosystem in open source.

License

Copyright (c) 2020 Jina AI Limited. All rights reserved.

Jina is licensed under the Apache License, Version 2.0. See LICENSE for the full license text.