GitPlanet

Cheap and reliable Node.js hosting starts at $3/month, and $1/month static HTML hosting

Created with love in Canada, visit hostnodejs.com today

Feel like to post an Ad? Learn Details
All Projects → OpenASR → idear

OpenASR / idear

Licence: Apache-2.0 license
🎙️ Handsfree Audio Development Interface

Programming Languages

kotlin
9241 projects
shell
77523 projects

Labels

accessibilityintellijspeechspeech-synthesisspeech-recognitionintellij-pluginvoice-controlvoice-assistantvosk-api

Projects that are alternatives of or similar to idear

anycontrol
Voice control for your websites and applications
Stars: ✭ 53 (-36.9%)
Mutual labels:  speech, speech-recognition, voice-control, voice-assistant
react-native-spokestack
Spokestack: give your React Native app a voice interface!
Stars: ✭ 53 (-36.9%)
Mutual labels:  speech-synthesis, speech-recognition, voice-control, voice-assistant
spokestack-android
Extensible Android mobile voice framework: wakeword, ASR, NLU, and TTS. Easily add voice to any Android app!
Stars: ✭ 52 (-38.1%)
Mutual labels:  speech, speech-synthesis, speech-recognition, voice-assistant
KeenASR-Android-PoC
A proof-of-concept app using KeenASR SDK on Android. WE ARE HIRING: https://keenresearch.com/careers.html
Stars: ✭ 21 (-75%)
Mutual labels:  speech, speech-recognition, voice-control, voice-assistant
Sonus
💬 /so.nus/ STT (speech to text) for Node with offline hotword detection
Stars: ✭ 532 (+533.33%)
Mutual labels:  speech, speech-recognition, voice-control
leon
🧠 Leon is your open-source personal assistant.
Stars: ✭ 8,560 (+10090.48%)
Mutual labels:  speech-synthesis, speech-recognition, voice-assistant
spokestack-ios
Spokestack: give your iOS app a voice interface!
Stars: ✭ 27 (-67.86%)
Mutual labels:  speech-synthesis, speech-recognition, voice-assistant
musicologist
Music advice from a conversational interface powered by Algolia
Stars: ✭ 19 (-77.38%)
Mutual labels:  speech-recognition, voice-control, voice-assistant
porfir
Голосовой ассистент Порфирьевич
Stars: ✭ 23 (-72.62%)
Mutual labels:  speech-synthesis, speech-recognition, voice-assistant
Java Speech Api
The J.A.R.V.I.S. Speech API is designed to be simple and efficient, using the speech engines created by Google to provide functionality for parts of the API. Essentially, it is an API written in Java, including a recognizer, synthesizer, and a microphone capture utility. The project uses Google services for the synthesizer and recognizer. While this requires an Internet connection, it provides a complete, modern, and fully functional speech API in Java.
Stars: ✭ 490 (+483.33%)
Mutual labels:  speech, speech-synthesis, speech-recognition
Lingvo
Lingvo
Stars: ✭ 2,361 (+2710.71%)
Mutual labels:  speech, speech-synthesis, speech-recognition
Pydantic Pycharm Plugin
PyCharm plugin for pydantic. This plugin provides autocompletion, inspection, type-checking, inserting unfilled argument, and more.
Stars: ✭ 116 (+38.1%)
Mutual labels:  intellij, intellij-plugin
Caster
Dragonfly-Based Voice Programming and Accessibility Toolkit
Stars: ✭ 242 (+188.1%)
Mutual labels:  accessibility, voice-control
Mainframer Intellij Plugin
An intellij idea plugin for mainframer project
Stars: ✭ 125 (+48.81%)
Mutual labels:  intellij, intellij-plugin
Elm Plugin
Elm language support plugin for IntelliJ IDEA.
Stars: ✭ 137 (+63.1%)
Mutual labels:  intellij, intellij-plugin
Idea Conventional Commit
Context and template-based completion for conventional/semantic commits.
Stars: ✭ 115 (+36.9%)
Mutual labels:  intellij, intellij-plugin
Intellij Visual Studio Dark Theme
Visual Studio Dark Theme for IDEA Intellij
Stars: ✭ 125 (+48.81%)
Mutual labels:  intellij, intellij-plugin
Mavenhelper
IntelliJ plugin - https://plugins.jetbrains.com/plugin/7179
Stars: ✭ 138 (+64.29%)
Mutual labels:  intellij, intellij-plugin
Idea Php Toolbox
Collections of tools and improvements to make PhpStorm a little bit better
Stars: ✭ 147 (+75%)
Mutual labels:  intellij, intellij-plugin
Idea Gradle Dependencies Formatter
Gradle dependencies formatter for IntelliJ IDEA
Stars: ✭ 156 (+85.71%)
Mutual labels:  intellij, intellij-plugin
View All Similar Projects ➔

idiolect icon idiolect

Deploy

A general purpose voice user interface for the IntelliJ Platform, inspired by Tavis Rudd. Possible use cases: visually impaired and RSI users. Originally developed as part of a JetBrains hackathon, it is now a community-supported project. For background information, check out this presentation.

Usage

To get started, press the Voice control button in the toolbar, then speak a command, e.g. "Hi, IDEA!" Idiolect supports a simple grammar. For a complete list of commands, please refer to the wiki. Click the button once more to deactivate.

Building

For Linux or macOS users:

git clone https://github.com/OpenASR/idiolect && cd idiolect && ./gradlew runIde

For Windows users:

git clone https://github.com/OpenASR/idiolect & cd idiolect & gradlew.bat runIde

Recognition works with most popular microphones (preferably 16kHz, 16-bit). For best results, minimize background noise.

Contributing

Contributors who have IntelliJ IDEA installed can simply open the project. Otherwise, run the following command from the project's root directory:

./gradlew runIde -PluginDev

Architecture

Idiolect is implemented using the IntelliJ Platform SDK. For more information about the plugin architecture, please refer to the wiki page.

Integration with Idiolect

plugin.xml defines a number of <extensionPoint>s which would allow other plugins to integrate with or extend/customise the capabilities of Idiolect.

AsrProvider

Listens for audio input, recognises speech to text and returns an NlpRequest with possible utterances. Does not resolve the intent.

Possible alternative implementations could:

  • integrate with Windows SAPI 5 Speech API
  • integrate with Dragon/Nuance API

NlpProvider

Processes an NlpRequest. The default implementation invokes IdeService.invokeAction(ExecuteVoiceCommandAction, nlpRequest) and the action is handled by ExecuteVoiceCommandAction and ActionRecognizerManager.handleNlpRequest()

AsrSystem

Processes audio input, recognises speech to text and executes actions. The default implementation AsrControlLoop uses the AsrProvider and NlpProvider.

Some APIs such as AWS Lex implement the functionality of AsrProvider and NlpProvider in a single call.

IntentResolver

Processes an NlpRequest (utterance/alternatives) and resolves an NlpResponse with intentName and slots. ActionRecognizerManager.handleNlpRequest() iterates through the IntentResolvers until it finds a match.

The Idiolect implementations use either exact-match or regular expressions on the recognized text. Alternative implementations may use AI to resolve the intent.

CustomPhraseRecognizer

Many of the auto-generated trigger phrases are not suitable for voice activation. You can add your own easier to say and remember phrases in ~/.idea/phrases.properties

IntentHandler

Fulfills an NlpResponse (intent + slots), performing desired actions. ActionRecognizerManager.handleNlpRequest() iterates through the IntentHandlers until the intent is actioned.

TemplateIntentHandler

Handles two flavours of intent prefix:

  • Template.id.${template.id} eg: Template.id.maven-dependency
  • Template.${template.groupName}.${template.key} eg: Template.Maven.dep

template.id is often null. template.key is the "Abbreviation" that you would normally type before pressing TAB.

The default trigger phrases are generated from the template description or key and are often not suitable for voice activation. You can add your own trigger phrase -> live template mapping in ~/.idea/phrases.properties and it will be resolved by CustomPhraseRecognizer.

ttsProvider

Reads audio prompts/feedback to the user

org.openasr.idiolect.nlp.NlpResultListener

Any interfaces which are registered to the topic in plugin.xml under <applicationListeners> will be notified when

  • listening state changes
  • recognition is returned by the AsrProvider
  • request is fulfilled by an IntentHandler
  • there is a failure
  • a prompt/message is provided for the user

Plugin Actions

plugin.xml defines <action>s:

VoiceRecordControllerAction

This action is invoked when the user clicks on the Voice control button in the toolbar. This simply tells AsrService to activate or standby. When the AsrService is active, the AsrSystem,

by default ASRControlLoop (see below).

ExecuteActionFromPredefinedText

A debugging aid to use one of the ActionRecognizer extension classes configured in plugin.xml to generate an ActionCallInfo which is then runInEditor().

ExecuteVoiceCommandAction

Similar to ExecuteActionFromPredefinedText but uses the Idiolect.VoiceCommand.Text data attached to the invoking AnActionEvent.

IDEA Actions

There are many Actions (classes which extend AnAction) provided by IDEA:

ASRControlLoop

When AsrControlLoop detects an utterance, it invokes PatternBasedNlpProvider.processUtterance() which typically calls invokeAction() and/or one or more of the methods of IdeService

Programming By Voice

Maintainers

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].