All Projects → allegro → Embedded Elasticsearch

allegro / Embedded Elasticsearch

Licence: apache-2.0
Tool that ease up creation of integration tests with Elasticsearch

Programming Languages

java
68154 projects - #9 most used programming language

Projects that are alternatives of or similar to Embedded Elasticsearch

Elastix
A simple Elasticsearch REST client written in Elixir.
Stars: ✭ 231 (-10.12%)
Mutual labels:  elasticsearch
Eui
Elastic UI Framework 🙌
Stars: ✭ 3,248 (+1163.81%)
Mutual labels:  elasticsearch
Datashare
Better analyze information, in all its forms
Stars: ✭ 254 (-1.17%)
Mutual labels:  elasticsearch
Eland
Python Client and Toolkit for DataFrames, Big Data, Machine Learning and ETL in Elasticsearch
Stars: ✭ 235 (-8.56%)
Mutual labels:  elasticsearch
Retail Demo Store
AWS Retail Demo Store is a sample retail web application and workshop platform demonstrating how AWS infrastructure and services can be used to build compelling customer experiences for eCommerce, retail, and digital marketing use-cases
Stars: ✭ 238 (-7.39%)
Mutual labels:  elasticsearch
Elastic
An Elasticsearch REST API client for Rust
Stars: ✭ 248 (-3.5%)
Mutual labels:  elasticsearch
Springboot Learning Example
spring boot 实践学习案例,是 spring boot 初学者及核心技术巩固的最佳实践。
Stars: ✭ 14,640 (+5596.5%)
Mutual labels:  elasticsearch
K8s
Important production-grade Kubernetes Ops Services
Stars: ✭ 253 (-1.56%)
Mutual labels:  elasticsearch
Sist2
Lightning-fast file system indexer and search tool
Stars: ✭ 245 (-4.67%)
Mutual labels:  elasticsearch
Dsiem
Security event correlation engine for ELK stack
Stars: ✭ 255 (-0.78%)
Mutual labels:  elasticsearch
Esbulk
Bulk indexing command line tool for elasticsearch
Stars: ✭ 235 (-8.56%)
Mutual labels:  elasticsearch
Typo3 Docker Boilerplate
🍲 TYPO3 Docker Boilerplate project (NGINX, Apache HTTPd, PHP-FPM, MySQL, Solr, Elasticsearch, Redis, FTP)
Stars: ✭ 240 (-6.61%)
Mutual labels:  elasticsearch
Elastik Nearest Neighbors
Go to: https://github.com/alexklibisz/elastiknn
Stars: ✭ 249 (-3.11%)
Mutual labels:  elasticsearch
Relevant Search Book
Code and Examples for Relevant Search
Stars: ✭ 231 (-10.12%)
Mutual labels:  elasticsearch
Advanced Java
😮 Core Interview Questions & Answers For Experienced Java(Backend) Developers | 互联网 Java 工程师进阶知识完全扫盲:涵盖高并发、分布式、高可用、微服务、海量数据处理等领域知识
Stars: ✭ 59,142 (+22912.45%)
Mutual labels:  elasticsearch
Syncclient
syncClient,数据实时同步中间件(同步mysql到kafka、redis、elasticsearch、httpmq)!
Stars: ✭ 227 (-11.67%)
Mutual labels:  elasticsearch
Elasticsearch Analysis Stconvert
STConvert is analyzer that convert chinese characters between traditional and simplified.中文简繁體互相转换.
Stars: ✭ 247 (-3.89%)
Mutual labels:  elasticsearch
Search flip
Full-Featured ElasticSearch Ruby Client with a Chainable DSL
Stars: ✭ 256 (-0.39%)
Mutual labels:  elasticsearch
Elasticsearch
Free and Open, Distributed, RESTful Search Engine
Stars: ✭ 57,778 (+22381.71%)
Mutual labels:  elasticsearch
Rwa Trivia
Trivia App - Real World Angular series
Stars: ✭ 251 (-2.33%)
Mutual labels:  elasticsearch

This project is no longer maintained. We recommend using Testcontainers instead: https://www.testcontainers.org/modules/elasticsearch/

embedded-elasticsearch

Build Status Maven Central

Small utility for creating integration tests that use Elasticsearch. Instead of using Node it downloads Elasticsearch in specified version and starts it in a separate process. It also allows you to install required plugins which is not possible when using NodeBuilder. Utility was tested with 1.x, 2.x, 5.x and 6.x versions of Elasticsearch.

Introduction

All you need to do to use this tool is create EmbeddedElastic instance. To do so, use provided builder:

final embeddedElastic = EmbeddedElastic.builder()
        .withElasticVersion("5.0.0")
        .withSetting(PopularProperties.TRANSPORT_TCP_PORT, 9350)
        .withSetting(PopularProperties.CLUSTER_NAME, "my_cluster")
        .withPlugin("analysis-stempel")
        .withIndex("cars", IndexSettings.builder()
            .withType("car", getSystemResourceAsStream("car-mapping.json"))
            .build())
        .withIndex("books", IndexSettings.builder()
            .withType(PAPER_BOOK_INDEX_TYPE, getSystemResourceAsStream("paper-book-mapping.json"))
            .withType("audio_book", getSystemResourceAsStream("audio-book-mapping.json"))
            .withSettings(getSystemResourceAsStream("elastic-settings.json"))
            .build())
        .build()
        .start()

When you are done with creating it, you can start it real simple:

embeddedElastic.start()

And that's all, you can connect to your embedded-elastic instance on specified port and use it in your tests.

Available builder options

Method Description
withElasticVersion(String version) version of Elasticsearch; based on that version download url to official Elasticsearch repository will be created
withDownloadUrl(URL downloadUrl) if you prefer to download Elasticsearch from a different location than official repositories you can do that using this method
withInResourceLocation(String inResourcePath) you can also have your Elasticsearch package inside resource directory, specify it's location with this option
withSetting(String key, Object value) setting name and value as in elasticsearch.yml file
withPlugin(String expression) plugin that should be installed into Elasticsearch; treat expression as argument to ./elasticsearch-plugin install <expression> command; use multiple times for multiple plugins
withIndex(String indexName, IndexSettings indexSettings) specify index that should be created and managed by EmbeddedElastic
withTemplate(String templateName, String templateBody) specify a template that should be created and managed by EmbeddedElastic
withStartTimeout(long value, TimeUnit unit) specify timeout you give Elasticsearch to start
withInstallationDirectory(File installationDirectory) specify custom installation directory
withDownloadDirectory(File downloadDirectory) specify custom download directory where downloaded distribution packages will be saved
withCleanInstallationDirectoryOnStop(boolean cleanInstallationDirectoryOnStop) specify whether clean the installation directory after Elasticsearch stop
withEsJavaOpts(String javaOpts) value of ES_JAVA_OPTS variable to be set for Elasticsearch process
withJavaHome(JavaHomeOption javaHomeOption) select java environment to run in. For available options see below
getTransportTcpPort() get transport tcp port number used by Elasticsearch instance
getHttpPort() get http port number used by Elasticsearch instance
withDownloadProxy(Proxy proxy) proxy that should be used for downloading Elasticsearch package
withDownloaderConnectionTimeout(long value, TimeUnit unit) connection timeout that should be used by downloader
withDownloaderReadTimeout(long value, TimeUnit unit) socket timeout that should be used by downloader

Available IndexSettings.Builder options

Method Description
withType(String type, String mapping) specify type and it's mappings
withSettings(String settings) specify index settings

Availabe JavaHomeOption options

Method Description
useSystem() default behavior, lets elasticsearch startup script determine the JRE to run in
inheritTestSuite() use the same JRE as the process starting it
path(String path) manually set the path of the JRE to execute the embedded elastic

Available operations

EmbeddedElastic provides following operations:

Method Description
start() downloads Elasticsearch and specified plugins, setups everything and finally starts your Elasticsearch instance
stop() stops your Elasticsearch instance and removes all data
index index your document, comes with variants that take only document, or document and it's id
deleteIndex(String indexName), deleteIndices() deletes index with name specified during EmbeddedElastic creation
createIndex(String indexName), createIndices() creates index with name specified during EmbeddedElastic creation; note that this index is created during EmbeddedElastic startup, you will need this method only if you deleted your index using deleteIndex method
recreateIndex(String indexName), recreateIndices() combination of deleteIndex and createIndex
refreshIndices() refresh index; useful when you make changes in different thread, and want to check results instantly in tests
deleteTemplate(String templateName), deleteTemplates() deletes a template of the specified name/all templates during EmbeddedElastic creation
createTemplate(String templateName), createTemplates() creates a template with the specified name/all templates during EmbeddedElastic creation; note that this template is created during EmbeddedElastic startup, you will need this method only if you deleted your template using deleteTemplate method
recreateTemplate(String templateName), recreateTemplates() combination of deleteTemplate and createTemplate

Example

If you want to see example, look at this spec: pl.allegro.tech.search.embeddedelasticsearch.EmbeddedElasticSpec

Dependency

To start using embedded-elasticsearch in your project add it as a test dependency:

Gradle:

testCompile 'pl.allegro.tech:embedded-elasticsearch:2.7.0'

Maven:

<dependency>
    <groupId>pl.allegro.tech</groupId>
    <artifactId>embedded-elasticsearch</artifactId>
    <version>2.7.0</version>
    <scope>test</scope>
</dependency>

SBT:

libraryDependencies ++= Seq("pl.allegro.tech" % "embedded-elasticsearch" % "2.7.0" % "test")

Known problems

If you build your project on Travis, you may have problems with OOM errors when using default settings. You can change Elasticsearch memory settings using withEsJavaOpts method. Example (from spec pl.allegro.tech.embeddedelasticsearch.EmbeddedElasticSpec):

    static EmbeddedElastic embeddedElastic = EmbeddedElastic.builder()
            .withElasticVersion(ELASTIC_VERSION)
            .withSetting(TRANSPORT_TCP_PORT, TRANSPORT_TCP_PORT_VALUE)
            .withSetting(CLUSTER_NAME, CLUSTER_NAME_VALUE)
            .withEsJavaOpts("-Xms128m -Xmx512m")
            .withIndex(CARS_INDEX_NAME, CARS_INDEX)
            .withIndex(BOOKS_INDEX_NAME, BOOKS_INDEX)
            .withStartTimeout(1, MINUTES)
            .build()
            .start()

Running more then one Elasticsearch instance

There are cases where you might want to run more than one Elasticsearch instance e.g.:

  • running tests of one project in parallel (e.g. using gradle --parallel or mvn -T1C)
  • running tests of different projects on the same physical mashine (e.g. Jenkins jobs running on the same server)
  • running integration tests which require more than one Elasticsearch instance

In such situations you should use distinct values for following settings for each instance:

  • withSetting(PopularProperties.TRANSPORT_TCP_PORT, ...)
  • withSetting(PopularProperties.HTTP_PORT, ...)
  • withInstallationDirectory(...)

With such configuration embedded-elasticsearch will redownload elasticsearch installation package for every distinct installation directory. To avoid this behavior and reuse downloaded installation package you should set common location of downloaded files with withDownloadDirectory(...) for every embedded-elasticsearch configuration.

License

embedded-elasticsearch is published under Apache License 2.0.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].