Slack

element43

Element43 is a free and open web application providing market data for players of the MMORPG EVE Online. This includes the near real-time collection and processing of order book data of the entire game's economy as well as the calculation of various metrics for analytics based on historic price data. Data can be accessed via a modern web-based UI or a simple HTTP API serving JSON while internal communication is based on gRPC. The application's backend is split into multiple modular components/services implemented in a variety of languages. Builds are executed automatically and the application can be deployed as a collection of Docker containers. Work on element43 was started by a small group of developers spread all over the globe in summer 2012.

Note: Currently there is no hosted version of Element43 available. However, you can easily self-host the components listed below by using the Docker images.

Contributing

Contributions are always welcome! Feel free to get in touch with us on Slack or file an issue in this repository. PRs can be made in individual service's repositories. We also have a getting started guide for new developers.

Components

This repo serves as the entry point into element43's infrastructure. Shared code such as the component's gRPC interface descriptions is also kept in the main repo. The components listed here are under active development and serve as the application's core. In Follow the links in the first column to access the component's code and documentation. Until we provide an updated version of our development environment and the getting started guide, just ping us on Slack if you have any questions. Running individual components is a matter of executing a Docker container configured as outlined in the individual service's docs.

A simplified diagram of element43's structure.

	CI Status	Container Image	Language	Description
vue43	-		TypeScript	Element43's SPA frontend based on VueJS/NuxtJS.
esi-markets			Rust	One of element43's core components. This service keeps an in-memory representation of EVE Online's global order book and records individual order's history.
jumpgate	-		Go	Proxies HTTP/JSON requests from the outside to gRPC/Protobuf calls to internal services.
static-data	-		Go	Proxies and caches batch-calls to various static data APIs (1st/3rd party).
top-stations			Go	Generates metrics for individual station's markets using data from esi-markets' API hourly
market-stats			Go	Generates price/volume statistics for the entire economy every night.
market-streamer	-		Go	Fetches market data from ESI, converts it into UUDIF and streams it via ZMQ. Drop-in replacement for EMDR. Not used in the main application, however still supported until the end of 2018 to help with the migration from EMDR to ESI.

Legacy Components

These are legacy components of the application's older iterations which no longer are under active development. They were superseeded by the components listed above.

	CI Status	Container Image	Language	Description
element43-django	-	-	Python 2 / Web	Legacy monolithic web application based on Django, switched to componentized structure due to reasons outlined here. Legacy docs can be found here.
node-43		-	JavaScript	The Django application's market data ingestion service. Takes data from EMDR and stores it in Postgres DB. Worked nicely, however integration of data/services via DB generates tight coupling between applications.
vagrant-element43	-	-	Vagrant/Ansible	A dev environment for the Django-based Element43 made with Vagrant and Ansible for getting started fast. Now everything is based on Docker and even faster.
emdr-to-nsq	-	-	Go	Takes market data from ZMQ stream, performs deduplication and pushes data onto a NSQ. Replaced by esi-markets.
order-server	-	-	Go	Stores order info from NSQ in Postgres DB and provides data as JSON via HTTP. Replaced by esi-markets.
search43	-	-	Python 3	A simple live-search API for Element43/EVE. Replaced by ESI's official endpoint.
emdr_consumer	-	-	Elixir	A market data consumer similar to emdr-to-nsq written in Elixir. Rewritten in Go because of library ecosystem and resource consumption.
orders	-	-	Elixir	A prototype of a market data storage backend similar to order-server written in Elixir.
static_data	-	-	Elixir	A prototype of a static data proxy similar to static-data written in Elixir.
crest	-	-	Elixir	A WIP Elixir client library for the now deprecated CREST API.
elixir-build-base	-	-	Docker	Base container image for Elixir-based builds. Not needed anymore.