Migrate Sanic Currency Exchange Rates Api to Use Redis Stack @Redis Hackathon

Use Redis Stack (RedisJson) to replace Postgres as main storage, plus some refactoring

• README
• Contributing

How it works

How the data is stored:

The data is store as a document, for each day of the currency rates.

{
   "2020-01-01": {
      "USD": 1,
      "EUR": 1.1,
      "AUD": 0.97
   }
}

How the data is accessed:

In python codes, the data is accessed through a simple get

r.json().get('2020-01-01')

Performance Benchmarks

Load test is used here to measure the performance.

• same Machine (my personal macbook laptop)
• same spec for DB
  • 10GB Volume
  • Same ARM64 Instance

The 2 key results here are iterations (higher is better) and iteration_duration(avg)(lower is better)

This performance benchmark is surprising to myself as well, Redis as a well performanced DB, in most of time, it give a very good result of performance.

In sum, this performance compare may not be objective, the reasons behind it requires more researching. These are some potential reasons I assume,

1.Min Specs for different DB are difference, even the spec for Postgres and Redis Stack here are the same, but their min spec requirements may be different.

• Postgres: 1 GHz processor. 2 GB of RAM. 512 MB of HDD.
• Redis: 2.6GHz(2 CPUs), 4 GB of Ram, 10 GD of HDD (Redis Stack supposed to be higher)

2.Decimal in Python as a class could not be saved directly as a JSON attribute in RedisJSOM, so it was stores as string, and it will be converted to Decimal when read. This may slow the APP down.

3.Query was not used (RedisSearch) was not used, so when query history data, a loop of GET are sent to DB, which is not efficient

In sum, this comparing result doesn't tell which is better between Postgres and RedisJSON, it only says that at current stage, at current DB Instance spec, for this API, postgres gives better performance, and the way of Decimal Convert and Loop of Get in Application level slows the app too. (If anyone knows better about Redis to solve these problems, please let me know). So this comparing seems not very fair.

How to run it locally?

Prerequisites

• python 3.10
• poetry >1.0
• redis stack, please make sure you have a remote Redis Instance running, feel free to get one free from Redis Stack database in the cloud.
• docker 20.10.xx (optional)

Local installation and development

Local Python App Development

0. clone the repo git clone https://github.com/tim-hub/sanic-currency-exchange-rates-api.git
1. export REDIS_URL = 'redis://localhost:6379' (use your own redis stack url)
2. export USE_REDIS = 1
3. poetry install
4. poetry run python src/main.py

Through Docker

docker build -t rate-api . && docker run --name rate-api -t -i -e REDIS_URL=redis://your-remote-redis:6379 -e USE_REDIS=1 rate-api

or

Use a docker image.

docker run --name rate-api -p 8000:8000 -e REDIS_URL=redis://your-remote-redis:6379 -e USE_REDIS=1  ghcr.io/tim-hub/sanic-currency-exchange-rates-api:v2.3.0

Others

• How to Use Postgres DB at Contributing

Deployment

This app is wrapped in a docker container, so it could be deployer to any platform (which supports docker), From AWS, Heroku to K8s,

the only requirements for it are 2 environment variables.

• REDIS_URL
• USE_REDIS

For example run in local machine:

docker run --name rate-api -p 8000:8000 -e REDIS_URL=redis://your-remote-redis:6379 -e USE_REDIS=1  ghcr.io/tim-hub/sanic-currency-exchange-rates-api:v2.3.0

More Information about Redis Stack

Here some resources to help you quickly get started using Redis Stack. If you still have questions, feel free to ask them in the Redis Discord or on Twitter.

Getting Started

1. Sign up for a free Redis Cloud account using this link and use the Redis Stack database in the cloud.
2. Based on the language/framework you want to use, you will find the following client libraries:
   • Redis OM .NET (C#)
     • Watch this getting started video
     • Follow this getting started guide
   • Redis OM Node (JS)
     • Watch this getting started video
     • Follow this getting started guide
   • Redis OM Python
     • Watch this getting started video
     • Follow this getting started guide
   • Redis OM Spring (Java)
     • Watch this getting started video
     • Follow this getting started guide

The above videos and guides should be enough to get you started in your desired language/framework. From there you can expand and develop your app. Use the resources below to help guide you further:

1. Developer Hub - The main developer page for Redis, where you can find information on building using Redis with sample projects, guides, and tutorials.
2. Redis Stack getting started page - Lists all the Redis Stack features. From there you can find relevant docs and tutorials for all the capabilities of Redis Stack.
3. Redis Rediscover - Provides use-cases for Redis as well as real-world examples and educational material
4. RedisInsight - Desktop GUI tool - Use this to connect to Redis to visually see the data. It also has a CLI inside it that lets you send Redis CLI commands. It also has a profiler so you can see commands that are run on your Redis instance in real-time
5. Youtube Videos
   • Official Redis Youtube channel
   • Redis Stack videos - Help you get started modeling data, using Redis OM, and exploring Redis Stack
   • Redis Stack Real-Time Stock App from Ahmad Bazzi
   • Build a Fullstack Next.js app with Fireship.io
   • Microservices with Redis Course by Scalable Scripts on freeCodeCamp