Keep-Current

Recommendation Engine

This repository is the documents personalized recommendation engine of the Keep-Current project.

Potential tools to implement

We lean heavily on existing tools as well as developing new methods. We are colaborating through Google Colab notebooks.

• TextBlob
• spaCy
• NLTK
• Vowpal Wabbit

Getting started

for running this project locally, you need first to install the dependency packages. To install them, you can use

• pipenv
• anaconda
• virtualenv

Installation using pipenv (which combines virtualenv with pip)

Install pipenv

# if you haven't installed pip
sudo easy_install pip
# install pipenv
pip install pipenv

# with homebrew (on macOS)
brew install pipenv

Install the packages and run the server

# install all packages
pipenv install
# run the server
pipenv run python manage.py server

Installing using Anaconda

If you have anaconda installed, it's recommended to create an environment for the project, and install the dependencies in it.

# create the environment
conda create -q -n web-miner python=3.6

# activate the environment
source activate web-miner

# install the packages
pip install -r requirements.txt

and test your installation by running the web server:

# start server
python manage.py server

Architecture

Project Architecture

We follow the clean architecture style and structure the codebase accordingly.

Image credit to Uncle Bob

Most important rule:

Source code dependencies can only point inwards. Nothing in an inner circle can know anything at all about something in an outer circle. In particular, the name of something declared in an outer circle must not be mentioned by the code in the an inner circle. That includes, functions, classes. variables, or any other named software entity.

Who are we?

This project intends to be a shared work of meetup members, with the purpose, beside the obvious result, to also be used as a learning platform, while advancing the Natural Language Processing / Machine Learning field by exploring, comparing and hacking different models.

Please visit

• the project board on Github
• the repository board on Github
• our chat room on Slack. If you're new, you can join using this link
• our facebook group where we discuss and share current topics also outside of the project

for more.

How to Contribute

You can find our Project board here on GitHub and we use Slack as our communication channel. If you're new, you can join using this link

We welcome anyone who would like to join and contribute.

Please see our contribute guide.

We meet regularly every month in Vienna through

• the Data Science Cafe meetup of the VDSG or
• the WeAreDevelopers :: Keep-Current meetup

to show our progress and discuss the next steps.

Keep-Current Project

After studying a topic, keeping current with the news, published papers, advanced technologies and such proved to be a hard work. One must attend conventions, subscribe to different websites and newsletters, go over different emails, alerts and such while filtering the relevant data out of these sources.

In this project, we aspire to create a platform for students, researchers, professionals and enthusiasts to discover news on relevant topics. The users are encouraged to constantly give a feedback on the suggestions, in order to adapt and personalize future results.

The goal is to create an automated system that scans the web, through a list of trusted sources, classify and categorize the documents it finds, and match them to the different users, according to their interest. It then presents it as a timely summarized digest to the user, whether by email or within a site.

If you wish to assist in different aspects (Data Engineering / Web development / DevOps), we have divided the project to several additional repositories focusing on these topics:

• The mining of data is done in the Web-Miner repository
• Web Development & UI/UX experiments can be found in our App repository
• Data Engineering tasks are more than welcomed in our Data Engineering repository
• Devops tasks are all across the project. This project is developed mostly in a serverless architecture. Using Docker and Kubernetes enables freedom in deploying it on different hosting providers and plans.

Feel free to join the discussion and provide your input!

About Machine Learning

Machine Learning & Natural Language Processing

If you're new to Machine Learning, we suggest reading the following sources:

• Machine learning collection
• Natural Language Processing in Python Book
• Keon's awesome-nlp!