ClinWiki

This application allows you to rate clinical trials. You can retrieve trials to rank by searching for NCT ID or MeSH Term

Setup

You can use one of the following options to start ClinWiki on your own system:

• Docker Compose (easiest method)
• Locally in development mode
• Locally in production mode

Docker Compose

1. Clone the ClinWiki repository

2. Create a .env file in the project's root directory (use .example.env for inspiration)

   Reach out to @williamhoos for AACT database URL and credentials for other third party services

3. Install Docker

   • Windows and Mac users install Docker Desktop

   • Linux users follow the official install instructions for Docker for your distribution

4. Only for macOS: Install and start docker-sync

   • Bring up a separate terminal
   • Install docker-sync via gem install docker-sync
   • Run docker-sync start --config=docker-sync.yml

   This will fix file mounting issues in OS X and Docker that create significant performance problems using the default config. Additionally, you will want to run the docker-compose commands with additional arguments (included below for each step).

5. Build images

   docker-compose build

   macOS: docker-compose -f docker-compose.yml -f docker-compose-osx.yml build

6. Run ClinWiki server on http://localhost:3000

   docker-compose up or docker-compose up -d to run as daemon in background

   macOS: docker-compose -f docker-compose.yml -f docker-compose-osx.yml up or docker-compose -f docker-compose.yml -f docker-compose-osx.yml up -d

7. Bootstrap search

   compose/bin/search_bootstrap

8. Open a bash shell inside the clinwiki Docker container

   docker-compose exec clinwiki bash

9. Seed the database

   rake db:seed

10. Start Rails console (still inside Docker container)

    rails c

11. Create new user

    User.create!(first_name: "CHANGEME", last_name: "CHANGEME", email: "CHANGEME", password: "CHANGEME", password_confirmation: "CHANGEME")

12. To access the sub-site config and bulk update admin features, add admin role after creating a user

    User.find_by(email: "CHANGEME").add_role(:admin)

13. Exit Rails console

    exit

14. Exit Docker container

    exit

15. Build and serve frontend (on http://localhost:3001):

    • cd /front
    • yarn install
    • yarn start

    NOTE: Changes to local .ts/.tsx files will be automatically applied to the running system

16. Go to http://localhost:3001 and sign in using the user email and password created in the Rails console

17. Create the default site by clicking on Sites under the profile dropdown on the top right hand side. Then click create site, fill out the form and click save.

    Sites can be accessed locally by navigating to a subdomain against your localhost. By default, http://mysite.localhost:3001 and http://test.localhost:3001 are white-listed for CORS requests. This means you can configure a site with the subdomain mysite and a site with the subdomain test without additional CORS configuration.

• Extras
  • Logs: docker-compose logs -f clinwiki
  • Console: docker-compose exec clinwiki bundle exec rails c
  • Run and build: docker-compose up -d --build The front end is not built yet so you will only see the rails start page but you can browse to http://localhost:3000/graphiql to send graphql queries.

CW_MODE

You can use the CW_MODE environment variable to configure how docker-compose starts ClinWiki.

• CW_MODE=PROD - the default, builds the front end and runs rails
• CW_MODE=FAST - runs rails without building the front end for a quicker startup
• CW_MODE=DEV - starts the 'clinwiki' container without running rails so you can start/restart it manually during development
• CW_MODE=WORKER - runs the sidekiq process instead of rails

To specify the variable changes the command line a little bit depending on if you're on Windows or Linux/Mac:

Linux/Mac:

CW_MODE=FAST docker-compose up -d

Windows:

set CW_MODE=FAST
docker-compose up -d

Run Locally

You are encouraged to use rvm for Ruby version and gemset management. Once RVM is installed, switch to the correct Ruby and gemset as follows:

rvm use ruby-2.5.3@clinwiki --install --create

Then, from the project root directory:

bundle install
bundle exec rake db:create
bundle exec rake db:migrate

Once the dependencies are installed and the database is initialized, you can run the server with:

bundle exec rails s

Note that we are using dotenv to make the handling of database variables easier.

Environment Variables

We use a handful of environment variables to run the server. You can expose these when running locally by creating a .env file in the project root level directory. Here's an example:

# Needed to connect to AACT
AACT_DATABASE_URL=postgres://aact:[email protected]:5432/aact
REDIS_URL=redis://127.0.0.1:6379/0
DATABASE_URL=postgres://postgres@localhost:5432/clinwiki

# Optional

JWT_EXPIRATION_TIME_SECS=86400 # JWT expiration time

NOTE: The use of the DATABASE_URL environment variable should prevent the need for a database.yml file in when running the server in development mode.

Run Locally in Production Mode

1. Create a .env file in the project's root directory (use .example.env for inspiration)

2. Precompile assets

   export $(cat .env | xargs) && RAILS_ENV=production rake assets:precompile

3. Start server

   export $(cat .env | xargs) && RAILS_ENV=production rails s

Data Access

Databases

The AACT Clinical Trials Database provides data the data we use for clinical trials.

While the database is available online, you are encouraged to download a copy of the database for ease of access.

We maintain a separate postgres database for data related to users. Please refer to the Installation section for additional info.

If you don't have postgres installed locally, you can do so using homebrew:

brew install postgres

Elasticsearch

We use searchkick to enable search.

Follow the instructions in the searchkick docs for setting up Elasticsearch locally.

Indexing

Indexing is performed in the background by Sidekiq. You can run Sidekiq as follows:

bundle exec sidekiq -C config/sidekiq.yml

Note that with about 32 concurrent workers, it should take roughly 8 hours to fully reindex.

It's also worth noting that we run a background job every ten seconds to handle batch reindexing. This is important because it optimizes queries using ActiveRecord's includes functionality. Refer to the Study model's search_import scope for more details.

Scheduled jobs can be configured in ./config/schedule.yml.

To disable Auto indexing set env variables for sidekiq process:

AUTO_INDEX_LAST_DAYS=0

A full reindex can be performed by running:

bundle exec rake search:index

A development environments can bootstrap a subset of search results as follows:

bundle exec rake search:bootstrap_dev

Searching

Searching for studies is handled by Searchkick through the Studies model. Results are directly brokered through the Elasticsearch index, which brokers data from the annotation database along with the AACT database.

Running Imports

1. Generate your CSV

   CSVs expect a header with the following format:

   `nct_id,Type,Value,Action,user_email

   For crowd values, there are two types of crowd values and both can use the 'Add' and 'Remove' options. For native fields that are crowd overwriteable, Add will add or overwrite a crowd definition for that field, and Remove will remove the crowd definition.(not sure if still supported) For crowd values that produce custom facets, the "Type" in the csv ("label" value in the UI) will be the name of the facet. The "Value" in the csv ("description" in the UI) is the value(s) in the facet. For multiple values of the crowd facet, enter as multiple lines or separated on a single line by '|' (example Type = "Karnofsky Score Allowed", Value = "100|90|80|70")

2. Commit your CSV to the appropriate subfolder

   The CSV should be exported to the imports/ within the root project folder.

   You can commit the CSV with the following workflow, which assumes you have set up heroku's command line tools, and that the heroku remote points to the clinwiki-prod project:

   git add imports
   git commit -m 'Added import file'  # you can write any comment you'd like
   git push origin master  # add to clinwiki-org/clinwiki
   git push heroku master  # add to heroku -- this should kick off a deploy

3. Run the CSV import Rake task

   # if your file is in imports/foo.csv, $MY_FILE should be foo.csv
   heroku run -a clinwiki-prod rake import:csv["$MY_FILE"]
   (in heroku web, rake import:csv["$MY_FILE"])

Running Exports

heroku run -a clinwiki-prod rake export:front_matter_csv > my-front-matter.csv

Additional Data Seeding

When using the standard seeding of AACT data and clinwiki data, we are still working to establish a more complete development seeded database. Some functionality for workflows, crowd values and bulk update will not be available until at least a few studies have crowd keys and values. To complete set-up, add at least the following keys and values spread across at least 5 studies. Having any values for crowd key/values seeds this funcitnoality in the application. Having any keys labeled "WF_" ensures workflows function. Having at least one study with WF_bulk key enables configuration of the bulk update functionality. [this fuctionality will be replaced with alternative seed files and functionality]

• tags
• • Organization 1
• • Organization 2
• • Organization 3
• WF_bulk
• • Needs Review
• WF_TestA
• • TestA Value 1
• • TestA Value 2
• WF_TestB
• • TestB Value 1
• • TestB Value 2
• • TestB Value 3