Who owns what in nyc?

The Who owns What project is a new resource for community organizers and tenant leaders to demystify property ownership and shell company networks across New York City.

With this website, you can find crucial information about who is responsible for your building. The site utilizes a database of 160k other properties to connect the dots and discover other properties that your landlord might own or be associated with. Use this tool to discover what buildings in your neighborhood to organize in, what communities your landlord might be targeting, and if your building might be financially overleveraged.

This project is currently in active development!

Architecture

This site is built on top of the critical work done by @aepyornis on the nycdb project, which is used to cleanly extract, sanitize, and load HPD Registration data into a PostgreSQL instance.

Backend logic and data manipulation is largely handled by making calls to PostgreSQL functions and prebuilding results into tables whenever possible to avoid complex queries made per-request. for the SQL code that provides this functionality, see:

• the hpd-registration scripts of nycdb, and
• the sql directory of this repository.

Backend

The backend of the app is a simple Django app that connects to Postgres.

Frontend

The frontend of the app (/client) is built on top of create-react-app. See /client/README.md for all the info you might need.

Setup

In order to set things up, you'll need to copy .env.sample to .env and edit it as needed:

cp .env.sample .env

In particular, make sure you configure the DATABASE_URL environment variable.

Then you'll want to set up and enter a Python 3 virtual environment:

python3 -m venv venv
source venv/bin/activate  # Or 'venv\Scripts\activate' on Windows
pip install -r requirements-dev.txt

Then you'll need to load data into the database. If you want to use real data, which takes a long time to load, you can do so with:

python dbtool.py builddb

Alternatively, you can load a small test dataset with:

python dbtool.py loadtestdata

After that, make sure you have Node 12 or higher installed as well as yarn, and then run:

cd client
yarn

This will grab dependencies for the client.

Running in development

You will need to run two separate terminals; one for the back-end and another for the front-end.

To run the back-end API:

python manage.py runserver

The server will listen at http://localhost:8000 by default, though you probably won't need to visit it unless you're manually testing out the API.

To run the front-end:

cd client
yarn start

You can visit your local dev instance at http://localhost:3000.

Alternative: Docker-based development

As an alternative to the aforementioned setup, you can use Docker.

First create an .env file and edit it as needed:

cp .env.sample .env

Note that you don't need to change DATABASE_URL if you just want to use the test database.

Now run:

docker-compose run app python dbtool.py loadtestdata

This will build a nycdb with test data, which is must faster than downloading the whole nycdb. You can, however, opt to download the whole thing by running docker-compose run app python dbtool.py builddb, but be prepared, as it will take a while!

Once you've done that, run:

bash docker-update.sh

(You will want to re-run that whenever you update your git repository, too.)

Then start up the server:

docker-compose up

Eventually, you should see a message that says "You can now view client in the browser."

Visit http://localhost:3000 and you should be good to go! If you installed test data, you can see useful results by clicking on the "All Year Management" portfolio on the home page.

Note: If you would like to connect your Docker instance to an external postgres database, you can update the DATABASE_URL server-side env variable with your remote db's connection URI.

Tests

Back-end tests can be run via the Python virtualenv:

pytest

If you're using Docker, this can be done via docker-compose run app pytest.

See /client/README.md for more details on front-end tests.

Black

Black is a formatting tool similar to Prettier, but for Python code.

Before committing or pushing to GitHub, you may want to run the following to ensure that any files you've changed are properly formatted:

black .

Note that if you don't either use this or some kind of editor plug-in before pushing to GitHub, continuous integration will fail.

Deploying

Package client-side assets through:

cd client && yarn build

You will need to deploy client/build to a static file server.

Cross-browser testing

We use BrowserStack Live to make sure that our sites work across browsers, operating systems, and devices.

Updating data

Updating WoW's data is straighforward, unless a new dataset is needed or the schema of an existing dataset changes. Previously this was necessary every year with new versions of the PLUTO dataset (now there is a version on Open Data with automatic updates and a stable schema), but can also happen unpredicitably when an agency decides to change the schema of an existing dataset.

To use new data, you'll need to update a few things:

1. Update the NYCDB revision WoW and its test suite use at requirements-dev.txt.
2. Update the list of NYCDB datasets WoW depends on at who-owns-what.yml.
3. Update any SQL to refer to the new dataset's tables.
4. Any new or updated datasets may need new scaffolding for WoW's test suite to continue functioning. This means you may need to run the tests/generate_factory_from_csv.py tool to create new factories in the tests/factories folder. You may also need to add new test data to the tests/data directory in order for tests to continue working.

An example of all this in practice can be seen in #209, which upgrades WoW from PLUTO 18v2 to 19v2.

Note also that the justfixnyc/nycdb-k8s-loader project may be useful for keeping the WoW database up-to-date on a day-to-day basis.

License

JustFix uses the GNU General Public License v3.0 Open-Source License. See LICENSE.md file for the full text.

Code of Conduct

Read about JustFix's code of conduct as an organization on our Mission page.