Partridge
Partridge is a Python 3.6+ library for working with GTFS feeds using pandas DataFrames.
Partridge is heavily influenced by our experience at Remix analyzing and debugging every GTFS feed we could find.
At the core of Partridge is a dependency graph rooted at trips.txt. Disconnected data is pruned away according to this graph when reading the contents of a feed.
Feeds can also be filtered to create a view specific to your needs. It's most common to filter a feed down to specific dates (service_id) or routes (route_id), but any field can be filtered.
Philosophy
The design of Partridge is guided by the following principles:
As much as possible
- Favor speed
- Allow for extension
- Succeed lazily on expensive paths
- Fail eagerly on inexpensive paths
As little as possible
- Do anything other than efficiently read GTFS files into DataFrames
- Take an opinion on the GTFS spec
Installation
pip install partridgeGeoPandas support
pip install partridge[full]Usage
Setup
import partridge as ptg
inpath = 'path/to/caltrain-2017-07-24/'Examples
The following is a collection of gists containing Jupyter notebooks with transformations to GTFS feeds that may be useful for intake into software applications.
- Find the busiest week in a feed and reduce its file size
- Combine routes by route_short_name
- Merge GTFS with shapefile geometries
- Merge multiple agencies into one
- Rewrite a feed to clean up formatting issues
- If a feed has stop_code, replace the contents of stop_id with the contents of stop_code
- Diff the number of service hours in two feeds
- Investigate the the distance in meters of each stop to the closest point on a shape
- Convert frequencies.txt to an equivalent trips.txt
- Calculate headway for a stop
Inspecting the calendar
The date with the most trips
date, service_ids = ptg.read_busiest_date(inpath)
# datetime.date(2017, 7, 17), frozenset({'CT-17JUL-Combo-Weekday-01'})The week with the most trips
service_ids_by_date = ptg.read_busiest_week(inpath)
# {datetime.date(2017, 7, 17): frozenset({'CT-17JUL-Combo-Weekday-01'}),
# datetime.date(2017, 7, 18): frozenset({'CT-17JUL-Combo-Weekday-01'}),
# datetime.date(2017, 7, 19): frozenset({'CT-17JUL-Combo-Weekday-01'}),
# datetime.date(2017, 7, 20): frozenset({'CT-17JUL-Combo-Weekday-01'}),
# datetime.date(2017, 7, 21): frozenset({'CT-17JUL-Combo-Weekday-01'}),
# datetime.date(2017, 7, 22): frozenset({'CT-17JUL-Caltrain-Saturday-03'}),
# datetime.date(2017, 7, 23): frozenset({'CT-17JUL-Caltrain-Sunday-01'})}Dates with active service
service_ids_by_date = ptg.read_service_ids_by_date(path)
date, service_ids = min(service_ids_by_date.items())
# datetime.date(2017, 7, 15), frozenset({'CT-17JUL-Caltrain-Saturday-03'})
date, service_ids = max(service_ids_by_date.items())
# datetime.date(2019, 7, 20), frozenset({'CT-17JUL-Caltrain-Saturday-03'})Dates with identical service
dates_by_service_ids = ptg.read_dates_by_service_ids(inpath)
busiest_date, busiest_service = ptg.read_busiest_date(inpath)
dates = dates_by_service_ids[busiest_service]
min(dates), max(dates)
# datetime.date(2017, 7, 17), datetime.date(2019, 7, 19)Reading a feed
_date, service_ids = ptg.read_busiest_date(inpath)
view = {
'trips.txt': {'service_id': service_ids},
'stops.txt': {'stop_name': 'Gilroy Caltrain'},
}
feed = ptg.load_feed(path, view)Read shapes and stops as GeoDataFrames
service_ids = ptg.read_busiest_date(inpath)[1]
view = {'trips.txt': {'service_id': service_ids}}
feed = ptg.load_geo_feed(path, view)
feed.shapes.head()
# shape_id geometry
# 0 cal_gil_sf LINESTRING (-121.5661454200744 37.003512297983...
# 1 cal_sf_gil LINESTRING (-122.3944115638733 37.776439059278...
# 2 cal_sf_sj LINESTRING (-122.3944115638733 37.776439059278...
# 3 cal_sf_tam LINESTRING (-122.3944115638733 37.776439059278...
# 4 cal_sj_sf LINESTRING (-121.9031703472137 37.330157067882...
minlon, minlat, maxlon, maxlat = feed.stops.total_bounds
# -122.412076, 37.003485, -121.566088, 37.77639Extracting a new feed
outpath = 'gtfs-slim.zip'
service_ids = ptg.read_busiest_date(inpath)[1]
view = {'trips.txt': {'service_id': service_ids}}
ptg.extract_feed(inpath, outpath, view)
feed = ptg.load_feed(outpath)
assert service_ids == set(feed.trips.service_id)Features
- Surprisingly fast :)
- Load only what you need into memory
- Built-in support for resolving service dates
- Easily extended to support fields and files outside the official spec (TODO: document this)
- Handle nested folders and bad data in zips
- Predictable type conversions
Thank You
I hope you find this library useful. If you have suggestions for improving Partridge, please open an issue on GitHub.