All Projects → jbusecke → cmip6_preprocessing

jbusecke / cmip6_preprocessing

Licence: MIT license
Analysis ready CMIP6 data in python the easy way with pangeo tools.

Programming Languages

Jupyter Notebook
11667 projects
python
139335 projects - #7 most used programming language

Projects that are alternatives of or similar to cmip6 preprocessing

intake-esm
An intake plugin for parsing an Earth System Model (ESM) catalog and loading assets into xarray datasets.
Stars: ✭ 78 (-38.1%)
Mutual labels:  cmip6, pangeo
pcmdi metrics
Self contained packages to run PCMDI Metrics
Stars: ✭ 44 (-65.08%)
Mutual labels:  climate-models, climate-analysis
ATLAS
Datasets, code and virtual workspace for the Climate Change ATLAS
Stars: ✭ 56 (-55.56%)
Mutual labels:  cmip6
dmriprep
dMRIPrep is a robust and easy-to-use pipeline for preprocessing of diverse dMRI data. The transparent workflow dispenses of manual intervention, thereby ensuring the reproducibility of the results.
Stars: ✭ 55 (-56.35%)
Mutual labels:  preprocessing
preprocessy
Python package for Customizable Data Preprocessing Pipelines
Stars: ✭ 34 (-73.02%)
Mutual labels:  preprocessing
3D Ground Segmentation
A ground segmentation algorithm for 3D point clouds based on the work described in “Fast segmentation of 3D point clouds: a paradigm on LIDAR data for Autonomous Vehicle Applications”, D. Zermas, I. Izzat and N. Papanikolopoulos, 2017. Distinguish between road and non-road points. Road surface extraction. Plane fit ground filter
Stars: ✭ 55 (-56.35%)
Mutual labels:  preprocessing
multi-imbalance
Python package for tackling multi-class imbalance problems. http://www.cs.put.poznan.pl/mlango/publications/multiimbalance/
Stars: ✭ 66 (-47.62%)
Mutual labels:  preprocessing
esmtools
🐙 a toolbox for Earth System Model analysis 🐙
Stars: ✭ 24 (-80.95%)
Mutual labels:  climate-analysis
Geoweaver
a web system to allow users to automatically record history and manage complicated scientific workflows in web browsers involving the online spatial data facilities, high-performance computation platforms, and open-source libraries.
Stars: ✭ 32 (-74.6%)
Mutual labels:  pangeo
climate-data-science
Climate Data Science and Earth Observation with Python.
Stars: ✭ 103 (-18.25%)
Mutual labels:  pangeo
prospectr
R package: Misc. Functions for Processing and Sample Selection of Spectroscopic Data
Stars: ✭ 26 (-79.37%)
Mutual labels:  preprocessing
arcsi
Software to automate the production of optical analysis ready data (ARD) from Landsat, Sentinel-2 and others.
Stars: ✭ 22 (-82.54%)
Mutual labels:  analysis-ready-data
ClimateTools.jl
Climate science package for Julia
Stars: ✭ 108 (-14.29%)
Mutual labels:  climate-analysis
cmor
Climate Model Output Rewriter
Stars: ✭ 42 (-66.67%)
Mutual labels:  cmip6
AutoTS
Automated Time Series Forecasting
Stars: ✭ 665 (+427.78%)
Mutual labels:  preprocessing
Start maja
To process a Sentinel-2 time series with MAJA cloud detection and atmospheric correction processor
Stars: ✭ 47 (-62.7%)
Mutual labels:  preprocessing
torcharrow
High performance model preprocessing library on PyTorch
Stars: ✭ 566 (+349.21%)
Mutual labels:  preprocessing
Machine Learning
A repository of resources for understanding the concepts of machine learning/deep learning.
Stars: ✭ 29 (-76.98%)
Mutual labels:  preprocessing
NVTabular
NVTabular is a feature engineering and preprocessing library for tabular data designed to quickly and easily manipulate terabyte scale datasets used to train deep learning based recommender systems.
Stars: ✭ 797 (+532.54%)
Mutual labels:  preprocessing
SeqTools
A python library to manipulate and transform indexable data (lists, arrays, ...)
Stars: ✭ 42 (-66.67%)
Mutual labels:  preprocessing

Documentation Status Anaconda Cloud conda-forge Pypi Build Status Full Archive CI codecov License:MIT DOI

BLM

Science is not immune to racism. Academia is an elitist system with numerous gatekeepers that has mostly allowed a very limited spectrum of people to pursue a career. I believe we need to change that.

Open source development and reproducible science are a great way to democratize the means for scientific analysis. But you can't git clone software if you are being murdered by the police for being Black!

Free access to software and hollow diversity statements are hardly enough to crush the systemic and institutionalized racism in our society and academia.

If you are using this package, I ask you to go beyond just speaking out and donate here to Data for Black Lives and Black Lives Matter Action.

I explicitly welcome suggestions regarding the wording of this statement and for additional organizations to support. Please raise an issue for suggestions.

xmip (formerly cmip6_preprocessing)

This package facilitates the cleaning, organization and interactive analysis of Model Intercomparison Projects (MIPs) within the Pangeo software stack.

Are you interested in CMIP6 data, but find that is is not quite analysis ready? Do you just want to run a simple (or complicated) analysis on various models and end up having to write logic for each seperate case, because various datasets still require fixes to names, coordinates, etc.? Then this package is for you.

Developed during the cmip6-hackathon this package provides utility functions that play nicely with intake-esm.

We currently support the following functions

  1. Preprocessing CMIP6 data (Please check out the tutorial for some examples using the pangeo cloud). The preprocessig includes: a. Fix inconsistent naming of dimensions and coordinates b. Fix inconsistent values,shape and dataset location of coordinates c. Homogenize longitude conventions d. Fix inconsistent units
  2. Creating large scale ocean basin masks for arbitrary model output

The following issues are under development:

  1. Reconstruct/find grid metrics
  2. Arrange different variables on their respective staggered grid, so they can work seamlessly with xgcm

Check out this recent Earthcube notebook (cite via doi: 10.1002/essoar.10504241.1) for a high level demo of xmip and xgcm.

Installation

Install xmip via pip:

pip install xmip

or conda:

conda install -c conda-forge xmip

To install the newest main from github you can use pip aswell:

pip install git+pip install git+https://github.com/jbusecke/xmip.git

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].