All Projects → jkthompson → Pyspark Pictures

jkthompson / Pyspark Pictures

Licence: mit
Learn the pyspark API through pictures and simple examples

Projects that are alternatives of or similar to Pyspark Pictures

Pyrevolution
Python tutorials and puzzles to share with the world!
Stars: ✭ 151 (-0.66%)
Mutual labels:  jupyter-notebook
Santander Customer Transaction Prediction
2nd Place Solution 💰🥈
Stars: ✭ 152 (+0%)
Mutual labels:  jupyter-notebook
Pytorch Pose Estimation
PyTorch Implementation of Realtime Multi-Person Pose Estimation project.
Stars: ✭ 152 (+0%)
Mutual labels:  jupyter-notebook
Competition baselines
开源的各大比赛baseline
Stars: ✭ 150 (-1.32%)
Mutual labels:  jupyter-notebook
Amitt framework
Repo replaced by cogsec-collaborative/AMITT
Stars: ✭ 152 (+0%)
Mutual labels:  jupyter-notebook
Stanford Cs229
Python solutions to the problem sets of Stanford's graduate course on Machine Learning, taught by Prof. Andrew Ng
Stars: ✭ 151 (-0.66%)
Mutual labels:  jupyter-notebook
Nbviewer
nbconvert as a web service: Render Jupyter Notebooks as static web pages
Stars: ✭ 1,954 (+1185.53%)
Mutual labels:  jupyter-notebook
Cognitive Vision Python
Jupyter Notebook with Python samples for the Cognitive Services Computer Vision API
Stars: ✭ 152 (+0%)
Mutual labels:  jupyter-notebook
Artificial Intelligence Projects
Collection of Artificial Intelligence projects.
Stars: ✭ 152 (+0%)
Mutual labels:  jupyter-notebook
3 Min Pytorch
<펭귄브로의 3분 딥러닝, 파이토치맛> 예제 코드
Stars: ✭ 152 (+0%)
Mutual labels:  jupyter-notebook
Covid 19 Data Science
Welcome to Glacier Data Project. A post-wuhan2020 project for data science
Stars: ✭ 152 (+0%)
Mutual labels:  jupyter-notebook
Simfin Tutorials
Tutorials for SimFin - Simple financial data for Python
Stars: ✭ 150 (-1.32%)
Mutual labels:  jupyter-notebook
Netgan
Implementation of the paper "NetGAN: Generating Graphs via Random Walks".
Stars: ✭ 152 (+0%)
Mutual labels:  jupyter-notebook
Ipyplot
IPyPlot is a small python package offering fast and efficient plotting of images inside Python Notebooks. It's using IPython with HTML for faster, richer and more interactive way of displaying big numbers of images.
Stars: ✭ 152 (+0%)
Mutual labels:  jupyter-notebook
Ssd keras
简明 SSD 目标检测模型 keras version(交通标志识别 训练部分见 dev 分支)
Stars: ✭ 152 (+0%)
Mutual labels:  jupyter-notebook
Pydata2015
Stars: ✭ 151 (-0.66%)
Mutual labels:  jupyter-notebook
Mine pytorch
MINE: Mutual Information Neural Estimation in pytorch (unofficial)
Stars: ✭ 152 (+0%)
Mutual labels:  jupyter-notebook
Alphalens
Performance analysis of predictive (alpha) stock factors
Stars: ✭ 2,130 (+1301.32%)
Mutual labels:  jupyter-notebook
Pennai
PennAI: AI-Driven Data Science
Stars: ✭ 152 (+0%)
Mutual labels:  jupyter-notebook
Improved Seam Carving
A numpy implementation of forward energy from the paper “Improved Seam Carving for Video Retargeting" (2008)
Stars: ✭ 152 (+0%)
Mutual labels:  jupyter-notebook

pyspark-pictures

Learn the pyspark API through pictures and simple examples

View on NBViewer

RDD Example:

example image

# flatMap
x = sc.parallelize([1,2,3])
y = x.flatMap(lambda x: (x, 100*x, x**2))
print(x.collect())
print(y.collect())

[1, 2, 3]
[1, 100, 1, 2, 200, 4, 3, 300, 9]

Install (for interactive use)

  1. install Spark
  2. install IPython notebook

Quick Start

  1. start pyspark inside IPython notebook

    IPYTHON_OPTS="notebook" pyspark
    
  2. open browser to notebook link

  3. open pyspark-pictures.ipynb or pyspark-pictures-dataframes.ipynb

  4. edit example code, press: ctrl + enter to run each cell

References

pyspark API

Contribute

Contributors are welcome
Original images are here, download to pdf, convert to svg with: genSVD (pdf2svg)

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].