All Projects → microsoft → MCW-Big-data-analytics-and-visualization

microsoft / MCW-Big-data-analytics-and-visualization

Licence: MIT License
MCW Big data analytics and visualization

Programming Languages

javascript
184084 projects - #8 most used programming language
Jupyter Notebook
11667 projects
CSS
56736 projects
C#
18002 projects

Projects that are alternatives of or similar to MCW-Big-data-analytics-and-visualization

Spark
.NET for Apache® Spark™ makes Apache Spark™ easily accessible to .NET developers.
Stars: ✭ 1,721 (+900.58%)
Mutual labels:  spark-sql, hdinsight
databricks-notebooks
Collection of Databricks and Jupyter Notebooks
Stars: ✭ 19 (-88.95%)
Mutual labels:  power-bi, spark-sql
spark2-etl-examples
A project with examples of using few commonly used data manipulation/processing/transformation APIs in Apache Spark 2.0.0
Stars: ✭ 23 (-86.63%)
Mutual labels:  spark-sql
pbi-tools
Power BI DevOps & Source Control Tool
Stars: ✭ 60 (-65.12%)
Mutual labels:  power-bi
Azure.DataFactory.LocalEnvironment
This code allows you to load any existing Azure Data Factory project file (*.dfproj) and perform further actions like "Export to ARM Template" or to debug custom C# activities
Stars: ✭ 25 (-85.47%)
Mutual labels:  azure-data-factory
wow-spark
🔆 spark自学手册,包含了例如spark core、spark sql、spark streaming、spark-kafka、delta-lake,以及scala基础练习,还有一些例如master、shuffle源码分析,总结及翻译。
Stars: ✭ 20 (-88.37%)
Mutual labels:  spark-sql
vsts-publish-adf
This extension adds Azure Data Factory release tasks to Azure Pipelines.
Stars: ✭ 23 (-86.63%)
Mutual labels:  azure-data-factory
spark-vcf
Spark VCF data source implementation for Dataframes
Stars: ✭ 15 (-91.28%)
Mutual labels:  spark-sql
spark-data-sources
Developing Spark External Data Sources using the V2 API
Stars: ✭ 36 (-79.07%)
Mutual labels:  spark-sql
SparkProgrammingInScala
Apache Spark Course Material
Stars: ✭ 57 (-66.86%)
Mutual labels:  spark-sql
recsys spark
Spark SQL 实现 ItemCF,UserCF,Swing,推荐系统,推荐算法,协同过滤
Stars: ✭ 76 (-55.81%)
Mutual labels:  spark-sql
spark-sql-internals
The Internals of Spark SQL
Stars: ✭ 331 (+92.44%)
Mutual labels:  spark-sql
Movies-Analytics-in-Spark-and-Scala
Data cleaning, pre-processing, and Analytics on a million movies using Spark and Scala.
Stars: ✭ 47 (-72.67%)
Mutual labels:  spark-sql
big data
A collection of tutorials on Hadoop, MapReduce, Spark, Docker
Stars: ✭ 34 (-80.23%)
Mutual labels:  spark-sql
PowerBI-book-ru
Проект открытого методического пособия по Power BI, Power Query, Power Pivot
Stars: ✭ 76 (-55.81%)
Mutual labels:  power-bi
automating-azure-analysis-services
Code samples for a SQL Saturday presentation on the topic of Automating Azure Analysis Services
Stars: ✭ 53 (-69.19%)
Mutual labels:  azure-data-factory
Spark
Apache Spark is a fast, in-memory data processing engine with elegant and expressive development API's to allow data workers to efficiently execute streaming, machine learning or SQL workloads that require fast iterative access to datasets.This project will have sample programs for Spark in Scala language .
Stars: ✭ 55 (-68.02%)
Mutual labels:  spark-sql
AzureDataFactoryHOL
Azure Data Factory Hands On Lab - Step by Step - A Comprehensive Azure Data Factory and Mapping Data Flow step by step tutorial
Stars: ✭ 43 (-75%)
Mutual labels:  azure-data-factory
spark learning
尚硅谷大数据Spark-2019版最新 Spark 学习
Stars: ✭ 42 (-75.58%)
Mutual labels:  spark-sql
AzureSMR
AzureSMR is no longer being actively developed. For ongoing support of Azure in R, see: https://github.com/Azure/AzureR
Stars: ✭ 60 (-65.12%)
Mutual labels:  hdinsight

Big data analytics and visualization

Margie's Travel (MT) provides concierge services for business travelers. In an increasingly crowded market, they are always looking for ways to differentiate themselves and provide added value to their corporate customers.

They are looking to pilot a web app that their internal customer service agents can use to provide additional valuable information to the traveler during the flight booking process. They want to enable their agents to enter in the flight information and produce a prediction as to whether the departing flight will encounter a 15-minute or longer delay, considering the weather forecast for the departure hour.

November 2021

Target audience

  • Application developers
  • Data scientists
  • Data engineers
  • Data architects

Abstracts

Workshop

In this workshop, you will deploy a web app using Machine Learning Services to predict travel delays given flight delay data and weather conditions. Plan a bulk data import operation, followed by preparation, such as cleaning and manipulating the data for testing, and training your machine learning model.

At the end of this workshop, you will be better able to build a complete machine learning model in Azure Databricks for predicting if an upcoming flight will experience delays. In addition, you will learn to store the trained model in Azure Machine Learning Model Management, then deploy to Docker containers for scalable on-demand predictions, use Azure Data Factory (ADF) for data movement and operationalizing ML scoring, summarize data with Azure Databricks and Spark SQL, and visualize batch predictions on a map using Power BI.

Whiteboard design session

In this whiteboard design session, you will work with a group to design a solution for ingesting and preparing historic flight delay and weather data and creating, training, and deploying a machine learning model that can predict flight delays.

At the end of this whiteboard design session, you will have learned how to include a web application that obtains weather forecasts from a 3rd party, collects flight information from end-users, and sends that information to the deployed machine learning model for scoring. Part of the exercise will include providing visualizations of historic flight delays and orchestrating the collection and batch scoring of historic and new flight delay data.

Hands-on lab

This hands-on lab is designed to provide exposure to many of Microsoft's transformative line of business applications built using Microsoft big data and advanced analytics.

By the end of the lab, you will be able to show an end-to-end solution, leveraging many of these technologies but not necessarily doing work in every component possible.

Azure services and related products

  • Azure Databricks
  • Azure Data Factory (ADF)
  • Azure Storage
  • Power BI Desktop
  • Azure App Service (optional)

Related references

Help & Support

We welcome feedback and comments from Microsoft SMEs & learning partners who deliver MCWs.

Having trouble?

  • First, verify you have followed all written lab instructions (including the Before the Hands-on lab document).
  • Next, submit an issue with a detailed description of the problem.
  • Do not submit pull requests. Our content authors will make all changes and submit pull requests for approval.

If you are planning to present a workshop, review and test the materials early! We recommend at least two weeks prior.

Please allow 5 - 10 business days for review and resolution of issues.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].