GitPlanet

Cheap and reliable Node.js hosting starts at $3/month, and $1/month static HTML hosting

Created with love in Canada, visit hostnodejs.com today

Feel like to post an Ad? Learn Details
All Projects → LinMingQiang → flink-learn

LinMingQiang / flink-learn

Licence: other
Learning Flink : Flink CEP,Flink Core,Flink SQL

Programming Languages

java
68154 projects - #9 most used programming language
scala
5932 projects
FreeMarker
481 projects
shell
77523 projects

Labels

sqlstreambigdataflink

Projects that are alternatives of or similar to flink-learn

Flinkstreamsql
基于开源的flink，对其实时sql进行扩展；主要实现了流与维表的join，支持原生flink SQL所有的语法
Stars: ✭ 1,682 (+2302.86%)
Mutual labels:  stream, bigdata, flink
Flink Boot
懒松鼠Flink-Boot 脚手架让Flink全面拥抱Spring生态体系，使得开发者可以以Java WEB开发模式开发出分布式运行的流处理程序，懒松鼠让跨界变得更加简单。懒松鼠旨在让开发者以更底上手成本（不需要理解分布式计算的理论知识和Flink框架的细节）便可以快速编写业务代码实现。为了进一步提升开发者使用懒松鼠脚手架开发大型项目的敏捷的度，该脚手架默认集成Spring框架进行Bean管理，同时将微服务以及WEB开发领域中经常用到的框架集成进来，进一步提升开发速度。比如集成Mybatis ORM框架，Hibernate Validator校验框架,Spring Retry重试框架等，具体见下面的脚手架特性。
Stars: ✭ 209 (+198.57%)
Mutual labels:  bigdata, flink
Flinkx
Based on Apache Flink. support data synchronization/integration and streaming SQL computation.
Stars: ✭ 2,651 (+3687.14%)
Mutual labels:  bigdata, flink
Athenax
SQL-based streaming analytics platform at scale
Stars: ✭ 1,178 (+1582.86%)
Mutual labels:  stream, flink
coolplayflink
Flink: Stateful Computations over Data Streams
Stars: ✭ 14 (-80%)
Mutual labels:  bigdata, flink
Hadoopcryptoledger
Hadoop Crypto Ledger - Analyzing CryptoLedgers, such as Bitcoin Blockchain, on Big Data platforms, such as Hadoop/Spark/Flink/Hive
Stars: ✭ 126 (+80%)
Mutual labels:  bigdata, flink
Reddit sse stream
A Server Side Event stream to deliver Reddit comments and submissions in near real-time to a client.
Stars: ✭ 39 (-44.29%)
Mutual labels:  stream, bigdata
Bigdataguide
大数据学习，从零开始学习大数据，包含大数据学习各阶段学习视频、面试资料
Stars: ✭ 817 (+1067.14%)
Mutual labels:  bigdata, flink
Azure Event Hubs Spark
Enabling Continuous Data Processing with Apache Spark and Azure Event Hubs
Stars: ✭ 140 (+100%)
Mutual labels:  stream, bigdata
Bigdata practice
大数据分析可视化实践
Stars: ✭ 166 (+137.14%)
Mutual labels:  stream, bigdata
TiBigData
TiDB connectors for Flink/Hive/Presto
Stars: ✭ 192 (+174.29%)
Mutual labels:  bigdata, flink
Flink Notes
flink学习笔记
Stars: ✭ 106 (+51.43%)
Mutual labels:  bigdata, flink
Bigdata Notebook
Stars: ✭ 100 (+42.86%)
Mutual labels:  bigdata, flink
Ecommercerecommendsystem
商品大数据实时推荐系统。前端：Vue + TypeScript + ElementUI，后端 Spring + Spark
Stars: ✭ 139 (+98.57%)
Mutual labels:  bigdata, flink
Bigdata Interview
🎯 🌟[大数据面试题]分享自己在网络上收集的大数据相关的面试题以及自己的答案总结.目前包含Hadoop/Hive/Spark/Flink/Hbase/Kafka/Zookeeper框架的面试题知识总结
Stars: ✭ 857 (+1124.29%)
Mutual labels:  bigdata, flink
Alchemy
给flink开发的web系统。支持页面上定义udf，进行sql和jar任务的提交；支持source、sink、job的管理；可以管理openshift上的flink集群
Stars: ✭ 264 (+277.14%)
Mutual labels:  stream, flink
God Of Bigdata
专注大数据学习面试，大数据成神之路开启。Flink/Spark/Hadoop/Hbase/Hive...
Stars: ✭ 6,008 (+8482.86%)
Mutual labels:  bigdata, flink
dockerfiles
Multi docker container images for main Big Data Tools. (Hadoop, Spark, Kafka, HBase, Cassandra, Zookeeper, Zeppelin, Drill, Flink, Hive, Hue, Mesos, ... )
Stars: ✭ 29 (-58.57%)
Mutual labels:  bigdata, flink
bigdata-doc
大数据学习笔记，学习路线，技术案例整理。
Stars: ✭ 37 (-47.14%)
Mutual labels:  bigdata, flink
hadoopoffice
HadoopOffice - Analyze Office documents using the Hadoop ecosystem (Spark/Flink/Hive)
Stars: ✭ 56 (-20%)
Mutual labels:  bigdata, flink
View All Similar Projects ➔

Flink Learn

Flink scala version Kafka version
1.14.2 2.12+ 0.10+ 
Maven 模板 : https://www.mvnjar.com/org.apache.flink/flink-quickstart-scala/jar.html <br>
注意 ： 当在idea上调试时，先将 es-shade install。然后 ignore Project：es-shade ；否则有冲突问题

Flink Submit

！：1.10之后去掉 -yn ，加上会报错
1：单独资源： flink run -m yarn-cluster --yarnname _wc -p 5 -yjm 1024 -ytm 1024 -ys 2 WordCount.jar
2：共享空间： yarn-session.sh -n 5 -jm 1024 -tm 1024 -s 2 // 在yarn提前开辟一个资源空间 application_1567318548013_0001。 <br>
在开辟的空间上提交任务： flink run -m yarn-cluster --yarnname flink_wc -p 5 -yn 4 -yjm 1024 -ytm 1024 -ys 2 -yid application_1567318548013_0001 WordCount.jar <br>

测试的内容包括

Demo和一些注解
1: Flink DDL
2: FLink Connector
3: Flink Calcite
4: Flink Join
5: FLink 维表
6: Flink Window
7: FLink Datalake
8: Flink Hive
9: Flink CDC
10: FLink 时态表
11: Flink State
12: Flink CEP
13: Flink StateProcessApi
14: Flink Bug

Flink API
官方文档 ： Table DataStream 互转

StreamExecutionEnvironment : 流式相关。不能使用SQL的API。如果要在流里面用SQL，使用下面的
StreamTableEnvironment ： 流式SQL相关。可以使用 SQL的API。如果要用Stream相关的，需要将tableData.toRetractStream[Row]

mvn编译参数加上： -Xmx2g -XX:MaxMetaspaceSize=1024m
window 系统在 assembly.xml 里面使用 <outputDirectory>./</outputDirectory>
否则报错：
OS=Windows and the assembly descriptor contains a *nix-specific root-relative-reference (starting with slash)
Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].