实验二:Flink初级编程实践
一、实验目的:
基本的Flink编程方法;
用Eclipse工具编写Flink程序的方法。
二、实验设备与器件
PC机、流式计算开发环境
三、实验内容
基本的Flink编程方法:
用Eclipse工具编写Flink程序的方法。
设计题目
1. Maven安装包路径:/opt/apache-maven-3.6.3-bin.zip
。
注意:
建议Maven安装完成后设置为阿里云镜像仓库:https://maven.aliyun.com/repository/public
2. 使用Eclipse工具开发WordCount程序,并打包成JAR文件,提交到Flink中运行,Jar上传路径:/usr/local/hadoop/FlinkWordCount-0.0.1-SNAPSHOT.jar
,需要统计的原始数据如下:
(1)public static final String[] WORDS=new String[]{"To be, or not to be,--that is the question:--", "Whether \'tis nobler in the mind to suffer", "The slings and arrows of outrageous fortune", "Or to take arms against a sea of troubles,", "And by opposing end them?--To die,--to sleep,--", "No more; and by a sleep to say we end", "The heartache, and the thousand natural shocks", "That flesh is heir to,--\'tis a consummation", "Devoutly to be wish\'d. To die,--to sleep;--", "To sleep! perchance to dream:--ay, there\'s the rub;", "For in that sleep of death what dreams may come,", "When we have shuffled off this mortal coil,", "Must give us pause: there\'s the respect", "That makes calamity of so long life;", "For who would bear the whips and scorns of time,", "The oppressor\'s wrong, the proud man\'s contumely,", "The pangs of despis\'d love, the law\'s delay,", "The insolence of office, and the spurns", "That patient merit of the unworthy takes,", "When he himself might his quietus make", "With a bare bodkin? who would these fardels bear,", "To grunt and sweat under a weary life,", "But that the dread of something after death,--", "The undiscover\'d country, from whose bourn", "No traveller returns,--puzzles the will,", "And makes us rather bear those ills we have", "Than fly to others that we know not of?", "Thus conscience does make cowards of us all;", "And thus the native hue of resolution", "Is sicklied o\'er with the pale cast of thought;", "And enterprises of great pith and moment,", "With this regard, their currents turn awry,", "And lose the name of action.--Soft you now!", "The fair Ophelia!--Nymph, in thy orisons", "Be all my sins remember\'d."};
注意:
Maven项目的依赖及版本请按照如下配置:
3.
如果Maven设置了阿里云
或其他外网镜像仓库,但无法下载依赖,很可能代表你处在内网环境下,无法访问到外网的镜像仓库,可以设置Maven
为我们提前准备好的私有镜像仓库:http://nexus3.plugins.svc/repository/maven-public/
,然后再尝试下载所需依赖。数据流词频统计
使用Linux系统自带的NC程序模拟生成数据流,不断产生单词并发送出去。编写Flink程序对NC程序发来的单词进行实时处理,计算词频,并把词频统计结果输出。要求打成JAR包部署到Flink中运行,Jar上传路径:/usr/local/hadoop/FlinkWordCount2-0.0.1-SNAPSHOT.jar
。
测试说明
在按编程要求
完成操作后,请点击评测
按钮,系统会自动对你的操作进行评测。
当你的结果与预期输出一致时,即为通过。
预期输出:
