这篇文章主要讲解了“flink mysql数据接入的方法”，文中的讲解内容简单清晰，易于学习与理解，下面请大家跟着小编的思路慢慢深入，一起来研究和学习“flink mysql数据接入的方法”吧！
一.api方式接入
1.添加依赖
<dependency>
<groupId>com.alibaba.ververica</groupId>
<artifactId>flink-connector-mysql-cdc</artifactId>
<version>
1.1
.
0
</version>
</dependency>
|
2.API代码
public
static
void
main(String[] args)
throws
Exception {
SourceFunction<String> sourceFunction = MySQLSource.<String>builder()
.hostname(
"localhost"
)
.port(
3306
)
.databaseList(
"test"
)
.tableList(
"test"
)
.deserializer(
new
StringDebeziumDeserializationSchema())
.build();
StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
env.addSource(sourceFunction)
.print().setParallelism(
1
);
env.execute();
}
|
二.sql方式接入
1.添加jar包至lib下
flink-sql-connector-mysql-cdc_1.1.0.jar
2.mysql中创建表
create
table
test(
id
INT
,
name
varchar
(100),
description
varchar
(100),
weight
DECIMAL
(10,3)
)
|
3.创建元数据
CREATE
TABLE
mysql_test (
id
INT
NOT
NULL
,
name
STRING,
description STRING,
weight
DECIMAL
(10,3)
)
WITH
(
'connector'
=
'mysql-cdc'
,
'hostname'
=
'localhost'
,
'port'
=
'3306'
,
'username'
=
'root'
,
'password'
=
'root'
,
'database-name'
=
'test'
,
'table-name'
=
'test'
);
|
4.使用查询sql
SELECT
id,
UPPER
(
name
), description, weight
FROM
mysql_test;
|
5.增加和删除表字段测试
增加不影响
删除表字段后，会出错
[ERROR] Could not execute SQL statement. Reason:
org.apache.kafka.connect.errors.DataException: name is not a valid field name
注：mysql的版本如果是8.0，flink端链接会出错
com.github.shyiko.mysql.binlog.network.AuthenticationException: Client does not support authentication protocol requested by server; consider upgrading MySQL client
出现上述问题的原因是：mysql8 之前的版本中加密规则是mysql_native_password,而在mysql8之后,加密规则是caching_sha2_password 把mysql用户登录密码加密规则还原成mysql_native_password
解决方案：在mysql中执行以下命令
alter user 'root'@'%' identified with mysql_native_password by 'root'; 修改认证规则
flush privileges; 刷新权限
感谢各位的阅读，以上就是“flink mysql数据接入的方法”的内容了，经过本文的学习后，相信大家对flink mysql数据接入的方法这一问题有了更深刻的体会，具体使用情况还需要大家实践验证。这里是天达云，小编将为大家推送更多相关知识点的文章，欢迎关注！