canal的kafka的kerberos认证,Could not find a ‘KafkaClient’ entry in the JAAS configuration. System proper

2023年1月18日07:56:36

关于canal同步数据到kafka的kerberos认证,操作过程:

1、修改canal.properties

canal.mq.kafka.kerberos.enable = true
canal.mq.kafka.kerberos.krb5FilePath = ../conf/kerberos/krb5.conf
canal.mq.kafka.kerberos.jaasFilePath = ../conf/kerberos/jaas.conf

2、在conf目录下创建kerberos文件夹

添加krb5.conf文件
添加jaas.conf文件
添加kafka.keytab文件
-----------在kerberos下有三个文件

3、krb5.conf文件和jaas.conf文件

##jaas.conf文件
KafkaClient {
   com.sun.security.auth.module.Krb5LoginModule required
   useKeyTab=true
   keyTab="../conf/kerberos/kafka.keytab"
   principal="kafka/master1-dev.hadoop.com.cn@HADOOP.COM.CN";
};
##krb5.conf文件
includedir /etc/krb5.conf.d/

[logging]
 default = FILE:/var/log/krb5libs.log
 kdc = FILE:/var/log/krb5kdc.log
 admin_server = FILE:/var/log/kadmind.log

[libdefaults]
 default_realm = HADOOP.COM.CN
 dns_lookup_realm = false
 dns_lookup_kdc = false
 ticket_lifetime = 24h
 renew_lifetime = 7d
 forwardable = true
 rdns = false

[realms]
 HADOOP.COM.CN = {
  kdc = master1-dev.hadoop.com.cn
  admin_server = master1-dev.hadoop.com.cn
 }

[domain_realm]
 .hadoop.com.cn = HADOOP.COM.CN
 hadoop.com.cn = HADOOP.COM.CN

4、启动报错

[main] ERROR com.alibaba.otter.canal.server.CanalMQStarter - ## Something goes wrong when starting up the canal MQ workers:
org.apache.kafka.common.KafkaException: Failed to construct kafka producer
        at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:456) ~[kafka-clients-1.1.1.jar:na]
        at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:303) ~[kafka-clients-1.1.1.jar:na]
        at com.alibaba.otter.canal.kafka.CanalKafkaProducer.init(CanalKafkaProducer.java:81) ~[canal.server-1.1.4.jar:na]
        at com.alibaba.otter.canal.server.CanalMQStarter.start(CanalMQStarter.java:51) ~[canal.server-1.1.4.jar:na]
        at com.alibaba.otter.canal.deployer.CanalStarter.start(CanalStarter.java:101) [canal.deployer-1.1.4.jar:na]
        at com.alibaba.otter.canal.deployer.CanalLauncher.main(CanalLauncher.java:115) [canal.deployer-1.1.4.jar:na]
Caused by: java.lang.IllegalArgumentException: Could not find a 'KafkaClient' entry in the JAAS configuration. System property 'java.security.auth.login.config' is /data/home/apps/canal_1.1.4_local/bin/../conf/kerberos/jaas.conf
        at org.apache.kafka.common.security.JaasContext.defaultContext(JaasContext.java:133) ~[kafka-clients-1.1.1.jar:na]
        at org.apache.kafka.common.security.JaasContext.load(JaasContext.java:98) ~[kafka-clients-1.1.1.jar:na]
        at org.apache.kafka.common.security.JaasContext.loadClientContext(JaasContext.java:84) ~[kafka-clients-1.1.1.jar:na]
        at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:119) ~[kafka-clients-1.1.1.jar:na]
        at org.apache.kafka.common.network.ChannelBuilders.clientChannelBuilder(ChannelBuilders.java:65) ~[kafka-clients-1.1.1.jar:na]
        at org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:88) ~[kafka-clients-1.1.1.jar:na]
        at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:413) ~[kafka-clients-1.1.1.jar:na]
        ... 5 common frames omitted

确定配置了jass并且也确定没有问题,但就是启动失败。

最后在启动脚本./bin/startup.sh里面添加一句:

JAVA_OPTS=" $JAVA_OPTS -Djava.security.auth.login.config=$base/conf/kerberos/jaas.conf"

如果还是不行,再加一句。

JAVA_OPTS="  $JAVA_OPTS -Djava.security.krb5.conf=$base/conf/kerberos/krb5.conf"

就OK了!

  • 作者:遗梦孤魂
  • 原文链接:https://blog.csdn.net/weixin_40126236/article/details/103492815
    更新时间:2023年1月18日07:56:36 ,共 3054 字。