admin管理员组

文章数量:1125590

When Pyspark is used to write data to the hudi table and the options content is as follows：

hudi_options = {   'hoodie.datasource.write.keygenerator.class':'org.apache.hudi.keygen.ComplexKeyGenerator', 
'hoodie.datasource.hive_sync.database': 'dingpan_test_mrs', 
'hoodie.table.name': 'dwd_design_designasset_function_offline_wide', 'hoodie.datasource.write.table.name': 'dwd_design_designasset_function_offline_wide', 'hoodie.datasource.write.recordkey.field':'id', 'hoodie.datasource.write.precombine.field': 'modified_date', 'hoodie.datasource.write.table.type':'COPY_ON_WRITE', 'hoodie.datasource.write.operation': 'upsert', 
'hoodie.upsert.shuffle.parallelism': 1, 
'hoodie.insert.shuffle.parallelism': 1 
} 

new_data.write
.format("hudi")
.options(**hudi_options)
.mode("append")
.save(obs_path)

the error message：

Could not load key generator class org.apache.hudi.keygen.ComplexKeyGenerator.

How should I adjust my code？Please

本文标签： pyspark Could not load key generator class orgapachehudikeygenComplexKeyGeneratorStack Overflow

版权声明：本文标题：pyspark Could not load key generator class org.apache.hudi.keygen.ComplexKeyGenerator - Stack Overflow 内容由网友自发贡献，该文观点仅代表作者本人， 转载请联系作者并注明出处：http://www.betaflare.com/web/1736661400a1946433.html， 本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如发现本站有涉嫌抄袭侵权/违法违规的内容，一经查实，本站将立刻删除。