java - KafkaClusterTestKit in test failure when switching to EmbeddedKafkaKraftBroker - Stack Overflow

admin管理员组

文章数量:1400097

We have some test classes that were based on EmebeddedKafkaZKBroker for testing, now that we are moving to Kraft, we uplifted our Kafka-client to version 4.0.0 and spring kafka to the latest version 3.3.4. Now since there are two different ways to use the @EmbeddedKafka annotation, we want to use the Kraft one so I added the config Kraft = true but still the test with @EmbeddedKafka annotation are failing... Our Dependencies

dependencies {
    implementation ".springframework.boot:spring-boot-starter-websocket:${springBootStarterVersion}"
    implementation group: 'com.nimbusds', name: 'nimbus-jose-jwt', version: '9.47'
    implementation group: 'com.google.guava', name: 'guava', version: '33.4.0-jre'
    implementation('.projectlombok:lombok:1.18.36')
    annotationProcessor('.projectlombok:lombok:1.18.36')
    implementation(".apache.kafka:kafka-clients:${kafkaVersion}")
    implementation(".springframework.kafka:spring-kafka:${springKafkaVersion}")
    implementation(".springframework.boot:spring-boot-starter-actuator:${springBootStarterVersion}")
    // This change is included to force gradle to take a specific version of Apache Tomcat while building
    implementation(".springframework.boot:spring-boot-starter-web:${springBootStarterVersion}"){
        exclude group: '.springframework.boot', module: 'spring-boot-starter-tomcat'
    }
    implementation(".apache.tomcat.embed:tomcat-embed-core:${apacheTomcatVersion}")
    implementation(".apache.tomcat.embed:tomcat-embed-el:${apacheTomcatVersion}")
    implementation(".apache.tomcat.embed:tomcat-embed-websocket:${apacheTomcatVersion}")
    // Logging dependencies
    implementation(".springframework.boot:spring-boot-starter-log4j2:${springBootStarterVersion}")
    annotationProcessor('.apache.logging.log4j:log4j-core:2.24.2')

    // Test implementations
    testImplementation ".springframework.boot:spring-boot-starter-test:${springBootStarterVersion}"
    testImplementation(".springframework.kafka:spring-kafka-test:${springKafkaVersion}") {
        exclude group: '.apache.zookeeper', module: 'zookeeper'
        exclude group: '.apache.kafka', module: 'zookeeper-jute'
    }

    testImplementation('.junit.jupiter:junit-jupiter-api:5.11.4')
    testImplementation(".junit.platform:junit-platform-runner:1.11.4")
    testImplementation(".junit.platform:junit-platform-commons:1.11.4")
    testRuntimeOnly('.junit.jupiter:junit-jupiter-engine:5.11.4')
    testRuntimeOnly('.junit.vintage:junit-vintage-engine:5.11.4')
    // Added as a part of kafka-clients uplift to version 3.9.0
    testImplementation(".apache.kafka:kafka_2.13:${kafkaVersion}")
    testImplementation(".apache.kafka:kafka-server:${kafkaVersion}")
    testImplementation(".apache.kafka:kafka-server-common:${kafkaVersion}")
    testImplementation(".apache.kafka:kafka-storage:${kafkaVersion}")
    testImplementation(".apache.kafka:kafka-raft:${kafkaVersion}")
    testImplementation(".springframework.kafka:spring-kafka-test:${springKafkaVersion}")
    {
        exclude group: '.apache.kafka', module: 'kafka_2.12'
    }
}

The EmbeddedKafka setting:

@EmbeddedKafka(
    partitions = 1,
    controlledShutdown = false,
    brokerProperties = {
        "listeners=PLAINTEXT://localhost:9092,CONTROLLER://localhost:9093",
        "auto.create.topics.enable=true",
        "process.roles=broker,controller",
        "controller.quorum.voters=1@localhost:9093",
        "node.id=1",
        "controller.listener.names=CONTROLLER"
    },
    kraft = true
)

Is this any extra configuration I am missing or any more dependencies that needs to be added for the test classes to work ?

We have some test classes that were based on EmebeddedKafkaZKBroker for testing, now that we are moving to Kraft, we uplifted our Kafka-client to version 4.0.0 and spring kafka to the latest version 3.3.4. Now since there are two different ways to use the @EmbeddedKafka annotation, we want to use the Kraft one so I added the config Kraft = true but still the test with @EmbeddedKafka annotation are failing... Our Dependencies

dependencies {
    implementation ".springframework.boot:spring-boot-starter-websocket:${springBootStarterVersion}"
    implementation group: 'com.nimbusds', name: 'nimbus-jose-jwt', version: '9.47'
    implementation group: 'com.google.guava', name: 'guava', version: '33.4.0-jre'
    implementation('.projectlombok:lombok:1.18.36')
    annotationProcessor('.projectlombok:lombok:1.18.36')
    implementation(".apache.kafka:kafka-clients:${kafkaVersion}")
    implementation(".springframework.kafka:spring-kafka:${springKafkaVersion}")
    implementation(".springframework.boot:spring-boot-starter-actuator:${springBootStarterVersion}")
    // This change is included to force gradle to take a specific version of Apache Tomcat while building
    implementation(".springframework.boot:spring-boot-starter-web:${springBootStarterVersion}"){
        exclude group: '.springframework.boot', module: 'spring-boot-starter-tomcat'
    }
    implementation(".apache.tomcat.embed:tomcat-embed-core:${apacheTomcatVersion}")
    implementation(".apache.tomcat.embed:tomcat-embed-el:${apacheTomcatVersion}")
    implementation(".apache.tomcat.embed:tomcat-embed-websocket:${apacheTomcatVersion}")
    // Logging dependencies
    implementation(".springframework.boot:spring-boot-starter-log4j2:${springBootStarterVersion}")
    annotationProcessor('.apache.logging.log4j:log4j-core:2.24.2')

    // Test implementations
    testImplementation ".springframework.boot:spring-boot-starter-test:${springBootStarterVersion}"
    testImplementation(".springframework.kafka:spring-kafka-test:${springKafkaVersion}") {
        exclude group: '.apache.zookeeper', module: 'zookeeper'
        exclude group: '.apache.kafka', module: 'zookeeper-jute'
    }

    testImplementation('.junit.jupiter:junit-jupiter-api:5.11.4')
    testImplementation(".junit.platform:junit-platform-runner:1.11.4")
    testImplementation(".junit.platform:junit-platform-commons:1.11.4")
    testRuntimeOnly('.junit.jupiter:junit-jupiter-engine:5.11.4')
    testRuntimeOnly('.junit.vintage:junit-vintage-engine:5.11.4')
    // Added as a part of kafka-clients uplift to version 3.9.0
    testImplementation(".apache.kafka:kafka_2.13:${kafkaVersion}")
    testImplementation(".apache.kafka:kafka-server:${kafkaVersion}")
    testImplementation(".apache.kafka:kafka-server-common:${kafkaVersion}")
    testImplementation(".apache.kafka:kafka-storage:${kafkaVersion}")
    testImplementation(".apache.kafka:kafka-raft:${kafkaVersion}")
    testImplementation(".springframework.kafka:spring-kafka-test:${springKafkaVersion}")
    {
        exclude group: '.apache.kafka', module: 'kafka_2.12'
    }
}

The EmbeddedKafka setting:

@EmbeddedKafka(
    partitions = 1,
    controlledShutdown = false,
    brokerProperties = {
        "listeners=PLAINTEXT://localhost:9092,CONTROLLER://localhost:9093",
        "auto.create.topics.enable=true",
        "process.roles=broker,controller",
        "controller.quorum.voters=1@localhost:9093",
        "node.id=1",
        "controller.listener.names=CONTROLLER"
    },
    kraft = true
)

Is this any extra configuration I am missing or any more dependencies that needs to be added for the test classes to work ?

• java
• spring-boot
• apache-kafka
• spring-kafka
• spring-kafka-test

Share Improve this question Follow asked Mar 24 at 7:48 BloodFuryBloodFury 3955 bronze badges

Add a comment  | 

1 Answer 1

Sorted by: Reset to default 2

We don't claim that Spring Kafka 3.3.x is going to work with Kafka client 4.0.0, therefore we are working on Spring Kafka 4.0.0 for November release train and the work in progress for such an integration: https://github/spring-projects/spring-kafka/pull/3815. Keep in mind, though, the KafkaClusterTestKit still does not allow to use a fixed port. So, listeners and voters might not work as you would expect.

You may consider to use Testcontainers instead when you can specify fixed ports since that one is based on a real Kafka broker, not test kit.

本文标签： javaKafkaClusterTestKit in test failure when switching to EmbeddedKafkaKraftBrokerStack Overflow