Apache NIFI 2.1.0 displays duplicated controller service related to redis after adding my custom nar file - Stack Overflow

admin管理员组

文章数量:1126062

Environment:

• Apache NIFI 2.1.0

I am developing a custom processor for Apache NIFI 2.1.0. This processor needs to use RedisConnectionPool/RedisConnectionPoolService. Therefore I add some dependencies related to nifi-redis.

My processor (JJMyRedisProcessor.java):

import org.apache.nifiponents.PropertyDescriptor;
import org.apache.nifi.flowfile.FlowFile;
import org.apache.nifi.annotation.behavior.ReadsAttribute;
import org.apache.nifi.annotation.behavior.ReadsAttributes;
import org.apache.nifi.annotation.behavior.WritesAttribute;
import org.apache.nifi.annotation.behavior.WritesAttributes;
import org.apache.nifi.annotation.lifecycle.OnScheduled;
import org.apache.nifi.annotation.documentation.CapabilityDescription;
import org.apache.nifi.annotation.documentation.SeeAlso;
import org.apache.nifi.annotation.documentation.Tags;
import org.apache.nifi.processor.AbstractProcessor;
import org.apache.nifi.processor.ProcessContext;
import org.apache.nifi.processor.ProcessSession;
import org.apache.nifi.processor.ProcessorInitializationContext;
import org.apache.nifi.processor.Relationship;
import org.apache.nifi.processor.util.StandardValidators;
import org.apache.nifi.redis.util.RedisUtils;
import java.util.List;
import java.util.Set;

@Tags({"example"})
@CapabilityDescription("Provide a description")
@SeeAlso({})
@ReadsAttributes({@ReadsAttribute(attribute="", description="")})
@WritesAttributes({@WritesAttribute(attribute="", description="")})
public class JJMyRedisProcessor extends AbstractProcessor {

    public static final PropertyDescriptor MY_PROPERTY = new PropertyDescriptor
            .Builder()
            .name("My Property")
            .displayName("My Property")
            .description("Example Property")
            .required(false)
            .addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
            .build();

    public static final Relationship REL_SUCCESS = new Relationship.Builder()
            .name("success")
            .description("Example success relationship")
            .build();

    private List<PropertyDescriptor> descriptors;

    private Set<Relationship> relationships;

    @Override
    protected void init(final ProcessorInitializationContext context) {
        descriptors = List.of(MY_PROPERTY, RedisUtils.REDIS_CONNECTION_POOL);

        relationships = Set.of(REL_SUCCESS);
    }

    @Override
    public Set<Relationship> getRelationships() {
        return this.relationships;
    }

    @Override
    public final List<PropertyDescriptor> getSupportedPropertyDescriptors() {
        return descriptors;
    }

    @OnScheduled
    public void onScheduled(final ProcessContext context) {

    }

    @Override
    public void onTrigger(final ProcessContext context, final ProcessSession session) {
        FlowFile flowFile = session.get();
        if (flowFile == null) {
            return;
        }
        // TODO implement

        session.transfer(flowFile, REL_SUCCESS);
    }
}

Below is my pom.xml in processors folder

<project xmlns=".0.0" xmlns:xsi="; xsi:schemaLocation=".0.0 .0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <parent>
        <groupId>com.fet</groupId>
        <artifactId>redistest</artifactId>
        <version>1.0</version>
    </parent>

    <artifactId>nifi-fettest-processors</artifactId>
    <packaging>jar</packaging>

    <dependencies>
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-api</artifactId>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-utils</artifactId>
            <version>2.1.0</version>
        </dependency>
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-mock</artifactId>
            <version>2.1.0</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-simple</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-api</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-engine</artifactId>
            <scope>test</scope>
        </dependency>
        <!-- =============== I added below dependencies ============================= -->
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-ssl-context-service-api</artifactId>
            <version>2.1.0</version>

        </dependency>
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-record-serialization-service-api</artifactId>
            <version>2.1.0</version>

        </dependency>
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-redis-service-api</artifactId>
            <version>2.1.0</version>
        </dependency>
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-redis-utils</artifactId>
            <version>2.1.0</version>
        </dependency>
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-redis-extensions</artifactId>
            <version>2.1.0</version>
        </dependency>
    </dependencies>
</project>

After mvn clean compile package suffessfully and copy the target generated .nar file to Apache NIFI 2.1.0's nar_extenstions/ folder, restart NIFI. I realize that when Add Controller Service and filter by 'Redis', NIFI shows duplicated items with different version as below:

Now you can see that there are two RedisConnectionPoolService with two version, 1.0 and 2.1.0. So I guess that maybe I should change some dependencies's scope to provided. Then, I change my pom.xml to below (dependencies portion only)

    <dependency>
        <groupId>org.apache.nifi</groupId>
        <artifactId>nifi-redis-service-api</artifactId>
        <version>2.1.0</version>
        <scope>provided</scope>
    </dependency>
    <dependency>
        <groupId>org.apache.nifi</groupId>
        <artifactId>nifi-redis-utils</artifactId>
        <version>2.1.0</version>
        <scope>provided</scope>
    </dependency>
    <dependency>
        <groupId>org.apache.nifi</groupId>
        <artifactId>nifi-redis-extensions</artifactId>
        <version>2.1.0</version>
        <scope>provided</scope>
    </dependency>

mvn clean compile package again. Now mvn appears error:

  Caused by: java.lang.NoClassDefFoundError: org/apache/nifi/redis/util/RedisUtils

Then, I adjust dependencies in pom.xml to below:

    <dependency>
        <groupId>org.apache.nifi</groupId>
        <artifactId>nifi-redis-service-api</artifactId>
        <version>2.1.0</version>
        <scope>provided</scope>
    </dependency>
    <dependency>
        <groupId>org.apache.nifi</groupId>
        <artifactId>nifi-redis-utils</artifactId>
        <version>2.1.0</version>
        <!-- <scope>provided</scope> -->
    </dependency>
    <dependency>
        <groupId>org.apache.nifi</groupId>
        <artifactId>nifi-redis-extensions</artifactId>
        <version>2.1.0</version>
        <scope>provided</scope>
    </dependency>

mvn compile compile package tells me below error:

Caused by: java.lang.NoClassDefFoundError: org/springframework/data/redis/core/RedisTemplate
    at com.fet.processors.fettest.JJMyRedisProcessor.init (JJMyRedisProcessor.java:50)
....
[INFO] Reactor Summary for redistest 1.0:
[INFO]
[INFO] redistest .......................................... SUCCESS [  1.145 s]
[INFO] nifi-fettest-processors ............................ SUCCESS [  4.212 s]
[INFO] nifi-2.1.0-fettest-nar ............................. FAILURE [  8.614 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE

Line 50 in JJMyRedisProcessor.java is:

        descriptors = List.of(MY_PROPERTY, RedisUtils.REDIS_CONNECTION_POOL)

I add org.springframework.data:spring-data-redis:3.4.0 to dependencies:

    <dependency>
        <groupId>org.springframework.data</groupId>
        <artifactId>spring-data-redis</artifactId>
        <version>3.4.0</version>
    </dependency>

mvn compile package and tells me error:

Caused by: java.lang.NoClassDefFoundError: org/apache/commons/pool2/impl/GenericObjectPoolConfig
    at com.fet.processors.fettest.JJMyRedisProcessor.init (JJMyRedisProcessor.java:50)

Add org.apachemons:commons-pool2:2.11.1 to dependencies

    <dependency>
        <groupId>org.apachemons</groupId>
        <artifactId>commons-pool2</artifactId>
        <version>2.11.1</version>
    </dependency>

mvn compile package and tells me error:

Caused by: java.lang.NoClassDefFoundError: redis/clients/jedis/JedisPoolConfig
    at com.fet.processors.fettest.JJMyRedisProcessor.init (JJMyRedisProcessor.java:50)

Add redis.clients:jedis:5.2.0 to dependencies

    <dependency>
        <groupId>redis.clients</groupId>
        <artifactId>jedis</artifactId>
        <version>5.2.0</version>
    </dependency>

mvn compile package and tells me error:

Caused by: java.lang.NoClassDefFoundError: org/apache/nifi/redis/RedisConnectionPool
    at org.apache.nifi.redis.util.RedisUtils.<clinit> (RedisUtils.java:64)

org.apache.nifi.redis.RedisConnectionPool is in package nifi-redis-service-api. So, that is to say, mvn turns back to tell me that i should not set nifi-redis-service-api scope to 'provided'

Now, I have no idea how to adjust pom.xml so that the Controller List in NIFI won't appear the duplicate items like above.

Any instruction is appreciated.

本文标签：