Spring Cloud Data Flow Streams Error: Unable to access jarfile - spring-boot

Question

I launch Spring cloud data flow with docker-compose base on this website.
https://dataflow.spring.io/docs/installation/local/docker/
I created 3 apps, Source, Processor & Sink.
I registered them on the Spring Data Flow Cloud Dashboard.
Then I created a stream with the source connecting to processor connecting to sink.
When I deployed the apps, and opened http://localhost:9393/streams/logs/{name-of-stream},
I get the following error,
{
logs: {
testdir-stream.sender1-v3: "Error: Unable to access jarfile /testdir/data-sender-0.0.1-SNAPSHOT.jar ",
testdir-stream.sink1-v3: "Error: Unable to access jarfile /testdir/data-sink-0.0.1-SNAPSHOT.jar ",
testdir-stream.processor1-v3: "Error: Unable to access jarfile /testdir/data-processor-0.0.1-SNAPSHOT.jar "
}
}
Apparently, the Spring Cloud Data Flow (SCDF) server cannot access(or executing it or something?) the jarfile. I have chmod +x the entire testdir folder in the docker container such that the permissions is 'drwxr-xrwx'.
However, SCDF is still unable to access the jar. How can I solve this problem? Thanks.
EDIT
I am running docker-compose -f ./docker-compose.yml -f ./docker-compose-prometheus.yml up
docker-compose.yml
version: '3'
services:
mysql:
image: mysql:5.7.25
container_name: dataflow-mysql
environment:
MYSQL_DATABASE: dataflow
MYSQL_USER: root
MYSQL_ROOT_PASSWORD: rootpw
expose:
- 3306
kafka-0:
image: confluentinc/cp-kafka:5.2.1
container_name: dataflow-kafka-0
expose:
- "9092"
environment:
- KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://kafka-0:9092
- KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181
- KAFKA_ADVERTISED_HOST_NAME=kafka-0
- KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1
depends_on:
- zookeeper
kafka-1:
image: confluentinc/cp-kafka:5.2.1
container_name: dataflow-kafka-1
expose:
- "9093"
environment:
- KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://kafka-1:9093
- KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181
- KAFKA_ADVERTISED_HOST_NAME=kafka-1
- KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1
depends_on:
- zookeeper
zookeeper:
image: confluentinc/cp-zookeeper:5.2.1
container_name: dataflow-kafka-zookeeper
expose:
- "2181"
environment:
- ZOOKEEPER_CLIENT_PORT=2181
dataflow-server:
image: springcloud/spring-cloud-dataflow-server:${DATAFLOW_VERSION:?DATAFLOW_VERSION is not set!}
container_name: dataflow-server
volumes:
- 'C:\Users\Desmond\Desktop\Desmond\Project\springcloud:/root/apps'
ports:
- "9393:9393"
environment:
- spring.cloud.dataflow.applicationProperties.stream.spring.cloud.stream.kafka.binder.brokers=PLAINTEXT://kafka-0:9092,PLAINTEXT://kafka-1:9093
- spring.cloud.dataflow.applicationProperties.stream.spring.cloud.stream.kafka.streams.binder.brokers=PLAINTEXT://kafka-0:9092,PLAINTEXT://kafka-1:9093
- spring.cloud.dataflow.applicationProperties.stream.spring.cloud.stream.kafka.binder.zkNodes=zookeeper:2181
- spring.cloud.dataflow.applicationProperties.stream.spring.cloud.stream.kafka.streams.binder.zkNodes=zookeeper:2181
- spring.cloud.skipper.client.serverUri=http://skipper-server:7577/api
- SPRING_DATASOURCE_URL=jdbc:mysql://mysql:3306/dataflow
- SPRING_DATASOURCE_USERNAME=root
- SPRING_DATASOURCE_PASSWORD=rootpw
- SPRING_DATASOURCE_DRIVER_CLASS_NAME=org.mariadb.jdbc.Driver
depends_on:
- kafka-0
- kafka-1
entrypoint: "./wait-for-it.sh mysql:3306 -- java -jar /maven/spring-cloud-dataflow-server.jar"
app-import:
image: springcloud/openjdk:2.0.0.RELEASE
container_name: dataflow-app-import
depends_on:
- dataflow-server
command: >
/bin/sh -c "
./wait-for-it.sh -t 180 dataflow-server:9393;
wget -qO- 'http://dataflow-server:9393/apps' --post-data='uri=${STREAM_APPS_URI:-https://dataflow.spring.io/kafka-maven-latest&force=true}';
echo 'Stream apps imported'
wget -qO- 'http://dataflow-server:9393/apps' --post-data='uri=${TASK_APPS_URI:-https://dataflow.spring.io/task-maven-latest&force=true}';
echo 'Task apps imported'"
skipper-server:
image: springcloud/spring-cloud-skipper-server:${SKIPPER_VERSION:?SKIPPER_VERSION is not set!}
container_name: skipper
ports:
- "7577:7577"
- "9000-9010:9000-9010"
- "20000-20105:20000-20105"
environment:
- SPRING_CLOUD_SKIPPER_SERVER_PLATFORM_LOCAL_ACCOUNTS_DEFAULT_PORTRANGE_LOW=20000
- SPRING_CLOUD_SKIPPER_SERVER_PLATFORM_LOCAL_ACCOUNTS_DEFAULT_PORTRANGE_HIGH=20100
- SPRING_DATASOURCE_URL=jdbc:mysql://mysql:3306/dataflow
- SPRING_DATASOURCE_USERNAME=root
- SPRING_DATASOURCE_PASSWORD=rootpw
- SPRING_DATASOURCE_DRIVER_CLASS_NAME=org.mariadb.jdbc.Driver
entrypoint: "./wait-for-it.sh mysql:3306 -- java -Djava.security.egd=file:/dev/./urandom -jar /spring-cloud-skipper-server.jar"
docker-compose-prometheus.yml
version: '3'
# Extends the default docker-compose.yml with Prometheus/Grafana monitoring configuration
# Usage: docker-compose -f ./docker-compose.yml -f ./docker-compose-prometheus.yml up
services:
dataflow-server:
environment:
- |
SPRING_APPLICATION_JSON=
{
"spring.cloud.dataflow.applicationProperties":{
"task.management.metrics.export.prometheus":{
"enabled":true,
"rsocket.enabled":true,
"rsocket.host":"prometheus-rsocket-proxy",
"rsocket.port":7001
},
"stream.management.metrics.export.prometheus":{
"enabled":true,
"rsocket.enabled":true,
"rsocket.host":"prometheus-rsocket-proxy",
"rsocket.port":7001
}
},
"spring.cloud.dataflow.grafana-info.url":"http://localhost:3000"
}
prometheus-rsocket-proxy:
image: micrometermetrics/prometheus-rsocket-proxy:0.8.0
container_name: prometheus-rsocket-proxy
expose:
- '9096'
- '7001'
- '8081'
ports:
- '9096:9096'
- '7001:7001'
- '8081:8081'
environment:
- server.port=9096
grafana:
image: springcloud/spring-cloud-dataflow-grafana-prometheus:${DATAFLOW_VERSION:?DATAFLOW_VERSION is not set! Use 'export DATAFLOW_VERSION=local-server-image-tag'}
container_name: grafana
ports:
- '3000:3000'
prometheus:
image: springcloud/spring-cloud-dataflow-prometheus-local:${DATAFLOW_VERSION:?DATAFLOW_VERSION is not set! Use 'export DATAFLOW_VERSION=local-server-image-tag'}
container_name: prometheus
ports:
- '9090:9090'
Pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.1.7.RELEASE</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<groupId>com.example</groupId>
<artifactId>data-sender</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>data-sender</name>
<description>Test Proj</description>
<properties>
<java.version>1.8</java.version>
<spring-cloud.version>Greenwich.SR2</spring-cloud.version>
<mockito-junit-jupiter.version>2.19.0</mockito-junit-jupiter.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-stream</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-stream-binder-kafka</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.kafka</groupId>
<artifactId>spring-kafka</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-stream-test-support</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.kafka</groupId>
<artifactId>spring-kafka-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<version>1.18.4</version>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-engine</artifactId>
<version>${junit-jupiter.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-junit-jupiter</artifactId>
<version>${mockito-junit-jupiter.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-classic</artifactId>
</dependency>
<!-- https://mvnrepository.com/artifact/com.googlecode.json-simple/json-simple -->
<dependency>
<groupId>com.googlecode.json-simple</groupId>
<artifactId>json-simple</artifactId>
<version>1.1.1</version>
</dependency>
<dependency>
<groupId>net.lingala.zip4j</groupId>
<artifactId>zip4j</artifactId>
<version>1.3.2</version>
</dependency>
<!-- https://mvnrepository.com/artifact/commons-io/commons-io -->
<dependency>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
<version>2.6</version>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
<version>3.9</version>
</dependency>
<!-- prometheus-rsocket monitoring -->
<dependency>
<groupId>org.springframework.cloud.stream.app</groupId>
<artifactId>app-starters-micrometer-common</artifactId>
</dependency>
<dependency>
<groupId>io.micrometer</groupId>
<artifactId>micrometer-registry-prometheus</artifactId>
</dependency>
<dependency>
<groupId>io.micrometer.prometheus</groupId>
<artifactId>prometheus-rsocket-spring</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-cloudfoundry-connector</artifactId>
</dependency>
</dependencies>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>${spring-cloud.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>

Answer 1

For accessing custom applications from your local machine, you need to configure both the dataflow-server and the skipper-skipper Docker containers to access to your local file system. You have already mounted the source folder to dataflow-server but you need to do the same with the skipper-server definition like this:
skipper-server:
image: springcloud/spring-cloud-skipper-server:${SKIPPER_VERSION:?SKIPPER_VERSION is not set!}
container_name: skipper
volumes:
- 'C:\Users\Desmond\Desktop\Desmond\Project\springcloud:/root/apps'
...
We will clarify this in our documentation #128
Two additional points regarding prometheus-rsocket monitoring:
Since you are using the new prometheus-rsocket configuration you will need to import the Einstein-BUILD-SNAPSHOT-stream-applications-kafka-maven app starters instead of kafka-maven-latest. You can do this by setting the following property before starting docker-compose:
export STREAM_APPS_URI=https://dataflow.spring.io/Einstein-BUILD-SNAPSHOT-stream-applications-kafka-maven
The SR4 app starter release is expected very soon, but until then you need to use the BUILD-SNAPSHOT to leverage the prometheus metrics.
Can you please share how you have implemented your custom apps? Do they contain the following dependencies?
<dependencyManagement>
<dependencies>
...
<dependency>
<groupId>org.springframework.cloud.stream.app</groupId>
<artifactId>app-starters-core-dependencies</artifactId>
<version>2.1.2.BUILD-SNAPSHOT</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
....
<dependency>
<groupId>org.springframework.cloud.stream.app</groupId>
<artifactId>app-starters-micrometer-common</artifactId>
</dependency>
<dependency>
<groupId>io.micrometer</groupId>
<artifactId>micrometer-registry-prometheus</artifactId>
</dependency>
<dependency>
<groupId>io.micrometer.prometheus</groupId>
<artifactId>prometheus-rsocket-spring</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-cloudfoundry-connector</artifactId>
</dependency>
</dependencies>
<repositories>
<repository>
<snapshots>
<enabled>true</enabled>
</snapshots>
<id>spring-snapshots</id>
<name>Spring Snapshots</name>
<url>https://repo.spring.io/snapshot</url>
</repository>
...
</repositories>
If your apps's parent POM is org.springframework.cloud.stream.app:app-starters-build:2.1.2.BUILD-SNAPSHOT above dependencies are already injected. Otherwise you have to add them manually.
UPDATE -- adding a simple pom configuration to enable prometheus-rsocket metrics:
<properties>
<spring-cloud.version>Greenwich.SR2</spring-cloud.version>-->
<prometheus-rsocket-spring.version>0.8.0</prometheus-rsocket-spring.version>
</properties>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>io.micrometer.prometheus</groupId>
<artifactId>prometheus-rsocket-spring</artifactId>
<version>${prometheus-rsocket-spring.version}</version>
</dependency>
<dependency>
<groupId>io.micrometer.prometheus</groupId>
<artifactId>prometheus-rsocket-client</artifactId>
<version>${prometheus-rsocket-spring.version}</version>
</dependency>

Answer 2

I believe container configs are wrong for skipper server as that's the one running those containers if local setup is used. It should work if same volume is used in skipper as it's now done with dataflow server.
For getting those logs, dataflow simply requests those from a skipper and error originates from there.