I am trying to register a spring cloud task jar which is residing in a ftp server , from a spring cloud data flow container which is running on PCF.
Can somebody help me in terms of how to do a "app register" from a ftp location from SCDF if SCDF supports that ? How do i register a ftp location as remote repository with user name & passwd ?
Thanks!
Nilanjan
As discussed in the email, currently, in SCDF, we support file://, maven://, http://, docker://, and hdfs:// as the file resolution strategies. Besides, for cloud platforms, it'd be recommended that you resolve and register artifacts from maven or docker to be more resilient.
If you're running SCDF on PCF, you could bind the SCDF-server to volume-services. By having all the custom apps in volume-services, SCDF-server could then resolve artifacts from the designated directory via file:// resolution - this is another good alternative for PCF deployments.
Related
I am creating a spring boot microservice project with intelij IDEA.
Currently I have developed three seperate spring boot rest services as customer service, vehicle service and spring cloud config server. Spring cloud config server is pointing to a github repository.
The issue is sometimes above projects take more than 10 minutes to run and sometimes does't run and give an error message as "failed to check application readystate intellij attached provider for the vm is not found". I have no idea why this happens ?
There are two possible causes:
1. IntelliJ IDEA and the Spring application are running in different JVMs.
There is a bug for IntelliJ IDEA regarding that:
https://youtrack.jetbrains.com/issue/IDEA-210665
Here is short summary:
IntelliJ IDEA uses local JMX connector for retrieving Spring Boot actuator endpoint's data by default. However, it could be impossible to get local JMX connector address via attach api if Spring Boot application and IntelliJ IDEA are run by different JVMs. In this case, add the following lines to VM options of your Spring Boot run configuration:
-Dcom.sun.management.jmxremote.port={some_port}
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false
As mentioned in the official Oracle documentation, this configuration is insecure. Any remote user who knows (or guesses) your port number and host name will be able to monitor and control your Java applications and platform.
2. Prolonged time to retrieve local hostname
You can check that time using inetTester. Normally it should take only several milliseconds to complete. If it takes a long time then you can add the hostname returned by inetTester to /etc/hosts file like this:
127.0.0.1 localhost winsky
::1 localhost winsky
I'm deploying Spring Cloud Dataflow 1.4.0.RELEASE on my org's Cloud Foundry. I took the CF deployable server and used a manifest to deploy it. We use it mostly for tasks developed using Spring Cloud Task. In the documentation (that uses PCF) suggests this configuration:
SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_TASK_SERVICES my_mysql
Where my_mysql is the DB to keep track of task related information. Documentation suggests using this as CF's cups but that won't work since my org uses Vault to store keys and secrets so DB connections are not allowed in such way.
Since I did not provided any value for SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_TASK_SERVICES in my manifest when I start SPDF it throws an error indicating that bean "taskService" could not be created.
Haven't found a workaround for this. Is this the only way to specify DB connection for tasks in version 1.4?
You can also pass a deployment property when launching the task:
task launch --name mytask --properties "deployer.mytask.cloudfoundry.services=-Duser.timezone=my_mysql"
This way, you don't have to use the server level environment property SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_TASK_SERVICES
Let's say I have started to use the spring-cloud-config-server and get it working (using a git repository in background). So now I will deploy that config-server on a cluster (mesos cluster or AWS cloud etc.)
So for reliability etc. I would like to start two instances of the same config-service within a cluster. By using a service registry all other services can now connect to that config-server and get their configuration.
But the question: How can the synchronisation between those config-servers being handled... So for example If I change the configuration in the git repository and now there is some lack of time between both instance will deliver the exact same information...
Does there exist a solution for that ? Some kind of a raft census protocol / setup etc. ? Or is there the only solution not to use spring-config-server and use etcd instead or other solutions ?
Update:
It might be an option to make a fore-update option for the git repositories. This makes sure to get the most recent state with the drawback of performance.
In Spring Cloud Services v3.1.2 and later, you can use the periodic parameter when configuring a Config Server service instance to cause the mirror service to automatically refresh a Git repository mirror periodically
pcf link
I am new to SCDF and am trying to get started with a RabbitMQ transport layer and SCDF version 1.2.2. I have setup RabbitMQ in a separate VM and have the SCDF local server and SCDF shell jar in one VM. Can someone suggest how I can specify the server details of my RabbitMQ (which is in a different host in the same network) for SCDF to use as a transport.
For reasons outside my control I need to use the MQ setup in a different machine. Please advise.
SCDF doesn't require RabbitMQ and I think you are trying to use RabbitMQ as the binder for your Spring Cloud Stream applications that are orchestrated via SCDF.
You would need to configure the properties mentioned here
You can find more information here on how to specify these properties at SCDF.
I have configured my application with config server and github supported external config files. It works fine when I am having single instance of my application in cloud foundry.
But for multiple instance it is said to implement spring cloud bus to apply external config changes to all the instances. For this I have bind my config server with rabbit MQ instance available on Pivotal Cloud foundry. have added spring.cloud.starter.bus.amqp jar in my build.gradle file.
Problem: But when I am hitting POST request to client app at:
http://server:port/bus/refresh the call goes to controller rather than refreshing all the instances and failing as no mapping for same.
Please let me know if I am missing any configuration to make spring-cloud-bus work.
Thanks in advance!
application.properties(Client application):
spring.profiles=cloud
spring.rabbitmq.host= 10.17.128.102
spring.rabbitmq.port= 5672
spring.rabbitmq.virtual-host= *****
spring.rabbitmq.username= ******
spring.rabbitmq.password= *****
rabbit.exchangeName= demoPartyServiceexc
rabbit.routingKey= demoPartyService
rabbit.queueName= demoPartyServicequeue
logging.level.ROOT= ERROR
bootstrap.properties(Client application):
spring.application.name=demo-api
spring.cloud.bus.enabled=true
spring.cloud.config.bus.enabled=true
spring.cloud.bus.amqp.enabled=true
spring.cloud.bus.refresh.enabled=true
spring.cloud.bus.env.enabled=true
spring.cloud.config.uri=https://config-a5e99419-8179-47f7-8c23-62ed5b38db0b.cf.com
spring.cloud.config.server.bootstrap=true
spring.cloud.config.server.git.uri= My Github repository URI
spring.cloud.config.server.git.username= ********
spring.cloud.config.server.git.password= ********
application.properties file in GIT repo:
logging.level.ROOT=WARN
What are server.servlet-path and management.context-path in client app set to?
I think you might need to send the request to http://host/<management.context-path>/bus/refresh.
Or better yet, to http://config-server-host/monitor so that it publishes a message to a topic in RabbitMQ and all client apps get notified.
Configuration snippets, source code and more details could be found at my blog post: Refreshable Configuration using Spring Cloud Config Server, Spring Cloud Bus, RabbitMQ and Git