I just deployed my first microservice. My microservice is working fine. All routes are working. But the service class inside the microservice is not working properly. The service class is not reading data from the CSV file.
Below is the code I am using to read data from CSV file.
public class ReadCsvUtil {
private static final Logger LOGGER = LoggerFactory.getLogger(ReadCsvUtil.class);
public List<String[]> readData() throws IOException {
String file = ".\\src\\main\\resources\\pensioners.csv";
List<String[]> content = new ArrayList<>();
try(BufferedReader br = new BufferedReader(new FileReader(file))) {
String line = "";
while ((line = br.readLine()) != null) {
content.add(line.split(","));
}
} catch (FileNotFoundException e) {
LOGGER.debug(e.getMessage());
}
return content;
}
}
The service class invokes the above function to get details of all the people.
The above code is working fine on my desktop and I am able to get details but code is not working on AWS. Also, I tried to remove the CSV and manually enter the values and it's working in AWS. So I am 99% sure there is some problem in reading the CSV files.
Is there anyway I can fix this?
If the path to the file one directory up, instead of
String file = ".\\src\\main\\resources\\pensioners.csv";
try,
String file = "..\src\main\resources\pensioners.csv";
I presume the AWS sever is ubuntu and your local is windows OS.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed last year.
This post was edited and submitted for review 10 months ago and failed to reopen the post:
Original close reason(s) were not resolved
Improve this question
I am looking to send log data from the application to Splunk. I came to know that there is nothing to do with spring, it's just Splunk needs some configurations to read Application's Logs files. I want to know how we can make Splunk read Applications Log files.
Please help me out with Splunk integration with Spring Boot. It will be great if you provided any code snippets or references.
In terms of integration, what are you after? Are you looking to bring data in from Splunk for use in your Sprint Boot application, or are you looking to send data from your application into Splunk?
For logging into Splunk, I suggest you look at the following:
https://github.com/splunk/splunk-library-javalogging
https://docs.spring.io/autorepo/docs/spring-integration-splunk/0.5.x-SNAPSHOT/reference/htmlsingle/
https://github.com/barrycommins/spring-boot-splunk-sleuth-demo
If you are looking to interact with the Splunk application and run queries against it, look at the Splunk Java SDK, https://dev.splunk.com/enterprise/docs/java/sdk-java/howtousesdkjava/
Here are the steps which I have followed to integrate Splunk successfully into my Spring Boot application:
Set up the repository in the pom.xml file by adding the following:
<repositories>
<repository>
<id>splunk-artifactory</id>
<name>Splunk Releases</name>
<url>https://splunk.jfrog.io/splunk/ext-releases-local</url>
</repository>
</repositories>
Add the maven dependency for Splunk jar, within the dependencies tags, which will download and setup the Splunk jar file in the project (In my case the jar file is splunk-1.6.5.0.jar):
<dependency>
<groupId>com.splunk</groupId>
<artifactId>splunk</artifactId>
<version>1.6.5.0</version>
</dependency>
Configure and run the Splunk query from your controller / service / main class:
package com.my.test;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.util.HashMap;
import java.util.Map;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.splunk.Args;
import com.splunk.HttpService;
import com.splunk.Job;
import com.splunk.SSLSecurityProtocol;
import com.splunk.Service;
#SpringBootApplication
public class Main {
public static String username = "your username";
public static String password = "your password";
public static String host = "your splunk host url like - splunk-xx-test.abc.com";
public static int port = 8089;
public static String scheme = "https";
public static Service getSplunkService() {
HttpService.setSslSecurityProtocol(SSLSecurityProtocol.TLSv1_2);
Map<String, Object> connectionArgs = new HashMap<>();
connectionArgs.put("host", host);
connectionArgs.put("port", port);
connectionArgs.put("scheme", scheme);
connectionArgs.put("username", username);
connectionArgs.put("password", password);
Service splunkService = Service.connect(connectionArgs);
return splunkService;
}
/* Take the Splunk query as the argument and return the results as a JSON
string */
public static String getQueryResultsIntoJsonString(String query) throws IOException {
Service splunkService = getSplunkService();
Args queryArgs = new Args();
//set "from" time of query. 1 = from beginning
queryArgs.put("earliest_time", "1");
//set "to" time of query. now = till now
queryArgs.put("latest_time", "now");
Job job = splunkService.getJobs().create(query);
while(!job.isDone()) {
try {
Thread.sleep(500);
} catch(InterruptedException ex) {
ex.printStackTrace();
}
}
Args outputArgs = new Args();
//set format of result set as json
outputArgs.put("output_mode", "json");
//set offset of result set (how many records to skip from the beginning)
//Default is 0
outputArgs.put("offset", 0);
//set no. of records to get in the result set.
//Default is 100
//If you put 0 here then it would be set to "no limit"
//(i.e. get all records, don't truncate anything in the result set)
outputArgs.put("count", 0);
InputStream inputStream = job.getResults(outputArgs);
//Now read the InputStream of the result set line by line
//And return the final result into a JSON string
//I am using Jackson for JSON processing here,
//which is the default in Spring boot
BufferedReader in = new BufferedReader(new InputStreamReader(inputStream));
String resultString = null;
String aLine = null;
while((aLine = in.readLine()) != null) {
//Convert the line from String to JsonNode
ObjectMapper mapper = new ObjectMapper();
JsonNode jsonNode = mapper.readTree(aLine);
//Get the JsonNode with key "results"
JsonNode resultNode = jsonNode.get("results");
//Check if the resultNode is array
if (resultNode.isArray()) {
resultString = resultNode.toString();
}
}
return resultString;
}
/*Now run your Splunk query from the main method (or a RestController or a Service class)*/
public static void main(String[] args) {
try {
getQueryResultsIntoJsonString("search index=..."); //your Splunk query
} catch (IOException e) {
e.printStackTrace();
}
}
}
Inside my JHipster (version 6.4.1) application in resources I have directory called static, where I put JSON file which is required for one of services. File is called standards.json.
In my service I want to read this file in quite simple way:
try {
ClassPathResource cpr = new ClassPathResource("static/standards.json");
byte[] bdata = FileCopyUtils.copyToByteArray(cpr.getInputStream());
String content = new String(bdata, StandardCharsets.UTF_8);
Gson g = new Gson();
data = g.fromJson(content, StandardLevel[].class);
// here I am doing something with data
} catch (IOException e) {
LOGGER.error(e.getMessage());
}
But unfortunately I am getting runtime error:
class path resource [static/standards.json] cannot be opened because it does not exist
It is strange, because when I was doing it in this same way in "clean" Spring Boot application, without JHipster, everything was working correctly.
Any ideas why it is not working here? Or how should I use static JSOn files, which are required for my backend side?
All
I created a jar file with the following MANIFEST.MF inside:
Manifest-Version: 1.0
Ant-Version: Apache Ant 1.8.3
Created-By: 1.6.0_25-b06 (Sun Microsystems Inc.)
Main-Class: my.Main
Class-Path: . lib/spring-core-3.2.0.M2.jar lib/spring-beans-3.2.0.M2.jar
In its root there is a file called my.config which is referenced in my spring-context.xml like this:
<bean id="..." class="...">
<property name="resource" value="classpath:my.config" />
</bean>
If I run the jar, everything looks fine escept the loading of that specific file:
Caused by: java.io.FileNotFoundException: class path resource [my.config] cannot be resolved to absolute file path because it does not reside in the file system: jar:file:/D:/work/my.jar!/my.config
at org.springframework.util.ResourceUtils.getFile(ResourceUtils.java:205)
at org.springframework.core.io.AbstractFileResolvingResource.getFile(AbstractFileResolvingResource.java:52)
at eu.stepman.server.configuration.BeanConfigurationFactoryBean.getObject(BeanConfigurationFactoryBean.java:32)
at eu.stepman.server.configuration.BeanConfigurationFactoryBean.getObject(BeanConfigurationFactoryBean.java:1)
at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:142)
... 22 more
classes are loaded the from inside the jar
spring and other dependencies are loaded from separated jars
spring context is loaded (new ClassPathXmlApplicationContext("spring-context/applicationContext.xml"))
my.properties is loaded into PropertyPlaceholderConfigurer ("classpath:my.properties")
if I put my .config file outside the file system, and change the resource url to 'file:', everything seems to be fine...
Any tips?
If your spring-context.xml and my.config files are in different jars then you will need to use classpath*:my.config?
More info here
Also, make sure you are using resource.getInputStream() not resource.getFile() when loading from inside a jar file.
In the spring jar package, I use new ClassPathResource(filename).getFile(), which throws the exception:
cannot be resolved to absolute file path because it does not reside in the file system: jar
But using new ClassPathResource(filename).getInputStream() will solve this problem. The reason is that the configuration file in the jar does not exist in the operating system's file tree,so must use getInputStream().
I know this question has already been answered. However, for those using spring boot, this link helped me - https://smarterco.de/java-load-file-classpath-spring-boot/
However, the resourceLoader.getResource("classpath:file.txt").getFile(); was causing this problem and sbk's comment:
That's it. A java.io.File represents a file on the file system, in a
directory structure. The Jar is a java.io.File. But anything within
that file is beyond the reach of java.io.File. As far as java is
concerned, until it is uncompressed, a class in jar file is no
different than a word in a word document.
helped me understand why to use getInputStream() instead. It works for me now!
Thanks!
The error message is correct (if not very helpful): the file we're trying to load is not a file on the filesystem, but a chunk of bytes in a ZIP inside a ZIP.
Through experimentation (Java 11, Spring Boot 2.3.x), I found this to work without changing any config or even a wildcard:
var resource = ResourceUtils.getURL("classpath:some/resource/in/a/dependency");
new BufferedReader(
new InputStreamReader(resource.openStream())
).lines().forEach(System.out::println);
I had similar problem when using Tomcat6.x and none of the advices I found was helping.
At the end I deleted work folder (of Tomcat) and the problem gone.
I know it is illogical but for documentation purpose...
I was having an issue recursively loading resources in my Spring app, and found that the issue was I should be using resource.getInputStream. Here's an example showing how to recursively read in all files in config/myfiles that are json files.
Example.java
private String myFilesResourceUrl = "config/myfiles/**/";
private String myFilesResourceExtension = "json";
ResourceLoader rl = new ResourceLoader();
// Recursively get resources that match.
// Big note: If you decide to iterate over these,
// use resource.GetResourceAsStream to load the contents
// or use the `readFileResource` of the ResourceLoader class.
Resource[] resources = rl.getResourcesInResourceFolder(myFilesResourceUrl, myFilesResourceExtension);
// Recursively get resource and their contents that match.
// This loads all the files into memory, so maybe use the same approach
// as this method, if need be.
Map<Resource,String> contents = rl.getResourceContentsInResourceFolder(myFilesResourceUrl, myFilesResourceExtension);
ResourceLoader.java
import java.io.IOException;
import java.io.InputStream;
import java.nio.charset.Charset;
import java.util.HashMap;
import java.util.Map;
import org.springframework.core.io.Resource;
import org.springframework.core.io.support.PathMatchingResourcePatternResolver;
import org.springframework.core.io.support.ResourcePatternResolver;
import org.springframework.util.StreamUtils;
public class ResourceLoader {
public Resource[] getResourcesInResourceFolder(String folder, String extension) {
ResourcePatternResolver resolver = new PathMatchingResourcePatternResolver();
try {
String resourceUrl = folder + "/*." + extension;
Resource[] resources = resolver.getResources(resourceUrl);
return resources;
} catch (IOException e) {
throw new RuntimeException(e);
}
}
public String readResource(Resource resource) throws IOException {
try (InputStream stream = resource.getInputStream()) {
return StreamUtils.copyToString(stream, Charset.defaultCharset());
}
}
public Map<Resource, String> getResourceContentsInResourceFolder(
String folder, String extension) {
Resource[] resources = getResourcesInResourceFolder(folder, extension);
HashMap<Resource, String> result = new HashMap<>();
for (var resource : resources) {
try {
String contents = readResource(resource);
result.put(resource, contents);
} catch (IOException e) {
throw new RuntimeException("Could not load resource=" + resource + ", e=" + e);
}
}
return result;
}
}
For kotlin users, I solved it like this:
val url = ResourceUtils.getURL("classpath:$fileName")
val response = url.openStream().bufferedReader().readText()
The answer by #sbk is the way we should do it in spring-boot environment (apart from #Value("${classpath*:})), in my opinion. But in my scenario it was not working if the execute from standalone jar..may be I did something wrong.
But this can be another way of doing this,
InputStream is = this.getClass().getClassLoader().getResourceAsStream(<relative path of the resource from resource directory>);
I was having an issue more complex because I have more than one file with same name, one is in the main Spring Boot jar and others are in jars inside main fat jar.
My solution was getting all the resources with same name and after that get the one I needed filtering by package name.
To get all the files:
ResourceLoader resourceLoader = new FileSystemResourceLoader();
final Enumeration<URL> systemResources = resourceLoader.getClassLoader().getResources(fileNameWithoutExt + FILE_EXT);
In Spring boot 1.5.22.RELEASE Jar packaging this worked for me:
InputStream resource = new ClassPathResource("example.pdf").getInputStream();
"example.pdf" is in src/main/resources.
And then to read it as byte[]
FileCopyUtils.copyToByteArray(resource);
I had the same issue, ended up using the much more convenient Guava Resources:
Resources.getResource("my.file")
While this is a very old thread, but I also faced the same issue while adding FCM in a Spring Boot Application.
In development, the file was getting opened and no errors but when I deployed the application to AWS Elastic beanstalk , the error of FileNotFoundException was getting thrown and FCM was not working.
So here's my solution to get it working on both development env and jar deployment production.
I have a Component class FCMService which has a method as follows:
#PostConstruct
public void initialize() {
log.info("Starting FCM Service");
InputStream inputStream;
try {
ClassPathResource resource = new ClassPathResource("classpath:fcm/my_project_firebase_config.json");
URL url = null;
try {
url = resource.getURL();
} catch (IOException e) {
}
if (url != null) {
inputStream = url.openStream();
} else {
File file = ResourceUtils.getFile("classpath:fcm/my_project_firebase_config.json");
inputStream = new FileInputStream(file);
}
FirebaseOptions options = FirebaseOptions.builder().setCredentials(GoogleCredentials.fromStream(inputStream))
.build();
FirebaseApp.initializeApp(options);
log.info("FCM Service started");
} catch (IOException e) {
log.error("Error starting FCM Service");
e.printStackTrace();
}
}
Hope this helps someone looking for a quick fix with implementing FCM.
Can be handled like:
var serviceAccount = ClassLoader.getSystemResourceAsStream(FB_CONFIG_FILE_NAME);
FirebaseOptions options = new FirebaseOptions.Builder()
.setCredentials(GoogleCredentials.fromStream(serviceAccount))
.build();
Where FB_CONFIG_FILE_NAME is name of file in your 'resources' folder.
I'm writing a jar intended to be used with Spring and Ehcache. Spring requires that there be a cache defined for each element, so I was planning to have an Ehcache defined for the jar, preferably as a resource in the jar that could be imported into the primary Ehcache configuration for the app. However, my reading of the example Ehcache config file and my Google searches have not turned up any way to import a sub Ehcache config file.
Is there a way to import a sub Ehcache config file, or is there some other way to solve this problem?
What I did to do something similar (replace some placeholders in my Ehcache xml file - a import statement is more or less a placeholder if you will) is to extend (more or less copy to be honest) from Springs EhCacheManagerFactoryBean and create the final Ehcache xml config file on the fly.
For creating the CacheManager instance in afterPropertiesSet() you just hand over a InputStream which points to your config.
#Override
public void afterPropertiesSet() throws IOException, CacheException {
if (this.configLocation != null) {
InputStreamSource finalConfig = new YourResourceWrapper(this.configLocation); // put your custom logic here
InputStream is = finalConfig.getInputStream();
try {
this.cacheManager = (this.shared ? CacheManager.create(is) : new CacheManager(is));
} finally {
IOUtils.closeQuietly(is);
}
} else {
// ...
}
// ...
}
For my filtering stuff I internally used a ByteArrayResource to keep the final config.
data = IOUtils.toString(source.getInputStream()); // get the original config (configLocation) as string
// do your string manipulation here
Resource finalConfigResource = new ByteArrayResource(data.getBytes());
For "real" templating one could also think of using a real template engine like FreeMarker (which Spring has support for) to do more fancy stuff.