Fat Jar throwing File not found exception when trying to access text file within the jar - maven

I have built a Spring boot MVC application with a Tree data structure in place of an actual database. The program reads from a text file and stores words in the tree. originally I used a the CommandLineRunner class to populate the tree, which works... but after creating a fat jar and running the jar, I get a file not found exception. how can I build a fat jar with maven that includes the text file with maven?
the file is currently in the project root.
here is the logic to generate the tree:
#Component
#Order(value = Ordered.HIGHEST_PRECEDENCE)
public class GenerateTree implements CommandLineRunner {
#Autowired
TreeRepository trie = new TreeRepository();
#Autowired
FileReader fileReader = new FileReader();
#Override
public void run(String... args) throws Exception {
for (String s : fileReader.readFile("wordList1.txt")){
trie.add(s);
}
}
}
here is the logic that reads in the file:
#Component
public class FileReader {
List<String> readFile(String filename){
List<String> list = new ArrayList<>();
try (Stream<String> stream = Files.lines(Paths.get(filename))) {
list = stream
.filter(line -> line.matches("[a-zA-Z]+"))
.collect(Collectors.toList());
} catch (IOException e) {
e.printStackTrace();
}
return list;
}
}

You cannot access a File inside a jar (see https://stackoverflow.com/a/8258308/4516887).
Put the wordlist.txt into the src/main/resources directory and read its contents using a [ClassPathResource][1]:
ClassPathResource resource = new ClassPathResource("worldlist.txt");
try (InputStream in = resource.getInputStream();
BufferedReader reader = new BufferedReader(new InputStreamReader(in))) {
String line;
while((line = reader.readLine()) != null) {
...
}
}

Related

List Files from Templates Directory in Spring Boot

I would like to generate a blog posts overview. For that I want to read the html files from a folder inside the templates folder in the resources folder where Spring Boot stores its templates.
I tried that but it doesnt return an error but also list no files.
What is the way to go here?
Thanks
#Controller
public class Route {
#Autowired
private ResourceLoader resourceLoader;
#RequestMapping("/")
public String home() throws IOException {
final String path = "templates/blog";
final Resource res = resourceLoader.getResource("templates/blog");
try (final BufferedReader reader = new BufferedReader(new InputStreamReader(res.getInputStream()))) {
reader.lines().forEachOrdered(System.out::println);
}
return "blog/a";
}
}
#Controller
public class Route {
#Value("classpath:templates/blog/*")
private Resource[] resources;
#RequestMapping("/")
public String home() throws IOException {
for (final Resource res : resources) {
System.out.println(res.getFilename());
}
return "blog/a";
}
}
did the trick to me.
You should be able to achieve this using NIO2.
In order for NIO2 to work, it requires the concept of FileSystem, and one can be created from the jar URI. Then this file system can be used with Files/Paths.
The code below contains two branches - the first handles loading the files from inside Jar, the second branch - when the code runs from IDE or via "mvn spring-boot:run".
All streams are being used via try-with-resources so they will be auto-closed.
The find function starts from the top of the file system and recursively searches for html files.
public static void readFile(String location) throws URISyntaxException {
URI uri = Objects.requireNonNull(ReadFromJar.class.getClassLoader().getResource(location)).toURI();
if (uri.getScheme().equals("jar")) { //inside jar
try (FileSystem fs = FileSystems.newFileSystem(uri, Collections.emptyMap())) { //build a new FS that represents the jar's contents
Files.find(fs.getPath("/"), 10, (path, fileAttr) -> // control the search depth (e.g. 10)
fileAttr.isRegularFile() //match only files
&& path.toString().contains("blog") //match only files in paths containing "blog"
&& path.getFileName().toString().matches(".*\\.html")) // match only html files
.forEach(ReadFromJar::printFileContent);
} catch (IOException ex) {
ex.printStackTrace();
}
}
else { //from IDE or spring-boot:run
final Path path = Paths.get(uri);
try (DirectoryStream<Path> dirStream = Files.newDirectoryStream(path)) {
dirStream.forEach(ReadFromJar::printFileContent);
} catch (IOException e) {
e.printStackTrace();
}
}
}
private static void printFileContent(final Path file) {
try {
System.out.println("Full path: " + file.toAbsolutePath().toString());
Files.lines(file).forEach(System.out::println);
} catch (IOException e) {
e.printStackTrace();
}
}

How can I put the port and host in the property file in Spring?

I have this url
private static final String PRODUCTS_URL = "http://localhost:3007/catalog/products/";
And this methods:
public JSONObject getProductByIdFromMicroservice(String id) throws IOException, JSONException {
return getProductsFromProductMicroservice(PRODUCTS_URL + id);
}
public JSONObject getProductsFromProductMicroservice(String url) throws IOException, JSONException {
CloseableHttpClient productClient = HttpClients.createDefault();
HttpGet getProducts = new HttpGet(url);
CloseableHttpResponse microserviceResponse = productClient.execute(getProducts);
HttpEntity entity = microserviceResponse.getEntity();
BufferedReader br = new BufferedReader(new InputStreamReader((entity.getContent())));
StringBuilder sb = new StringBuilder();
String line;
while ((line = br.readLine()) != null) {
sb.append(line);
}
br.close();
System.out.println(sb.toString());
JSONObject obj = new JSONObject(sb.toString());
System.out.println(obj);
return obj;
}
I want to put the port and host in a separate property file. I have already seen examples using properties and the yml file. But I do not understand how then my methods will work using this port when creating an instance of the class, which I will indicate in the properties file. Can you tell?
You can put your properties in a properties file in the resource directory for example
PRODUCTS_URL="http://localhost:3007/catalog/products/"
and add #PropertySource("YOUR_RESOURCE_FILE_HERE.properties") in your main class (Application.java)
#SpringBootApplication
#PropertySource("products.properties")
public class Application {...}
and then use #Value("${YOUR_PROPERTY_NAME}") to load it:
#Value("${PRODUCTS_URL}")
private String PRODUCTS_URL;
Check this tutorial
This is how i do it :
CONFIG FILE
#Database Server Properties
dbUrl=jdbc:sqlserver://localhost:1433;database=Something;
dbUser=sa
dbPassword=SomePassword
Then i annotate a config class with this :
#PropertySource("file:${ENV_VARIABLE_TO_PATH}/config.properties")
Then autowire this field :
#Autowired
private Environment environment;
Then create the data source :
#Bean
public DataSource dataSource()
{
HikariDataSource dataSource = new HikariDataSource();
try
{
dataSource.setDriverClassName("com.microsoft.sqlserver.jdbc.SQLServerDriver");
dataSource.setConnectionTestQuery("SELECT 1");
dataSource.setMaximumPoolSize(100);
String dbUrl = environment.getProperty("dbUrl");
if (dbUrl != null)
{
dataSource.setJdbcUrl(dbUrl);
}
else
{
throw new PropertyNotFoundException("The dbUrl property is missing from the config file!");
}
String dbUser = environment.getProperty("dbUser");
if (dbUser != null)
{
dataSource.setUsername(dbUser);
}
else
{
throw new PropertyNotFoundException("The dbUser property is missing from the config file!");
}
String dbPassword = environment.getProperty("dbPassword");
if (dbPassword != null)
{
dataSource.setPassword(dbPassword);
}
else
{
throw new PropertyNotFoundException("The dbPassword property is missing from the config file!");
}
logger.debug("Successfully initialized datasource");
}
catch (PropertyNotFoundException ex)
{
logger.fatal("Error initializing datasource : " + ex.getMessage());
}
return dataSource;
}
I know this is not exactly your scenario but perhaps you can find inspiration from this code to suit your specific needs?
Other answers here mention using #PropertySource annotation to specify path of config files. Also if this is a test code (unit/integration) you can also make use of another annotation #TestPropertySource.
With this, we can define configuration sources that have higher precedence than any other source used in the project.
See here: https://www.baeldung.com/spring-test-property-source

Spring Batch: FlatFileItemWriter to InputSteam/Byte array

So I've created a batch job which generates reports (csv files). I have been able to generate the the files seamlessly using FlatFileItemWriter but my end goal is to create an InputStream to call a rest service which will store the document or a byte array to store it in the database.
public class CustomWriter implements ItemWriter<Report> {
#Override
public void write(List<? extends Report> reports) throws Exception {
reports.forEach(this::writeDataToFile);
}
private void writeDataToFile(final Report data) throws Exception {
FlatFileItemWriter writer = new FlatFileItemWriter();
writer.setResource(new FileSystemResource("C:/reports/test-report.csv"));
writer.setLineAggregator(getLineAggregator();
writer.afterPropertiesSet();
writer.open(new ExecutionContext());
writer.write(data);
writer.close();
}
private DelimitedLineAggregator<Report> getLineAggregator(final Report report) {
DelimitedLineAggregator<Report> delimitedLineAgg = new DelimitedLineAggregator<Report>();
delimitedLineAgg.setDelimiter(",");
delimitedLineAgg.setFieldExtractor(getFieldExtractor());
return delimitedLineAgg;
}
private FieldExtractor<Report> getFieldExtractor() {
BeanWrapperFieldExtractor<Report> fieldExtractor = new BeanWrapperFieldExtractor<Report>();
fieldExtractor.setNames(COLUMN_HEADERS.toArray(new String[0]));
return fieldExtractor;
}
}
One way I could do this is to store the file locally temporarily and create a new step to pick the generated files up and do the sending/storing but I would really like to skip this step and send/store it in the first step.
How do I go about doing this?

Reading property file in Spring MVC app

I'm trying to read a property file using below code, basically I'm having a Spring Boot app and I'm trying to read the below non spring bean class.The property file is in src/main/resource directory.
public class VisaProperties {
static Properties properties;
static {
try {
properties = new Properties();
String propertiesFile = System.getProperty("ftproperties");
if (propertiesFile == null) {
properties.load(VisaProperties.class.getResourceAsStream("motoconfig.cybersource.properties"));
} else {
properties.load(new FileReader(propertiesFile));
}
} catch (IOException e) {
throw new RuntimeException(e);
}
}
public static String getProperty(Property property) {
return (String) properties.get(property.getValue());
}
}
and trying call the end point property using below code getting null. How can I call the property?
VisaProperties.getProperty(Property.END_POINT)
You can simplify the code as:
final Properties properties = new Properties();
try (final InputStream stream =
this.getClass().getResourceAsStream("config.properties")) {
properties.load(stream);
}
Note: Use "try with resources" so that stream will be automatically
closed when the try {} block exits.
Done, using the code below:
Properties properties = new Properties();
InputStream inputStream = VisaProperties.class
.getClassLoader()
.getResourceAsStream("config.properties");
properties.load(inputStream);
inputStream.close();

How to externalize the queries to xml files using spring

I am using spring and their JDBC template to do read/write operations to the database. I am facing a problem in my reporting module that i have to frequently change the query sqls to cater to frequent changes.
Though using spring jdbc ORM, is there a way to externalize my query parameters such that i just change it in the XML & restart and there is no need to rebuild my source again for deployment. Any approach ORM (preferred) or simple Sql will do.
As of now i have to change the query again and again ,rebuild the source and deploy.
I am not sure if Spring provides some out of the box solutions to implement what you want. But here is one way to get it done, which i had implemented ones. So i will try to reduce some hardwork for you.
You might need to implement a utility to load from resources xml file. Something like this.
public final class LoadFromResourceFileUtils {
public static String loadQuery(final String libraryPath,
final String queryName) {
final InputStream is = StreamUtils
.streamFromClasspathResource(libraryPath);
if (is == null) {
throw new RuntimeException(String.format(
"The SQL Libary %s could not be found.", libraryPath));
}
final Document doc = XMLParseUtils.parse(is);
final Element qryElem = (Element) doc.selectSingleNode(String.format(
"SQLQueries/SQLQuery[#name='%s']", queryName));
final String ret = qryElem == null ? null : qryElem.getText();
return ret;
}
}
You would need to store your queries in an XML say queries.xml and keep it in your classpath, for e.g
<?xml version="1.0" encoding="UTF-8"?>
<SQLQueries>
<SQLQuery name="myQuery">
<![CDATA[
your query
]]>
</SQLQuery>
</SQLQueries>
And in your DAO you can do this to get the query
String query = LoadFromResourceFileUtils.loadQuery(
"queries.xml", "myQuery");
XMLParseUtils and StreamUtils for your reference
public final class XMLParseUtils {
public static Document parse(final InputStream inStream) {
Document ret = null;
try {
if (inStream == null) {
throw new RuntimeException(
"XML Input Stream for parsing is null");
}
final SAXReader saxReader = new SAXReader();
ret = saxReader.read(inStream);
} catch (final DocumentException exc) {
throw new RuntimeException("XML Parsing error", exc);
}
return ret;
}
}
public final class StreamUtils {
public static InputStream streamFromClasspathResource(
final String resourceClassPath) {
final Class<StreamUtils> clazz = StreamUtils.class;
final ClassLoader clLoader = clazz.getClassLoader();
final InputStream inStream = clLoader
.getResourceAsStream(resourceClassPath);
if (inStream == null) {
if(LOGGER.isDebugEnabled()){
LOGGER.debug(String.format("Resource %s NOT FOUND.",
resourceClassPath));
}
}
return inStream;
}
}

Resources