I am building a spring boot application where i need to read json files for my component tests. I have a utility method which takes the name of the file and reads the content using ResourceUtils. Here is the code:
public static String getContent(String path) throws IOException {
File file = ResourceUtils.getFile(MyTest.class.getResource(path));
String content = new String(Files.readAllBytes(file.toPath()));
return content;
}
The checkmarx is reporting the above code as "This may cause a Path
Traversal vulnerability."
How to fix this?
Thanks
See this example for path traversal vulnerability Path Traversal
To fix this change it something like
private static final String BASE_PATH ="/yourbasepath/somewherewherefileisstored";
public static String getContent(String path) throws IOException {
File file = new File(BASE_PATH, path);
if (file.getCanonicalPath().startsWith(BASE_PATH)){
String content = new String(Files.readAllBytes(file.toPath()));
return content;
}
else{
//throw some error
}
}
Related
I just deployed my first microservice. My microservice is working fine. All routes are working. But the service class inside the microservice is not working properly. The service class is not reading data from the CSV file.
Below is the code I am using to read data from CSV file.
public class ReadCsvUtil {
private static final Logger LOGGER = LoggerFactory.getLogger(ReadCsvUtil.class);
public List<String[]> readData() throws IOException {
String file = ".\\src\\main\\resources\\pensioners.csv";
List<String[]> content = new ArrayList<>();
try(BufferedReader br = new BufferedReader(new FileReader(file))) {
String line = "";
while ((line = br.readLine()) != null) {
content.add(line.split(","));
}
} catch (FileNotFoundException e) {
LOGGER.debug(e.getMessage());
}
return content;
}
}
The service class invokes the above function to get details of all the people.
The above code is working fine on my desktop and I am able to get details but code is not working on AWS. Also, I tried to remove the CSV and manually enter the values and it's working in AWS. So I am 99% sure there is some problem in reading the CSV files.
Is there anyway I can fix this?
If the path to the file one directory up, instead of
String file = ".\\src\\main\\resources\\pensioners.csv";
try,
String file = "..\src\main\resources\pensioners.csv";
I presume the AWS sever is ubuntu and your local is windows OS.
I have a folder structure /data/reports on a file system, which contains all reports.
How can I configure a SpringBoot application to serve the contents of this file sytem.
Currently I have tried few options, but none working
#Configuration
#EnableWebMvc
public class AppConfig implements WebMvcConfigurer {
#Value(value = "${spring.resources.static-locations:#{null}}")
private String fileSystem;
#Override
public void addResourceHandlers(ResourceHandlerRegistry registry) {
registry
.addResourceHandler("/data/reports/**")
.addResourceLocations(fileSystem)
.setCachePeriod(3600)
.resourceChain(true)
.addResolver(new PathResourceResolver());
}
}
and in application.properties I have defined
spring.resources.static-locations=file:///data/reports
server.servlet.jsp.init-parameters.listings=true
But in both cases, when I try
http://host:port/application/data/reports
I'm getting 404
What am I missing ?
Based on the suggestions given, I realized that one mistake I'm doing is to access the reports via
http://host:port/application/data/reports
instead of
http://host:port/data/reports
if I use application in the request, those calls will go through RequestDispatcher and will try to find for a matching RequestMapping, which does not exist. I think I'm convinced so far.
But the problem I'm seeing now is, I'm getting SocketTimeoutException while trying to read from the resource listed in the URL. I had put some breakpoints in Spring source "ResourceHttpMessageConverter.java"
protected void writeContent(Resource resource, HttpOutputMessage outputMessage)
throws IOException, HttpMessageNotWritableException {
try {
InputStream in = resource.getInputStream(); //It is timing out here
try {
StreamUtils.copy(in, outputMessage.getBody());
}
catch (NullPointerException ex) {
// ignore, see SPR-13620
}
The resource is a small text file with 1 line "Hello World". Yet it is timing out.
The resource in the above class is a FileUrlResource opened on file:///c:/data/reports/sample.txt
On the other hand, I tried to read that resource as
File file = new File("c:/data/reports/sample.txt");
System.out.println(file.exists());
URL url = file.toURI().toURL();
URLConnection con = url.openConnection();
InputStream is = con.getInputStream(); //This works
Thanks
I use Apache Solr so that I can work with files, I can add regular text fields via Spring, but I don’t know how to add TXT / pdf
#SolrDocument(solrCoreName = "accounting")
public class Accounting {
#Id
#Field
private String id;
#Field
private File txtFile;
#Field
private String docType;
#Field
private String docTitle;
public Accounting() {
}
public Accounting(String id, String docType, String docTitle) {
this.id = id;
this.docTitle = docTitle;
this.docType = docType;
}
here is the problem with the txtFile field
<field name="docTitle" type="strings"/>
<field name="docType" type="strings"/>
These fields that I manually added to schema.xml, I can not figure out how to add a field here that will be responsible for the file, for example, I will add here a txt file, how to do it? Thank you very much. And do I correctly declare the field private File txtFile; in the entity for the file?
Solr will not store the actual file anywhere. Depending on your config it can store the binary content though. Using the extract request handler Apache Solr which relies on Apache Tika to extract the content from the document.
You can try something like below code. The current code is not using anything from the springboot. Here the content is read from the pdf document and then the data is indexed into solr along with id and filename. I have used the tika apis to extract the content of the pdf.
public static void main(final String[] args) throws IOException, TikaException, SAXException {
String urlString = "http://localhost:8983/solr/TestCore1";
SolrClient solr = new HttpSolrClient.Builder(urlString).build();
BodyContentHandler handler = new BodyContentHandler();
Metadata metadata = new Metadata();
File file = new File("C://Users//abhijitb//Desktop//TestDocument.pdf");
FileInputStream inputstream = new FileInputStream(file);
ParseContext pcontext = new ParseContext();
// parsing the document using PDF parser
PDFParser pdfparser = new PDFParser();
pdfparser.parse(inputstream, handler, metadata, pcontext);
// getting the content of the document
//System.out.println("Contents of the PDF :" + handler.toString());
try {
String fileName = file.getName();
SolrInputDocument document = new SolrInputDocument();
document.addField("id", "123456");
document.addField("title", fileName);
document.addField("text", handler.toString());
solr.add(document);
solr.commit();
} catch (SolrServerException | IOException e) {
e.printStackTrace();
}
}
Once you index the data, it can be verified on the solr admin page by querying for it.
Please find the image for your reference.
I would like to generate a blog posts overview. For that I want to read the html files from a folder inside the templates folder in the resources folder where Spring Boot stores its templates.
I tried that but it doesnt return an error but also list no files.
What is the way to go here?
Thanks
#Controller
public class Route {
#Autowired
private ResourceLoader resourceLoader;
#RequestMapping("/")
public String home() throws IOException {
final String path = "templates/blog";
final Resource res = resourceLoader.getResource("templates/blog");
try (final BufferedReader reader = new BufferedReader(new InputStreamReader(res.getInputStream()))) {
reader.lines().forEachOrdered(System.out::println);
}
return "blog/a";
}
}
#Controller
public class Route {
#Value("classpath:templates/blog/*")
private Resource[] resources;
#RequestMapping("/")
public String home() throws IOException {
for (final Resource res : resources) {
System.out.println(res.getFilename());
}
return "blog/a";
}
}
did the trick to me.
You should be able to achieve this using NIO2.
In order for NIO2 to work, it requires the concept of FileSystem, and one can be created from the jar URI. Then this file system can be used with Files/Paths.
The code below contains two branches - the first handles loading the files from inside Jar, the second branch - when the code runs from IDE or via "mvn spring-boot:run".
All streams are being used via try-with-resources so they will be auto-closed.
The find function starts from the top of the file system and recursively searches for html files.
public static void readFile(String location) throws URISyntaxException {
URI uri = Objects.requireNonNull(ReadFromJar.class.getClassLoader().getResource(location)).toURI();
if (uri.getScheme().equals("jar")) { //inside jar
try (FileSystem fs = FileSystems.newFileSystem(uri, Collections.emptyMap())) { //build a new FS that represents the jar's contents
Files.find(fs.getPath("/"), 10, (path, fileAttr) -> // control the search depth (e.g. 10)
fileAttr.isRegularFile() //match only files
&& path.toString().contains("blog") //match only files in paths containing "blog"
&& path.getFileName().toString().matches(".*\\.html")) // match only html files
.forEach(ReadFromJar::printFileContent);
} catch (IOException ex) {
ex.printStackTrace();
}
}
else { //from IDE or spring-boot:run
final Path path = Paths.get(uri);
try (DirectoryStream<Path> dirStream = Files.newDirectoryStream(path)) {
dirStream.forEach(ReadFromJar::printFileContent);
} catch (IOException e) {
e.printStackTrace();
}
}
}
private static void printFileContent(final Path file) {
try {
System.out.println("Full path: " + file.toAbsolutePath().toString());
Files.lines(file).forEach(System.out::println);
} catch (IOException e) {
e.printStackTrace();
}
}
We are running Java-Cucumber-Maven automated test scripts from TeamCity. Up to now this has worked very well but our current test requires that several .CSV files are downloaded for analysis by the script. There's a
ReadCsvFiles.java
file in our code and that dictates where the downloaded files are located....easy for running tests locally but we're having real problems with the path to the build agent in TeamCity..
public class ReadCsvFiles {
//public static String sFileLocation = "C:\\Users\\ankit.bisht\\Downloads\\";
public static String sFileLocation = "C:\\BuildAgent\\temp\\buildTmp\\";
//public static String sFileLocation = "//users/redmayned//Downloads/";
public static void depositsHeldReports() throws Throwable {
String csvFile = sFileLocation + "DepositsHeld.csv";
CSVReader reader = null;
try {
reader = new CSVReader(new FileReader(csvFile));
String[] column;
boolean doesExist = false;
while ((column = reader.readNext()) != null) {
The commented out entries are for local windows and local OSX. The middile one is just one of many options we've tried...all resulting in an error message;
java.io.FileNotFoundException: C:\BuildAgent\temp\buildTmp\DepositsHeld.csv (The system cannot find the file specified)
Can anyone save my sanity please?
Thanks
David