I am adding a POST endpoint to a Spring Boot REST service to allow it to accept file uploads from web apps and other sources:
#PostMapping("/fileUpload")
public ResponseEntity uploadFile(#RequestParam("file") MultipartFile file) {
// ...
}
I'd like to run a security check against this file and see if there's anything malicious inside of it. Does Spring have any tools or libs to help with such an effort?
Spring Security does not provide any type of scanning for malicious files. You will need to use an anti virus tool that provides a Java API that you can use in your application. Off the top of my head I know Symantec offers a Java API, have a look here.
https://www.symantec.com/connect/articles/how-use-symantec-scan-engine-52-content-scanning-technologies-direct-integration-your-appli
Add the ClamAV Java library (ClamAV4J) as a dependency in your project's build file.
Create a service class that uses the ClamAV4J library to scan files for viruses.
#Service
public class VirusScanService {
private final ClamAVClient client;
public VirusScanService(ClamAVClient client) {
this.client = client;
}
public boolean isFileInfected(MultipartFile file) throws IOException {
return client.scan(file.getInputStream()).isInfected();
}
}
#RestController
public class FileUploadController {
private final VirusScanService virusScanService;
public FileUploadController(VirusScanService virusScanService) {
this.virusScanService = virusScanService;
}
#PostMapping("/upload")
public String handleFileUpload(#RequestParam("file") MultipartFile file) throws IOException {
if (virusScanService.isFileInfected(file)) {
return "The file is infected!";
} else {
// save the file and return a success message
return "File uploaded successfully!";
}
}
}
Related
I have a folder structure /data/reports on a file system, which contains all reports.
How can I configure a SpringBoot application to serve the contents of this file sytem.
Currently I have tried few options, but none working
#Configuration
#EnableWebMvc
public class AppConfig implements WebMvcConfigurer {
#Value(value = "${spring.resources.static-locations:#{null}}")
private String fileSystem;
#Override
public void addResourceHandlers(ResourceHandlerRegistry registry) {
registry
.addResourceHandler("/data/reports/**")
.addResourceLocations(fileSystem)
.setCachePeriod(3600)
.resourceChain(true)
.addResolver(new PathResourceResolver());
}
}
and in application.properties I have defined
spring.resources.static-locations=file:///data/reports
server.servlet.jsp.init-parameters.listings=true
But in both cases, when I try
http://host:port/application/data/reports
I'm getting 404
What am I missing ?
Based on the suggestions given, I realized that one mistake I'm doing is to access the reports via
http://host:port/application/data/reports
instead of
http://host:port/data/reports
if I use application in the request, those calls will go through RequestDispatcher and will try to find for a matching RequestMapping, which does not exist. I think I'm convinced so far.
But the problem I'm seeing now is, I'm getting SocketTimeoutException while trying to read from the resource listed in the URL. I had put some breakpoints in Spring source "ResourceHttpMessageConverter.java"
protected void writeContent(Resource resource, HttpOutputMessage outputMessage)
throws IOException, HttpMessageNotWritableException {
try {
InputStream in = resource.getInputStream(); //It is timing out here
try {
StreamUtils.copy(in, outputMessage.getBody());
}
catch (NullPointerException ex) {
// ignore, see SPR-13620
}
The resource is a small text file with 1 line "Hello World". Yet it is timing out.
The resource in the above class is a FileUrlResource opened on file:///c:/data/reports/sample.txt
On the other hand, I tried to read that resource as
File file = new File("c:/data/reports/sample.txt");
System.out.println(file.exists());
URL url = file.toURI().toURL();
URLConnection con = url.openConnection();
InputStream is = con.getInputStream(); //This works
Thanks
I want to upload a multipart file to AWS S3. So, i have to convert it.
But new File method needs a local location to get the file.
I am able to do in local. But running this code in every machine seems like a issue.
Please find both scenarios.
Working
private File convertMultiPartToFile(MultipartFile multipartFile) throws IOException {
File convFile = new File("C:\\Users\\" + multipartFile.getOriginalFilename());
multipartFile.transferTo(convFile);
return convFile;
}
Not working
private File convertMultiPartToFile(MultipartFile multipartFile) throws IOException {
File convFile = new File(multipartFile.getOriginalFilename());
multipartFile.transferTo(convFile);
return convFile;
}
Error received :
java.io.FileNotFoundException: newbusiness.jpg (Access is denied)
at java.io.FileOutputStream.open0(Native Method)
at java.io.FileOutputStream.open(FileOutputStream.java:270)
at java.io.FileOutputStream.<init>(FileOutputStream.java:213)
at java.io.FileOutputStream.<init>(FileOutputStream.java:162)
You could use Spring Content S3. This will hide the implementation details so you don't need to worry about them.
There are Spring Boot starter alternatives but as you are not using Spring Boot add the following dependency to your pom.xml
pom.xml
<dependency>
<groupId>com.github.paulcwarren</groupId>
<artifactId>spring-content-s3</artifactId>
<version>0.0.11</version>
</dependency>
Add the following configuration that creates a SimpleStorageResourceLoader bean:
#Configuration
#EnableS3Stores
public class S3Config {
#Autowired
private Environment env;
public Region region() {
return Region.getRegion(Regions.fromName(env.getProperty("AWS_REGION")));
}
#Bean
public BasicAWSCredentials basicAWSCredentials() {
return new BasicAWSCredentials(env.getProperty("AWS_ACCESS_KEY_ID"), env.getProperty("AWS_SECRET_KEY"));
}
#Bean
public AmazonS3 client(AWSCredentials awsCredentials) {
AmazonS3Client amazonS3Client = new AmazonS3Client(awsCredentials);
amazonS3Client.setRegion(region());
return amazonS3Client;
}
#Bean
public SimpleStorageResourceLoader simpleStorageResourceLoader(AmazonS3 client) {
return new SimpleStorageResourceLoader(client);
}
}
Create a "Store":
S3Store.java
public interface S3Store extends Store<String> {
}
Autowire this store into where you need to upload resources:
#Autowired
private S3Store store;
WritableResource r = (WritableResource)store.getResource(getId());
InputStream is = // plug your input stream in here
OutputStream os = r.getOutputStream();
IOUtils.copy(is, os);
is.close();
os.close();
When your application starts it will see the dependency on spring-content-s3 and your S3Store interface and inject an implementation for you, therefore, you don't need to worry about implementing this yourself.
IF you writing some sort of web application or microservice and you need a REST API then you can also add this dependency:
<dependency>
<groupId>com.github.paulcwarren</groupId>
<artifactId>spring-content-rest</artifactId>
<version>0.0.11</version>
</dependency>
Update your S3Config.java as follows:
#Configuration
#EnableS3Stores
#Import(RestConfiguration.class)
public class S3Config {
...
Update your store as follows:
S3Store.java
#StoreRestResource(path="s3docs")
public interface S3Store extends Store<String> {
}
Now when your application starts it will see your Store interface and also inject an #Controller implementation that will forward REST request onto your store. This replaces the autowiring code above obviously.
Then:
curl -X POST /s3docs/example-doc
with a multipart/form-data request will store the image in s3.
curl /s3docs/example-doc
will fetch it again and so on. This controller supports full CRUD and video streaming by the way.
If you want to associate this "content" with JPA Entity or something like that then you can have your S3Store extend AssociateStore or ContentStore and you have additional methods available that provide for associations.
There are a couple of getting started guides here. The s3 reference guide is here. And there is a tutorial video here. The coding bit starts about 1/2 way through.
HTH
Since it needs a temporary location to place files. Below code worked after deploying war on AWS.
private File convertMultiPartToFile(MultipartFile multipartFile) throws IOException {
File convFile = new File(System.getProperty("java.io.tmpdir") + System.getProperty("file.separator") +
multipartFile.getOriginalFilename());
multipartFile.transferTo(convFile);
return convFile;
}
You have problems with relative Paths
You can do this
public class UploadStackoverflow {
private String location = "upload-dir";
private Path rootLocation;
public File convertFile(MultipartFile file) throws IOException {
rootLocation = Paths.get(location);
Files.createDirectories(rootLocation);
String filename = StringUtils.cleanPath(file.getOriginalFilename());
InputStream inputStream = file.getInputStream();
Files.copy(inputStream, this.rootLocation.resolve(filename),
StandardCopyOption.REPLACE_EXISTING);
return new File(this.rootLocation.resolve(filename).toAbsolutePath().toString());
}
}
Using Springboot 1.5.x, Spring Cloud, and JAX-RS:
I could use a second pair of eyes since it is not clear to me whether the Spring configured, Javanica HystrixCommand works for all use cases or whether I may have an error in my code. Below is an approximation of what I'm doing, the code below will not actually compile.
From below WebService lives in a library with separate package path to the main application(s). Meanwhile MyWebService lives in the application that is in the same context path as the Springboot application. Also MyWebService is functional, no issues there. This just has to do with the visibility of HystrixCommand annotation in regards to Springboot based configuration.
At runtime, what I notice is that when a code like the one below runs, I do see "commandKey=A" in my response. This one I did not quite expect since it's still running while the data is obtained. And since we log the HystrixRequestLog, I also see this command key in my logs.
But all the other Command keys are not visible at all, regardless of where I place them in the file. If I remove CommandKey-A then no commands are visible whatsoever.
Thoughts?
// Example WebService that we use as a shared component for performing a backend call that is the same across different resources
#RequiredArgsConstructor
#Accessors(fluent = true)
#Setter
public abstract class WebService {
private final #Nonnull Supplier<X> backendFactory;
#Setter(AccessLevel.PACKAGE)
private #Nonnull Supplier<BackendComponent> backendComponentSupplier = () -> new BackendComponent();
#GET
#Produces("application/json")
#HystrixCommand(commandKey="A")
public Response mainCall() {
Object obj = new Object();
try {
otherCommandMethod();
} catch (Exception commandException) {
// do nothing (for this example)
}
// get the hystrix request information so that we can determine what was executed
Optional<Collection<HystrixInvokableInfo<?>>> executedCommands = hystrixExecutedCommands();
// set the hystrix data, viewable in the response
obj.setData("hystrix", executedCommands.orElse(Collections.emptyList()));
if(hasError(obj)) {
return Response.serverError()
.entity(obj)
.build();
}
return Response.ok()
.entity(healthObject)
.build();
}
#HystrixCommand(commandKey="B")
private void otherCommandMethod() {
backendComponentSupplier
.get()
.observe()
.toBlocking()
.subscribe();
}
Optional<Collection<HystrixInvokableInfo<?>>> hystrixExecutedCommands() {
Optional<HystrixRequestLog> hystrixRequest = Optional
.ofNullable(HystrixRequestLog.getCurrentRequest());
// get the hystrix executed commands
Optional<Collection<HystrixInvokableInfo<?>>> executedCommands = Optional.empty();
if (hystrixRequest.isPresent()) {
executedCommands = Optional.of(hystrixRequest.get()
.getAllExecutedCommands());
}
return executedCommands;
}
#Setter
#RequiredArgsConstructor
public class BackendComponent implements ObservableCommand<Void> {
#Override
#HystrixCommand(commandKey="Y")
public Observable<Void> observe() {
// make some backend call
return backendFactory.get()
.observe();
}
}
}
// then later this component gets configured in the specific applications with sample configuraiton that looks like this:
#SuppressWarnings({ "unchecked", "rawtypes" })
#Path("resource/somepath")
#Component
public class MyWebService extends WebService {
#Inject
public MyWebService(Supplier<X> backendSupplier) {
super((Supplier)backendSupplier);
}
}
There is an issue with mainCall() calling otherCommandMethod(). Methods with #HystrixCommand can not be called from within the same class.
As discussed in the answers to this question this is a limitation of Spring's AOP.
I'm experiencing a little issue that is wasting a lot of my time...
I've created, for demonstration purposes, a simple SpringBoot application using the Eclipse New > Spring Starter Project.
Here is my Application class:
package it.asirchia;
//All needed imports
#SpringBootApplication
public class Application {
public static HashMap<Long,Book> books = new HashMap<Long, Book>();
public static HashMap<Long,Editor> editors = new HashMap<Long, Editor>();
public static HashMap<Long,Person> authors = new HashMap<Long, Person>();
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
}
Then I've created the EditorsApis Controller:
package it.asirchia.apis;
//All needed imports
#RestController
#RequestMapping(value="/editors")
public class EditorsApis {
private static long counter = 0;
#RequestMapping(value="/", method=RequestMethod.GET)
public HashMap<Long, Editor> getAllEditor(){
return Application.editors;
}
#RequestMapping(value="/", method=RequestMethod.POST)
public void postNewEditor(#RequestBody Editor editor){
Application.editors.put(counter++, editor);
}
#RequestMapping(value="/{editorid}", method=RequestMethod.PUT)
public void updateEditor(#PathVariable long editorid,
#RequestBody Editor editor){
Application.editors.put(editorid, editor);
}
#RequestMapping(value="/{editorid}", method=RequestMethod.GET)
public Editor getEditor(#PathVariable long editorid){
return Application.editors.get(editorid);
}
#RequestMapping(value="/{editorid}", method=RequestMethod.DELETE)
public void deleteEditor(#PathVariable long editorid){
Application.editors.remove(editorid);
}
}
And an AuthorsApis and a BooksApis controllers that are very similar to the EditorApis one.
Of course I've created too all the three Pojos:
Editor.class, Person.class and Book.class
I've started up the Eclipse embedded Spring runtime and I can see that all the paths are properly mapped:
INFO [main] s.w.s.m.m.a.RequestMappingHandlerMapping Mapped "
{[/authors/],methods=[GET]}" onto public java.util.HashMap it.asirchia.apis.AuthorsApis.getAllAuthors()
And so on and so forth for all the other Rest APIs I've implemented.
The last three lines of the log are:
Starting beans in phase 0
Tomcat started on port(s): 8080 (http)
Started Application in 5.547 seconds (JVM running for 6.169)
Ok, for me wverything is properly configured, up and running. But when I try to invoke
GET /authors HTTP/1.1
Host: localhost:8080
I obtain:
{
"timestamp": 1507286437765,
"status": 404,
"error": "Not Found",
"message": "No message available",
"path": "/authors"
}
And the same happens for ALL the REST APIs I've implemented.
Any idea about the reason of this problem?
Thank you.
The following mapping will work localhost:8080/authors/ for you.Since in your method mapping GET you have added the "/" so you should provide the trailing slash in URL also. If you want mapping like this localhost:8080/authors then follow the below code,
#RequestMapping(value={"","/"}, method=RequestMethod.GET)
public HashMap<Long, Editor> getAllEditor(){
return Application.editors;
}
The above will accept,
1) localhost:8080/editors
2) localhost:8080/editors/
Hope this will help.
Can you just try to add a action content in value.Here only specifying only a "/".Like
#RequestMapping(value="/updateEditor", method=RequestMethod.GET)
If you need to add any path variable,you can modify the same with following,
#RequestMapping(value="/updateEditor/{editorid}", method=RequestMethod.PUT)
Just try this method also.
I'm developing a Spring webapp, using spring boot and spring batch frameworks.
We have a set of complex & different json files, and we need to:
read each file
slightly modify its content
finally store them in mongodb.
The question: It makes sense to use spring batch for this task? As I can see in tutorials examples etc, spring batch is the right tool for line by line processing, but what about file by file?
I don't have problems with the writer (MongoItemWritter) and processer, but I do not see how to implement the reader.
Thanks!
yes you can definetly use Spring Batch.
The item for your Reader can be a File.
public class CustomItemReader implements InitializingBean{
private List<File> yourFiles= null;
public File read() {
if ((yourFiles!= null) && (yourFiles.size() != 0)) {
return yourFiles.remove(0);
}
return null;
}
//Reading Items from Service
private void reloadItems() {
this.yourItems= new ArrayList<File>();
// populate the items
}
#Override
public void afterPropertiesSet() throws Exception {
reloadItems();
}
}
A custom Processor :
public class MyProcessor implements ItemProcessor<File, File> {
#Override
public File process(File arg0) throws Exception {
// Apply any logic to your File before transferring it to the writer
return arg0;
}
}
And A custom Writer :
public class MyWriter{
public void write(File file) throws IOException {
}
}