I'm trying to implement a rest api consuming excel file. I'm using spring-boot and code is available here.
Code works fine when using FileSystemResource for payload. But i'm not able to make the code work with ByteArrayResource in replacement of FileSystemResource:
RestApi.java:
#RestController
public class RestApi {
private static final Logger LOGGER = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
#PostMapping("/api/upload")
public ResponseEntity<?> uploadFile(#RequestParam("file") MultipartFile uploadfile) {
LOGGER.debug("Single file upload!");
try {
LOGGER.info("\n\n ****** File name: {}, type {}! ************", uploadfile.getOriginalFilename(), uploadfile.getContentType());
this.processExcelFile(uploadfile.getInputStream());
} catch (Exception e) {
return new ResponseEntity<>(HttpStatus.BAD_REQUEST);
}
return new ResponseEntity<>("Successfully uploaded - " + uploadfile.getOriginalFilename(), new HttpHeaders(), HttpStatus.OK);
}
private List<String> processExcelFile(InputStream stream) throws Exception {
List<String> result = new ArrayList<String>();
//Create Workbook instance holding reference to .xlsx file
try(XSSFWorkbook workbook = new XSSFWorkbook(stream);) {
//Get first/desired sheet from the workbook
XSSFSheet sheet = workbook.getSheetAt(0);
//Iterate through each rows one by one
Iterator<Row> rowIterator = sheet.iterator();
while (rowIterator.hasNext()) {
Row row = rowIterator.next();
String cellValue = row.getCell(0).getRichStringCellValue().toString();
result.add(cellValue);
LOGGER.info("\n\n ****** Cell value: {} ************", cellValue);
}
return result;
}
}
}
RestApiTest:
#RunWith(SpringRunner.class)
#SpringBootTest(webEnvironment = WebEnvironment.RANDOM_PORT)
public class RestApiTest {
#Autowired
private TestRestTemplate restTemplate;
#Autowired
private ResourceLoader loader;
#Test
public void testUploadFile() throws Exception {
Resource resource = this.loader.getResource("classpath:test.xlsx");
MultiValueMap<String, Object> parts = new LinkedMultiValueMap<>();
// parts.add("file", new FileSystemResource(resource.getFile()));
parts.add("file", new ByteArrayResource(IOUtils.toByteArray(resource.getInputStream())));
String response = this.restTemplate.postForObject("/api/upload", parts, String.class);
Assertions.assertThat(response).containsIgnoringCase("success");
}
}
I'm getting following error when running test:
java.lang.AssertionError:
Expecting:
<"{"timestamp":1487852597527,"status":400,"error":"Bad Request","exception":"org.springframework.web.multipart.support.MissingServletRequestPartException","message":"Required request part 'file' is not present","path":"/api/upload"}">
to contain:
<"success">
(ignoring case)
Any idea?
when using loader.getResource(...) you must use resource itself as answered above. So you don't need ByteArrayResource. I got this problem, but I'm not using resource from classpath. So if someone really need to use ByteArrayResource, here is my workaround
public class FileNameAwareByteArrayResource extends ByteArrayResource {
private String fileName;
public FileNameAwareByteArrayResource(String fileName, byte[] byteArray, String description) {
super(byteArray, description);
this.fileName = fileName;
}
#Override
public String getFilename() {
return fileName;
}
}
and then use it
parts.add("file", new FileNameAwareByteArrayResource("filename", byteArray));
Related
In the project am using olingo 2.0.12 jar in the java code.
During the create Entity service call ,
Is there a way to check for which entity data insert requested and,
Alter column values / append new column values before data persisted?
Is there a way to add above?
Code snippet given below,
public class A extends ODataJPADefaultProcessor{
#Override
public ODataResponse createEntity(final PostUriInfo uriParserResultView, final InputStream content,
final String requestContentType, final String contentType) throws ODataJPAModelException,
ODataJPARuntimeException, ODataNotFoundException, EdmException, EntityProviderException {
// Need to check the entity name and need to alter/add column values
}
}
Yes one of the possible ways would be to create your own CustomODataJPAProcessor which extends ODataJPADefaultProcessor.
You will have to register this in JPAServiceFactory by overriding the method
#Override
public ODataSingleProcessor createCustomODataProcessor(ODataJPAContext oDataJPAContext) {
return new CustomODataJPAProcessor(this.oDataJPAContext);
}
Now Olingo will use CustomODataJPAProcessor which can implement the following code to check the entities and transform them if needed
Sample code of CustomODataJPAProcessor
public class CustomODataJPAProcessor extends ODataJPADefaultProcessor {
Logger LOG = LoggerFactory.getLogger(this.getClass());
public CustomODataJPAProcessor(ODataJPAContext oDataJPAContext) {
super(oDataJPAContext);
}
#Override
public ODataResponse createEntity(final PostUriInfo uriParserResultView, final InputStream content,
final String requestContentType, final String contentType) throws ODataException {
ODataResponse oDataResponse = null;
oDataJPAContext.setODataContext(getContext());
InputStream forwardedInputStream = content;
try {
if (uriParserResultView.getTargetEntitySet().getName().equals("Students")) {
LOG.info("Students Entity Set Executed");
if (requestContentType.equalsIgnoreCase(ContentType.APPLICATION_JSON.toContentTypeString())) {
#SuppressWarnings("deprecation")
JsonElement elem = new JsonParser().parse(new InputStreamReader(content));
Gson gson = new GsonBuilder().setFieldNamingPolicy(FieldNamingPolicy.UPPER_CAMEL_CASE).create();
Student s = gson.fromJson(elem, Student.class);
// Change some values
s.setStudentID("Test" + s.getStudentID());
forwardedInputStream = new ByteArrayInputStream(gson.toJson(s).getBytes());
}
}
Object createdJpaEntity = jpaProcessor.process(uriParserResultView, forwardedInputStream,
requestContentType);
oDataResponse = responseBuilder.build(uriParserResultView, createdJpaEntity, contentType);
} catch (JsonIOException | JsonSyntaxException e) {
throw new RuntimeException(e);
} finally {
close();
}
return oDataResponse;
}
}
In Summery
Register your custom org.apache.olingo.odata2.service.factory Code Link
Create your own CustomODataJPAProcessor Code Link
Override createCustomODataProcessor in JPAServiceFactory to use the custom processor Code Link
I am attempting to write a TOML PropertySourceLoader implementation. I took a look at some other examples on GitHub and stackoverflow, all of which seem to eventually parse the result out to a map and then return an OriginTrackedMapPropertySource, which is what I tried below:
public final class TomlPropertySourceFactory implements PropertySourceFactory {
#Override
public PropertySource<?> createPropertySource(String name, EncodedResource resource) throws IOException {
if (!ClassUtils.isPresent("com.fasterxml.jackson.dataformat.toml.TomlFactory", null)) {
throw new IllegalStateException(
"Attempted to load " + name + " but jackson-dataformat-toml was not found on the classpath");
}
if (!ClassUtils.isPresent("com.fasterxml.jackson.core.io.ContentReference", null)) {
throw new IllegalStateException(
"Attempted to load " + name + " but jackson-core was either not found on the classpath or below version 2.13.0");
}
final ObjectMapper mapper = new ObjectMapper(new TomlFactory());
final Map<String, Object> resultMap = mapper.convertValue(mapper.readTree(resource.getInputStream()), new TypeReference<Map<String, Object>>(){});
return new OriginTrackedMapPropertySource(Optional.ofNullable(name).orElseGet(resource.getResource()::getFilename), resultMap);
}
}
public final class TomlPropertySourceLoader implements PropertySourceLoader {
#Override
public String[] getFileExtensions() {
return new String[]{"tml", "toml"};
}
#Override
public List<PropertySource<?>> load(final String name, final Resource resource) throws IOException {
final EncodedResource encodedResource = new EncodedResource(resource);
return Collections.singletonList(new TomlPropertySourceFactory().createPropertySource(name, encodedResource));
}
}
This code does seem to more or less do what is expected; it is executed when application.toml is present, it loads and parses the file out to a <String, Object> map, but from there, none of the actual properties seem to be present in the application — be it when using #Value, #ConfigurationProperties or even when attempting to set stuff like the port Tomcat runs on.
There's not a ton of information available on the internet, without digging into the depths of Spring, about what exactly it is expecting. I'm not sure if the problem is due to how my map is structured or perhaps due to something with the name.
Below you can find my application.toml file:
[spring.datasource]
url = "jdbc:hsqldb:file:testdb"
username = "sa"
[spring.thymeleaf]
cache = false
[server]
port = 8085
[myapp]
foo = "Hello"
bar = 42
aUri = "https://example.org/hello"
targetLocale = "en-US"
[myapp.configuration]
endpoints = ["one", "two", "three"]
[myapp.configuration.connectionSettings]
one = "hello"
[myapp.configuration.connectionSettings.two]
two_sub = "world!"
And my configuration classes:
#Data
#NoArgsConstructor
#Component
#ConfigurationProperties("myapp")
public class AppConfig {
private String foo;
private int bar;
private URI aUri;
private Locale targetLocale;
private SubConfiguration configuration;
}
#Data
#NoArgsConstructor
public class SubConfiguration {
private List<String> endpoints;
private Map<String, Object> connectionSettings;
}
As well as my testing controller:
#RestController
#RequiredArgsConstructor
public final class TomlDemoController {
private final AppConfig appConfig;
#GetMapping("/api/config")
AppConfig getConfig() {
return appConfig;
}
}
The issue was the structure of the property map. The keys have to be flattened in order to work. As an example, for a given table:
[server]
port = 8085
Rather than producing a nested map:
Properties = {
"server" = {
"port" = 8085
}
}
Spring is expecting something more like:
Properties = {
"server.port" = 8085
}
A quick solution using the ObjectToMapTransformer found in Spring Integration:
#Override
#SuppressWarnings("unchecked")
public PropertySource<?> createPropertySource(String name, EncodedResource resource) throws IOException {
if (!ClassUtils.isPresent("com.fasterxml.jackson.dataformat.toml.TomlFactory", null)) {
throw new IllegalStateException(
"Attempted to load " + name + " but jackson-dataformat-toml was not found on the classpath");
}
if (!ClassUtils.isPresent("com.fasterxml.jackson.core.io.ContentReference", null)) {
throw new IllegalStateException(
"Attempted to load " + name + " but jackson-core was either not found on the classpath or below version 2.13.0");
}
final ObjectMapper mapper = new ObjectMapper(new TomlFactory());
final Message<JsonNode> message = new GenericMessage<>(mapper.readTree(resource.getInputStream()));
final ObjectToMapTransformer transformer = new ObjectToMapTransformer();
transformer.setShouldFlattenKeys(true);
Map<String,Object> resultMap = (Map<String, Object>) transformer.transform(message).getPayload();
return new OriginTrackedMapPropertySource(Optional.ofNullable(name).orElseGet(resource.getResource()::getFilename), resultMap);
}
I am trying to unit test a put request which takes a file and some json data as request body. following is the method i am trying to test:
#RequestMapping(
value = "/{id}",
method = RequestMethod.PUT,
produces = { "application/json" }
)
public ResponseEntity<UpdateT1Output> update(#PathVariable String id, #ModelAttribute #Valid UpdateT1Input t1) {
// implementation here
}
UpdateT1Input.java
public class UpdateT1Input {
private char[] ca;
private byte[] file;
public void setFile(MultipartFile mpfile) {
try {
file = mpfile.getBytes();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
private List<Double> flpa;
private List<Double> fpa;
#NotNull(message = "id Should not be null")
private Long id;
private String str;
private Long versiono;
}
test setup
#Test
public void UpdateT1_T1Exists_ReturnStatusOk() throws Exception {
// create entity obj with default values
T1Entity entity = createUpdateEntity();
entity.setVersiono(0L);
UpdateT1Input t1Input = new UpdateT1Input();
t1Input.setId(entity.getId());
t1Input.setFlpa(entity.getFlpa());
t1Input.setStr(entity.getStr());
ObjectWriter ow = new ObjectMapper()
.registerModule(new JavaTimeModule())
.disable(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS)
.writer()
.withDefaultPrettyPrinter();
String json = ow.writeValueAsString(t1Input);
MockMultipartHttpServletRequestBuilder builder =
MockMvcRequestBuilders.multipart("/t1/" + entity.getId());
builder.with(request -> {
request.setMethod("PUT");
return request;
});
mvc.perform(builder
.file("file", "ABC".getBytes("UTF-8"))
.content(json)
.contentType(MediaType.APPLICATION_JSON)
)
.andExpect(status().isOk());
}
but in controller only id and file fields are set in input dto all other fields are null. i am using #ModelAttribute to avoid dividing request into file and data parts. so is there a way that to get all the fields in single object?
To store images I'm using Spring Content JPA strategy.
My test profile has HSQLDB in-memory implementation.
Is there a more convenient way to populate DB with images?
For now, I have a solution to create a folder with images and then
upload them manually on startup. As I understand, to get rid of the image folder I can upload them to SQLite and then fetch data from it, but maybe there is a better way?
CarrentalApplication
#SpringBootApplication
#EnableJpaRepositories
#EnableJpaStores
public class CarrentalApplication {
private static final String IMAGE_FOLDER = "classpath:static/img/test-cars/";
private final Map<Integer, String> cars = new HashMap<>() {{
put(100010, "merc_benz");
put(100011, "volvo_s60");
put(100012, "suz_swift");
put(100013, "volks16v");
put(100014, "ford150");
put(100015, "lambo610");
put(100016, "bmvx5");
put(100017, "audis6");
}};
public static void main(String[] args) {
SpringApplication.run(CarrentalApplication.class, args);
}
#Bean
public CommandLineRunner uploadImages(CarService carService, CarRepository carRepository,
CarImageStore imageStore) {
return (args) -> cars.forEach((carId, name) -> {
try {
Car car = carService.get(carId);
File file = ResourceUtils.getFile(IMAGE_FOLDER + name + ".jpg");
FileInputStream input = new FileInputStream(file);
imageStore.setContent(car, input);
carRepository.save(car);
} catch (Exception e) {
e.printStackTrace();
}
});
}
}
CarImageStore
#StoreRestResource(path = "data")
#Repository
public interface CarImageStore extends ContentStore<Car, String> {
}
Car content fields:
#ContentId
private String contentId;
#ContentLength
private Long contentLength = 0L;
#MimeType
private String mimeType = "text/plain";
Another way to do this is move the images onto the classpath; i.e. into /src/test/resources (assuming maven) and load them from there with:
imageStore.setContent(car, this.getClass().getResourceAsStream("/" + name + ".jpg"));
I'd like to have your opinion on a error always throwed in my spring boot rest controller. I got the following first controller accepting reading requests :
#RequestMapping(value="/read/{file:.+}" , method = RequestMethod.GET)
public ResponseEntity myFunction(#PathVariable("file") String file) {
String[] parts = file.split("\\.");
String extension = parts[1];
List<SousBloc> resWord;
List<SousBloc> resPdf;
List<CvAvecBlocs> resExcel;
RestTemplate rt = new RestTemplate();
rt.getMessageConverters().add(new StringHttpMessageConverter());
if(extension.equals("xlsx")){
resExcel = rt.getForObject("http://localhost:8080/readExcel/"+file, List.class, 200);
return new ResponseEntity<>(resExcel, HttpStatus.OK);
}
else if(extension.equals("pdf")){
resPdf = rt.getForObject("http://localhost:8080/readPdf/"+file, List.class, 200);
return new ResponseEntity<>(resPdf, HttpStatus.OK);
}
else if(extension.equals("docx")){
resWord = rt.getForObject("http://localhost:8080/readWord/"+file, List.class, 200);
return new ResponseEntity<>(resWord, HttpStatus.OK);
}
return null;
}
There is my Reading Word Controller :
#Controller
public class ReadWordController {
private static String UPLOADED_FOLDER = "C:\\cvsUploades\\";
#Autowired
ReadWord readWord;
#RequestMapping(value="/readWord/{file:.+}" , method = RequestMethod.GET)
public ResponseEntity readingWord(#PathVariable("file") String file) throws IOException {
String path = UPLOADED_FOLDER+file;
List<SousBloc> sousBlocs = readWord.extract(path);
return new ResponseEntity<>(sousBlocs, HttpStatus.OK);
}
}
Well this controller works fine and does the job.
Now there is my Reading Pdf Controller :
#Controller
public class ReadPdfController {
private static String UPLOADED_FOLDER = "C:\\cvsUploades\\";
#Autowired
ReadPdf readPdf;
#RequestMapping(value="/readPdf/{file:.+}" , method = RequestMethod.GET)
public ResponseEntity readingPdf(#PathVariable("file") String file) throws IOException {
String path = UPLOADED_FOLDER+file;
List<SousBloc> blocs = readPdf.extract(path);
return new ResponseEntity<>(blocs, HttpStatus.OK);
}
}
It is contructed on the same model of the Reading Word Controller but it does not work. In debug, the program works fine until the return new ResponseEntity<>(blocs, HttpStatus.OK); that throws a 406 error null...
Do you know why ?
EDIT: I tried something strange and it worked... I put the following code :
#Controller
public class ReadWordController {
private static String UPLOADED_FOLDER = "C:\\cvsUploades\\";
#Autowired
ReadWord readWord;
#Autowired
ReadPdf readPdf;
#RequestMapping(value="/readWord/{file:.+}" , method = RequestMethod.GET)
public ResponseEntity readingWord(#PathVariable("file") String file) throws IOException {
/*String path = UPLOADED_FOLDER+file;
List<SousBloc> sousBlocs = readWord.extract(path);
return new ResponseEntity<>(sousBlocs, HttpStatus.OK);*/
String path = "C:\\cvsUploades\\file.pdf";
List<SousBloc> blocs = readPdf.extract(path);
return new ResponseEntity<>(blocs, HttpStatus.OK);
}
}