Spring Boot - Handle CSV as well as Excel Multipart file - spring-boot

I have a REST API in Spring Boot Application that takes in a param of type Multipart file.
There is possibility that user may import either CSV file or Excel(.xlsx / .xsl) file of huge size which needs to be handled.
I am using Apache POI to read Excel type file and it is working fine. To my existing code, how do I efficiently handle CSV file reading also
Below is Excel file Reading Code:
#RequestMapping(value="/read", method = RequestMethod.POST)
#Transactional
public Map<String, String> read(#RequestParam("file") MultipartFile file) {
Map<String, String> response = new ArrayList();
if (!file.isEmpty()) {
ByteArrayInputStream stream;
Workbook wb;
StringBuilder contentSb = new StringBuilder();
try {
stream = new ByteArrayInputStream(file.getBytes());
wb = WorkbookFactory.create(stream);
org.apache.poi.ss.usermodel.Sheet sheet = wb.getSheetAt(wb.getActiveSheetIndex());
Iterator<Row> rowIterator = sheet.rowIterator();
System.out.println("Processing Excel file");
for (int rowIndex = 0; rowIndex <= sheet.getLastRowNum(); rowIndex++) {
Row row = sheet.getRow(rowIndex);
if (row != null) {
Cell cell = row.getCell(0);
if (cell != null) {
contentSb.append(cell.getStringCellValue()+",");
}
}
}
System.out.println("Processed Excel file");
return response;
} catch (Exception e) {
e.printStackTrace();
}
}
else {
return response;
}
}
Thank you in advance!

Related

Return a new thymeleaf page fragment and some file from spring controller

Tell me how to properly implement the spring controller. I want it to return a new thymeleafe page fragment and Excel file at the same time.
I know how to implement a controller to return a page fragment:
#GetMapping("/some_url")
public String updateFragment(Model model) {
model.addAttribute("attribute", someAtribute);
return "some_page :: fragment";
}
And I know how to implement a controller for downloading files:
#GetMapping("some_url")
public void getReport(HttpServletResponse response) throws IOException {
response.setContentType("application/octet-stream");
String headerKey = "Content-Disposition";
String headerValue = "attachment; filename=file.xlsx";
response.setHeader(headerKey, headerValue);
try {
FileInputStream inputStream = new FileInputStream(new File("path_to_file"));
Workbook workbook = WorkbookFactory.create(inputStream);
Sheet sheet = workbook.getSheetAt(0);
int row = 12;
Row exampleRow = sheet.getRow(12);
int count = 1;
for (SIZForKomplex s : objectList) {
if (s.getNumber() != 0) {
sheet.getRow(row).setRowStyle(exampleRow.getRowStyle());
sheet.getRow(row).setHeight(exampleRow.getHeight());
for (int i = 0; i < 8; i++) {
sheet.getRow(row).getCell(i).setCellStyle(exampleRow.getCell(i).getCellStyle());
}
sheet.getRow(row).getCell(0).setCellValue(count);
sheet.getRow(row).getCell(1).setCellValue(s.getNomenclatureNumber());
sheet.getRow(row).getCell(2).setCellValue(s.getNamesiz());
sheet.getRow(row).getCell(3).setCellValue(s.getSize());
sheet.getRow(row).getCell(4).setCellValue(s.getHeight());
sheet.getRow(row).getCell(5).setCellValue(s.getNumber());
sheet.getRow(row).getCell(6).setCellValue(sizRepository.findById(s.getId()).orElseThrow().getEd_izm());
sheet.getRow(row).getCell(7).setCellValue(" ");
row++;
count++;
}
}
inputStream.close();
ServletOutputStream outputStream = response.getOutputStream();
workbook.write(outputStream);
workbook.close();
outputStream.close();
} catch (IOException | EncryptedDocumentException
ex) {
ex.printStackTrace();
}
}
But how do I combine this. I need to update the data on the page and download the Excel file

Send multiple files from angular typescript to spring and return as zip folder for downloading?

I want to send multiple files in an array to spring and create a zip folder for downloading
UploadController:
#Autowired
StorageService storageService;
#PostMapping("/upload")
public ResponseEntity<ResponseMessage> uploadFiles(#RequestParam("files") MultipartFile[] files) {
String message = "";
try {
storageService.zip(files);
message = "Uploaded the files successfully";
return ResponseEntity.status(HttpStatus.OK).body(new ResponseMessage(message));
} catch (Exception e) {
message = "Fail to upload files!";
return ResponseEntity.status(HttpStatus.EXPECTATION_FAILED).body(new ResponseMessage(message));
}
}
StorageService
public void zip(MultipartFile[] files) {
List<Path> filepaths = new ArrayList();
for (MultipartFile file : files) {
Path filepath = Paths.get("my/tmp/dir", file.getOriginalFilename());
filepaths.add(filepath);
try (OutputStream os = Files.newOutputStream(filepath)) {
os.write(file.getBytes());
}
}
File zip = new File("path/to/my/zip");
try { zip.createNewFile(); }
FileOutputStream output = null;
try { output = new FileOutputStream(zip); }
ZipOutputStream out = new ZipOutputStream(output);
try {
for (Path filepath : filepaths) {
File f = new File(filepath);
FileInputStream input = new FileInputStream(f);
ZipEntry e = new ZipEntry(f.getName());
out.putNextEntry(e);
byte[] bytes = new byte[1024];
int length;
while((length = input.read(bytes)) >= 0) {
out.write(bytes, 0, length);
}
input.close();
}
out.close();
output.close();
}
}

Check if Spring Multipart File uploaded has a form field

We were using the apache file uploads for uploading a file and below code was used for the same.
ServletFileUpload upload = new ServletFileUpload(itemFactory);
List<FileItem> items = upload.parseRequest(request);
Iterator<FileItem> iterator = items.iterator();
while (iterator.hasNext()) {
FileItem item = iterator.next();
if (item.isFormField()) {
String name = item.getFieldName();
String value = item.getString();
conf.put(name, value);
} else {
InputStream is = item.getInputStream();
byte[] bytes = ByteStreams.toByteArray(is);
String query = new String(bytes, "UTF-8");
conf.put("test", query);
}
}
But we recently moved to Spring boot and trying to use spring multipart to upload a file.The code which will replace the above code will be
DefaultMultipartHttpServletRequest requestMain = (DefaultMultipartHttpServletRequest) request;
Iterator<String> fileNameIterator = requestMain.getFileNames();
ListIterator<MultipartFile> iterator = null;
while (fileNameIterator.hasNext()) {
multipartFiles = requestMain.getFiles(fileNameIterator.next());
iterator = multipartFiles.listIterator();
while (iterator.hasNext()) {
MultipartFile item = iterator.next();
if (item.isEmpty()) {
// String name = item.getFieldName();
// String value = item.getString();
// conf.put(name, value);
} else {
InputStream is = item.getInputStream();
byte[] bytes = ByteStreams.toByteArray(is);
String query = new String(bytes, "UTF-8");
conf.put("query", query);
}
}
}
But I am not able to figure out how to check if the file has a form field in spring boot.In apache file upload it was achieved using item.isFormField() method.
You don't need to check if the item is form field.
The requestMain.getFiles(fileNameIterator.next()); method will return a MultipartFile so you know is not a form field.
To get the form fields use
requestMain.getParameterMap()
Or, for a specific field use
requestMain.getParameter("field")

Load Json Data using Pig

I am trying to extract data from below mention json format by pig using jsonLoader():
{"Partition":"10","Key":"618897","Properties2":[{"K":"A","T":"String","V":"M "}, {"K":"B","T":"String","V":"N"}, {"K":"D","T":"String","V":"O"}]}
{"Partition":"11","Key":"618900","Properties2":[{"K":"A","T":"String","V":"W”"},{"K":"B","T":"String","V":"X"}, {"K":"C","T":"String","V":"Y"},{"K":"D","T":"String","V":"Z"}]}
Right now I am able to extract data from “partition” ,“key” and “V” for every array objects with the following code:
A= LOAD '/home/hduser/abc.jon' Using JsonLoader('Partition:chararray,Key:chararray,Properties2:{(K:chararray,T:chararray,V:chararray)},Timestamp:chararray');
B= foreach A generate $0,$1,BagToString(Properties2.V,'\t') as vl:chararray;
store B into './Result/outPut2';
From above code I am getting "Properties2" array value on the sequence basis not column basis, it is creating problem whenever sequence changed or new object comes in existence.
Please help me to extract data on the basis of column( K values.)
My Output
Expected Output
Thanks In Advance
You have two options here
1.Use elephant-bird which will give you a map of key and value.
A = LOAD '/apps/pig/json_sample' USING com.twitter.elephantbird.pig.load.JsonLoader('-nestedLoad') as (json:map[]);
B = FOREACH A GENERATE json#'Partition',json#'Key',json#'Properties2';
dump B;
will give you an output of :
(10,618897,{([T#String,K#A,V#M ]),([T#String,K#B,V#N]),([T#String,K#D,V#O])})
(11,618900,{([T#String,K#A,V#W”]),([T#String,K#B,V#X]),([T#String,K#C,V#Y]),([T#String,K#D,V#Z])})
Or you have to write a custom loader which has to do this
a).It should know what is the correct order of values that will be coming
for the key K
b).Go through each of these values and see if the json is missing any of this key and return an empty/null char for that location.
Am posting the getNext() method of the CustomJsonLoader which will do the same:
#Override
public Tuple getNext() throws IOException {
// TODO Auto-generated method stub
try {
boolean notDone = in.nextKeyValue();
if (!notDone) {
return null;
}
Text value = (Text) in.getCurrentValue();
List<String> valueList = new ArrayList<String>();
if (value != null) {
String jsonString = value.toString();
System.out.println(jsonString);
JSONParser parser = new JSONParser();
JSONObject obj = null;
try {
obj = (JSONObject) parser.parse(jsonString);
} catch (ParseException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
System.out.println("obj is "+obj);
if (obj != null) {
String partition = (String) obj.get("Partition");
String key = (String) obj.get("Key");
valueList.add(partition);
valueList.add(key);
JSONArray innArr = (JSONArray) obj.get("Properties2");
char[] innKeys = new char[] { 'A', 'B', 'C', 'D' };
Map<String,String> keyMap = new HashMap<String,String>();
for (Object innObj : innArr) {
JSONObject jsonObj = (JSONObject) innObj;
keyMap.put(jsonObj.get("K")+"",jsonObj.get("V")+"");
}
for (int i = 0; i < innKeys.length; i++) {
char ch = innKeys[i];
if (keyMap.containsKey(ch+"")) {
valueList.add(keyMap.get(ch+""));
}else{
valueList.add("");
}
}
Tuple t = tupleFactory.newTuple(valueList);
return t;
}
}
return null;
} catch (InterruptedException e) {
}
}
and register it and run :
REGISTER udf/CustomJsonLoader.jar
A = LOAD '/apps/pig/json_sample' USING CustomJsonLoader();
DUMP A;
(10,618897,M,N,,O)
(11,618900,W,X,Y,Z)
Hope this helps!

multi lining files in IsolatedStorage # Windows Phone 7

i have created some files in the IO
in the "car" files, i would like to put some other reference like model, color etc...
so my question is : is it possible to have a multi-lining files in the IO
if yes how can i get them in the streamreader
// i want to storage many parameters in a file and find them again with the streamreader
protected override void OnNavigatedTo(System.Windows.Navigation.NavigationEventArgs e)
{
//reception des parametres de la listbox
base.OnNavigatedTo(e);
string parameter = this.NavigationContext.QueryString["parameter"];
this.tbTitre.Text = parameter;
try
{
//Create a new StreamReader
StreamReader editionDevisReader = null;
IsolatedStorageFile probyOrange = IsolatedStorageFile.GetUserStoreForApplication();
//Read the file from the specified location.
editionDevisReader = new StreamReader(new IsolatedStorageFileStream("devis\\"+parameter+".txt", FileMode.Open, probyOrange));
//Read the contents of the file .
string textFile = editionDevisReader.ReadLine();
//Write the contents of the file to the TextBlock on the page.
tbTitre.Text = textFile;
while (editionDevisReader != null)
{
RowDefinition rowdefinition = new RowDefinition();
TextBlock textblock = new TextBlock();
textblock.HorizontalAlignment = new System.Drawing.Size(48, 20);
}
editionDevisReader.Close();
}
catch
{
//If the file hasn't been created yet.
tbTitre.Text = "veuillez d abord creer le fichier";
}
thx a lot all
Yes, you can save anything (up to a point) in a file:
using (var store = IsolatedStorageFile.GetUserStoreForApplication())
{
using (var isfs = new IsolatedStorageFileStream("myfile.txt", FileMode.OpenOrCreate, store))
{
using (var sw = new StreamWriter(isfs))
{
sw.Write("anything really. Here it's just a string but could be a serialized object, etc.");
sw.Close();
}
}
}
You can then read the file with:
var result = string.Empty;
try
{
using (var store = IsolatedStorageFile.GetUserStoreForApplication())
{
if (!store.FileExists("myfile.txt"))
{
return result;
}
using (var isfs = new IsolatedStorageFileStream("myfile.txt", FileMode.Open, store))
{
using (var sr = new StreamReader(isfs))
{
string lineOfData;
while ((lineOfData = sr.ReadLine()) != null)
{
result += lineOfData;
}
}
}
}
}
catch (IsolatedStorageException)
{
result = string.Empty; // may have partial data/file before error
}
return result;
You can use
StreamReader.Writeline
and
StreamRead.ReadLine
to write and read blocks of text seperated by line feeds.

Resources