How to return File downloaded as DataBuffer - spring

I am downloading a file like below:
private File downloadAndReturnFile(String fileId, String destination) {
log.info("Downloading file.. " + fileId);
Path path = Paths.get(destination);
Flux<DataBuffer> dataBuffer = webClient.get().uri("/the/download/uri/" + fileId + "").retrieve()
.bodyToFlux(DataBuffer.class)
.doOnComplete(() -> log.info("{}", fileId + " - File downloaded successfully"));
//DataBufferUtils.write(dataBuffer, path, StandardOpenOption.CREATE).share().block();
return ???? // What should I do here to return above DataBuffer as file?
}
How can I return the dataBuffer as a File? Or, how can I convert this dataBuffer to a File object?

You could use DataBufferUtils.write method. To do that, you should
instantiate a File object (maybe using fileId and destination) which is also the return value you want
create OutputStream or Path or Channels object from the File object
call DataBufferUtils.write(dataBuffer, ....).share().block() to write the DataBuffer into the file
viz. (all thrown exceptions are omitted),
...
File file = new File(destination, fileId);
Path path = file.toPath();
DataBufferUtils.write(dataBuffer, path, StandardOpenOption.CREATE).share().block();
return file;
Or,
...
Path path = Paths.get(destination);
DataBufferUtils.write(dataBuffer, path, StandardOpenOption.CREATE).share().block();
return path.toFile();

In case someone looking for a full code. Just sharing Tonny's solution to the fullest.
private File downloadAndReturnFile(String fileId, String destination) throws FileNotFoundException {
log.info("Downloading file.. " + fileId);
Flux<DataBuffer> dataBuffer = webClient.get().uri("/the/download/uri/" + fileId + "").retrieve()
.bodyToFlux(DataBuffer.class)
.doOnComplete(() -> log.info("{}", fileId + " - File downloaded successfully"));
Path path = Paths.get(destination);
DataBufferUtils.write(dataBuffer, path, StandardOpenOption.CREATE).share().block();
return path.toFile();
}

Related

Unable to upload files greater than 7KB via ajax call to servlet over https

I am running JBoss 6.3 portal and have deployed a war file containing following two files
1) DocUpload.jsp
Containing following code snippet making an ajax call to send the mentioned fields in data2 along with file object fd.
fd.append('file', document.getElementById('file1').files[0]);
data2 = encodeURIComponent(document.getElementById("lob").value)+'#'+encodeURIComponent(document.getElementById("loantype").value)+
'#'+encodeURIComponent(document.getElementById('docType').value)+'#'+encodeURIComponent(document.getElementById('docName').value)+
'#'+encodeURIComponent(document.getElementById('entity').value)+'#'+encodeURIComponent(document.getElementById('userName').value)+
'#'+encodeURIComponent(document.getElementById('PartyName').value)+'#'+encodeURIComponent(document.getElementById('loanAccount').value)+
'#'+encodeURIComponent(document.getElementById('LoanAmount').value)+'#'+encodeURIComponent(e4)+'#'+encodeURIComponent(document.getElementById('hiddenWIName').value)+'#'+encodeURIComponent(e3);
alert("data2 "+data2);
var ret = doPostAjax("${pageContext.request.contextPath}/AddDocumentsServlet?data="+data2,fd);
2) AddDocumentsServlet.java
Containing code for handling the request
File path = new File(RootFolderPath + File.separator + "Portal_TmpDoc" + File.separator + todayAsString + File.separator + lMilliSecondsCurrent);
UploadPath = path.getAbsolutePath();
if (!path.isDirectory()) {
path.mkdirs();
}
if (isMultipart) {
System.out.println("Inside if isMultipart");
DiskFileItemFactory factory = new DiskFileItemFactory();
factory.setSizeThreshold(MEMORY_THRESHOLD);
factory.setRepository(new File(System.getProperty("java.io.tmpdir")));
ServletFileUpload upload = new ServletFileUpload(factory);
upload.setFileSizeMax(MAX_FILE_SIZE);
upload.setSizeMax(MAX_REQUEST_SIZE);
try {
//List multiparts = upload.parseRequest(request);
System.out.println("Before parsing request");
List<FileItem> multiparts = upload.parseRequest(request);
System.out.println("multiparts :::"+multiparts);
for (Iterator iterator = multiparts.iterator(); iterator.hasNext();) {
FileItem item = (FileItem) iterator.next();
logger.info(item);
if (!item.isFormField()) {
String fileobject = item.getFieldName();
System.out.println(request.getParameter("data"));
String[] fileArray = request.getParameter("data").split("#");
name = new File(item.getName()).getName();
name = name.substring(name.lastIndexOf(File.separatorChar) + 1);
ext = name.substring(name.lastIndexOf(".") + 1);
logger.info(name);
File directory = new File(UploadPath);
File[] afile;
int j = (afile = directory.listFiles()).length;
for (int i = 0; i < j; i++) {
File f = afile[i];
if (f.getName().startsWith(filename))
f.delete();
}
item.write(new File(UploadPath + File.separator + filename + "." + ext));
System.out.println("File Uploaded");
}
}
My problem is when I use http connection for the above requests and getting session the program runs just fine. But while using https and uploading file greater than 7 KB the page becomes unresponsive.
On further analysis I found that the program flow gets stuck on this line
List<FileItem> multiparts = upload.parseRequest(request);
and despite having this line in a try block no exception is caught.

FSDataOutputStream.writeUTF() adds extra characters at the start of the data on hdfs. How to avoid this extra data?

What I am trying to is to convert a sequence file on hdfs which has xml data into .xml files on hdfs.
Searched on Google and found the below code. I made modifications according to my need and the following is the code..
public class SeqFileWriterCls {
public static void main(String args[]) throws Exception {
System.out.println("Reading Sequence File");
Path path = new Path("seq_file_path/seq_file.seq");
Configuration conf = new Configuration();
FileSystem fs = FileSystem.get(conf);
SequenceFile.Writer writer = null;
SequenceFile.Reader reader = null;
FSDataOutputStream fwriter = null;
OutputStream fowriter = null;
try {
reader = new SequenceFile.Reader(fs, path, conf);
//writer = new SequenceFile.Writer(fs, conf,out_path,Text.class,Text.class);
Writable key = (Writable) ReflectionUtils.newInstance(reader.getKeyClass(), conf);
Writable value = (Writable) ReflectionUtils.newInstance(reader.getValueClass(), conf);
while (reader.next(key, value)) {
//i am just editing the path in such a way that key will be my filename and data in it will be the value
Path out_path = new Path(""+key);
String string_path = out_path.toString();
String clear_path=string_path.substring(string_path.lastIndexOf("/")+1);
Path finalout_path = new Path("path"+clear_path);
System.out.println("the final path is "+finalout_path);
fwriter = fs.create(finalout_path);
fwriter.writeUTF(value.toString());
fwriter.close();
FSDataInputStream in = fs.open(finalout_path);
String s = in.readUTF();
System.out.println("file has: -" + s);
//fowriter = fs.create(finalout_path);
//fowriter.write(value.toString());
System.out.println(key + " <===> :" + value.toString());
System.exit(0);
}
} catch (Exception e) {
e.printStackTrace();
} finally {
IOUtils.closeStream(reader);
fs.close();
}
}
I am using "FSDataOutputStream" to write the data to HDFS and the method is used is "writeUTF" The issue is that when i write to the hdfs file some additional characters are getting in the starting of data. But when i print the data i couldnt see the extra characters.
i tried using writeChars() but even taht wont work.
is there any way to avoid this?? or is there any other way to write the data to HDFS???
please help...
The JavaDoc of the writeUTF(String str) method says the followings:
Writes a string to the underlying output stream using modified UTF-8 encoding in a machine-independent manner.
First, two bytes are written to the output stream as if by the writeShort method giving the number of bytes to follow. This value is the number of bytes actually written out, not the length of the string. Following the length, each character of the string is output, in sequence, using the modified UTF-8 encoding for the character. (...)
Both the writeBytes(String str) and writeChars(String str) methods should work fine.

How to obtain file name from the path of the file stored in the distributed cache of HDFS

public void configure(JobConf job) {
inputFile = job.get("map.input.file");
Path[] cacheFiles = new Path[2];
try {
Path file0 = DistributedCache.getLocalCacheFiles(job)[0];
Path file1 = DistributedCache.getLocalCacheFiles(job)[1];
} catch (IOException ioe) {
System.err.println("Caught exception while getting cached files: " + StringUtils.stringifyException(ioe));
}
}
I am writing the code in the configure function.
Now how do I get the file names from the Paths file0 and file1? I need the file names because I need to store the data from both the files into two separate Hashmaps.
Try following:
Path file0 = DistributedCache.getLocalCacheFiles(job)[0];
Path file1 = DistributedCache.getLocalCacheFiles(job)[1];
String filename0 = file0.getName();
String filename1 = file1.getName();

how to add a folder in the apk

I wanna to know how to add a folder in the apk.don't give me a solution about using the compress software directly. The apk was recompiled by the 'apktool.jar'. I need some code to achieve this problem. Hope one can solve my question as soon as possible.Thank you~
public static int zipMetaInfFolderToApk(String apkName, String folderName) throws IOException {
if(!new File(apkName).exists()){
return ConstantValue.ISQUESTION;
}
String zipName = apkName.substring(0, apkName.lastIndexOf(".")) + "."
+ "zip";
String bak_zipName = apkName.substring(0, apkName.lastIndexOf("."))
+ "_bak." + "zip";
FileUtils.renameFile(apkName, zipName);
ZipFile war = new ZipFile(zipName);
ZipOutputStream append = new ZipOutputStream(new FileOutputStream(
bak_zipName));
Enumeration<? extends ZipEntry> entries = war.entries();
while (entries.hasMoreElements()) {
ZipEntry e = entries.nextElement();
System.out.println("copy: " + e.getName());
append.putNextEntry(e);
if (!e.isDirectory()) {
copy(war.getInputStream(e), append);
}
append.closeEntry();
}
String name = "";
if(folderName.equals("")){
name = "META-INF/" + folderName;
}else{
name = "META-INF/" + folderName + "/";
}
ZipEntry e = new ZipEntry(name);
try{
append.putNextEntry(e);
}catch(ZipException e1){
append.closeEntry();
war.close();
append.close();
FileUtils.renameFile(zipName,apkName);
FileUtils.deleteFolder(bak_zipName);
e1.printStackTrace();
return ConstantValue.ISQUESTION;
}
append.closeEntry();
war.close();
append.close();
FileUtils.deleteFolder(zipName);
FileUtils.renameFile(bak_zipName, apkName);
return ConstantValue.ISNORMAL;
}

How to get the Uploaded Files in MVC3?

Hi all i am uploading the files uploaded by the user in this path
string savefilename = Path.Combine(Server.MapPath("~/Content/UploadedFiles/"),
Path.GetFileName());
And i am saving the Url in the Database in the Url Column in this
~/Content/UploadedFiles/BugTrackerDataBase.xlsx
and i am trying to retrieve the file Uploaded by the user by a link in my grid view
my retrieve method looks like this
public ActionResult ViewAttachments(string AttachmentName)
{
try
{
AttachmentName = Session["AttachmentUrl"].ToString();
var fs = System.IO.File.OpenRead(Server.MapPath("'" + AttachmentName + "'"));
return File(fs, "application/doc", AttachmentName);
}
catch
{
throw new HttpException(404, "Couldn't find " + AttachmentName);
}
}
and i have the Excepiton
"Could not find a part of the path 'D:\AnilWork\BugTracker\BugTracker\ViewBug\'UploadedFiles\BugTrackerDataBase.xlsx''."
can any one tell me where am i doing wrong or the write procedure to do this
That is because you have " ' " in your path.
\BugTracker\ViewBug\'UploadedFiles\BugTrackerDataBase.xlsx''
Remove them an it should work. Like this
var fs = System.IO.File.OpenRead(Server.MapPath(AttachmentName));
try
var fs = System.IO.File.OpenRead(Server.MapPath(" + AttachmentName + "));
instead of
var fs = System.IO.File.OpenRead(Server.MapPath("'" + AttachmentName + "'"));
it shoud be replaced with (Server.MapPath(""+ AttachmentName + ""))

Resources