how to save CoreDocument in Stanford nlp to disk 2 - stanford-nlp

Followed Professor Manning's suggestion to use the ProtobufAnnotationSerializer and did something wrong.
used serializer.writeCoreDocument on the correctly working document; Later read written file with pair = serializer.read; then used pair.second InputStream p2 = pair.second; p2 was empty resulting in a null pointer when running Pair pair3 = serializer.read(p2);
public void writeDoc(CoreDocument document, String filename ) {
AnnotationSerializer serializer = new
ProtobufAnnotationSerializer();
FileOutputStream fos = null;
try {
OutputStream ks = new FileOutputStream(filename);
ks = serializer.writeCoreDocument(document, ks);
ks.flush();
ks.close();
}catch(IOException ioex) {
logger.error("IOException "+ioex);
}
}
public void ReadSavedDoc(String filename) {
// Read
byte[]kb = null;
try {
File initialFile = new File(filename);
InputStream ks = new FileInputStream(initialFile);
ProtobufAnnotationSerializer serializer = new
ProtobufAnnotationSerializer();
InputStream kis = new
ByteArrayInputStream(ks.readAllBytes());
ks.close();
Pair<Annotation, InputStream> pair = serializer.read(kis);
InputStream p2 = pair.second;
int nump2 = p2.available();
logger.info(nump2);
byte[] ba = p2.readAllBytes();
Annotation readAnnotation = pair.first;
Pair<Annotation, InputStream> pair3 = serializer.read(p2);
kis.close();
} catch (IOException e) {
e.printStackTrace();
} catch (ClassNotFoundException e) {
e.printStackTrace();
} catch (ClassCastException e) {
e.printStackTrace();
} catch(Exception ex) {
logger.error("Exception: "+ex);
ex.printStackTrace();
}
}

This line is unnecessary and should be deleted:
Pair<Annotation, InputStream> pair3 = serializer.read(p2);
If you have set up readAnnotation correctly that's the end of the read/write process. p2 is empty because you have read all its contents already.
There is a clear example of how to use serialization here:
https://github.com/stanfordnlp/CoreNLP/blob/master/itest/src/edu/stanford/nlp/pipeline/ProtobufSerializationSanityITest.java
You will have to also build a CoreDocument from an Annotation.
CoreDocument readDocument = new CoreDocument(readAnnotation);

Related

is it possible to read the content of the file present in the ftp server? [duplicate]

This is re-worded from a previous question (which was probably a bit unclear).
I want to download a text file via FTP from a remote server, read the contents of the text file into a string and then discard the file. I don't need to actually save the file.
I am using the Apache Commons library so I have:
import org.apache.commons.net.ftp.FTPClient;
Can anyone help please, without simply redirecting me to a page with lots of possible answers on?
Not going to do the work for you, but once you have your connection established, you can call retrieveFile and pass it an OutputStream. You can google around and find the rest...
FTPClient ftp = new FTPClient();
...
ByteArrayOutputStream myVar = new ByteArrayOutputStream();
ftp.retrieveFile("remoteFileName.txt", myVar);
ByteArrayOutputStream
retrieveFile
Normally I'd leave a comment asking 'What have you tried?'. But now I'm feeling more generous :-)
Here you go:
private void ftpDownload() {
FTPClient ftp = null;
try {
ftp = new FTPClient();
ftp.connect(mServer);
try {
int reply = ftp.getReplyCode();
if (!FTPReply.isPositiveCompletion(reply)) {
throw new Exception("Connect failed: " + ftp.getReplyString());
}
if (!ftp.login(mUser, mPassword)) {
throw new Exception("Login failed: " + ftp.getReplyString());
}
try {
ftp.enterLocalPassiveMode();
if (!ftp.setFileType(FTP.BINARY_FILE_TYPE)) {
Log.e(TAG, "Setting binary file type failed.");
}
transferFile(ftp);
} catch(Exception e) {
handleThrowable(e);
} finally {
if (!ftp.logout()) {
Log.e(TAG, "Logout failed.");
}
}
} catch(Exception e) {
handleThrowable(e);
} finally {
ftp.disconnect();
}
} catch(Exception e) {
handleThrowable(e);
}
}
private void transferFile(FTPClient ftp) throws Exception {
long fileSize = getFileSize(ftp, mFilePath);
InputStream is = retrieveFileStream(ftp, mFilePath);
downloadFile(is, buffer, fileSize);
is.close();
if (!ftp.completePendingCommand()) {
throw new Exception("Pending command failed: " + ftp.getReplyString());
}
}
private InputStream retrieveFileStream(FTPClient ftp, String filePath)
throws Exception {
InputStream is = ftp.retrieveFileStream(filePath);
int reply = ftp.getReplyCode();
if (is == null
|| (!FTPReply.isPositivePreliminary(reply)
&& !FTPReply.isPositiveCompletion(reply))) {
throw new Exception(ftp.getReplyString());
}
return is;
}
private byte[] downloadFile(InputStream is, long fileSize)
throws Exception {
byte[] buffer = new byte[fileSize];
if (is.read(buffer, 0, buffer.length)) == -1) {
return null;
}
return buffer; // <-- Here is your file's contents !!!
}
private long getFileSize(FTPClient ftp, String filePath) throws Exception {
long fileSize = 0;
FTPFile[] files = ftp.listFiles(filePath);
if (files.length == 1 && files[0].isFile()) {
fileSize = files[0].getSize();
}
Log.i(TAG, "File size = " + fileSize);
return fileSize;
}
You can just skip the download to local filesystem part and do:
FTPClient ftpClient = new FTPClient();
try {
ftpClient.connect(server, port);
ftpClient.login(user, pass);
ftpClient.enterLocalPassiveMode();
ftpClient.setFileType(FTP.BINARY_FILE_TYPE);
InputStream inputStream = ftpClient.retrieveFileStream("/folder/file.dat");
BufferedReader reader = new BufferedReader(new InputStreamReader(inputStream, "Cp1252"));
while(reader.ready()) {
System.out.println(reader.readLine()); // Or whatever
}
inputStream.close();
} catch (IOException ex) {
ex.printStackTrace();
} finally {
try {
if (ftpClient.isConnected()) {
ftpClient.logout();
ftpClient.disconnect();
}
} catch (IOException ex) {
ex.printStackTrace();
}
}

how to copy OdadataSourceHandle from one design report to another design report?

String targetRptFile = System.getenv("LOCAL_BIRT_INPUT") + "/"+report.getMergeRptTemplates().getTargetTemplate()+".rptdesign";
String attachRptFile = System.getenv("LOCAL_BIRT_INPUT") + "/"+report.getMergeRptTemplates()
.getAttachTemplate()+".rptdesign";
DesignConfig dConfig = new DesignConfig();
IDesignEngineFactory factory = (IDesignEngineFactory) Platform.createFactoryObject(IDesignEngineFactory.EXTENSION_DESIGN_ENGINE_FACTORY);
IDesignEngine dEngine = factory.createDesignEngine(dConfig);
SessionHandle session = dEngine.newSessionHandle(null);
ReportDesignHandle targetRptDesignHandle = null;
ReportDesignHandle attachRptDesignHandle = null;
try {
targetRptDesignHandle = session.openDesign(targetRptFile);
attachRptDesignHandle = session.openDesign(attachRptFile);
} catch (DesignFileException e) {
e.printStackTrace();
}
OdaDataSourceHandle attachOdaDataSourceHandle = (OdaDataSourceHandle)attachRptDesignHandle.getAllDataSources()
.get(0);
targetRptDesignHandle.getAllDataSources().add(attachOdaDataSourceHandle);
String newTargetRptFile = System.getenv("LOCAL_BIRT_INPUT") + "/"+report
.getMergeRptTemplates().getTargetTemplate()+"-merge"+".rptdesign";
try {
targetRptDesignHandle.saveAs(newTargetRptFile);
targetRptDesignHandle.close();
attachRptDesignHandle.close();
session.closeAll(true);
} catch (IOException e) {
e.printStackTrace();
}
}
Never did this, but I remember a similar problem when I wanted to copy a JDBCDataSource. I worked around this by setting all the properties manually instead of trying to copy the whole DS.

Hadoop compression

I was trying to compress a file using the following code. The compression works fine when the size of the file is small(say 1 GB). But when the size of the file is around 5GB the program does not fail rather it keeps on running for 2 days with out any result.
Based on the info message I get it seems like cluster issue although I am not sure enough.
Following is the code the error I am getting:
Error
Code I am using
public void compressData(final String inputFilePath,final String outputPath) throws DataFabricAppendException {
CompressionOutputStream compressionOutputStream = null;
FSDataOutputStream fsDataOutputStream = null;
FSDataInputStream fsDataInputStream = null;
CompressionCodec compressionCodec = null;
CompressionCodecFactory compressionCodecFactory = null;
try {
compressionCodecFactory = new CompressionCodecFactory(conf);
final Path compressionFilePath = new Path(outputPath);
fsDataOutputStream = fs.create(compressionFilePath);
compressionCodec = compressionCodecFactory
.getCodecByClassName(BZip2Codec.class.getName());
compressionOutputStream = compressionCodec
.createOutputStream(fsDataOutputStream);
fsDataInputStream = new FSDataInputStream(fs.open(new Path(
inputFilePath)));
IOUtils.copyBytes(fsDataInputStream, compressionOutputStream, conf,
false);
compressionOutputStream.finish();
} catch (IOException ex) {
throw new DataFabricAppendException(
"Error while compressing non-partitioned file : "
+ inputFilePath, ex);
} catch (Exception ex) {
throw new DataFabricAppendException(
"Error while compressing non-partitioned file : "
+ inputFilePath, ex);
} finally {
try {
if (compressionOutputStream != null) {
compressionOutputStream.close();
}
if (fsDataInputStream != null) {
fsDataInputStream.close();
}
if (fsDataOutputStream != null) {
fsDataOutputStream.close();
}
} catch (IOException e1) {
LOG.warn("Could not close necessary objects");
}
}
}

File is getting locked on read/write operation in C#

A third party application is reading the output text file of my windows application,it is getting paused.I think,it is happening because of file gets locked either on writing or reading.how can i make this work fine.Please see my code.
List<string> fileList = new List<string>();
System.Timers.Timer timer;
string currentfilename;
string currentcontent;
public Service1()
{
//
timer = new System.Timers.Timer();
timer.AutoReset = false;
timer.Elapsed += new System.Timers.ElapsedEventHandler(DoStuff);
}
void timer_Elapsed(object sender, System.Timers.ElapsedEventArgs e)
{
throw new NotImplementedException();
}
private void DoStuff(object sender, System.Timers.ElapsedEventArgs e)
{
DateTime LastChecked = DateTime.Now;
try
{
string[] files = System.IO.Directory.GetFiles(#"C:\Journal", "*.jrn", System.IO.SearchOption.AllDirectories);
foreach (string file in files)
{
if (!fileList.Contains(file))
{
currentfilename = file;
fileList.Add(file);
copywarehouse(file);
try
{
string sourcePath = #"C:\Journal";
string sourceFile = System.IO.Path.Combine(sourcePath, file);
using (FileStream fs= new FileStream(sourceFile, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
{
StreamReader sr = new StreamReader(fs);
currentcontent = sr.ReadToEnd();
sr.Close();
}
}
catch (Exception ex)
{
throw (ex);
}
}
}
try
{
string sourcePath1 = #"C:\Journal";
string currentfilecontent;
string sourceFile1 = System.IO.Path.Combine(sourcePath1, currentfilename);
using (FileStream fs = new FileStream(sourceFile1, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
{
StreamReader sr = new StreamReader(fs);
currentfilecontent = sr.ReadToEnd();
sr.Close();
}
if (currentfilecontent != currentcontent)
{
if (currentfilecontent.Contains(currentcontent))
{
string originalcontent = currentfilecontent.Substring(currentcontent.Length);
File.WriteAllText(#"C:\Journal\tempfile.txt", originalcontent + "\r\n");
using (FileStream fs = new FileStream(#"C:\Journal\tempfile.txt", FileMode.OpenOrCreate))
{
StreamWriter sw = new StreamWriter(fs);
sw.Write(originalcontent);
sw.Close();
}
currentcontent = currentfilecontent;
}
}
}
//}
catch (Exception ex)
{
throw (ex);
}
TimeSpan ts = DateTime.Now.Subtract(LastChecked);
TimeSpan MaxWaitTime = TimeSpan.FromMilliseconds(100);
if (MaxWaitTime.Subtract(ts).CompareTo(TimeSpan.Zero) > -1)
timer.Interval = MaxWaitTime.Subtract(ts).Milliseconds;
else
timer.Interval = 1;
timer.Start();
}
catch (Exception ex)
{
throw (ex);
}
}
private void copywarehouse(string filename)
{
string sourcePath = #"C:\Journal";
string targetPath = #"C:\Journal";
string copycontent=null;
try
{
string sourceFile = System.IO.Path.Combine(sourcePath, filename);
string destFile = System.IO.Path.Combine(targetPath, "tempfile.txt");
using (FileStream fs = new FileStream(sourceFile, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
{
StreamReader sr = new StreamReader(fs);
copycontent = sr.ReadToEnd();
sr.Close();
}
using (FileStream fs = new FileStream(destFile, FileMode.Append))
{
StreamWriter sw = new StreamWriter(fs);
sw.Write(copycontent);
sw.Close();
}
}
catch (Exception ex)
{
throw (ex);
}
}
Here the problem is with third party application which is reading my application output as input.it is not allowing access to more than one file content..

How to make a save action that checks whether a 'save-as' has already been performed

I have researched and tried to refer back to my fileChooser.getSeletedFile() in my save as action but can not work out how to check whether or not a file has been created. Here is my attempted code so far:
Save as code(works well):
public void Save_As() {
fileChooserTest.setApproveButtonText("Save");
int actionDialog = fileChooserTest.showOpenDialog(this);
File fileName = new File(fileChooserTest.getSelectedFile() + ".txt");
try {
if (fileName == null) {
return;
}
BufferedWriter outFile = new BufferedWriter(new FileWriter(fileName));
outFile.write(this.jTextArea2.getText());//put in textfile
outFile.flush(); // redundant, done by close()
outFile.close();
} catch (IOException ex) {
}
}
"Save" code doesn't work:
private void SaveActionPerformed(java.awt.event.ActionEvent evt) {
File f = fileChooserTest.getSelectedFile();
try {
if (f.exists()) {
BufferedWriter bw1 = new BufferedWriter(new FileWriter(fileChooserTest.getSelectedFile() + ".txt"));
bw1 = new BufferedWriter(new FileWriter(fileChooserTest.getSelectedFile() + ".txt"));
String text = ((JTextArea) jTabbedPane1.getSelectedComponent()).getText();
bw1.write(text);
bw1.close();
} else {
Save_As();
}
} catch (Exception ex) {
ex.printStackTrace();
}
}
Instead of storing an instance to the JFileChooser rather store an instance to the File (wich will be null before any save has been performed). In your SaveActionPerformed method check if the file is null. If it is null then do a Save_As and store the selected file in your file variable, if it is not null then do a normal save into the file.

Resources