I am trying to load trained xgboost model to be used in custom UDF written in Java. File is in zip format and stored in hdfs.
I have tried to read it using Path class but it's not working.
import org.apache.hadoop.fs.Path;
public EasyPredictModelWrapper loadModel(String xgBoostModelFile) {
if (model == null) {
synchronized (_lockObject) {
if (model == null) {
log.info("Model has not been loaded, loading ...");
try {
Path path = new Path(xgBoostModelFile);
model = new EasyPredictModelWrapper(MojoModel.load(path)); // Doesn't compile since MojoModel only takes string as an input.
} catch (IOException e) {
log.error("Got an exception while trying to load xgBoostModel \n", e);
}
}
}
}
return model;
}
I Want to successfully load model.zip
Got answer in H20 slack community.
FileSystem fs = FileSystem.get(new Configuration());
Path path = new Path(xgBoostModelFile);
FSDataInputStream inputStream = fs.open(path);
MojoReaderBackend mojoReaderBackend = MojoReaderBackendFactory.createReaderBackend(inputStream,CachingStrategy.MEMORY);
model = new EasyPredictModelWrapper(MojoModel.load(mojoReaderBackend));
Related
I have a REST API in Spring Boot Application that takes in a param of type Multipart file.
There is possibility that user may import either CSV file or Excel(.xlsx / .xsl) file of huge size which needs to be handled.
I am using Apache POI to read Excel type file and it is working fine. To my existing code, how do I efficiently handle CSV file reading also
Below is Excel file Reading Code:
#RequestMapping(value="/read", method = RequestMethod.POST)
#Transactional
public Map<String, String> read(#RequestParam("file") MultipartFile file) {
Map<String, String> response = new ArrayList();
if (!file.isEmpty()) {
ByteArrayInputStream stream;
Workbook wb;
StringBuilder contentSb = new StringBuilder();
try {
stream = new ByteArrayInputStream(file.getBytes());
wb = WorkbookFactory.create(stream);
org.apache.poi.ss.usermodel.Sheet sheet = wb.getSheetAt(wb.getActiveSheetIndex());
Iterator<Row> rowIterator = sheet.rowIterator();
System.out.println("Processing Excel file");
for (int rowIndex = 0; rowIndex <= sheet.getLastRowNum(); rowIndex++) {
Row row = sheet.getRow(rowIndex);
if (row != null) {
Cell cell = row.getCell(0);
if (cell != null) {
contentSb.append(cell.getStringCellValue()+",");
}
}
}
System.out.println("Processed Excel file");
return response;
} catch (Exception e) {
e.printStackTrace();
}
}
else {
return response;
}
}
Thank you in advance!
I'm using MEF in order to execute plugins code in my project.
1. I'm loading my dll sources :
public void AssembleComponents()
{
try
{
//Creating an instance of aggregate catalog. It aggregates other catalogs
var aggregateCatalog = new AggregateCatalog();
//Build the directory path where the parts will be available
var directoryPath =
string.Concat(Path.GetDirectoryName(Assembly.GetExecutingAssembly().CodeBase)
.Split('\\').Reverse().Skip(4).Reverse().Aggregate((a, b) => a + "\\" + b)
, "\\", "ExportComponents\\Components");
string localPath = new Uri(directoryPath).LocalPath;
//Load parts from the available dlls in the specified path using the directory catalog
var directoryCatalog = new DirectoryCatalog(localPath, "*.dll");
//Load parts from the current assembly if available
var asmCatalog = new AssemblyCatalog(Assembly.GetExecutingAssembly());
//Add to the aggregate catalog
aggregateCatalog.Catalogs.Add(directoryCatalog);
aggregateCatalog.Catalogs.Add(asmCatalog);
//Crete the composition container
var container = new CompositionContainer(aggregateCatalog);
// Composable parts are created here i.e. the Import and Export components assembles here
container.ComposeParts(this);
}
catch (Exception ex)
{
throw ex;
}
}
I'm going through my plugins and executing a "Validate" method :
public List<string> Validate(string operationType)
{
List<string> res = null;
foreach (System.Lazy<IValidationRules, IPluginMetadata> plugin in ChekcsPlugins)
{
if (plugin.Metadata.DisplayName == operationType)
{
res = plugin.Value.Validate();
break;
}
}
return res;
}
I'm know how to export back a returned value once the "Validate" is done but what i need is to return values at run time back during the method execution.
Is it possible?
I searched around for this but I could not find a soultion.
Sorry about my bad description. Im not very good at this.
I have a UI class
Its calling a "lotto" class.
That lotto classes constructor is called a method named readData()
readData is reading from a file using BufferedReader
Im not getting an error message but its just not reading.
It gets stuck at BufferedReader fr = new BufferedReader... and goes to the catch thing.
If its a file not found problem how would i make it track where my file is. Im using eclipse and the program is stored on my usb. I need to hand it in to my teacher so i cant just put a location in. Is there code that tracks where my program is then takes the file from that folder?
Here is the code being used.
import java.io.*;
//contructor
public Lotto()
{
try
{
readData();
nc = new NumberChecker();
}
catch(IOException e)
{
System.out.println("There was a problem");
}
}
private void readData() throws IOException
{
//this method reads winning tickets date and pot from a file
BufferedReader file = new BufferedReader (new FileReader("data.txt"));
for(int i=0;i<5;i++)
{
System.out.println("in "+i);
winningNums[i] = file.readLine();
winningDates[i] = file.readLine();
weeksMoney[i] = Integer.parseInt(file.readLine());
System.out.println("out "+i);
}
file.close();
}
if you get an error in this line of code
BufferedReader file = new BufferedReader (new FileReader("data.txt"));
Then it is probably a FileNotFoundException
Make sure that the data.txt file is in the same folder as your compiled .class file and not the .java source.
It would be best to use a proper root to your file ex. c:\my\path\data.txt
And don't forget the \
Try surrounding the BufferedReader in a try catch and look for a file not found exception as well as IO exception. Also try putting in the fully qualified path name with double backslashes.
BufferedReader file;
try {
file = new BufferedReader (new FileReader("C:\\filepath\\data.txt"));
for(int i=0;i<5;i++)
{
System.out.println("in "+i);
winningNums[i] = file.readLine();
winningDates[i] = file.readLine();
weeksMoney[i] = Integer.parseInt(file.readLine());
System.out.println("out "+i);
}
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
because use zk upload component to upload a image,then insert the context path of the image to the CKEditor is too complex,
and at http://ckeditor.com/demo, you can see that CKEditor can upload image and flash etc,
but in zk, the CKEditor don't have this feature,
is that mean CKEditor in zk can't upload file?
I'm afraid this is not possible with zk.
I wrote a workaround to do this. You have to add a button to your GUI and add this EventListener to the button:
private class onUpload implements EventListener
{
#Override
public void onEvent(Event event) throws Exception
{
Media media = ((UploadEvent) event).getMedia();
if (media.getContentType().contains("image"))
{
reader.upload(media.getStreamData(), media.getName());
String description = edDescription.getValue();
description += "<img alt=\"\" src=\"/" + media.getName() + "\" />";
edDescription.setValue(description);
}
else
{
new Messagebox().show(_T("You can only upload images!"), _T("Not an image!"), Messagebox.OK, Messagebox.ERROR);
}
}
}
Reader is my class which handles file transfers and is used to write the data to the docroot. In my case the docroot of glassfish 3.1 can be located with the following code. I wrote the method getDocFolder() for ist because it also adds subfolders for each user if they don't already exists.
File file = new File("../docroot/");
This is the code for the upload method of the reader:
InputStream inputStream = null;
try
{
inputStream = new ByteArrayInputStream(imageStream);
String filename = getDocFolder()+"/"+imageName;
File file = new File(filename);
OutputStream out=new FileOutputStream(file);
byte buf[]=new byte[1024];
int len;
while((len = inputStream.read(buf)) > 0)
out.write(buf,0,len);
out.close();
inputStream.close();
}
catch (Exception ex)
{
Logger.getLogger(ImageReader.class.getName()).log(Level.SEVERE, "Error writing image", ex);
}
finally
{
try
{
inputStream.close();
}
catch (IOException ex) {}
}
I hope this helps
I want to get the XML in atom format of a GoogleDocs spreadsheet using the [generateAtom(..,..)][1] method of the class BaseEntry which a SpreadsheetEntry inherits. But I don't understand the the second parameter in the method, ExtensionProfile. What is it and will this method call suffice if I just want to get the XML in atom format?
XmlWriter x = new XmlWriter();
spreadSheetEntry.generateAtom(x,new ExtensionProfile());
[1]: http://code.google.com/apis/gdata/javadoc/com/google/gdata/data/BaseEntry.html#generateAtom(com.google.gdata.util.common.xml.XmlWriter, com.google.gdata.data.ExtensionProfile)
From the JavaDoc for ExtensionProfile:
A profile is a set of allowed
extensions for each type together with
additional properties.
Usually if you've got a service, you can ask that for its extension profile using Service.getExtensionProfile().
Elaborating Jon Skeet's answer, you need to instanciate a service like this:
String developer_key = "mySecretDeveloperKey";
String client_id = "myApplicationsClientId";
YouTubeService service = new YouTubeService(client_id, developer_key);
Then you can write to a file using the extension profile of your service:
static void write_video_entry(VideoEntry video_entry) {
try {
String cache_file_path = Layout.get_cache_file_path(video_entry);
File cache_file = new File(cache_file_path);
Writer writer = new FileWriter(cache_file);
XmlWriter xml_writer = new XmlWriter(writer);
ExtensionProfile extension_profile = service.getExtensionProfile();
video_entry.generateAtom(xml_writer, extension_profile);
xml_writer.close();
writer.close();
} catch (IOException e) {
e.printStackTrace();
}
}
Analogously, you can read a file using the extension profile of your service:
static VideoFeed read_video_feed(File cache_file_file) {
VideoFeed video_feed = new VideoFeed();
try {
InputStream input_stream = new FileInputStream(cache_file_file);
ExtensionProfile extension_profile = service.getExtensionProfile();
try {
video_feed.parseAtom(extension_profile, input_stream);
} catch (ParseException e) {
e.printStackTrace();
}
input_stream.close();
} catch (IOException e) {
e.printStackTrace();
}
return video_feed;
}