I am aware of the TransferManager and the .uploadFileList() and .uploadFileDirectory() methods, however they accept java.io.File types as arguments. I have a collection of byte array input streams containing jpeg image data. I don't want to create in-memory files to store this data before I upload it either.
So what I need is essentially what the S3 client's PutObjectRequest does but for a collection of InputStream objects. Also, if one upload fails, I want to abort the whole thing and not upload anything, much like how a database transaction will reverse the changes if something goes wrong along the way.
Is this possible with the Java SDK?
Before I share an answer, please consider upgrading...
fyi - TransferManager is deprecated, now supported as TransferManagerBuilder in JAVA AWS SDK, please consider upgrading if TransferManagerBuilder Object suits your needs.
now since you asked about TransferManager, you could either 1) copy the code below and replace the functionality/arguments with your custom in memory handling of the input stream and handle it in your custom function... or; 2) further below is another sample, try to use this as-is...
Github source modify with with inputstream and issue listed here
private def uploadFile(is: InputStream, s3ObjectName: String, metadata: ObjectMetadata) = {
try {
val putObjectRequest = new PutObjectRequest(bucketName, s3ObjectName,
is, metadata)
// TransferManager supports asynchronous uploads and downloads
val upload = transferManager.upload(putObjectRequest)
upload.addProgressListener(ExceptionReporter.wrap(UploadProgressListener(putObjectRequest)))
} catch {
case e: Exception => throw new RuntimeException(e)
}
}
Bonus, Nice custom answer here using sequence input streams
public void combineFiles() {
List<String> files = getFiles();
long totalFileSize = files.stream()
.map(this::getContentLength)
.reduce(0L, (f, s) -> f + s);
try {
try (InputStream partialFile = new SequenceInputStream(getInputStreamEnumeration(files))) {
ObjectMetadata resultFileMetadata = new ObjectMetadata();
resultFileMetadata.setContentLength(totalFileSize);
s3Client.putObject("bucketName", "resultFilePath", partialFile, resultFileMetadata);
}
} catch (IOException e) {
LOG.error("An error occurred while combining files. {}", e);
}
}
private Enumeration<? extends InputStream> getInputStreamEnumeration(List<String> files) {
return new Enumeration<InputStream>() {
private Iterator<String> fileNamesIterator = files.iterator();
#Override
public boolean hasMoreElements() {
return fileNamesIterator.hasNext();
}
#Override
public InputStream nextElement() {
try {
return new FileInputStream(Paths.get(fileNamesIterator.next()).toFile());
} catch (FileNotFoundException e) {
System.err.println(e.getMessage());
throw new RuntimeException(e);
}
}
};
}
I am busy writing an application where the user needs to capture a lot of images and then they get packaged together with some text data and then they get uploaded to a local server. I want to implement the uploading on the Android platform through an Intent Service but I cannot find a good Xamarin Forms PCL example to show me how.
This is the method where I initialize the Intent to pass to the IntentService:
public async Task<bool> UploadAsync(Uri serviceAddress,
CaptureEntity capture,
List<ImageEntity> images)
{
try
{
Intent uploadIntent = new Intent();
uploadIntent.PutExtra("serviceAddress", serviceAddress.ToString());
uploadIntent.PutExtra("captureId", capture.WorkflowId.ToString());
StartService(uploadIntent);
return true;
}
catch (Exception exc)
{
App.logger.LogError(DateTime.Now, "Uploader", exc.ToString());
throw exc;
}
}
And this is the IntentService itself.
[Service]
public class ServiceIntent : IntentService
{
public ServiceIntent() : base("ServiceIntent")
{
}
//[return: GeneratedEnum]
public override StartCommandResult OnStartCommand(Intent intent, [GeneratedEnum] StartCommandFlags flags, int startId)
{
return base.OnStartCommand(intent, flags, startId);
}
public override void OnCreate()
{
base.OnCreate();
}
protected override void OnHandleIntent(Intent intent)
{
Uri serviceAddress = new Uri(intent.GetStringExtra("serviceAddress"));
Guid captureId = Guid.Parse(intent.GetStringExtra("captureId"));
CaptureEntity capture = new DatabaseConnection_Android().CreateConnection().Query<CaptureEntity>("SELECT * FROM [CaptureEntity]").Single(c => c.WorkflowId == captureId);
var images = new DatabaseConnection_Android().CreateConnection().Query<ImageEntity>("SELECT * FROM [ImageEntity]").Where(i => i.CaptureEntityId == capture.Id);
try
{
MultipartFormDataContent content = new MultipartFormDataContent();
StringContent strContent = new StringContent(
capture.XmlData,
Encoding.UTF8,
"text/xml");
IImageHandler handler = new ImageHandler_Droid();
HttpRequestMessage request = new HttpRequestMessage();
request.Headers.Add("workflow", capture.WorkflowId.ToString());
request.Method = HttpMethod.Post;
request.RequestUri = serviceAddress;
foreach (var image in images)
{
byte[] imageByte = handler.ReadAllBytes(image.ImagePath);
ByteArrayContent byteContent = new ByteArrayContent(imageByte);
byteContent.Headers.Add("Content-Type", "image/jpeg");
content.Add(byteContent, "file", image.ImageName);
}
content.Add(strContent, "text/xml");
request.Content = content;
using (HttpClient client = new HttpClient())
{
client.Timeout = TimeSpan.FromSeconds(180);
var response = client.SendAsync(
request,
HttpCompletionOption.ResponseContentRead).Result;
var readResponse = response.Content.ReadAsStringAsync().Result;
if (readResponse == "File uploaded.")
MessagingCenter.Send<CaptureEntity, string>(
capture,
"Completed",
"Success");
else if (readResponse.Contains("An error has occurred."))
MessagingCenter.Send<CaptureEntity, string>(
capture,
"Uploader",
String.Format(CultureInfo.InvariantCulture,
"Failed: {0}",
readResponse));
else
MessagingCenter.Send<CaptureEntity, string>(
capture,
"Uploader",
String.Format(CultureInfo.InvariantCulture,
"Failed: {0}",
readResponse));
}
}
catch (WebException webExc)
{
MessagingCenter.Send<string, string>("Uploader", "Failed",
String.Format(CultureInfo.InvariantCulture,
"{0} upload failed.\n{1}",
capture.DisplayName,
webExc.Message));
}
catch (TimeoutException timeExc)
{
MessagingCenter.Send<string, string>("Uploader", "Failed",
String.Format(CultureInfo.InvariantCulture,
"{0} upload failed.\n{1}",
capture.DisplayName,
timeExc.Message));
}
catch (Exception exc)
{
MessagingCenter.Send<string, string>("Uploader", "Failed",
String.Format(CultureInfo.InvariantCulture,
"{0} upload failed.\n{1}",
capture.DisplayName,
exc.Message));
}
}
}
Can anyone tell me what I am doing wrong as I am getting the following error when I want to start the service:
Java.Lang.NullPointerException: Attempt to invoke virtual method 'android.content.ComponentName android.content.Context.startService(android.content.Intent)' on a null object reference
In your Intent declaration you need to tell the service you want to call
Something like this:
var uploadIntent = new Intent(this, typeof(ServiceIntent));
Note: this represents the Context.
Update:
As mentioned in the comments your interface implementation cannot derive from Activity class. In order to have access to the Context to be able to call the StartService method and also create your Intent you can make it in two ways:
Using the Xamarin.Forms.Forms.Context:
public async Task<bool> UploadAsync(Uri serviceAddress,
CaptureEntity capture,
List<ImageEntity> images)
{
try
{
var context = Xamarin.Forms.Forms.Context;
var uploadIntent = new Intent(context, typeof(ServiceIntent));
uploadIntent.PutExtra("serviceAddress", serviceAddress.ToString());
uploadIntent.PutExtra("captureId", capture.WorkflowId.ToString());
context.StartService(uploadIntent);
return true;
}
catch (Exception exc)
{
App.logger.LogError(DateTime.Now, "Uploader", exc.ToString());
throw exc;
}
}
If you are using latest versions of Xamarin.Forms this global context was deprecated and they suggest to you local context instead. You can still use it though but in future updates of XF your app might break.
using CurrentActivity plugin:
public async Task<bool> UploadAsync(Uri serviceAddress,
CaptureEntity capture,
List<ImageEntity> images)
{
try
{
var context = CrossCurrentActivity.Current.Activity;
var uploadIntent = new Intent(context, typeof(ServiceIntent));
uploadIntent.PutExtra("serviceAddress", serviceAddress.ToString());
uploadIntent.PutExtra("captureId", capture.WorkflowId.ToString());
context.StartService(uploadIntent);
return true;
}
catch (Exception exc)
{
App.logger.LogError(DateTime.Now, "Uploader", exc.ToString());
throw exc;
}
}
This plugin can be installed from nugget and the setup is very straight forward. Basically it gives you access to the current activity and you can use it as your context to call the IntentService
Hope this helps.-
Here is the IntentService.
IntentService is a base class for Services that handle asynchronous requests (expressed as Intents) on demand. Clients send requests through startService(Intent) calls; the service is started as needed, handles each Intent in turn using a worker thread, and stops itself when it runs out of work.
In Android, we usually use IntentService to do asynchronous operator. As we all know, thread is also used to do asynchronous operator. The difference between IntentService and Thread is IntentService is Service which belongs to Android Component. So, the priority of IntentService is higher than Thread.
For example, there is a ActivityA which has a IntentService, and there is a ActivityB which has a Thread, both IntentService and Thread are working, and both ActivityA and ActivityB are al background Activity. Now, if your phone's system doesn't have extra resources, your ActivityB will be killed firstly.
About the Exception:
Java.Lang.NullPointerException: Attempt to invoke virtual method 'android.content.ComponentName android.content.Context.startService(android.content.Intent)' on a null object reference
That means you should use android.content.Context to call the StartService method. In Android, there are three kinds of Context. Application, Activity and Service. So you can call the StartService method in these three classes directly. If you are not in these three classes, you need pass the Context to your class, and then use the Context to call StartService.
I added Activity for this class' inheritance.
If you do this, your class will be a Activity, and you need to register it in your manifiest, add layout for your class, and it should have the lifecycle, and etc. It will not be what you want to get class. In Android, Activity is a Component, not normal class, so you can't inherit it unless you want your class to be a Activity.
Demo:
I have made a demo for you,
I have an interceptor written for Flume code is below:
public Event intercept(Event event) {
byte[] xmlstr = event.getBody();
InputStream instr = new ByteArrayInputStream(xmlstr);
//TransformerFactory factory = TransformerFactory.newInstance(TRANSFORMER_FACTORY_CLASS,TRANSFORMER_FACTORY_CLASS.getClass().getClassLoader());
TransformerFactory factory = TransformerFactory.newInstance();
Source xslt = new StreamSource(new File("removeNs.xslt"));
Transformer transformer = null;
try {
transformer = factory.newTransformer(xslt);
} catch (TransformerConfigurationException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
Source text = new StreamSource(instr);
OutputStream ostr = new ByteArrayOutputStream();
try {
transformer.transform(text, new StreamResult(ostr));
} catch (TransformerException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
event.setBody(ostr.toString().getBytes());
return event;
}
I'm removing NameSpace from my source xml with removeNs.xslt file. So that I can store that data into HDFS and later put into hive. When my interceptor run it throw below error :
ERROR org.apache.flume.source.jms.JMSSource: Unexpected error processing events
java.lang.NullPointerException
at test.intercepter.App.intercept(App.java:59)
at test.intercepter.App.intercept(App.java:82)
at org.apache.flume.interceptor.InterceptorChain.intercept(InterceptorChain.java:62)
at org.apache.flume.channel.ChannelProcessor.processEventBatch(ChannelProcessor.java:146)
at org.apache.flume.source.jms.JMSSource.doProcess(JMSSource.java:258)
at org.apache.flume.source.AbstractPollableSource.process(AbstractPollableSource.java:54)
at org.apache.flume.source.PollableSourceRunner$PollingRunner.run(PollableSourceRunner.java:139)
at java.lang.Thread.run(Thread.java:745)*
Can you suggest me what and where is the problem?
I found the solution. The problem was not anything else than new File("removeNs.xslt"). It was not able to find the location as I's not sure where to keep this file but later I get the flume agent path but as soon as I restart the flume agent it deletes all files which I kept in the flume agent dir. So I changed the code and kept the file material into my java code.
I'm working with Birt reports and, when i want to generate a pdf file, it never ends. The problem is in the line.
IRunAndRenderTask.run. I have no exception when I create the IBirtEngine.
Here is the code to create the BirtEngine and the reports designs.
private static IReportEngine birtEngine = null;
private static IReportRunnable examenAuditifReportDesign = null;
private static IReportRunnable bilanCollectifReportDesign = null;
private static Properties configProps = new Properties();
static{
loadEngineProps();//Read the birt configuration properties
EngineConfig config = new EngineConfig();
if( configProps != null){
config.setLogConfig(configProps.getProperty("logDirectory"), Level.INFO);
config.setBIRTHome(configProps.getProperty("birtHome"));
config.setResourcePath(configProps.getProperty("ressourcePath"));
config.setEngineHome(configProps.getProperty("birtHome"));
config.setProperty("birtReportsHome", configProps.getProperty("birtReportsHome"));
}
try {
RegistryProviderFactory.releaseDefault();
Platform.startup( config );
} catch ( BirtException e ) {
e.printStackTrace( );
}
IReportEngineFactory factory = (IReportEngineFactory) Platform.createFactoryObject(IReportEngineFactory.EXTENSION_REPORT_ENGINE_FACTORY );
birtEngine = factory.createReportEngine( config );
try {
examenAuditifReportDesign = birtEngine.openReportDesign(
new FileInputStream(birtEngine.getConfig().getProperty("birtReportsHome") + "/examenAuditif.rptdesign"));
bilanCollectifReportDesign = birtEngine.openReportDesign(
new FileInputStream(birtEngine.getConfig().getProperty("birtReportsHome") + "/bilanCollectif.rptdesign"));
} catch (EngineException e) {
e.printStackTrace();
} catch (FileNotFoundException e) {
e.printStackTrace();
}
}
And here's the code to execute the reports.
IRunAndRenderTask task = birtEngine.createRunAndRenderTask(examenAuditifReportDesign);
task.setParameterValue("RPT_ID_Travailleur", t.getId());
task.setParameterValue("RPT_LATEST_HA", params.getId_latest_ha());
task.setParameterValue("RPT_LATEST_EXAMN", params.getId_latest_examen());
task.setParameterValue("RPT_PREVIOUS_HA", params.getId_previous_ha());
task.setParameterValue("RPT_PREVIOUS_EXAMN", params.getId_previous_examen());
PDFRenderOption options = new PDFRenderOption();
options.setOutputStream(outputStream);
options.setOutputFormat("pdf");
task.setRenderOption(options);
task.run();
task.close();
and is in the task.run(); line that it takes forever, and i tried for about 1 hour or 1 hour and a half and it does not end.
If anyone can help it will be really apreciated.
Bye and thank you.
If this report works in a birt Eclipse designer then it should work with a birt runtime API, the issue is in your code.
You need to close yourself the outputStream sent to the PDFRenderOption object, when the task has terminated
Add this line to your code:
options.setSupportedImageFormats("PNG;JPG;BMP");
I found the problem and it wasn't neither of the two API's.
We were trying to generate a pdf file for worker, and we were zipping all the pdf file of all the workers, and this zip was saved in a blob in the database. That was the problem.
If the zip is saved in the File System it works ok.
thank you all.
I try to get my code working with catching certain errors. I store the token for a user after he or she grants permission to my app (this is a WP7 app). When I try to post on the wall by using the stored token it works. When I remove the permissions on facebook it throws an OAuthException. I can't catch it it seems. My app just crashes. This is the code I used:
private object PostToFacebook()
{
_fbApp = new FacebookClient(_appsettings.faceBookToken);
FacebookAsyncCallback callback = new FacebookAsyncCallback(this.postResult);
var parameters = new Dictionary<string, object>();
parameters.Add("message", "message on wall");
try
{
_fbApp.PostAsync("me/feed", parameters, callback);
}
catch (Exception ex)
{
}
return null;
}
private void postResult(FacebookAsyncResult asyncResult)
{
if (asyncResult.Error == null)
{
status = "succes";
}
else
{
status = "error" + asyncResult.Error.Message;
}
}
The try catch doesn't catch anything and the generic exception handler in my app.xaml.cs either.
Any ideas how to catch this error so I can ask the user to authenticate again?
Put your try..catch in the callback.
You can also catch exceptions globally by handling the UnhandledException event on the App object.