I am trying to use the RotatingServerAdvice to poll new files from multiple resource. I have a new requirement that for every new client, a new directory is created in the sftp server and I want to add that directory to the poll as well. So I will be using int-sftp:outbound-gateway to query all directories and check whether a new directory is created then add the new directory to the key directories. Is it possible that way? Thank you.
#Bean
public RotatingServerAdvice advice() {
List<RotationPolicy.KeyDirectory> keyDirectories = new ArrayList<>();
keyDirectories.add(new RotationPolicy.KeyDirectory("one", "foo"));
keyDirectories.add(new RotationPolicy.KeyDirectory("one", "bar"));
keyDirectories.add(new RotationPolicy.KeyDirectory("two", "baz"));
keyDirectories.add(new RotationPolicy.KeyDirectory("two", "qux"));
keyDirectories.add(new RotationPolicy.KeyDirectory("three", "fiz"));
keyDirectories.add(new RotationPolicy.KeyDirectory("three", "buz"));
return new RotatingServerAdvice(delegatingSf(), keyDirectories);
}
It is not possible with the default StandardRotationPolicy; you need to provide a custom RotationPolicy.
/**
* Construct an instance that rotates according to the supplied
* {#link RotationPolicy}.
* #param rotationPolicy the policy.
*/
public RotatingServerAdvice(RotationPolicy rotationPolicy) {
Assert.notNull(rotationPolicy, "'rotationPolicy' cannot be null");
this.rotationPolicy = rotationPolicy;
}
You can probably start with the StandardRotationPolicy as a model.
Related
I have a problem with spring integration.
I want to make a request on an ftp server to retrieve the name of a file
(at the command line: ls "filename")
But I cannot recover the file name dynamically.
I understood that there was a story with payload or header but I can not
This is what I have:
Review my controller, I use this :
private FtpConfig.MyGateway gateway;
...
gateway.fichierExist(filename);
in my FTP file :
#Bean
public SessionFactory<FTPFile> ftpSessionFactory() {
DefaultFtpSessionFactory sf = new DefaultFtpSessionFactory();
sf.setHost("");
sf.setPort(21);
sf.setUsername("");
sf.setPassword("");
return new CachingSessionFactory<FTPFile>(sf);
}
#Bean
#ServiceActivator(inputChannel = "ftpChannelExist")
public MessageHandler handler2() {
FtpOutboundGateway ftpOutboundGateway =
new FtpOutboundGateway(ftpSessionFactory(), "ls");
ftpOutboundGateway.setOptions("-a -1")
FtpSimplePatternFileListFilter filter = new FtpSimplePatternFileListFilter("filename"); //on filtre sur le nom
return ftpOutboundGateway;
}
#MessagingGateway
public interface MyGateway {
#Gateway(requestChannel = "ftpChannelExist")
ArrayList<String> fichierExist(String filename);
}
I tried with header too, but I can not do anything ...
Thanks.
(Sorry for my english, i'm french)
See LS command description in the Reference Manual:
In addition, filename filtering is provided, in the same manner as the inbound-channel-adapter.
The message payload resulting from an ls operation is a list of file names, or a list of FileInfo objects. These objects provide information such as modified time, permissions etc.
The remote directory that the ls command acted on is provided in the file_remoteDirectory header.
What you are missing in your configuration is a fact of the remote directory to fetch files from. Typically we suggest to have such a directory in the payload as you do with your fichierExist(String filename) and configure the third ctor arg for the FtpOutboundGateway:
FtpOutboundGateway ftpOutboundGateway =
new FtpOutboundGateway(ftpSessionFactory(), "ls", "payload");
According the logic in the FtpOutboundGateway that expression is serving as a source for the remote directory in the LS command. In your case this one is going to be an argument of your fichierExist(String filename) gateway.
You indeed can use there a FtpSimplePatternFileListFilter, but be sure to specify a proper pattern to filter remote files.
In the end the names of the remote files in the requested directory, after filtering are going to be returned to the ArrayList<String> of your gateway. That's correct.
Otherwise your question isn't clear.
Thanks for your reply.
I have change my FtpOutboundGateway for add "payload" but I can't use payload for my FtpSimplePatternFileListFilter.
I've try :
FtpSimplePatternFileListFilter filter = new FtpSimplePatternFileListFilter("filename");
FtpSimplePatternFileListFilter filter = new FtpSimplePatternFileListFilter("payload");
FtpSimplePatternFileListFilter filter = new FtpSimplePatternFileListFilter("payload.filename");
FtpSimplePatternFileListFilter filter = new FtpSimplePatternFileListFilter("payload['filename']");
Consider the scenario where I have below code.
MqttConnectOptions connOpt = new MqttConnectOptions();
connOpt.setServerURIs(new String[]{"tcp://localhost:1883", "tcp://some-other-host:1883"});
connOpt.setAutomaticReconnect(true);
client.setCallback( new TemperatureSubscriber() );
client.connect(connOpt);
So when I say connect, it connects to localhost.
Then I get connection lost, due to say any reason. So at this point of time, since automaticReconnect is true, will it connect to localhost or some-other-host ?
Let me show how to find such answers yourself -
First you visit the Github repository for Paho source code.
Then you enter setAutomaticReconnect into the search field:
This is of course just the public name. You need to spot the corresponding private member.
In the MqttConnectOptions.java with the very simple code you find that member:
private boolean automaticReconnect = false;
Then you perform another search, this time for the automaticReconnect word:
And that leads you to the hint in the MqttAsyncClient.java file -
// Insert our own callback to iterate through the URIs till the connect
// succeeds
MqttToken userToken = new MqttToken(getClientId());
ConnectActionListener connectActionListener = new ConnectActionListener(this, persistence, comms, options,
userToken, userContext, callback, reconnecting);
userToken.setActionCallback(connectActionListener);
userToken.setUserContext(this);
Finally, in the ConnectActionListener.java file you can confirm that the URLs are being tried one after each other:
/**
* The connect failed, so try the next URI on the list.
* If there are no more URIs, then fail the overall connect.
*
* #param token the {#link IMqttToken} from the failed connection attempt
* #param exception the {#link Throwable} exception from the failed connection attempt
*/
public void onFailure(IMqttToken token, Throwable exception) {
int numberOfURIs = comms.getNetworkModules().length;
int index = comms.getNetworkModuleIndex();
...
...
comms.setNetworkModuleIndex(index + 1);
I am trying to rename google drive file resource. I guess that I just am missing something since all other actions like getting list of files, inserting files, moving files between directories are working.
Precondition: trying to rename file resource using this doc https://developers.google.com/drive/v2/reference/files/update with java (with only JDK stuff). Also, I do not use gdrive java sdk, apache http client or other libraries... Just clean JDK tools.
So what I do:
Here is the file metadata I am trying to send.
Modify title property in this metadata
Here is the code:
URLConnection urlConnection = new URL("https://www.googleapis.com/drive/v2/files/" + fileId).openConnection();
if (urlConnection instanceof HttpURLConnection) {
HttpURLConnection httpURLConnection = (HttpURLConnection) urlConnection;
httpURLConnection.setRequestMethod("PUT");
httpURLConnection.setDoOutput(true);
httpURLConnection.setRequestProperty("Authorization", "Bearer " + accessToken);
DataOutputStream outputStream = new DataOutputStream(httpURLConnection.getOutputStream());
outputStream.writeBytes(FILE_RESOURCE_METADATA_WITH_CHANGED_TITLE_IN_JSON);
outputStream.flush();
outputStream.close();
}
After making an actual call to API I receive 200 status code and File resource in response body (as expected) but title remains the same. So I got no error no changed title.
Moreover, the google drive api ignores any change in the file resource. It just returns same file resource without any changes applied (tried with title, description, originalFileName, parents properties).
What I tried also so far:
Sending only the properties that should be changed, like
{"title":"some_new_name"}
Result is same.
Changing PUT to PATCH. Unfortunately, PATCH is not supported by HttpURLConnection but workarounds gave same results. Changes are ignored.
Used google api exlorer (which can be found on the right side of API reference page) - and... it works. Filled only fileId and title property in request body and it worked. File is renamed.
What I am missing ?
Found the solution...
Adding this request property fixed the problem.
httpURLConnection.setRequestProperty("Content-Type", "application/json")
Try the sample java code given in the documentation.
Since the code deals to update existing file's metadata and content.
From the code, you will find file.setTitle(newTitle) which I think the one what you want to implement.
import com.google.api.client.http.FileContent;
import com.google.api.services.drive.Drive;
import com.google.api.services.drive.model.File;
import java.io.IOException;
// ...
public class MyClass {
// ...
/**
* Update an existing file's metadata and content.
*
* #param service Drive API service instance.
* #param fileId ID of the file to update.
* #param newTitle New title for the file.
* #param newDescription New description for the file.
* #param newMimeType New MIME type for the file.
* #param newFilename Filename of the new content to upload.
* #param newRevision Whether or not to create a new revision for this
* file.
* #return Updated file metadata if successful, {#code null} otherwise.
*/
private static File updateFile(Drive service, String fileId, String newTitle,
String newDescription, String newMimeType, String newFilename, boolean newRevision) {
try {
// First retrieve the file from the API.
File file = service.files().get(fileId).execute();
// File's new metadata.
file.setTitle(newTitle);
file.setDescription(newDescription);
file.setMimeType(newMimeType);
// File's new content.
java.io.File fileContent = new java.io.File(newFilename);
FileContent mediaContent = new FileContent(newMimeType, fileContent);
// Send the request to the API.
File updatedFile = service.files().update(fileId, file, mediaContent).execute();
return updatedFile;
} catch (IOException e) {
System.out.println("An error occurred: " + e);
return null;
}
}
// ...
}
Hope this give you some points.
I am running a Java program on a remote computer and trying to read the split data using RecordReader object but instead getting:
Exception in thread "main" java.io.IOException: job information not found in JobContext. HCatInputFormat.setInput() not called?
I already have called the following:
_hcatInputFmt = HCatInputFormat.setInput(_myJob, db,tbl);
and then creating the RecordReader object as:
_hcatInputFmt.createRecordReader(hSplit, taskContext)
On debugging it fails while searching for the value of the key: HCAT_KEY_JOB_INFO in job configuration object, while trying to create a RecordReader object.
How do I set this value? Any pointers will be helpful.
Thanks.
We have to use getConfiguration() method to get the configuration from the job object. The configuration object used in creating the job object won't do it.
I had the same problem, you shuold use:
HCatInputFormat.setInput(job, dbName, inputTableName);
HCatSchema inputschema = HCatBaseInputFormat.getTableSchema(job.getConfiguration());
not
HCatInputFormat.setInput(job, dbName, inputTableName);
HCatSchema inputschema = HCatBaseInputFormat.getTableSchema(getConf());
Because, when you use Job.getInstance(conf), it will copy the conf, you can't use the original conf. Here is the code:
/**
* A new configuration with the same settings cloned from another.
*
* #param other the configuration from which to clone settings.
*/
#SuppressWarnings("unchecked")
public Configuration(Configuration other) {
this.resources = (ArrayList<Resource>) other.resources.clone();
synchronized(other) {
if (other.properties != null) {
this.properties = (Properties)other.properties.clone();
}
if (other.overlay!=null) {
this.overlay = (Properties)other.overlay.clone();
}
this.updatingResource = new ConcurrentHashMap<String, String[]>(
other.updatingResource);
this.finalParameters = Collections.newSetFromMap(
new ConcurrentHashMap<String, Boolean>());
this.finalParameters.addAll(other.finalParameters);
}
synchronized(Configuration.class) {
REGISTRY.put(this, null);
}
this.classLoader = other.classLoader;
this.loadDefaults = other.loadDefaults;
setQuietMode(other.getQuietMode());
}
We are trying to migrate to use Microsoft Enterprise Library - Caching block. However, cache manager initialization seems to be pretty tied to the config file entries and our application creates inmemory "containers" on the fly. Is there anyway by which an instance of cache manager can be instantiated on the fly using pre-configured set of values (inmemory only).
Enterprise Library 5 has a fluent configuration which makes it easy to programmatically configure the blocks. For example:
var builder = new ConfigurationSourceBuilder();
builder.ConfigureCaching()
.ForCacheManagerNamed("MyCache")
.WithOptions
.UseAsDefaultCache()
.StoreInIsolatedStorage("MyStore")
.EncryptUsing.SymmetricEncryptionProviderNamed("MySymmetric");
var configSource = new DictionaryConfigurationSource();
builder.UpdateConfigurationWithReplace(configSource);
EnterpriseLibraryContainer.Current
= EnterpriseLibraryContainer.CreateDefaultContainer(configSource);
Unfortunately, it looks like you need to configure the entire block at once so you wouldn't be able to add CacheManagers on the fly. (When I call ConfigureCaching() twice on the same builder an exception is thrown.) You can create a new ConfigurationSource but then you lose your previous configuration. Perhaps there is a way to retrieve the existing configuration, modify it (e.g. add a new CacheManager) and then replace it? I haven't been able to find a way.
Another approach is to use the Caching classes directly.
The following example uses the Caching classes to instantiate two CacheManager instances and stores them in a static Dictionary. No configuration required since it's not using the container. I'm not sure it's a great idea -- it feels a bit wrong to me. It's pretty rudimentary but hopefully helps.
public static Dictionary<string, CacheManager> caches = new Dictionary<string, CacheManager>();
static void Main(string[] args)
{
IBackingStore backingStore = new NullBackingStore();
ICachingInstrumentationProvider instrProv = new CachingInstrumentationProvider("myInstance", false, false,
new NoPrefixNameFormatter());
Cache cache = new Cache(backingStore, instrProv);
BackgroundScheduler bgScheduler = new BackgroundScheduler(new ExpirationTask(null, instrProv), new ScavengerTask(0,
int.MaxValue, new NullCacheOperation(), instrProv), instrProv);
CacheManager cacheManager = new CacheManager(cache, bgScheduler, new ExpirationPollTimer(int.MaxValue));
cacheManager.Add("test1", "value1");
caches.Add("cache1", cacheManager);
cacheManager = new CacheManager(new Cache(backingStore, instrProv), bgScheduler, new ExpirationPollTimer(int.MaxValue));
cacheManager.Add("test2", "value2");
caches.Add("cache2", cacheManager);
Console.WriteLine(caches["cache1"].GetData("test1"));
Console.WriteLine(caches["cache2"].GetData("test2"));
}
public class NullCacheOperation : ICacheOperations
{
public int Count { get { return 0; } }
public Hashtable CurrentCacheState { get { return new System.Collections.Hashtable(); } }
public void RemoveItemFromCache(string key, CacheItemRemovedReason removalReason) {}
}
If expiration and scavenging policies are the same perhaps it might be better to create one CacheManager and then use some intelligent key names to represent the different "containers". E.g. the key name could be in the format "{container name}:{item key}" (assuming that a colon will not appear in a container or key name).
You can using UnityContainer:
IUnityContainer unityContainer = new UnityContainer();
IContainerConfigurator configurator = new UnityContainerConfigurator(unityContainer);
configurator.ConfigureCache("MyCache1");
IContainerConfigurator configurator2 = new UnityContainerConfigurator(unityContainer);
configurator2.ConfigureCache("MyCache2");
// here you can access both MyCache1 and MyCache2:
var cache1 = unityContainer.Resolve<ICacheManager>("MyCache1");
var cache2 = unityContainer.Resolve<ICacheManager>("MyCache2");
And this is an extension class for IContainerConfigurator:
public static void ConfigureCache(this IContainerConfigurator configurator, string configKey)
{
ConfigurationSourceBuilder builder = new ConfigurationSourceBuilder();
DictionaryConfigurationSource configSource = new DictionaryConfigurationSource();
// simple inmemory cache configuration
builder.ConfigureCaching().ForCacheManagerNamed(configKey).WithOptions.StoreInMemory();
builder.UpdateConfigurationWithReplace(configSource);
EnterpriseLibraryContainer.ConfigureContainer(configurator, configSource);
}
Using this you should manage an static IUnityContainer object and can add new cache, as well as reconfigure existing caching setting anywhere you want.