How can this be done? It works fine with one int-file:outbound-channel-adapter, but I could not make it work when I add another one. I actually added another, separate set of channel/adapter but it still did not work.
In int-file:outbound-channel-adapter tag, there is actually a "directory" attribute, but it only accepts a single directory path.
Here is the code I have tried:
<int-file:outbound-channel-adapter id="outputDirectory1"
directory="${output.directory1}"
channel="fileWriterChannel1"
filename-generator- expression="headers.get('filename')"
delete-source-files="true"/>
<int-file:outbound-channel-adapter id="outputDirectory2"
directory="${output.directory2}"
channel="fileWriterChannel2"
filename-generator-expression="headers.get('filename')"
delete-source-files="true"/>
Below are the channels, while the bean is the actual writer. Note that the two channels both refer to the bean (ref="messageTransformer"):
<int:transformer id="messageToStringTransformer1"
input-channel="messageTypeChannel"
output-channel="fileWriterChannel1"
ref="messageTransformer"
method="write"/>
<int:transformer id="messageToStringTransformer2"
input-channel="messageTypeChannel"
output-channel="fileWriterChannel2"
ref="messageTransformer"
method="write"/>
<bean id="messageTransformer" class="com.message.writer.DefaultMessageWriter"/>
If I do understand you correctly, do you want to write a Message payload to a collection of directories simultaneously? In order to have multiple file adapters listen to the same channel, you have to use a Publish Subscribe Channel using the element. For more information, please see: http://static.springsource.org/spring-integration/reference/html/messaging-channels-section.html#channel-configuration-pubsubchannel
When using a File Outbound Channel Adapter, you can also use the directory-expression attribute which is available since Spring Integration 2.2. It gives you full SpEL expression support. Thus, the directory you want to write to, can be for example a provided message header. For more information, please see:
http://static.springsource.org/spring-integration/reference/html/files.html#file-writing-output-directory
Related
I started using spring integration SFTP and I have some questions.
Filters not working. I have example configuration:
Sftp.inboundAdapter(ftpFileSessionFactory())
.preserveTimestamp(true)
.deleteRemoteFiles(false)
.remoteDirectory(integrationProperties.getRemoteDirectory())
.filter(sftpFileListFilter()) // doesn't work
.patternFilter("*.xlsx") // doesn't work
And my ChainFileListFilter:
private ChainFileListFilter<ChannelSftp.LsEntry> sftpFileListFilter() {
ChainFileListFilter<ChannelSftp.LsEntry> chainFileListFilter = new ChainFileListFilter<>();
chainFileListFilter.addFilter(new SftpPersistentAcceptOnceFileListFilter(metadataStore(), "INT"));
chainFileListFilter.addFilter(new SftpSimplePatternFileListFilter("*.xlsx"));
return chainFileListFilter;
}
If I understand correctly, only the XLSX file should be saved in the local directory. If yes it doesn't work with this configuration. Am I doing something wrong or misunderstood this?
How I can configure SFTP that each downloaded file emit message? I see in the doc two params max-messages-per-poll and max-fetch-size, but I don't know how to set it up so that every file emits a message. I would like to sync files once every 24 hours and produce batch job queue. Maybe there is a workaround?
Is there built-in filter which allow me fetch only files with changed content? The best solution would be to check the checksums of the files.
I will be grateful for your help and explanations.
You cannot combine filter() and patternFilter(). Only one of them can be used: the last one overrides whatever you used before. In other words: or filter() or patternFilter() - not both. By default the logic is like this:
public SftpInboundChannelAdapterSpec patternFilter(String pattern) {
return filter(composeFilters(new SftpSimplePatternFileListFilter(pattern)));
}
private CompositeFileListFilter<ChannelSftp.LsEntry> composeFilters(FileListFilter<ChannelSftp.LsEntry>
fileListFilter) {
CompositeFileListFilter<ChannelSftp.LsEntry> compositeFileListFilter = new CompositeFileListFilter<>();
compositeFileListFilter.addFilters(fileListFilter,
new SftpPersistentAcceptOnceFileListFilter(new SimpleMetadataStore(), "sftpMessageSource"));
return compositeFileListFilter;
}
So, technically you don't need your custom one, if you don't use external persistent MetadataStore. But if you do, think about flipping SftpSimplePatternFileListFilter with SftpPersistentAcceptOnceFileListFilter. Since it is better to check for the pattern before storing the file into MetadataStore.
It is the fact that every synched remote file, passed those filters, is stored into local dir and the message for that local file is emitted immediately when the poller does a request.
The maxFetchSize plays the role when we load remote files into a local dir. The maxMessagesPerPoll is used from the poller, but those are already built from the local files. The message is emitted per local file, not as a batch for all of them. That's not what messaging is designed for.
Please, share more info what does not work with files. The SftpPersistentAcceptOnceFileListFilter checks not only file name, but also mtime of the file. So, that it not about any checksum, but more last modified timestamp of the file.
We have web application and want to use ftp:inbound-channel-adapter to read the files from ftp & create a message & send it to a channel
When local-directory is set to local-directory="#servletContext.getRealPath('')}/temp/inFtp" it gives
Message generated is
GenericMessage [payload={applicationDeployPath}\temp\inFtp\xyz.stagging.dat, headers={timestamp=1444812027516, id=9b5f1581-bfd2-6690-d4ea-e7398e8ecbd6}]
But directory is not created, i checked i have full permission here.
The path [{applicationDeployPath}\temp\inFtp] does not denote a properly accessible directory.
In Message i want to send some more fields as payload having value from property file as per environment, How can I do that?
<int-ftp:inbound-channel-adapter id="ftpInboundAdaptor"
channel="ftpInboundChannel"
session-factory="ftpClientFactory"
filename-pattern="*.stagging.dat"
delete-remote-files="false"
remote-directory="/stagging/"
local-directory="#servletContext.getRealPath('')}/temp/inFtp"
auto-create-local-directory="true">
<int:poller fixed-rate="1000"/>
</int-ftp:inbound-channel-adapter>
thanks in advance
The local-directory is only evaluated once (and created if necessary), at initialization time, not for every message.
I am surprised the context even initializes; this syntax looks bad:
local-directory="#servletContext.getRealPath('')}/temp/inFtp"
If you mean
local-directory="#{servletContext.getRealPath('')}/temp/inFtp"
it would only work if you have some bean called servletContext.
If you mean
local-directory="#servletContext.getRealPath('')/temp/inFtp"
you would need a variable servletContext in the evaluation context.
it gives Message generated is GenericMessage [payload={applicationDeployPath}\temp\inFtp\xyz.stagging.dat, headers={timestamp=1444812027516, id=9b5f1581-bfd2-6690-d4ea-e7398e8ecbd6}]
If it's actually generating that message, it must have created that directory.
It's not clear what you mean by
I want to send some more fields as payload
The payload is a File.
If you want to change the payload to something else containing the file and other fields, use a transformer.
I'd like to utilize Spring Integration to initiate messages about files that appear in a remote location, without actually transferring them. All I require is the generation of a Message with, say, header values indicating the path to the file and filename.
What's the best way to accomplish this? I've tried stringing together an FTP inbound channel adapter with a service activator to write the header values I need, but this causes the file to be transferred to a local temp directory, and by the time the service activator sees it, the message consists of a java.io.File that refers to the local file and the remote path info is gone. It is possible to transform the message prior to this local transfer occurring?
We have similar problem and we solved it with filters. On inbound-channel-adapter you can set custom filter implementation. So before polling your filter will be called and you will have all informations about files, from which you can decide will that file be downloaded or not, for example;
<int-sftp:inbound-channel-adapter id="test"
session-factory="sftpSessionFactory"
channel="testChannel"
remote-directory="${sftp.remote.dir}"
local-directory="${sftp.local.dir}"
filter="customFilter"
delete-remote-files="false">
<int:poller trigger="pollingTrigger" max-messages-per-poll="${sftp.max.msg}"/>
</int-sftp:inbound-channel-adapter>
<beans:bean id="customFilter" class="your.class.location.SftpRemoteFilter"/>
Filter class is just implementation of the FileListFilter interface. Here it is dummy filter implementation.
public class SftpRemoteFilter implements FileListFilter<LsEntry> {
private static final Logger log = LoggerFactory.getLogger(SftpRemoteFilter.class);
#Override
public final List<LsEntry> filterFiles(LsEntry[] files) {
log.info("Here is files.");
//Do something smart
return Collections.emptyList();
}
}
But if you want to do that as you described, I think it is possible to do it by setting headers on payloads and then using same headers when you are using that payload, but in that case you should use Message<File> instead File in your service activator method.
I have this problem and I am not sure why it's happening and how to fix it. I have created an OSB peject. In the proxy service pipeline I am doing a Service Callout to a sync SOAP service in another application. The other service needs the request body as below:
<RequestSelectionValues xmlns="http://www.camstar.com/WebService/WSShopFloor">
<inputServiceData xmlns:q1="http://www.camstar.com/WebService/DataTypes" q1:type="OnlineQuery">
<OnlineQuerySetup>
<__CDOTypeName/>
<__name>xLot By FabLotNumber</__name>
</OnlineQuerySetup>
<Parameters>
<__listItem>
<Name>FabLotNumber</Name>
<DefaultValue>FAB_Lot_1</DefaultValue>
</__listItem>
<__listItem>
<Name>BLOCKOF200ROWS</Name>
<DefaultValue>1</DefaultValue>
</__listItem>
</Parameters>
</inputServiceData>
<queryOption xmlns:q2="http://www.camstar.com/WebService/DataTypes" q2:type="QueryOption">
<RowSetSize>1000</RowSetSize>
<StartRow>1</StartRow>
<QueryType>user</QueryType>
<ChangeCount>0</ChangeCount>
<RequestRecordCount>false</RequestRecordCount>
<RequestRecordSetAndCount>false</RequestRecordSetAndCount>
</queryOption>
<serviceInfo xmlns:q3="http://www.camstar.com/WebService/DataTypes" q3:type="OnlineQuery_Info">
<OnlineQuerySelection>
<RequestValue>false</RequestValue>
<RequestMetadata>false</RequestMetadata>
<RequestSubFieldValues>false</RequestSubFieldValues>
<RequestSelectionValues>true</RequestSelectionValues>
</OnlineQuerySelection>
</serviceInfo>
</RequestSelectionValues>
I am using an Assign to put the above expression in a variable.
Notice the line:
<serviceInfo xmlns:q3="http://www.camstar.com/WebService/DataTypes" q3:type="OnlineQuery_Info">
xmlns:q3="http://www.camstar.com/WebService/DataTypes" needs to be before q3:type="OnlineQuery_Info" for the other service to be called successfully otherwise the service call fails.
In the development it looks fine. I can test the assign of expression as well.
When I go to the OSB console to test the service I notice that in the Assign variable the namespace place switches and it becomes like this:
<serviceInfo q3:type="OnlineQuery_Info" xmlns:q3="http://www.camstar.com/WebService/DataTypes">
This makes the service calls to fail. I have tried putting the body payload in an xslt. Result is the same. I am not sure why it switches the type before namespace. The end result is that the service is not working as expected.
Any idea what I can do to fix this issue. How can I prevent the switching?
Thanks
I haven't found any settings in OSB that can prevent reordering of attributes for you. However, the above OSB behavior is completely XML standard compliant. In fact, the target service side should be XML compliant and treat the two variants mentioned above as the same, because according to XML standard, tow XML documents with only difference in attribute ordering should be treated as the same.
EDIT:
Please go here to download a modified config. My thoughts are:
Specify the business service to invoke in 'Text as Request' mode, as "CamstarLotQuery/business/CSWSShopFloor_Txt" shown below:
Manipulate messages as text, not XML, in your proxy service, as specified in "CamstarLotQuery/proxy/CamstarLotQueryTxt_Txt":
You might need to specify a SOAP Action in http header when calling a business service, depending on the target service.
One solution i can think of is to assign all the namespaces at the Parent Tag Level, and keep the attributes where they are applicable.
Example:
<RequestSelectionValues xmlns:q1="http://www.camstar.com/WebService/DataTypes" xmlns="http://www.camstar.com/WebService/WSShopFloor" xmlns:q2="http://www.camstar.com/WebService/DataTypes" xmlns:q3="http://www.camstar.com/WebService/DataTypes">
But the problem with this implementation is that since the namespace declaration is now Global, you have to declare your namespace prefixes (q1, q2, q3) to the blocks where the namespaces were previously defined.
Example:
<q3:serviceInfo q3:type="OnlineQuery_Info">
<q3:OnlineQuerySelection>
<q3:RequestValue>false</q3:RequestValue>
<q3:RequestMetadata>false</q3:RequestMetadata>
<q3:RequestSubFieldValues>false</q3:RequestSubFieldValues>
<q3:RequestSelectionValues>true</q3:RequestSelectionValues>
</q3:OnlineQuerySelection>
</q3:serviceInfo>
if this namespace prefix is not declared, then as per XML standards, the tag assume the 'default' namespace value - which will be the namespace of the parent.
However, even though this solution has a round-about way of implementation, this solution will definitely work.
Using Saml2SecurityTokenHandler to validate SAML2 bearer token from internal provider or from ACS. Able to programmatically configure the handler to validate just fine, but it doesn't seem to want to pick up configuration from the microsoft.IdentityModel section in my config file. Constructing a SecurityTokenHandlerCollectionManager seems to have no notion of the named configuration section either so I can't seem to use mySaml2SecurityTokenHandler .Configuration - mySecurityTokenHandlerCollectionManager["NAME"].Configuration.
Is there a good sample of setting this up somewhere?
To use file-based config, it turns out you simply rely on the FederatedAuthentication context, rather than explicitly constructing the Saml2SecurityTokenHandler:
var handlers = FederatedAuthentication.ServiceConfiguration.SecurityTokenHandlers;
var token = handlers.ReadToken(xmlReader);
var collection = handlers.ValidateToken(token);
This seems to be a known problem.
Have you tried FederatedClientCredentials --> SecurityTokenHandlerCollectionManager property --> SecurityTokenHandlerCollection --> replace the standard Saml2SecurityTokenHandler with whatever?