I am quite new to the world of Integration and IBM Integration Bus Toolkit. I am doing a small exercise with IBM Integration Bus Toolkit, whereby my small application picks up an .txt file in a specified directory and moving it into another one (I am using Message Flow to do it). I can even specify the file name for the output file
What I wanna do now is appending the file name of the output file by adding the current date to it (e.g. output-20180225.txt) but I am not quite sure how and where to do it.
I know there is a function CURRENT_DATEbut I am not so sure how to apply it to the file name.
Using Compute node and writing the following code:
SET OutputLocalEnvironment.WrittenDestination.File.Name = $yourfilename;
you should use local environment tree and in the you modify you file name like this :
Example File name -- Response_20180621035622333.txt
DECLARE currtime CHARACTER CAST(CURRENT_TIMESTAMP AS CHARACTER FORMAT 'yyyyMMddHHmmssSSS');
SET OutputLocalEnvironment.WrittenDestination.File.Name='ResponseFile_'||currtime||'.txt';
this will set the new file name every time with current time stamps.
Related
I want to read the file from FTP server to pentaho with this particular format.
tcms_oneed_YYYYMMDD.txt
YYYYMMDD is the sysdate.
You pass the date in a parameter to the transformation and use the parameter when indicating the name of the file to read or you can calculate the file name in previous steps and check the box indicating to accept filenames from previous steps, indicating the name of the column with the filename.
I believe you can use a get files and add a wildcard with regex to batch capture all files that are inside a directory.
PS: If you need to read one by one you can play inside a JOB and mark to read one line at a time, I hope that's what you need.
I'll leave a screenshot below of the demo of the test I did so you can view
Wildcard: tcms_oneed_\d{8}.*.txt
Step - PRINT of Get files Pentaho
I'm using Batch Loader to migrate a large number of files. I'm generating my hda file using a script on the source system. How can I specify the target folder? Everything I've migrated so far has no entry for Folder when I view it's info, and I can't find a way to specify it. Sample of my hda below.
# Batch load data for TEST_FILE_3 as jpg
Action=insert
dDocName=TEST_FILE_3_as_jpg
dDocType=Document
dDocTitle=TEST_FILE_3 as jpg title
dDocAuthor=usr_TEST_FILE_3
dSecurityGroup=Public
primaryFile=TEST_FILE_3.jpg
dInDate=01/29/2018
<<EOD>>
In Native UI, change User Profile Layout to top menus and do a search. When Item is returned, access its Info. When info is displayed, add: &IsJava=1 to the url and enter.
This shows all metadata field.s Ones you are looking for are one of these: fParentPath, fParentPathLocalized, ownerFilePath, ownerFilePathLocalized
Here are a few steps to assist you:
Place a file in the folder you want your document to end up in.
Find the document info page and append &IsJava to the url. Something like
http://.../cs/idcplg?IdcService=DOC_INFO&dID=110113&IsJava
Examine the output. You will find variables that indicate the path. Their names will differ depending on what components you are using, but a good guess is fParentPath
Copy name and value to your batch loader file
I have the following two challanges:
I'd like to assert that the filename of an xml file is always equal to a certain string in the file itself
I'd like to assert that in every folder called 'Foo' is a file called 'bar.xml'
How can I do this with sonar? Is there already a plugin for this available?
There's no plugin for that, you will have to write your own.
To do the first point, you can write a sensor that parses the XML files to find if the name of the files exists in the file itself, this should not be complicated.
For the second point, you would have to write a sensor that is executed only on folders.
You can check the "Extension Guide" documentation to find code samples on how to do that.
In Windows Explorer a file has three dates: "creation date", "last modified date" and "acquisition date?" (in Italian it is called "data acquisizione". It is important for me because it is the exact date at which, e.g., a photo is taken (the other two dates change upon trasmission, e.g. when sending zipped files. I know functions to get "creation date" and "last modified date", but I do not how to get the "acquisition date?" of a file. Can you help me? All the best, Ovidio Crocicchi
You are trying to obtain a metadata that is only contained in image files, like JPEG.
I had the same problem and I have not found a solution using standard API. Eventually I solve the problem using the metadata-extractor library that can extract EXIF metadata from image files. The library is really user-friendly, this is the code I used:
File file = new File("C:\\myFile.jpg");
Metadata metadata = ImageMetadataReader.readMetadata(file);
Date acquisitionDate = metadata.getDirectory(ExifIFD0Directory.class)
.getDate(ExifIFD0Directory.TAG_DATETIME);
Windows files have 3 file times, all are exposed in the windows API.
Creation Date, Modified Date, Last Accessed Date.
Any other date is an extended property.
There are S/O articles on reading and setting extended properities via C# using the .net framework.
You should also be able to right click on a file in explorer, select properties from the context menu and see all of the extended properties -- c.f. the MS article Retrieving Extended File Properties
ADDED
You mention that these attributes are not copied when the file is copied. This is a very common problem with these extended attribute as many programs do not copy these extended properties when copying a file because they are only loosely coupled to the file and are not copied automatically whenever the file is copied -- as such, they should not rely on them to store important information, and because of this, they are likely to be poorly supported essentially forever -- since programmers won't rely upon them for important information.
Depending upon the target filesystem, there may be no way to copy these extended attributes at all. E.g. drives using FAT32 (common for USB drives).
I have a scenario where I have to upload a unique named text file.I have used counter in the thread level and using the reference name as file name.say If I want to upload the File_1.txt, I have used something like this file_${counter_refname}.This is working like a charm in single machine but, Now the problem is when I run in the distributed mode slave_1 is uploading the File_1.txt and Slave 2 machine is also uploading the same file File_1.txt. The Target server will not allow to upload file with same name. How to resolve this problem??
Use a CSVData Set that will contain file names.
Create 1 file per distributed node referenced by it and ensure all these files never contain the same file name.
To do it, first create 1 file with unique names then split it in as many files as you have nodes.
Another option is to just generate a unique file name if sequence number is not important by using __UUID function:
http://jmeter.apache.org/usermanual/functions.html#__UUID
There is a couple of functions which can help to identify slave machine and parametrize file name.
__machineName() - returns hostname of machine running JMeter instance
__machineIP() - the same for IP address
Also you can look into __Random() or __RandomString() functions to generate unique files with random names on the fly i.e. with Beanshell PreProcessor as follows:
import org.apache.commons.io.FileUtils;
String filename = "file_${__Random(10000,99999,)}.txt";
FileUtils.writeStringToFile(new File(filename), "a quick brown fox", "UTF-8");
vars.put("filename", filename);
Refer to generated file name as ${filename} where required.