I have a requirement to generate the target file in Informatica with date/time appended to it. How will the Informatica FTP connection identify such dynamic file name with date appended to its name?
Also I would like to know if it is possible to FTP multiple files at a time via Informatica FTP connection. Please someone help me on this.
Its actually pretty simple, you just have to use the part of the file name that is constant and then place a *
for eg:
Myfile_20190607.txt
Myfile_20190507.txt
If i specify Myfile_2019* , this is good enough to pickup the files soecified above. You may have to play with the * and criteria to fit the files that you need.
Note: if you are sending files to third party, try to use SFTP instead of plain old ftp and most organization blocked ftp to outside ip's.
As far as I know until Informatica 9.x, it is neither possible to generate dynamic filename nor create multiple files using FTP connection. Only option was to create the files on Informatica server and then run a script to FTP them over to the destination server.
Here is how:
edit your workflow, choose variables tab, create a workflow
variable with datatype NSTRING; assume the variable name is
$wf_timestamp;
create an assignment task and assign TO_CHAR(SYSDATE,'YYYYMMDD')
to the variable in the assignment task;
edit session: Choose Mapping tab, choose your target; then
Connections; then edit FTP Value; then in the Remote Filename
attribute, enter your filename with the timestemp, eg,
myfile_$$$wf_timestamp.csv;
put your assignment before you session in your workflow.
that's it.
Related
I am using IBM Mainframe TSO to view files from a dataset. I recently have been told to start FTPing the latest generation dataset everyday to a folder on my desktop. The problem is that the FTP script I have only lets me FTP a file with the exact name I put. Everyday the dataset changes.
How can I write a script that will FTP the latest generation? Please see example below of how the dataset changes:
Dataset
8/30/18 - KIBI.AL242422.REPORT.G6441V00
8/31/18- KIBI.AL242422.REPORT.G6442V00
9/1/18 - KIBI.AL242422.REPORT.G6443V00
9/4/18 - KIBI.AL242422.REPORT.G6444V00
9/5/18 - KIBI.AL242422.REPORT.G6445V00
command.bat
ftp -i -s:Command.txt
quit
command.txt
open sc01.sample.com
USER NAME
PASSWORD
get 'KIBI.AL242422.REPORT.G6441V00'
What your referring to are Generation Data Groups. You can refer to the files in relative form where (0) is the most current. (-1) is the previous generation, etc. In your case you want to access the dataset by relative reference. In your FTP client do the following:
cd KIBI
get AL242422.REPORT(0)
The system will determine which of the datasets is the one you want. Its a nice feature.
I currently have access to a third party's FTP server which, upon login, automatically redirects me to a directory that does not contain the files I am trying to download.
ftp://ftp.fakehost.com -> ftp://ftp.fakehost.com/uselessDir
My files are in ftp://ftp.fakehost/usefulDir.
This ftp server does not support directory traversal so I cannot get to usefulDir by simply modifying my url. FileZilla works since I can execute specific ftp commands to get to the directory I want.
Can a Data Factory FTP service or dataset be customized to work around this problem since Data Factory cannot access the usefulDir directly ?
Please correct me if I doesn't understand your question correctly. Have you tried create a dataset and manually put the usefulDir in folderPath property directly, instead of using the Authoring UI to navigate to that folder (which is not possible based on your description.)
I have a quick project I am working on for one of our VPs.
We have a few thousand CAD jobs stored on a network file share. The file structure is such that there is a parent folder for the CAD job. Part of the folder name contains the job number. Inside the folder, there are 1 to many .ini text files that contain the connection information I need.
What I need is a programatic way to search through all the folders and extract the job number from the folder name, and all the connection values from the ini files.
For example for a folder named CM8252390-3, the job number is 8252390-3. Inside this folder are 3 ini files. Inside the ini files are that look like this:
[Connection]
Name=IMP_Acme_3.5
[Origin]
X=-15.044784
Y=19.620095
Z=44.621395
So my program needs to give me the following result
Job Connection
8252390-3 IMP_Acme1_3.5
8252390-3 IMP_Acme2_3.5
8252390-3 IMP_Acme3_3.5
8254260-1 IMP_Acme3_2.4
8254260-1 IMP_Acme3_4.1
...continued for all folders in the network share
Any suggestion on the best way to do this. I am primarily an Oracle PL/SQL developer, but have some basic Windows batch and Unix shell experience. If I can get the data loaded into Oracle tables, I can search using PL/SQL tools, but is there a better way using shell, batch, or other tools?
Thank you.
I think this is a job for Powershell or vbScript. It would be easy to use these tools to write the information you need to one file.
This file should be written to an Oracle directory.
grant read permission to a database user on this directory
use utl_file to read the file or treat the file as an external table and expose it as a view
schedule a regular OS job to refresh or rebuild the list
My orchestration receives a message that contains a file name and I want to pick that file from my FTP. I can configure FTP receive port to receive all files from some folder in the FTP, but how do I receive a file with specific name?
The process looks like
I would rather recommend your solution of writing a custom .NET component which will fetch the file from FTP location (you can call that component from your expression shape).
Dynamically creating Receive Ports/Receive locations and later removing them won't scale and possibly will get you into serious trouble.
I'm not sure if this link helps - specifically the CreateFtpReceiveLocation method - i.e. programatically adding a receive location (pseudo dynamic receive location).
You would also need to remove the location afterwards. I'm guessing that you can also set the FileMask on the Transport Properties of the 'dynamic' Location to the filename in the Custom Props of the TransportTypeData, and would need to remove the Location once you are done with the file.
I need to set up some sort of infrastructure to automatically FTP some files from one remote server to another. The FTP transaction will occur on a scheduled basis. Both these servers are Windows boxes and the location of the files that need to be FTP'ed will depend on the current date (the folder they sit in will be named the current day's date).
I would really hate to have to write something like that from scratch, so are there tools/utilities/the likes out there?
Windows command line ftp will read input from a text file. For example:
ftp -ni < ftpscript.txt
Where these options are used:
-n - Suppresses auto-login upon initial connection.
-i - Turns off interactive prompting during multiple file transfers.
The ftpscript.txt would look something like this:
open ftp.mycompany.com
user someuser password
binary
get /somedir/myfile.dat
It should be reasonably straightforward to write a script that outputs an FTP script containing the correct file name.
You would think that this would be an easy task. It isn't.
The best tool I've found is SyncBack and I think there is a free/lite version available.
If you're in a Windows environment consider using PowerShell and the System.Net.WebClient .NET Framework class like this post. This is a download example, but the WebClient object also provides an UploadFile method.