I have an Excel file at a shared location on windows environment. I have data stage server on Unix box. I want to read the excel file and load data to a Teradata table. I need help with reading the excel. One option for me is to transfer file to the server location and access it from there but can i read the excel from the shared folder in windows environment?
I tried to use ftp first in datastage. But getting the below error.
<FTP_Enterprise_18> Error occurred during initializeFromArgs().
<FTP_Enterprise_18> uri : ftp://server/path/file.xlsx is not valid remote file.
<main_program> Creation of a step finished with status = FAILED.
No - it is not possible to read it from a remote location so you will need to tranfer it first (if the shared location is not a SAMBA mount on the Unix maschine).
You can use the "Unstructured data" stage to read the Excel fle one it is on the Unix server.
Related
I would like to know if there is a way to read a file on a remote server using url? For example i have been shared a url to a file on a remote server, i need to read it using bash commands or any tool to retrieve data (eg: view first 50 rows and write to a file) without downloading the original file to local system.
The use case is to avoid downloading/uploading huge files located on the remote server to local systems instead access the file content using the url.
Any resources on this would help.
I am using IBM Mainframe TSO to view files from a dataset. I recently have been told to start FTPing the latest generation dataset everyday to a folder on my desktop. The problem is that the FTP script I have only lets me FTP a file with the exact name I put. Everyday the dataset changes.
How can I write a script that will FTP the latest generation? Please see example below of how the dataset changes:
Dataset
8/30/18 - KIBI.AL242422.REPORT.G6441V00
8/31/18- KIBI.AL242422.REPORT.G6442V00
9/1/18 - KIBI.AL242422.REPORT.G6443V00
9/4/18 - KIBI.AL242422.REPORT.G6444V00
9/5/18 - KIBI.AL242422.REPORT.G6445V00
command.bat
ftp -i -s:Command.txt
quit
command.txt
open sc01.sample.com
USER NAME
PASSWORD
get 'KIBI.AL242422.REPORT.G6441V00'
What your referring to are Generation Data Groups. You can refer to the files in relative form where (0) is the most current. (-1) is the previous generation, etc. In your case you want to access the dataset by relative reference. In your FTP client do the following:
cd KIBI
get AL242422.REPORT(0)
The system will determine which of the datasets is the one you want. Its a nice feature.
I have a quick project I am working on for one of our VPs.
We have a few thousand CAD jobs stored on a network file share. The file structure is such that there is a parent folder for the CAD job. Part of the folder name contains the job number. Inside the folder, there are 1 to many .ini text files that contain the connection information I need.
What I need is a programatic way to search through all the folders and extract the job number from the folder name, and all the connection values from the ini files.
For example for a folder named CM8252390-3, the job number is 8252390-3. Inside this folder are 3 ini files. Inside the ini files are that look like this:
[Connection]
Name=IMP_Acme_3.5
[Origin]
X=-15.044784
Y=19.620095
Z=44.621395
So my program needs to give me the following result
Job Connection
8252390-3 IMP_Acme1_3.5
8252390-3 IMP_Acme2_3.5
8252390-3 IMP_Acme3_3.5
8254260-1 IMP_Acme3_2.4
8254260-1 IMP_Acme3_4.1
...continued for all folders in the network share
Any suggestion on the best way to do this. I am primarily an Oracle PL/SQL developer, but have some basic Windows batch and Unix shell experience. If I can get the data loaded into Oracle tables, I can search using PL/SQL tools, but is there a better way using shell, batch, or other tools?
Thank you.
I think this is a job for Powershell or vbScript. It would be easy to use these tools to write the information you need to one file.
This file should be written to an Oracle directory.
grant read permission to a database user on this directory
use utl_file to read the file or treat the file as an external table and expose it as a view
schedule a regular OS job to refresh or rebuild the list
I'm using "COPY SELECT ... INTO file" statement from within application code. After file is ready, next step is to move the file to different location. The only problem is that file created by MonetDB has only root permissions so my application code can't touch it. Is there a way to configure MonetDB so dumps are saved as specified user? or my only solution is to iterate results in batches in application and save to file that way. Dumps can range from several MB to 1GB.
You could run MonetDB as the same user that your application server is configured for. Also, both your application server and MonetDB probably should not run as 'root'.
There is no direct support to export files with different permissions. You could try configuring the umask for the user that the starts the DB.
I'm trying to write a standalone java file to connect to hadoop db with HiveJDBC in a windows machine and load data for table from a local csv file.
I'm getting on error "Error in semantic analysis: Line 1:23 Invalid path 'd:/Sample2.txt': only "file" or "hdfs" file systems accepted".
What i'm trying is right or wrong? Can we load data from client windows machine for hadoop db.
I think you need to use something like:
file:d:/Sample2.txt
The error clearly states that the api accepts only file:// or hdfs://.
Check this question too - How to make your mapper write on local file system in hadoop