What is the best method for reading files from a Windows shared folder using a DAG on a remote Apache Airflow server?
My current process needs to be executed a remote Airflow server, but it needs to read and write files (which the client and my process update) that are stored on a shared net Windows folder.
Airflow is installed on remote Linux server (same network)
Windows folders are just standard UNC paths where people have access based on their NT ID. These users are saving files which I need to retrieve.
The files formats are .csv, .xls and .xlsx, which I plan to convert as Dataframes in my Airflow process
Best if I can save my user and password within an Airflow connection to access it with a conn_id
Related
I m trying to restore a MongoDB dump file present on my windows machine into MongoDB server created inside azure portal as azure virtual machine, I have created this virtual machine and named it mongodbserver then I m connecting via SSH with the client for this case is the power shell CMD that I'm connecting with form my windows machine, now my questions are:
1.In order for me to restore MongoDB dump file where should it be in the server side? or either on my windows machine still I will be able to get it from there and restore it successfully in the server?
2.Can I use Filezilla client to move the database dump file into the server but the problem with this i don't have the password for the virtual machine created since I'm connecting via SSH and Filezilla will require password for it to connect.
3.After restoring the database successfully how can I connect this virtual machine with the MongoDB compass and also how can i get the database Uri in order for me to connect it with my spring application.
Any suggestions will be much appreciated!
I have a windows service with an accompanying config file. The service is going to be installed on a bunch of lab computers and monitor things such as diskspace, network connectivity, logins, etc.
After a set interval, it will report the statistics to a Database. In order to connect to the DB, I currently have the application user's name and password stored in the connection string, which can be found in the config file.
What's the best way to secure this, such that regular users cannot access it? We do a similar thing here with our webapps using the Web.config, and we have IIS setup such that people can't browse into the directory and read it.
Is there a best practice to securing a config file stored locally on each machine for a windows service?
Use the same method that you would use for web.config
1- Go to the windows service folder, where configuration file is located
2- Rename the config file to web.config (back it up first)
3- Execute:
c:\Windows\Microsoft.NET\Framework64\[the asp versino]\aspnet_regiis.exe -pef "[name of the config file to encrypt]" .
4- Rename the web.config back to the original name
More info here
https://msdn.microsoft.com/en-us/library/zhhddkxy.aspx
I am trying to figure a way of connecting 2 web server's file systems together so they can access each others files natively. The servers are Windows 2012 and connected directly to the internet via public IPs. One server will be for storing large files, the other for the scripts and database (web server).
Essentially, I need a way to securely map a drive/folder between these servers so they show as folders e.g. the "d:\www\assets" folder is actually on the other sever (junction). As this link needs to be accessible to the SYSTEM (apache service) not a single user, a mapped drive is not ideal. Open, unsecured shares are also not a good idea.
Playing with junctions and links in the console doesn't show any method to provide a login/password to the remote system. The junction is created but inaccessible. If I map a drive, it is only for that user and not available to the SYSTEM account that Apache is using. If I run Apache as a user and map a drive as that user, it likely won't survive a reboot or work without being logged in on the console all the time.
Are there any native ways to hook these 2 servers together securely? I have full admin access on both servers and can create as many users as required, but they are not in a domain or potentially even on the same subnet.
You may be able to use to a directory symlink via cifs/windows share considering you have have access to the local disk on one of the servers and the your sharing the the folder you want to symlink
for example:
on server a:
1. navigate to server a's local disk: d:\www\
2. mklink /d assets \serverb\assets
option 2DFS (unconfirmed)
if you can create a dfs on server on one user windows boxes, i believe you set dfs target to point folder assestes to \serverb\assets
I want to know how to house keep(delete) the files which there in remote server by connecting using SFTP based retention period for n customers and each customer will have different directory.
Note: We don't have privileges to write script and run in remote server. We need to write script in our server to connect remote by using SFTP protocol.
Thanks,
Dasaradha
If you are on Linux sftp can run in batch mode and delete files on remote site: http://linux.die.net/man/1/sftp.
I need to use nodejs to write file to a password protected shared folder on a remote machine running Windows server 2008 r2, how can I do impersonation things within nodejs like .net app does?
Thx in advance!!!
PS:
Sorry for the unclear description. The folder is set to shared across the internal network on the remote windows server system, the folder is set to "readonly" for the windows account explicitly created by the admin. The nodejs server is on the same internal network and need to access the remote shared folder, thus I think nodejs need to impersonate the windows account on the remote machine to read contents from the shared folder. My question is how to do the sorry for the unclear description. The folder is set to shared across the internal network on the remote windows server system, the folder is set to "readonly" for the windows account explicitly created by the admin. The nodejs server is on the same internal network and need to access the remote shared folder, thus I think nodejs need to impersonate the windows account on the remote machine to read contents from the shared folder. My question is how to do the impersonation stuff for nodejs?
Late to the party - but since there is no accepted answer...
You'll need to launch your node process using the account that has access to the network share. How this is done differs based on your setup. For example, if you're runnning node on a lLinux server trying to connect to the remote Windows server, then it's not going to be straightforward with node.
However; if you're running node on a Windows server, trying to connect to a remote share on another Windows server (which is what it sounds like your situation probably is). Again, you'll have to run your node script as a user with access to the remote share. If you need to do it from code, I'd suggest taking a look at my module node-windows. It doesn't provide remote share access specifically, but it will allow you to run command line code with a different account from within your node script.