This might sound like a strange request, but I'm hoping I have more luck here than I've had googling for the same topic.
I'm searching for a Windows based application that allows me to upload files to an FTP server via the command line, across as many threads as possible.
I'm currently trialing WinSCP, which has a simple scripting interface, that I can invoke from the command line. However, whilst it's a) windows based, b) command line driven/scriptable it doesn't make use of any multi threading to synchronise uploads of large files.
It seems I'm forever limited to achieving 2 of my 3 goals. For example, FileZilla is a) windows based and b) multithreaded for uploads, but unfortunately lacks any command line or scripting capabilities :/
Does anyone know of anything that might be able to achieve all 3 of my desires?
Well, FileZilla is GPL, so you could fork it and create a command-line/scriptable version sans the GUI. You'd have to implement the scripting engine though.
Alternately, you could implement a client on top of Twisted FTP (in Python).
Related
This is question is about the general architecture, I do not require anyone to solve this little hack for me, although I won't be angry if someone does ;).
Suppose I have a web app that spawns standard unix processes (like Travis CI). While it seems simple enough to pick the stdout of such a process, I'd rather like to make the whole thing asynchronous (like e.g. Travis). So I thought of passing the whole output through a websocket and into some web-based terminal emulator.
However, the only emulators I could find were fully interactive (i.e. they allow for user input and thus have some custom server-side component). My goal would be to have a piece of client side code and just stuff the output into it.
So what is necessary to create a websocket, attach it to the stdout of a server-side process (preferably emulating a tty for colors and fancyness) and display a terminal client-side? I recon there are control codes to distinguish a tty from a text file and these control codes need to be encoded on the websocket somehow, but is there some documentation on this?
I have done this for .NET applications. I think this may be worth for you as example.
I have a small .NET project named NLog.Contrib.Targets.WebSocketServer that is a log watcher with WebSocket and AngularJS. Basically, it broadcasts the data that is being logged through a WebSocket, and there is an AngularJS directive that shows the data. How to highlight data is more a presentation stuff, so it will depend on the framework you use. Basically, this component attaches to whatever .NET application that uses NLog as logging framework, so you can try to find some extensibility point in Travis yourself and attach your thing there.
About attaching to stdout, I have a proof of concept about a web interactive CMD.exe also in .NET, although you can disregard the stdin part. If you use Mono, probably is the same thing than in Windows.
I think this is very similar to what you are looking for. If you have a more specific question let me know.
You can use STDWebsocket in order to achieve this. For examples, simply read the index.html script tag. It should solve your problem (or anyone that go through this question)
How can I access Windows Search index data from Emacs? Knowing this would be useful for example when writing a minor mode that integrates Windows Search into anything mode or ido-mode.
By Windows Search, I mean the Windows 7 feature that lets you find documents by pressing Start and typing part of document file names (or part of document contents).
Here is a little Python script providing a command line utility for Windows Search. You need to install Python for Windows extensions to use it.
Accessing Windows Search from within emacs is going to be a bit difficult because the API Microsoft provides is strongly skewed towards the Microsoft programming environment. Judging by the MSDN docs, the easiest path would be putting together a SQL query that Windows Search will accept and sending that to a PowerShell/VB script that knows how to send that query to Windows Search. You'd then tell anything/ido/icicles to incrementally send input to such a script, parse the results, and display those.
The task that you are attempting is very difficult, and much of the difficulty comes from the fact that you are trying to get two programs from very different worlds of programming to talk to one another. Completely apart from the FSF/GNU folks actively disliking Microsoft, the design of the Windows API means that the least-effort way of dealing with Windows is to use the Microsoft toolchain. This in contrast to the Unix "API" of sending plain text through intermediary programs, pipes, and sockets.
I'm probably dreaming here, but am wondering if there's any possibility of completely embedding a minimal CouchDB engine within a Windows application, such that the app can be run without requiring installation (of CouchDB/Erlang) on the user's computer.
I already provide this slimmed down / bundled ability - check here https://github.com/dch/couchdb/downloads and specifically the lean bundle at 16MiB erlang + all couch love here https://github.com/downloads/dch/couchdb/couchdb-1.1.0+COUCHDB-1152_otp_R14B03_lean.7z
Some brief notes on bundling and embedding couchdb on windows at wiki.apache.org/couchdb/Quirks_on_Windows including how to hide the erlang window (erl.exe -detached) at startup.
Ask on CouchDB #user mailing list if you want more info or help while you have a crack at this.
While not a code solution, you could use one of the bundling applications that can embed files and other files into one executable. One example would be BoxedApp.
Why bother. It is so easy to install Erlang on Windows. Just bundle up the whole thing including the erl.exe binary and have your installer unzip it into a folder. The only thing that you would need to change would be the batch files, or better yet, discard them and write your own batch file to start up CouchDb. Also, it is a good idea to use a different port that either the normal Erlang port (or the usual CouchDB port) and maybe even get Erlang to use localhost as its "shortname".
The CouchDB wiki does provide at least a few tips for Integrating CouchDB into your Windows Applications. YMMV, from what I can tell it's more or less just tips on creating a relocatable build. You'll want to likely generate a solid random admin user/password into the local.ini file during the install process and set up proper permissions on all created databases (to protect against any potential cross-site scripting vulnerabilities) in addition to ensuring the socket binding only happens on the default localhost interface.
When I develop web applications I'm frequently need to sync files from a working folder to external server or another folder. I like keeping my code separated from the web sever.
In open source world there is the eclipse with file sync that does the job pretty well. Unfortunately I can't find any good replacement for Visual Studio.
I've only found two generic solutions:
- Winscp which is pretty good but stucks when a file is locked and ask for confirmation. Which is quite annoying.
- DSynchronize which works pretty well (ie. doesn't ask questions) but doesn't have filters so I can't tell it not to sync my .svn files or web.conf :(.
Do you know any good way to achieve realtime synchronization in Visual Studio or windows?
I doens't have to have gui in fact I would love to see a command line solution like a powershell command that outputs modified files.
I've ended up using Mercurial (to skip the .svn files) and DSynchronize to sync files
I would give a try to immortal classic - rsync. There is cygwin enabled implementation for Windows called cwrsync: http://www.itefix.no/i2/node/10650 . With proper configuration (potentially with some fine tuning with scripting as well) it will do perfectly.
If you would like to have bi directional synchronization, the Unison may be the answer:
http://www.cis.upenn.edu/~bcpierce/unison/
If you are looking for something even fancier, you might give a try to one of distributed file systems available, like CODA (I'm afraid decent Windows systems aren't supported yet): http://www.coda.cs.cmu.edu or native DFS solution from Microsoft, however I'm afraid the set up is too hassling (if not impossible in your case) since it's targeted for enterprise solutions:
http://technet.microsoft.com/en-us/library/cc753479(WS.10).aspx
Of course DFS option probably won't support filtering you are interested in.
I often have to transfer large files >50GBs sometimes >100GBs between drives both internal and external during backups of our networks email servers. What is the best method of transferring these files? Command Line such as XCOPY? Possibly something robust enough to continue the transfer if interrupted due to time limits or network issues.
Check out robocopy. From Wikipedia:
robocopy, or "Robust File Copy", is a
command-line directory replication
command. It was available as part of
the Windows Resource Kit, and
introduced as a standard feature of
Windows Vista and Windows Server 2008.
For free, I use SyncToy (from Microsoft). That way if something fails it doesn't abort the whole transfer.
The next best for non-repetitive tasks IMHO is XCopy.
I have used Teracopy with good success.
I get asked this question every now and again and I always say the same thing. Microsoft Background Intelligent Transfer Service (BITS). This is the same technology used to deliver large service packs and such to workstations. Some of the features:
Network Throttling
Asynchronous Transfers
Auto-Resume
Priority Levels for Downloads
Proven Transfer Mechanism
For those not wanting to deal with the command line syntax you can explore wrapper applications, such as SharpBITS.NET, that provide a GUI interface.
I use CopyHandler and find it does the job well.
Well i use http://itrnsfr.com to transfer my big files online. I wish they extend the quote over 2 GB they currently offer to free users