It would be nice if I could download the source code of spark from github, then build it with sbt on my windows machine, and use IntelliJ to make little modifications to the code base. I have installed spark before on windows quite a few times, but I just use the packaged tarball and not the source code. Has anyone built the source code on a windows machine before?
You need to account also for the simple differences in \n\r and \n. So you should use dos2Unix utility for Linux and make sure that you are using an up to date version of Cygwin when installing and running hadoop utils.
I found the spark developer tools page and it was very helpful. I needed "build/sbt compile"
http://spark.apache.org/developer-tools.html#reducing-build-times
Related
I'm looking for a way to build an Erlang project on Windows. I have Erlang installed and all project files, including makefile, cloned from GitHub. I would like to build the project as if I was using make command on Linux and run it. What tools can I use to do that?
I'd try cygwin. https://www.cygwin.com
http://erlang.org/doc/installation_guide/INSTALL-WIN32.html
You can either build erlang with cygwin or use gnu-make in combination with a native windows build of erlang
If you're asking "how do I run make in Windows?" then you can use Cygwin, the newly-available Bash shell, or the MinGW tools. These will all give you some level of ability to run make, though not every makefile will work.
If you're asking "how can I build an Erlang project using a makefile?" then you are looking for erlang.mk. Note that Rebar3 (which is configured with Erlang terms and looks nothing like a make system) was recently selected as the "official" build tool, but erlang.mk is quite popular and is well-maintained.
For some reasons I prefer to avoid using Python.
But all I found is either Python scripts or unix specific instructions or something that doesn't have downloadable executables or unclear how to install it in windows.
Can you suggest any tool or service?
There is a plugin here for that.
Maybe just install Win-build of CLI-Mercurial or standalone TortoiseHG (plus HG-Git)?
I haven't Python per se on my box, only THG (which have Dulwich) and HG-Git anf Git-repos are pushable
I am trying to run Apache Spark on Windows 7. At first I have installed SBT by msi, then extracted files from spark-1.0.0 to program files by 7-zip. In the command line, I wrote the following:
spark-directory: sbt/sbt assembly
After a few seconds of processing, I got errors like:
-server access error: connection timed out
-could not retrieve jansi 1.1
-error during sbt execution: error retrieving required libraries
-unresolved dependency, jansi 1.1 not found
Could you please give me some advices about running Spark on Windows? I am looking for the right way because I am completely new with this technology. Regards.
You could use the pre-built spark from here
The scripts inside bin folder works in windows 7.
You need to set HADOOP_HOME variable in your path.
spark on windows for more information
If you are using building with sbt approach, then you'll need git also.
Install Scala, sbt and git on your machine. Download Spark source code and run following command
sbt assembly
In case,if you use prebuilt release,Here is the step by step process :
How to run Apache Spark on Windows7 in standalone mode
I know this may be a stupid question to ask but I want to be sure about using node.js on windows. Can I use node.js on windows with any way ?
Thanks
There's Windows binaries for Node itself, but it's crippled without the package manager npm.
Update: As of December 2011 there's a Node installer for Windows that bundles the package manager npm. How this rocks! http://nodejs.org/
Yes, there is currently official unstable version of node.js for windows. Check out Windows executable in download section.
The 0.5.x (unstable) branch comes with a pre-compiled node.exe. Odd branches are for development though, so it would be good enough for hacking around with (has some feature gaps), but you'll need to wait until 0.6.x for a stable "off-the-shelf" solution in windows.
Yes you can.
Download the node.js binaries from http://node-js.prcn.co.cc/
Unzip it to any location, you need 7zip to unzip it.
Then put the node.js bin directory into path.
Use terminal to run node
With 0.4.x and earlier versions it is better to use pre-compiled binaries from http://node-js.prcn.co.cc/ . To extract, you can use WinRar in addition to 7zip.
The 0.5.x branch barely works in Cygwin, and official support for Cygwin was dropped, so you will have to use the precompiled the native windows exe http://nodejs.org provides. NPM doesn't work with 0.5.x prior to 0.5.8, but for 0.5.8 and later there is an instruction how to get NPM running:
http://npmjs.org/doc/README.html#Installing-on-Windows-Experimental
I installed the Node.js Azure SDK from http://go.microsoft.com/fwlink/?LinkId=254279&clcid=0x409 . Open up a command prompt and type 'node' - you're good to go.
I have set this report on my experience setting a portable node.js ambient on Windows.
In order to leverage it you must be a full newbie.
It is not installed the Windows way, but instead it's made of expanded archives.
After doing the steps you end up with a command-line invironment capable of running node.js together with a few more tools usually found in UNIX-like environments.
It's here: http://bit.ly/YbGYVg
I've got some perl source code here, how do I build it on Windows, to get a windows binary that I can work with?
Usage of external tools normally comes with compatibility issues, random errors etc. You are better off using the inbuilt perl 'pp' tool. Install PAR::Packer (which includes the pp tool) module and then read the manual for it...
It allows you to pack your perl scripts to executables, and has options as what modules and dependencies to include, I've used it on winXP and win7 and never had an issue with any executabe produced.
pp manual
I've found Cava Packager to be just what I needed.
(source: cavapackager.com)
How to compile Perl scripts into EXEs
Download ActivePerl 5.10 for Windows.
Install it.
Restart your PC.
Download Cava Packager
Install it.
Open it.
Make a new project choosing a blank folder.
Scripts > Add..
Choose your .PL script file
Perl library > [...]
Choose "C:\Perl\bin\perl510.dll"
Add
Choose "C:\Perl\lib\"
Save
Build
You could use the Perl Development Kit from ActiveState to "compile" your script to a .exe file. I used it to create binaries of MRTG and a couple tools more to be deployed on windows servers running as a service. There used to be another product (from IndigoStar or something) called perl2exe I think to get the same result.
Just a note that Cava Packager also supports creating executables from Perl code on Linux and Mac OS X in addition to the original Windows version.
Note: As indicated by my name, I am affiliated with Cava Packager.