We're looking for a way to run someone with experience with ILMN's NextSeq platform. We're currently trying to convert native bcl files to fastq, but the platform doesn't provide that option by default.
ILMN offers this through the BaseSpace community website/appstore, but this is problematic for us b/c running bcl2fastq through BS doesn't allow us to specify any parameters.
There's a converter tool ILMN offers through their website, which runs on Linux. This is problematic for us, too, as we're essentially a highly regulated (think FDA-level, 21CFR.11 and all those things) Windows environment, so "just" putting a Linux box on-prem is not as simple as it sounds.
So we looked into running bcl2fastq through a VirtualBox VM on top of Windows, but that was rather disappointing; it takes roughly 2 days to run bcl2fastq on a VM, versus under one hour on a "native" Linux setup.
I don't think we've exhausted all of our options yet though to get bcl2fastq to work somehow in a Windows environment. Amongst others, the source code of bcl2fastq is actually publicly available, so perhaps we could re-compile the whole thing for a Windows architecture as well.
Related
This is a conceptual question and I hope it fits into Stackoverflow's question and answer style. I wonder what the concept of installing applications is good for. In my naive understanding of operating systems we do not need a registry and to use an application it should be enough to just copy the executable and files onto your drive and launch that.
Am a Windows user but also worked with Linux a bit and noticed that there are package managers instead of installers. But even those do more than just a copy instruction, I guess.
I do not think that all the installers exist only because the common user expects them out of steady habit. So what is the advantage of installers in contrast to developing applications which are designed to run out of a single folder and copy that over?
I would really like if someone could explain that concept.
Installing applications is a way to embed them in the OS. It's a kind of standard, you offer procedures like installing and uninstalling that should have the same functionalities for all applications (even "change" under Windows).
Countless times I've "installed" applications with a single shell script that came with them, and then had troubles removing such programs, having to look for single files. If the programmer uses the standard of the OS to make an executable that can be installed, that won't happen.
You can also easily view a list of the installed programs at any time.
Under Linux, additionally, if we're talking about a package manager, it is convenient for the user to have an easy way to download and install a program by just typing its name.
Last but not least, some applications are required to be installed and recognized by the OS (for example services in Windows).
For a university course I have to write a http server which is supposed to run on both Linux and Windows.
I have got a humble Linux machine which I don't think can handle any kind of heavy virtual environment, neither I'm willing to go through the hassle of installing it.
This is the first project of mine complex enough (I estimate ~1.5 months to develop) to require an environment sufficiently comfortable to alternate rapidly between short coding and testing sessions (the latter on both platforms, of course).
So, I was wondering what could be the best set up for this situation. I think testing it on Wine would be ok (it is not a real-world thing, after all), and I installed MinGW for the Windows-targeting part.
Basically, a simple well-written makefile could solve my problem... It should build both the Linux and Windows binaries and place them in the respective folders (the Windows one in the Wine sub-tree) and I'm all done! But I feel very inexperienced in this thing and I really don't know where to start. Maybe the make manual, ahah!:)
Thoughts, suggestions, anything I didn't think/know!
Thank you!
(PS. I'm planning to use emacs as editor, or maybe learn vim. Unless eclipse provide some kind of skynet-like plugin that entirely solve this problem...:)
You're on the right track. It's not that complicated, really, thanks to MinGW. You basically need two things:
The code has to be portable across the OSes. MinGW has some POSIX support, but you'll probably need to either use Cygwin in order to be able to use the POSIX interface or have your own compatibility layer for interfacing with the OS. I'd probably go for Cygwin as then you can code only against POSIX and won't have to test and debug your compatibility layer. Also, make sure you won't use any external libraries that are OS specific. Non-portable code often results in a compile error, but make sure you test the application thoroughly anyway.
The toolchains for targeting Linux and Windows. You already have them, you just need to use them correctly. Normally you'd use a variable like $(CROSS_COMPILE) as a prefix when calling the toolchain during cross compilation. So when compiling for Linux, you call gcc, ld, etc. (having the CROSS_COMPILE variable empty), and when compiling for Windows you call e.g. i486-mingw32-gcc, i486-mingw32-ld etc., i.e. CROSS_COMPILE=i486-mingw32-. Or just just define CC, LD etc. depending on the target.
I wrote a small game on Linux and made it run on Windows as well. If you browse the code, you can see the code has next to no #ifdef jungle (basically just some extra debugging features enabled for Linux), and the Makefile is simple as well, with no complicated handling for cross-compilation, just the possibility to override CC etc. like it should be. As lots of important open source software is written this way (especially software that's used by the desktop and embedded devices), you should also be able to find lots of other examples on how to set up the build environment correctly.
As for testing the application on Windows, I think the best option is if you can find a real Windows machine somehow. If you do everything correctly, it should run the same as on Linux and you won't need to continuously test your application on both OSes. If testing on a Windows machine is not possible, a VM would be the next best choice, though it would probably be more difficult to set it up. Wine is a good backup plan, but I don't think you can be sure your application works well on Windows if you only tested it on Wine.
Someone I used to work for emailed me out of the blue and said they want me to update a VB3 (!!!) program I wrote for them because customers are reporting having problems running it under Windows 7. They have lost the source code (natch). Is there a decompiler for VB3?
For decompiling VB "DoDi VB3 decompiler" is the best tool available. The results are pretty good.
However the original is anno 1997. VB3 is 16 bit code only + still some bugs + no source code. These makes that tool hard to get running on windows beyond Windows XP and limits it's use.
DoDi's VBDIS3 it self is also written in VB3 - so about 10 years later just for fun I cracked it to make the decompiler to decompile it self.
... and ported it to VB6!
So now you've the source code !!! :D
... and it's 32 Bit code + some bugfixes and improvements to the original:
DoDi's Visual Basic 3 Decompiler [Reloaded]:
http://vbdis4.angelfire.com
Screenshoot:
in case that website get lost Google for
'VBDIS3.67e_Reloaded_Rev3_DoDi_s_VB3Decompiler.7z'
or try the internet archive
http://web.archive.org/web/20090301170633/http://vbdis4.angelfire.com
I remember looking for Visual Basic decompilers/disassemblers a long time ago to no avail. There may be things out there now though. The most I could come up with was PE Explorer: http://www.heaventools.com/overview.htm
Have you suggested running your application under Windows Compatibility mode: http://www.sevenforums.com/tutorials/316-compatibility-mode.html
Edit: Further Googling: http://www.w7forums.com/visual-basic-3-0-compatible-64-bit-version-t5146.html
I maintain legacy 16-bit VB3 software
that is compatible with Windows 7 and
have done a lot of research in this
area.
16-bit applications will not natively
run in Windows 7 64-bit full stop.
This is because when a 64-bit capable
CPU is started in 64-bit mode, it
cannot change into 'real mode' (the
mode used by 16-bit applications)
without a hard reset. If a processor
is in 32-bit mode, it can swap between
real mode and protected mode at will.
The only workaround here is a virtual
machine- Windows 7 comes with a free
version of Windows XP which is usable
inside a custom version of Microsoft
Virtual PC. Google for "Windows 7
Virtual XP Mode". Once your copy of
Windows 7 has been validated as
authentic, it will allow you to
download Virtual PC and the XP Image.
It's definitely not a pretty solution,
but the only option for my customers
who have gone ahead and bought a
64-bit OS without checking to see if
their installed software is 64-bit
compatible.
Unlike the later versions which used native code and are generally not reliably decompilable, VB3 (and, I think, VB4) could be usually decompiled to almost original code. The keywords you need to search for are "DoDi VB3 decompiler" or "vb3dis". Here's a page that seems to have a copy.
Really? VB3, if it runs fine in XP, should be okay in the Windows compatibility box.
What I would do is to say that you couldn't find anything suitable (which is almost certainly true) but that you may be able to fix their problem for a moderate fee. Then, hopefully it's just a matter of setting a flag on the application (you may want to check this first to ensure it won't be too much work).
Money for Jam.
The other option is that you can offer to rewrite it for them in a more modern setting, using your vast knowledge of the application.
However, I'm surprised you don't have a copy lying around. I keep just about every piece of software I've ever written just in case (with permission for work-for-hire stuff of course), even down to the Fortran assignments I did at Uni back in the early 80s.
That's a good habit to get into, although I'm starting to wonder whether I should perhaps free up some space by ditching the Fortran :-)
There are a number of things you can do to help them without the source code. You could apply an external manifest if the application will only work when it is elevated. You could advise them to install the application somewhere other than Program Files (generally an awful idea, but might work in this case.) You could apply a compatibility setting or teach them how to install it into "XP mode" so it runs in an XP virtual machine.
Failing all of those, you could offer to rewrite it in VB.NET so they would get a more modern ui, Windows 7 features (not just capabilities) and would actually own the source code for their app. That might have value for them.
And yes, you should have kept the code. I have CDs burned from my old projects going back to the dawn of time (at some point I copied things from 3.5" backups to CDs while I still had some machines that could do both) and I have made more than one previous client happy by sending them the CD. It is a really cheap marketing investment, really.
It sounds like this is an application that is sold to customers, and is having trouble or doesn't work under Windows 7.
Consider advising the customer that the application should be:
modernized, thereby easing the ability to add features/maintained
otherwise run in XP Compatibility mode for their Windows 7 customers. Surely that number will only grow, and would become a support issue. Goto previous bullet.
Seriously though; your customer should realize that their software needs to be modernized. Hopefully you can persuade them of that long term goal; perhaps they already know that.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 7 years ago.
Improve this question
What are your recommendations for setting up a development environment in Windows, especially when not using an IDE. I am attempting to familiarize myself with Windows, and I feel a bit lost. What do developers commonly work on, especially when developing in multiple languages (e.g. Java, Python, C) in the same time.
So far, I have been a hardline using Mac+Linux environments where I love my command line a lot. I pretty much run all my compilations, testing, code repository commands in the terminal and have multiple terminals all the time. Terminal features (e.g. integrated bash completion, easy copy&paste, easy to setup environment variables) and package management tools (e.g. apt-get, port, fink) are quite handy. I dislike hunting down different websites to install their latest binary build.
Coming back to my question. My question is two fold:
What's commonly used? Do developers on Windows commonly use command line, or just be satisfied with an IDE?
For comers from Linux/Mac world: what do you recommend to get up to speed?
NOTE: I realize that a lot of Windows developers haven't used Linux, so they may not know what I'm talking about when it comes to Linux environment.
It's almost unheard of to not use an IDE for Windows development.
I started programming in the early 80's so grew up on the command line, but nothing beats a modern IDE for productivity.
By far the most common choice is Visual Studio, though I have also used #develop (open source) and find it a fine platform to get up to speed with.
I have used Eclipse extensively (on Linux and Windows) and find Visual Studio to be easier to use. I especially miss options for debugging under windows such as moving the instruction pointer around during debug and change-and-continue (change the code, within limits, while debugging, move the instruction pointer back if necessary, and keep debugging).
If you have used Eclipse, Visual Studio or #develop will not be that hard to get used to.
I tend to install cygwin, which is a unix emulation layer and includes many of the standard unix utilities (grep, awk, sed, etc). You can use bash or any other unix shell with cygwin to basically give yourself a unix environment on windows.
There are some downsides, paths are a good example. Windows programs expect windows paths while the unix tools expect unix paths. You can convert between the two using the cygpath program, but sometimes its tricky to know when to use it.
Another thing I do fairly often is create a bunch of batch files that load different programs onto my path. This allows me to have different version of say java installed and I can pick the version I want to use for any given shell session. I link a bunch of these together so that I have a full environment for the program I'm working on. For example, if I require java 1.5, maven, subversion then I would have a batch file to load each into the environment, then have a master file that loads all of them for a standard environment.
This approach gives a lot of flexibility and is really easy to maintain and work with different environment simultaneously.
Most windows developers that develop on the microsoft stack of products probably use Visual Studio. For windows development without Visual Studio, SharpDevelop is the current most popular alternative.
However if you are looking for a user experience more similar to linux you can always use windows command prompt and all of the command prompt compilers still exist. Just like with linux you'll have to modify your environmental variables to make everything work you you'd like it to.
If that still isn't close enough to the feel on linux, you can try out Cygwin.
Many of your common utilities from linux like gdb do have windows builds as well.
And of course there is the Eclipse IDE that is used for many languages, by many people, on multiple platforms. It is very extendable.
Some other tools you may be missing:
GCC - Available Via Cygwin
MS Build can give you similar functionality to what you had with make (I'm not sure if nmake is still used/supported)
Vi/Vim
Grep
SysInternals will have lots of various file/process monitoring utilities to hopefully adequately replace what you miss being able to simply get from /proc
Wireshark(or ethereal) to replace well... wireshark/ethereal/
Tail is available in the Windows Resource Kit
Emacs
Hopefully that covers most of your basic tasks.
Microsoft now has a real shell for Windows: Windows PowerShell.
In addition to Cygwin, there are ports for a lot of the GNU utilities and toolchain to Windows. GnuWin32 seems to be a more up-to-date version than UnixUtils. MSYS is essentially a port of BASH to Windows, but it's fairly useless without the MinGW userland.
C++ / .NET Development: Visual Studio 2008
Java / PHP Development: Eclipse IDE, which also supports C/C++.
For a non-IDE solution, Notepad++ is a very good code highlighter that supports many languages.
Simply install cygwin. The quality has improved dramatically in recent years. I'm currently running cygwin x64 on Vista, and it's great.
One thing to especially take note of in cygwin is your path. Most troubleshooting with scripts and installed software should begin there.
The other tip I'd give is to use the rxvt terminal over the standard issue cygwin terminal. It might be installed by default nowadays, but check to make sure.
Visual Studio for .net/C++ (even the express editions are useful)
The sysinternals tools rock, especially Procmon and process explorer.
If you do native/C++ work knowing windbg can be helpful
Notepad++ and gvim are my preferred editors
For doing command line/shell stuff I often use python to write short scripts (for anything but the simplest batch file)
If you are familiar with .net then learning powershell isn't much of a stretch and there is a ton of functionality available
I'm trying/thinking of making CppCMS - C++ Web Framework project little bit more cross platform.
Today I can easily support Linux, OpenSolaris, FreeBSD and even Cygwin. But when it comes to Native Windows it becomes really painful:
The overview of the situation:
I'm POSIX/Linux developer and I'm barely familiar with Native Windows development tools like Visual Studio and Win32 API. However I do some work for this platform so I understand the limitations and the fact that Windows is totally different world.
This is web project that uses APIs that popular in Unix world, like: CGI, FastCGI and SCGI that implemented in most UNIX web servers; but I understand that I would not be able to use it with IIS because it does not support FastCGI over TCP/IP (only Windows pipes).
So even when it would work it would probably run only with Windows port of Apache.
I relay heavily on POSIX API:
Pref-forking allows be keep high survivability in case of crashing (not supported under windows) so this feature would be missing.
I use some file-locking facilities (but I can probably give them up without forking)
I have intensive use of native pthreads, even I can replace them with Boost.Thread
I probably would never be able to support Visual Studio (maybe 2010 with C++0x support), because I relay on C++0x decltype/auto feature or typeof/__typeof__ extension that is supported by most compilers I worked with: gcc, intel, sun studio. (To be honest: I can work without them but it makes the life much easier to framework user.
I relay heavily on autotools and I can't replace them with CMake, bjam or friends, because when it comes to support of internationalization, cross copiling, package management, they just does not give me a solution.
There are many annoying points like missing gmtime_r, or localtime_r under windows and many others that just require from me to rewrite them or replace them with 3rd part libraries.
There are still many "UNIX like" libraries that ported to Win32 like: iconv, gcrypt and some others that are barely ported like libdbi that have many limitations on windows.
Bottom line:
There is lots of non-trivial work to do, and even when it would be complete, it would probably work only with MingW tools and not "native" tools that Windows programmers are
familiar with.
So, my questions are:
Does such MingW port worth an effort? Would this help to build bigger community?
Does anybody have experience on how painful porting big projects from POSIX environment to
Win32 API is?
Would it be useful for Windows developers at all?
Edit:
It is also important for me to understand, how many of windows developers prefer to use
Open source development tools, MingW over Microsoft development solutions like VS.
Edit #2: Clearification about "native" windows solusions and IIS.
In fact, running framework with IIS is really hard problem. I explain:
The project relates to standard web server API as FastCGI or SCGI that allows to accept many requests over sinlge socket. Thus, on application side, I accept new request proceed it and returns the answer. Sometimes several threads process several requests.
Thus, implementing one or two standard protocols I open communication with any existing server: Apache, lighttpd, nginx, cherokee... or any other servers; with small exception of IIS
IIS has implementation of FastCGI, but... It supports only 1 connection per local process only that controlled by web server...
So... there is absolutely no standard way to connect my application to IIS.
Please note, I implement standard Web server API, I do not implement Neither IIS proprietary ISAPI nor Apache proprietary API, even the second is more important as for targeting UNIX world.
So, just Windows IIS Web world is just not really ready for cooperation for such project, so if anybody would use it under Windows it would use it with more open web servers.
You should base your decision on user demand. Have users ever requested using the framework on Windows? If so, did they explain why they wanted to use Windows (e.g. what additional constraints they had, what webserver they wanted to use, etc.)?
Typically, Windows users do expect that things work the Windows way. That means Visual Studio support, IIS support, MSI installer, and so on. If something still feels like being Unix, I would rather use Unix proper, instead of fighting with a half-working port.
As a windows client app developer it sort of hurts me that the development environment division currently is essentially Win32 and everything else and that they are mostly incompatible. That's why I'm preparing to move to MinGW for my personal windows app projects and to try to make them cross-platform.
I would suggest gradually moving to more cross-platform libraries like, as you suggested, refactoring pthreads to boost::thread, or going from fork() to multi-process with IPC, probably also using boost's facilities. Date/time stuff can be dealt with Boost libs as well. As for database support, there are
Microsoft compiler support is not that important I think, as MinGW provides a decent build environment with all the IDEs that support it, Eclipse CDT and Dev-C++ being among the most popular. But if you are going to make your project msvc-compatible, make sure users will be able to use Express editions of Visual Studio 2010 (as soon as thay come out) - that way no one will have to fork out for a Visual Studio 2010 (upgrade) just to use your project and there will be no problem for you to require the latest in Microsoft technology.
Most likely you won't avoid some amount of ifdefs for a code base of your project's size, but surely the effort might be worth it, if not only for gaining valuable experience and expanding the community with a few new happy and grateful members.
Your saying that you can support Cygwin quite easily reminds me that I've seen commercial Windows software that simply bundled in cygwin1.dll to support some originally-Unix code. If adding cygwin1.dll to your installer is all it takes, try it out.
I think you only have to look at the questions asked on SO to work out that MinGW users on Windows (of which I am one) are in a minority in the development community - the vast majority of Windows developers are using MS tools. Anyway, the compiler is only half (or less) of the issue - if your architecture depends on forking lots of processes, using MinGW is not going to help you. My advice is, if you really want to do cross platform development:
look at how Apache do it
consider using the Apache libraries as your base
don't use very new or compiler-specific language features
use multi-threading rather than multi-process
Does such MingW port worth an effort? Would this help to build bigger community?
I am still working myself on this issue with my own large POSIX project and my conclusion is that if you need to later interface with Microsoft products, then its worth while, however then I would only use MingW if project is medium small, if it is very large, then I would go all the way with MSDN Microsoft development tools - Huge amount of help will be available there - however it will cost
Does anybody have experience on how painful porting big projects from POSIX environment to Win32 API is?
sofar my own conversion of my POSIX project have been constantly put on hold, because of the amount of time each issue takes to handle is enormous - not finished converting yet - If I ever will be
Would it be useful for Windows developers at all?
Sure working inside the Microsoft IDE using tools from MSDN will definately decrease development time, however it will increase your dependence on Microsoft libraries - something you need to decide from beginning if that is an issue
**
Actually you could just add the necessary cygwin dlls to your projects make and then you would beable to run it in windows
I managed to make my POSIX project run when I added following dlls
cygboost_filesystem.dll
cygboost_system-mt-1_53.dll
cygboost_thread.dll
cyggcc_s-1.dll
cygstdc++-6.dll
cygwin1.dll
Probably your project will have different dependencies, however if you think conversion is not worth it, then perhaps this is a solution for you
You could also add your libs as static, then you would end up with only having to provide the last cygwin1.dll