I would like to know where I could find good resources/documentation on configuring a new Windows10 installation using Powershell scripts. I know bash but I'm completely new to Powershell.
When I search google, all I can find about automatically configuring Windows relates to Windows Deployment Services. But I don't have and don't want a Windows Server and simply running a few scripts after each installation is sufficient for me.
I found a few scripts that solve some of the things I want to do:
https://github.com/FlatlanderWoman/winCleaner
https://github.com/hahndorf/Set-Privacy
But for everything else, when I look into the TechNet Library I find it very hard to find anything useful. And when I do find something, it looks outdated:
https://technet.microsoft.com/en-us/library/hh852115.aspx
The problem is: I known the GUI-way of configuring everything I want, but I don't know how to find the corresponding commandlets to do the same with Powershell.
Is there some kind of event listener I could use to find the Cmdlets? Or does anyone have some resources/documentation to recommend? Is the TechNet Library really the established way to find these commands?
Thank you.
Unfortunately PowerShell was only really implemented in Windows 7 (yes I know it was available for XP but not preinstalled) and even then it was kind of like an addon rather than part of the core OS. Windows 8 and 10 have further improved functionality but still for the most part do not use it for their own settings and functions as most home users would have no use for it.
However there is nearly always a way to do whatever you need to, I have a script that configures servers from scratch, renaming the server, installing requisite software and features, copying files, configuring VSS, right down to putting the Computer icon on the desktop. You just have to make a list of everything you want to do, then Google each one.
For example: https://www.google.co.uk/search?q=powershell+put+computer+on+desktop - at time of writing the first result is a TechNet script pointing at a registry key. Tidy as necessary, whack into your build script and move on to the next item.
As of yet there's nothing I've found I've been unable to do with PowerShell, but the vast majority of it has not been directly with cmdlets. There's a lot of registry tweaking and command line stuff like msiexec or schtasks, some COM objects and an awkward Type I had to create and use to set the DNS suffix.
Overall I think it's still easier to do all this in PowerShell than any other scripting language and it's more flexible than premade tools, not because it has so much functionality built-in but because it can access .NET and COM which gives you broad access to all the half-baked stuff MS have wedged in over the years.
All texts on how to create a compiler stop after explaining lexers and parsers. They don't explain how to create the machine code. I want to understand the end-to-end process.
Currently what I understand is that, the Windows exe file formats are called Portable Executable. I read about the headers it has and am yet to find a resource which explains this easily.
My next issue is, I don't see any resource which explains how machine code is stored in the file. Is it like 32-bit fixed length instructions stored one after another in the .text section?
Is there any place which at least explains how to create an exe file which does nothing (it has a No Op instruction). My next step then would be linking to dll files to print to console.
Nice question! I don't have much expertise on this specific question, but this is how I would start:
PE or ELF does not create pure machine code. It also contains some header info etc. Read more: Writing custom data to executable files in Windows and Linux
I assume you are looking for how does ELF/PE file hold the machine code, you can get that from this question (using objdump): How do you extract only contents of an ELF section
Now, if you want to know how the content part is generated in the first place, i.e. how is the machine code generated, then that's the task of the compiler's code generation.
Try out some resource editor like ResourceEditor to understand the exe or simply ildasm.
PS: These are mostly Unix solutions, but I am sure, PE should be doing something fundamentally similar.
I think the best way to approach it will be first try to analyze how existing PE/ELFs work, basically reverse engineering. And to do that, Unix machine will be a good point to start. And then do your magic :)
Not same but a similar question here.
Update:
I generated an object dump out of a sample c code. Now, I assume that's what you are targeting right? You need to know do you generate this file (a.out)?
https://gist.github.com/1329947
Take a look at this image, a life time of a c code.
Source
Now, just to be clear, you are looking to implement the final step, i.e. conversion of object code to executable code?
As in many of his articles, I'd say Matt Pietrek's piece about PE internals remains the best introdction to the matter more than a decade after being written.
Iv'e used "Wotsit's File Format" for years... all the way back to the days of MS-Dos :-) and back to when it was just a collection of text files you could download from most BBS systems called "The Game programmers file type encyclopaedia"
It's now owned by the people that run Gamedev.Net, and probably one of the best kept secrets on the internet.
You'll find the EXE format on this page : http://www.wotsit.org/list.asp?fc=5
Enjoy.
UPDATE June 2020 - The link above seems to be now dead, I've found the "EXE" page listed on this web archive page of the wotsit site: https://web.archive.org/web/20121019145432/http://www.wotsit.org/list.asp?al=E
UPDATE 2 - I'm keeping the edit as it was when I added the update erlier, thanks to those who wanted to edit it, but it's for a good reason I'm rejecting it:
1) Wotsit.org may at some point in the future come back online, if you actually try visiting the url, you'll find that it's not gone, it does still respond, it just responds with an error message. This tells me that someone is keeping the domain alive for whatever reason.
2) The archive links do seem to be a bit jittery, some work, some don't, sometimes they seem to work, then after a refresh they don't work, then they do work again. I remember from experience when wotsit was still online, they they had some very strange download/linking detection code in, and this probably caused archive.org to get some very wierd results, I do remember them taking this stance because of the huge number of 3rd party sites trying to cash in on their success, by pretending to be affiliate's and then direct linking to wotsit from an ad infested site.
Until the wotsit domain is removed entirely from the internet and not even the DNS responds, then would be the time to wrap everything up into single archive links, until then, this is the best way to maintain the link.
Not surprisingly the best sites for information about writing PE format files are all about creating viruses.
A search of VX Heavens for "PE" gives a whole bunch of tutorials for modifying PE files
Some information about making PE files as small as possible: Tiny PE.
The minimalistic way to mess around with code generation, if you're just looking to try a few simple things out, is to output MS-DOS .COM files, which have no header or metadata. Sadly, you'd be restricted to 16-bit code. This format is still somewhat popular for demos.
As for the instruction format, from what I recall the x86 instruction set is variable-length, including 1-byte instructions. RISC CPUs would probably have fixed-length instructions.
For Linux, one may read and run the examples from
"Programming from the Ground Up" by Jonathan Bartlett:
http://www.cs.princeton.edu/courses/archive/spr08/cos217/reading/ProgrammingGroundUp-1-0-lettersize.pdf
Then of course one may prefer to hack Windows programs. But perhaps the former
gives a better way to understand what really goes on.
Executable file format is dependent on the OS. For windows it is PE32(32 bit) or PE32+(64 bit).
The way the final executable look like depends on the ABI (application binary interface) of the OS. The ABI tells how the OS loader should load the exe and how it should relocate it, whether it is dll or plain executable etc..
Every object file(executable or dll or driver) contains a part called sections. This is where all of our code, data, jump tables etc.. are situated.
Now, to create an object file, which is what a compiler does, you should not just create the executable machine code, but also the headers, symbol table, relocation records, import/export tables etc..
The pure machine code generation part is completely dependent on how much optimized you want your code to be. But to actually run the code in the PC, you must have to create a file with all of the headers and related data(check MSDN for precise PE32+ format) and then put all of the executable machine code(which your compiler generated) into one of the sections(usually code resides in section called .text). If you have created the file conforming to the PE32+ format, then you have now successfully created an executable in windows.
Has anyone got any experience in doing this?
Specifically, I'd like to find out if any registry keys are being written and what files are going where when I run an MSI.
I was thinking of using ProcMon to see what the msiexec process is doing while I run through it but just thought I'd run it by here to see if anyone has a better method.
Bit rusty, but here's a few (maybe) helpful pointers.
There is a tool called Orca that you can use to edit MSI files.
There was also Wise for Windows, which is now called something else, and I'm not sure what you'll be able to do with the trial, it definitely had the ability to edit MSI files.
I was going to suggest FileMon and RegMon on their own, but I just saw they've actually been merged into ProcMon, shows how behind the times I am :)
Ideally, the setup author used only the Registry and COM tables so it's very easy to just look with Orca what's being done. However many setup authors produce less then idea installs. In those cases I use InstallWatch to snapshot the registry before and after to generate a difference.
InstallWatch Pro
You'll see other line noise from processes running on the machine but you learn to filter those with experience. ( E.g. the install didn't change the crypto seed or the MRU's and ShellBags )
When I develop web applications I'm frequently need to sync files from a working folder to external server or another folder. I like keeping my code separated from the web sever.
In open source world there is the eclipse with file sync that does the job pretty well. Unfortunately I can't find any good replacement for Visual Studio.
I've only found two generic solutions:
- Winscp which is pretty good but stucks when a file is locked and ask for confirmation. Which is quite annoying.
- DSynchronize which works pretty well (ie. doesn't ask questions) but doesn't have filters so I can't tell it not to sync my .svn files or web.conf :(.
Do you know any good way to achieve realtime synchronization in Visual Studio or windows?
I doens't have to have gui in fact I would love to see a command line solution like a powershell command that outputs modified files.
I've ended up using Mercurial (to skip the .svn files) and DSynchronize to sync files
I would give a try to immortal classic - rsync. There is cygwin enabled implementation for Windows called cwrsync: http://www.itefix.no/i2/node/10650 . With proper configuration (potentially with some fine tuning with scripting as well) it will do perfectly.
If you would like to have bi directional synchronization, the Unison may be the answer:
http://www.cis.upenn.edu/~bcpierce/unison/
If you are looking for something even fancier, you might give a try to one of distributed file systems available, like CODA (I'm afraid decent Windows systems aren't supported yet): http://www.coda.cs.cmu.edu or native DFS solution from Microsoft, however I'm afraid the set up is too hassling (if not impossible in your case) since it's targeted for enterprise solutions:
http://technet.microsoft.com/en-us/library/cc753479(WS.10).aspx
Of course DFS option probably won't support filtering you are interested in.
My objective is to write a program which will call another executable on a separate computer(all with win xp) with parameters determined at run-time, then repeat for several more computers, and then collect the results. In short, I'm working on a grid-computing project. The algorithm itself being used is already coded in FORTRAN, but we are looking for an efficient way to run it on many computers at once.
I suppose one way to accomplish this would be to upload a script to each computer and then run said script on each computer, all automatically and dependent on my own parameters. But how can I write a program which will write to, upload, and run a script on a separate computer?
I had considered GridGain, but the algorithm is already coded and in a different language, so that is ruled out.
My current guess at accomplishing this task is using Expect (wiki/Expect), but I have no knowledge of the tool.
Any advice appreciated.
You can use PsExec for this:
http://technet.microsoft.com/en-us/sysinternals/bb897553.aspx
You could also look at the open source alternative RemCom:
http://rce.sourceforge.net/
It's actually pretty simple to write your own as well but RCE will show you how to do it if you want. Although, using PsExec may just suffice your needs.
Have a look into PVM, it was made for the type of situation you're describing, but you may have to annotate your existing codebase and/or implement a wrapper application.