Is there a possibility to deploy or redeploy a SAS job (Data Integration Studio) via a shell script ?
Also is there a way to create its SPK file via script ?
You can deploy DI jobs from commandline, see here:
http://support.sas.com/documentation/cdl/en/etlug/65807/HTML/default/viewer.htm#p1jxhqhaz10gj2n1pyr0hbzozv2f.htm
I have imported and exported objects into SAS DIS via shell script use this sas ExportPackage utility. I personally find it way more convenient as compared to window method. However, for it to work you need to have X-windows Environment, i used Xming for it.
As for deploying Jobs, never tried it.
To redeploy jobs DI Studio versions 4.901 and higher have a DepoyJobs tool which is designed to perform this function: read more in the SAS documentation. It is available on the server. Older versions had a similar but much more restrictive client tool using ant.
Also see Paper 1067-2017 An Introduction to the Improved SAS® Data Integration Studio Batch Deployment Utility on UNIX by Jeff Dyson, The Financial Risk Group, which gives a run through on how to use it.
Related
On daily basis, I do write a lot of shell scripts for a different project.
I wanted to know, is there a way where I can write a lot of scripts project wise as we do create in IntelliJ/eclipse. As well as I could able to run those scripts from IDE only. I think this will create more clarity and my development speed will also increase.
Currently what i do, using FileZilla to connect to the server I open particular script as I set Sublime text as default file editor it opens into that only. I do make changes into the file and once I save it changes easily reflected into server using which to some extent solve my problem.
But does anyone knows!! is there any way/tool/software which could do all these steps/tasks as a whole.
That help to boost our productivity as IntelliJ and other IDEs are doing for many decades.
A better process is to have your scripts stored in git repository.
Clone the repository locally, edit your scripts using an editor/IDE of your choice. Test and once your changes are ready commit and push to the repository. Then update the repository copy on the server.
We're using database projects here at work and for our deployment to the production server, our current process is to manually run a compare using a saved .scmp file that compares the database project to our production database (using a read-only login), then generate a SQL script that we give to our I.T. support guy to run on production. We also do a build to generate our post-deployment script, and we give that one to our guy to run as well.
I'm trying to automate as much of this process as possible (to reduce the chances of mistakes and make it more efficient). I'd like to know if there's a way to automatically generate the sql change script using the predefined options in our .scmp file.
Additionally, is there an easy way to automate the appending of the post-deployment script to the end of the schema change script, so he just has one sql file to run?
Perhaps there's a nice way to do the whole thing with powershell or something.
Ok what you should do is use sqlpackage.exe to create your script from the dacpac that is produced by building the ssdt project.
Create a batch script to call it or make it a part of your CI process.
To filter the output there are some new options like exclude certain types in the latest (March 2015) release of ssdt or use a deployment filter like:
http://agilesqlclub.codeplex.com if you need more flexibility.
Using this you can filter the deployment like the compare and also the pre/post deploy scripts are pre/appended so you kill two birds with one stone!
Ed
It seems that the stock bootstrapping process is a bit lacking on Windows.
Linux has cloud-init which will install packages, store files, and run a bash script from user data.
Windows has ec2config but there is currently no support to run a cmd or powershell script when the system is "ready"--meaning that all the initial reboots are completed.
There seem to be third party options. For example RightScale has the RightLink agent which performs this function.
Are there open source options available?
Are there any plans to add this feature to Ec2Config?
Do I have to build this my self?
Am I missing something?
It appears that EC2Config on the Amazon-provided AMIs now supports "User Data Scripts" as of the 11-April-2012 updates.
The documentation has not yet been updated, so it's hard to tell if it supports PowerShell or just cmd.exe scripts. I've posted a question on the AWS forums to try and get some more detail, and will update here when I learn more.
UPDATE: It looks like cmd.exe batch syntax is supported, which can in turn invoke PowerShell. There's a new version of the EC2Config documentation included on the AMI. Quoting from it:
[EC2Config] will read in the user data specified for the instance and then check if it contain the tags <script> and </script>. If it finds both then it will take the information between those two tags and save it to a batch file located in the Settings folder of this application. It will then execute the batch file during the start of an instance.
The batch file will only be created and executed on the first launch of an instance after a sysprep. If you want to have the batch file created and executed again set the Ec2HandleUserdata plugin state to Enabled.
UPDATE 2: My interpretation is confirmed by Shon from the AWS Team
UPDATE 3: And as of the May-2012 AMIs, PowerShell is supported using the <powershell/> tag.
Cloudbase.it have opensourced a python windows service they call cloudbase-init which follows the configdrive and HTTP datasources.
http://www.cloudbase.it/cloud-init-for-windows-instances/
github here
https://github.com/stackforge/cloudbase-init/
I had to build one myself however it was very easy. Just made a service that reads the user-data when starts up and executes the file as a powershell script.
To get around the issue of not knowing when to start the service I just made the service start type as "delayed-auto" and that seemed to fix the problem. Depending on what you need to do to the system that may or may not work for you however in my case that was all I had to do.
I added a new codeplex project that already has this tool built for windows. Looking forward to some feedback.
http://cloudinitnet.codeplex.com/
We had to build it ourselves; we did it with a custom service and built our own AMIs. There's no provision currently within EC2Config to do it.
Even better, there is no easy way to determine when the instance is "ready". We had to do it by tailing the logfile of EC2Config.
I've recently found nssm (at nssm.cc) which easily wraps a simple batch file (or pretty much anything else) as a service. You can then us sc config servic1 depend= service0 to force the batch file to be run at a particular point in the service initialization sequence. I am using it in between ex2config and sql express to create a folder on d, for instance. You'll have to use the services tool to make it run as network services and change the AppExit property to Ignore using regedit, but it works once you get it all in place.
I'm currently working on some R&D for improving the process and practice that we approach development.
As a first iteration something I'd like to do is develop an easy way to set up a project e.g. Run an ant script that will, when provided with a project name etc,
Copy a template in svn
Create a database (if needed)
Automatically update the connection strings in config/properties files
Create an instance of trac
Email the user with all this information
I'm fine with the first 4 but I'm struggling to get a good process together for setting up Trac.
I've done it manually before but found it incredibily difficult.
Does anyone know of such a set up script or have any tips on doing this?
Cheers,
Rob
Have you looked at bitnami?
See the Trac bitnami bundle for example
I've just implemented build and deploy process which consists of java files, ant script and cmd files. In the process, a release manager will have to check out source, hit the build.cmd button and then carry a zip file over to a server.
I am wondering if it is worthwhile to make a GUI for it? So that the release manager does not need to check out source manually for example?
How do I start? I have quite limited knowledge of javax, but I very much like to learn.
Thanks,
Sarah
This sounds like something that could be handled by Hudson. It can check out source, run Ant scripts, etc., saving you the trouble of maintaining a GUI. I'd give that a shot before rolling your own.
I have helped develop the build process at my current company. The way we currently do it is with a script file. It checks out the latest code from the stable branch of our repository, performs some steps to get some data from a database (such as static SQL data that needs to be loaded at deployment), then compresses everything. The file is then distributed to our production servers and then the setup routine is executed. Everything is automatic and the script is written in Python. Python is great for these types of things because of the sheer number of libraries it has to help the developer.
Perhaps it may be useful to build a GUI for your deployment procedure -- typically this would be useful if the deployment requires user interaction to make decisions, such as "Which server shall I deploy to?", etc. But, if it's just a matter of doing things automatically, then a script file's the way to go. Choose your favourite language and dive in -- I of course recommend Python.
If you'd like to learn how to make a simple GUI in Java (since that seems to be what your company is familiar with), you should check out the stuff at this site:
http://java.sun.com/docs/books/tutorial/uiswing/index.html
I learned everything I know about Java from that site. The section on GUI programming is great.
Best of luck!
Shad