execute dashDB jobs from Workload Scheduler - jdbc

I have created some stored procedures on my dashDB instance on Bluemix, to manipulate data in tables in the same instance.
I can run these from Data Studio and they work as intended.
Next, I created a process in Workflow Scheduler, which I provisions as a service in the same app, where the dashDB is also a service.
While creating the job step in the process, I noticed a message in the dialog window. I have attached a screen shot here:
http://i.stack.imgur.com/EI2b7.jpg
When I did try to run the process step from Workflow Scheduler, the process failed with a JDBC not found error.
I do realize that the Workflow agent I'm using is hosted on Bluemix, so I am puzzled how I can install the JDBC client there.
Should I be setting up an agent on a local machine outside of bluemix, in a hybrid mode?

Currently there are 2 possibilities:
Open a ticket to ask for a dedicated cloud agent and then download
the JDBC driver on the agent.
Download and install an agent on a VM or on-prem.

It looks like you have the incorrect value for "JDBC jar class path" the correct value is /home/wauser/utils/
I'm not sure why it is required that we put enter this but I was able to get the connection to dashDB working with this change.
JDBC jar class path: /home/wauser/utils/

Related

Self-Hosted Integration Runtime times out after ADF pipeline loads a few tables

I have recently installed Integration Runtime for a local server which hosts an Access DB. The idea is to pull data from it and store in in Azure SQL DB. I have done the following:
Integration Runtime Services Installed on local machine hosting the MS Access DB and connected to it using ODBC
Created linked services in ADF to connect to the DB
Created Datasets for source and destination DB for each table required. One for the source msaccess and one for the target in azure sql db
Created a pipeline to copy the data from the source and sink into the asql db mentioned in step #3
Basically, all the connections work however when I trigger my pipeline to load around 10 of these tables, it runs and loads the first two and then fails afterwards by timing out. I must restart the Integration Runtime everytime to get it back up and running otherwise I can no longer query the tables.
To mitigate this I figured there was too much traffic and the server needed to rest between calls so I added wait timers in between each step of the pipelines but not much success. It did help a bit but that might be coincidence.
The error log in the monitor spits out at the failed step is :
Error: 2200
ErrorCode=UserErrorFailedToConnectOdbcSource,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=,Source=Microsoft.DataTransfer.Runtime.GenericOdbcConnectors,''Type=System.Data.Odbc.OdbcException,Message=,Source=,'
btw the integration runtime install service running is 5.12.7984.1 and the version of access installed is through office 365 x64. The exact MS access driver is 16.14430.20006. The OS is Windows Server 2019
I am getting the exact same error. To start, I did check the event viewer logs and saw some errors to do with access. So I gave the user running the IR more access to the registry keys/general log on as a service rights. This helped a little, but I am still stuck with the same problem.
When copying from an Access DB located on the SHIR itself to flat files in lake storage, I encountered the same error.
Removing Office 365 from the machine, then re-installing the Access runtime solved the problem.
This answer is from #ezaidi comments above.

Not terminating but stopping cloud build agent

TeamCity supports creating VMs in the cloud for running builds. It can also terminate instance after it is idle for defined period of time.
Is it possible not to terminate but shutdown the instance and to start it again when needed?
If your BA is a VM on Azure then I was able to achieve this by using Azure Automation account (There is a free version).
Basically, you trigger WebHooks for 2 runbooks you import from the gallery - StartAzureV2VM & StopAzureV2VM. The gallery can be found in "Automation Account-> Runbooks -> Browse Gallery"
Then on your TeamCity server you just periodically run some powershell which monitors the build queue on Teamcity and trigger appropriate webhook (with some timeouts etc.).
Here is the script I'm using, feel free to amend it to your needs:
https://gist.github.com/milanio/b300f23883afa9c6288f9365dfb98252

DbAdapter Plan Updates from Remote WLST connection

I am creating a Maven based remote deployment setup for weblogic and SOA.
I am using a python script which is executing remotely.
I have successfully managed to get it to create/update JDBC connections.
I now wish to update the DbAdapter deployment plan with the new JDBC connection. (Later I want to do this with the JMSAdapter as well)
I have found many links online which show how to do this via a local WLST connection.
https://redstack.wordpress.com/2012/08/09/automating-creation-of-jdbc-data-sources-and-dbadapter-plan-updates-and-redeployment/
I am getting an error with my remote script with the line:
plan = loadApplication(appPath, planPath)
The consensus online is that loadApplication will not work remotely.
A couple of different sources mention this but no source offers any workaround.
Is there a WLST based remote API I can use to add the configuration?
Can I download the plan.xml, alter it and re-upload it through WLST? (If so any tutorials?)
Is there an easier method that I am missing?
Thanks for any help.

AWS User data windows standard AMI

I'm trying to use codedeploy with autoscaling in order to automate the deployment of my application.
I have everything ready. When developing all the parts (hooks' scripts, roles etc) I installed the codedeploy agent manually. Now I want to make it production ready, which means that the codedeploy agent will be installed at sysprep (by providing the powershell commands via user data in launch configuration).
The problem is that it's not working. The script either runs and fails for some reason (are there any logs to confirm?) or it doesn't run at all. My AMI is based on a aws standard windows AMI. The EC2ConfigService is present.
Do you have any idea of what could be the problem or if I have some way to find what's the problem (logs)?
You could take a look at C:\Program Files\Amazon\Ec2ConfigService\Logs\Ec2ConfigLog.txt
On Linux AMIs you can also find the user data script execution logs in the ec2 console when you right click your instance -> Instance Settings -> Get System Log.

Teamcity server migration - agent changes

I'm planning to migrate my teamcity server onto a new physical location. The process is pretty straight forward, export the database, install a vanilla teamcity server and import the database via maintaindb.sh.
Since i have a large installation I decided to backup only server settings, projects and builds configurations, plugins. My point was that I can manually move build logs and artifacts later (i'd rather not try to restore from a 500GB zip file). However after importing the backup I was unable to see any build agents in the agent pool.
Any ideas? Do you have to install each build agent from scratch just because the server got migrated to a new location? Do you just have to point the agents to the new server and that's it (and if so why does the agent pool on the server seem empty)
Thanks,
If you are changing the servers URL in your migration, which from your question I am assuming that you are, then you will need to edit each build agent's properties.
In your ~TeamCity\Install\buildAgent\conf, you will have a buildAgent.properties file. You need to modify this file to point to your new Teamcity location via the serverURL value. Then you will want to restart the build agent server, and authorize and enable the build agent from your Teamcity interface.
There is an extremely brief explanation of this here at the bottom of the "Move TeamCity Installation to a New Machine" section.
And to answer your question as to why the agent pool seems empty - it is because the agent is not looking for the server at it's new location.

Resources