Attaching shared-libs to WebSphere application's module - websphere

Based on IBM documentation, I wrote a jython script that adds shared libs to an existing application.
# Application name
app = sys.argv[0]
dep = AdminConfig.getid('/Deployment:' + app + '/')
depObject = AdminConfig.showAttribute(dep, 'deployedObject')
classldr = AdminConfig.showAttribute(depObject, 'classloader')
for x in range(1, len(sys.argv)):
AdminConfig.create('LibraryRef', classldr,
[['libraryName', sys.argv[x]], ['sharedClassloader', 'true']])
AdminConfig.save()
Unfortunately, this is setting the shared library only for the application and not for the modules. How could I achieve setting the libraries for both ?
I tried to get the modules of an application but I cannot get the classloader of it.
BTW, what is the sharedClassloader attributes used for ?
Note: I know this is bad practice, but I inherited a bunch of legacy applications, so please don't advice to get rid of shared libs or to add a deployment.xml

Well, here is the working script addSharedLibrary.py <application-name> shared-lib [shared-lib...]
def addSharedLibrary(holder):
classldr = AdminConfig.showAttribute(holder, 'classloader')
for x in range(1, len(sys.argv)):
AdminConfig.create('LibraryRef', classldr, [['libraryName', sys.argv[x]], ['sharedClassloader', 'true']])
def handleWebModules(applicationName):
webModules = AdminConfig.list('WebModuleDeployment').split('\n')
for webModule in webModules:
if (webModule.find(applicationName) != -1):
print 'Setting for ' + webModule
addSharedLibrary(webModule)
dep = AdminConfig.getid('/Deployment:' + sys.argv[0] + '/')
addSharedLibrary(AdminConfig.showAttribute(dep, 'deployedObject'))
handleWebModules(app)
AdminConfig.save()

I think a good way to accomplish this is to create a shared library at the server level, and then create a classloader also on the server level in order to load the libraries.
set serv [$AdminConfig getid /Cell:mycell/Node:mynode/Server:server1/]
print AdminConfig.create('Library', serv, [['name', 'mySharedLibrary'], ['classPath',
'home/myProfile/mySharedLibraryClasspath']])
AdminConfig.create('Library', serv, [['name', 'mySharedLibrary'],
['classPath','test1.jar;test2.jar;test3.jar']])
appServer = AdminConfig.list('ApplicationServer', serv)
print appServer
classLoad = AdminConfig.showAttribute(appServer, 'classloaders')
cleanClassLoaders = classLoad[1:len(classLoad)-1]
classLoader1 = cleanClassLoaders.split(' ')[0]
classLoader1 = AdminConfig.create('Classloader', appServer, [['mode', 'PARENT_FIRST']])
print AdminConfig.create('LibraryRef', classLoader1, [['libraryName', 'MyshareLibrary']])
AdminConfig.save()
AdminNodeManagement.syncActiveNodes()

Thanks poussma for your jython script, it works on the previous version of WAS...
so i adjusted your code to make it work also on WAS8 :
Run the cmd line with : addSharedLibrary.py <application-name> [shared-lib...]
def addSharedLibrary(holder):
classldr = AdminConfig.showAttribute(holder, 'classloader')
for x in range(1, len(sys.argv)):
AdminConfig.create('LibraryRef', classldr, [['libraryName', sys.argv[x]], ['sharedClassloader', 'true']])
def handleWebModules(applicationName):
webModules = AdminConfig.list('WebModuleDeployment').split('\r')
webModules = map(lambda s: s.strip(), webModules)
for webModule in webModules:
if (webModule.find(applicationName+'.ear') != -1):
print 'Setting for ' + webModule
addSharedLibrary(webModule)
dep = AdminConfig.getid('/Deployment:' + sys.argv[0] + '/')
addSharedLibrary(AdminConfig.showAttribute(dep, 'deployedObject'))
handleWebModules(sys.argv[0])
AdminConfig.save()

Related

Trouble to access azure containers from Azure/databricks

I am having trouble to access Azure container from Azure/Databricks.
I follow instructions from this tuto, so I started to create my container and generate sas.
Then on a databricks notebook I delivered the following command
dbutils.fs.mount( source = endpoint_source, mount_point = mountPoint_folder, extra_configs = {config : sas})
where I replace endppoint_source, mountPoint_folder, sas by the following
container_name = "containertobesharedwithdatabricks"
storage_account_name = "atabricksstorageaccount"
storage_account_url = storage_account_name + ".blob.core.windows.net"
sas = "?sv=2021-06-08&ss=bfqt&srt=o&sp=rwdlacupiytfx&se=..."
endpoint_source = "wasbs://"+ storage_account_url + "/" + container_name
mountPoint_folder = "/mnt/projet8"
config = "fs.azure.sas."+ container_name + "."+ storage_account_url
but I ended with the following exception:
shaded.databricks.org.apache.hadoop.fs.azure.AzureException: shaded.databricks.org.apache.hadoop.fs.azure.AzureException: Container $root in account atabricksstorageaccount.blob.core.windows.net not found, and we can't create it using anoynomous credentials, and no credentials found for them in the configuration.
I cannot figure out why databricks cannot find the root container.
Any help would be mutch appreciated. Thanks in advance.
The storage account and folder exist, as can be seen from this capture, so I am puzzled out.
Using the same approach as yours, I got the same error:
Using the following code, I was able to mount successfully. Change the endpoint_source value to the format wasbs://<container-name>#<storage-account-name>.blob.core.windows.net.
endpoint_source = 'wasbs://data#blb2301.blob.core.windows.net'
mp = '/mnt/repro'
config = "fs.azure.sas.data.blb2301.blob.core.windows.net"
sas = "<sas>"
dbutils.fs.mount( source = endpoint_source, mount_point = mp, extra_configs = {config : sas})
My bad..., I put a "/" instead of "#" between the container_name and the storage_account_url and inverse the order, so the right synthax is:
endpoint_source = "wasbs://"+ container_name + "#" + storage_account_url

Generate Filename Before Downloading

I'm trying to download the latest backup of data during my chef run but it's trying to download the file before the filename is generated. What's the best approach for doing this. All I want to do is generate a filename based on the time and download it.
The below code gives the error undefined method 'latest_backup' for Custom resource aws_s3_file from cookbook aws.
ruby_block "generate file name" do
block do
require 'time'
latest_backup = "NOT-SET"
utc_now = Time.now.utc
utc_midday = Time.new(Time.new.year, Time.new.month, Time.new.day, 22, 00, 1 ).utc
utc_midnight = Time.new(Time.new.year, Time.new.month, Time.new.day, 10, 00, 1 ).utc
if (utc_now < utc_midday) && (utc_now > utc_midnight )
latest_backup = "data_" + Time.now.strftime("%Y%m%d") + "-00001.tgz"
elsif (utc_now > utc_midday ) && (utc_now < utc_midnight)
latest_backup = "data_" + Time.now.strftime("%Y%m%d") + "-120001.tgz"
end
end
action :create
end
aws_s3_file "/root/backup.tgz" do
remote_path "backup-dir/#{latest_backup}"
bucket "my-backups-bucket"
region "ap-southeast-2"
end
You can't set a local variable across contexts like that. Since nothing in that code requires waiting until converge time, you can just run the code outside of a ruby_block and have it be a normal local variable.

Download Images from list of urls

I have a list of urls in a text file.i want the images to be downloaded to a particular folder ,how i can do it.is there any addons available in chrome or any other program to download images from url
Create a folder in your machine.
Place your text file of images URL in the folder.
cd to that folder.
Use wget -i images.txt
You will find all your downloaded files in the folder.
On Windows 10/11 this is fairly trivial using
for /F "eol=;" %f in (filelist.txt) do curl -O %f
Note the inclusion of eol=; allows us to mask individual exclusions by adding ; at the start of those lines in filelist.txt that we do not want this time. If using above in a batch file GetFileList.cmd then double those %%'s
Windows 7 has a FTP command, but that can often throw up a firewall dialog requiring a User Authorization response.
Currently running Windows 7 and wanting to download a list of URLs without downloading any wget.exe or other dependency like curl.exe (which would be simplest as the first command) the shortest compatible way is a power-shell command (not my favorite for speed, but if needs must.)
The file with URLs is filelist.txt and IWR is the PS near equivalent of wget.
The Security Protocol first command ensures we are using modern TLS1.2 protocol
-OutF ... split-path ... means the filenames will be the same as remote filenames but in CWD (current working directory), for scripting you can cd /d folder if necessary.
PS> [Net.ServicePointManager]::SecurityProtocol = "Tls12" ; GC filelist.txt | % {IWR $_ -OutF $(Split-Path $_ -Leaf)}
To run as a CMD use a slightly different set of quotes around 'Tls12'
PowerShell -C "& {[Net.ServicePointManager]::SecurityProtocol = 'Tls12' ; GC filelist.txt | % {IWR $_ -OutF $(Split-Path $_ -Leaf)}}"
This needs to be made into a function with error handling but it repeatedly downloads images for image classification projects
import requests
urls = pd.read_csv('cat_urls.csv') #save the url list as a dataframe
rows = []
for index, i in urls.iterrows():
rows.append(i[-1])
counter = 0
for i in rows:
file_name = 'cat' + str(counter) + '.jpg'
print(file_name)
response = requests.get(i)
file = open(file_name, "wb")
file.write(response.content)
file.close()
counter += 1
import os
import time
import sys
import urllib
from progressbar import ProgressBar
def get_raw_html(url):
version = (3,0)
curr_version = sys.version_info
if curr_version >= version: #If the Current Version of Python is 3.0 or above
import urllib.request #urllib library for Extracting web pages
try:
headers = {}
headers['User-Agent'] = "Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.27 Safari/537.17"
request = urllib.request.Request(url, headers = headers)
resp = urllib.request.urlopen(request)
respData = str(resp.read())
return respData
except Exception as e:
print(str(e))
else: #If the Current Version of Python is 2.x
import urllib2
try:
headers = {}
headers['User-Agent'] = "Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.27 Safari/537.17"
request = urllib2.Request(url, headers = headers)
try:
response = urllib2.urlopen(request)
except URLError: # Handling SSL certificate failed
context = ssl._create_unverified_context()
response = urlopen(req,context=context)
#response = urllib2.urlopen(req)
raw_html = response.read()
return raw_html
except:
return"Page Not found"
def next_link(s):
start_line = s.find('rg_di')
if start_line == -1: #If no links are found then give an error!
end_quote = 0
link = "no_links"
return link, end_quote
else:
start_line = s.find('"class="rg_meta"')
start_content = s.find('"ou"',start_line+1)
end_content = s.find(',"ow"',start_content+1)
content_raw = str(s[start_content+6:end_content-1])
return content_raw, end_content
def all_links(page):
links = []
while True:
link, end_content = next_link(page)
if link == "no_links":
break
else:
links.append(link) #Append all the links in the list named 'Links'
#time.sleep(0.1) #Timer could be used to slow down the request for image downloads
page = page[end_content:]
return links
def download_images(links, search_keyword):
choice = input("Do you want to save the links? [y]/[n]: ")
if choice=='y' or choice=='Y':
#write all the links into a test file.
f = open('links.txt', 'a') #Open the text file called links.txt
for link in links:
f.write(str(link))
f.write("\n")
f.close() #Close the file
num = input("Enter number of images to download (max 100): ")
counter = 1
errors=0
search_keyword = search_keyword.replace("%20","_")
directory = search_keyword+'/'
if not os.path.isdir(directory):
os.makedirs(directory)
pbar = ProgressBar()
for link in pbar(links):
if counter<=int(num):
file_extension = link.split(".")[-1]
filename = directory + str(counter) + "."+ file_extension
#print ("Downloading image: " + str(counter)+'/'+str(num))
try:
urllib.request.urlretrieve(link, filename)
except IOError:
errors+=1
#print ("\nIOError on Image" + str(counter))
except urllib.error.HTTPError as e:
errors+=1
#print ("\nHTTPError on Image"+ str(counter))
except urllib.error.URLError as e:
errors+=1
#print ("\nURLError on Image" + str(counter))
counter+=1
return errors
def search():
version = (3,0)
curr_version = sys.version_info
if curr_version >= version: #If the Current Version of Python is 3.0 or above
import urllib.request #urllib library for Extracting web pages
else:
import urllib2 #If current version of python is 2.x
search_keyword = input("Enter the search query: ")
#Download Image Links
links = []
search_keyword = search_keyword.replace(" ","%20")
url = 'https://www.google.com/search?q=' + search_keyword+ '&espv=2&biw=1366&bih=667&site=webhp&source=lnms&tbm=isch&sa=X&ei=XosDVaCXD8TasATItgE&ved=0CAcQ_AUoAg'
raw_html = (get_raw_html(url))
links = links + (all_links(raw_html))
print ("Total Image Links = "+str(len(links)))
print ("\n")
errors = download_images(links, search_keyword)
print ("Download Complete.\n"+ str(errors) +" errors while downloading.")
search()
In this python project I make a search in unsplash.com, which brings me a list of URL, then I save a number of them (pre-defined by the user) to a pre-defined folder. Check it out.
On Windows, install wget - https://sourceforge.net/projects/gnuwin32/files/wget/1.11.4-1/
and add C:\Program Files (x86)\GnuWin32\bin to your environment path.
create a folder with a txt file of all the images you want to download.
in the location bar at the top of the file explorer type cmd
When the command prompt opens enter the following.
wget -i images.txt --no-check-certificate

Rally APIs: How to Move A Test Folder

I've worked with the script outlined in the following answer:
Rally APIs: How to copy Test Folder and member Test Cases
and it's handy, but what I really want to do is to move an entire Test Folder into a different project. This is next to impossible through the Rally User Interface. According to Rally Support, the only way to do this in the UI is:
Un-assign the Test Cases from their current Test Folder
Setup a Custom Grid app on your dashboard
Use the Custom Grid bulk edit to update the Project of the Test Cases
Lastly use the Custom Grid bulk edit to update the Test Folder - now that you're in the target Project, of the Test Cases
Even though the above process is clunky, it is easier now than it used to be before the advent of the bulk edit within the Custom Grids. Before you had to go through and edit each Test Case one-by-one which was very manual and slow.
However, we have several thousand Test Cases we need to move, and the Custom Grid has a fatal flaw for us. It will only show the first 200 records in a query. So we would have to manually change our grid query in a step wise manner to accomplish the move we need. This is barely better than editing Test Cases one-by-one. Is there a way to move a Test Folder with Test Cases from one Project to another, using a script? Please tell me there is.
The following script will perform this task - it will move all Test Cases from a Source Test Folder identified by FormattedID, to a Target Test Folder, also identified by FormattedID. The Source Test Folder and Target Test Folder can be in different Projects (although they must be within the same Workspace). Like the Copy script referenced in the question, the Target Test Folder must exist, i.e. the script will not create a Test Folder for you if the Target is not found.
For those needing to install and configure the Ruby REST Toolkit, links are here:
Developer Portal: Rally REST API for Ruby
Github
# Copyright 2002-2012 Rally Software Development Corp. All Rights Reserved.
require 'rally_api'
$my_base_url = "https://rally1.rallydev.com/slm"
$my_username = "user#company.com"
$my_password = "password"
$my_workspace = "My Workspace"
$my_project = "My Project"
$wsapi_version = "1.39"
# Test Folders
$source_test_folder_formatted_id = "TF8"
$target_test_folder_formatted_id = "TF11"
#==================== Make a connection to Rally ====================
config = {:base_url => $my_base_url}
config[:username] = $my_username
config[:password] = $my_password
config[:workspace] = $my_workspace
config[:project] = $my_project
config[:version] = $wsapi_version
#rally = RallyAPI::RallyRestJson.new(config)
begin
# Lookup source Test Folder
source_test_folder_query = RallyAPI::RallyQuery.new()
source_test_folder_query.type = :testfolder
source_test_folder_query.fetch = true
source_test_folder_query.query_string = "(FormattedID = \"" + $source_test_folder_formatted_id + "\")"
source_test_folder_result = #rally.find(source_test_folder_query)
# Lookup Target Test Folder
target_test_folder_query = RallyAPI::RallyQuery.new()
target_test_folder_query.type = :testfolder
target_test_folder_query.fetch = true
target_test_folder_query.query_string = "(FormattedID = \"" + $target_test_folder_formatted_id + "\")"
target_test_folder_result = #rally.find(target_test_folder_query)
if source_test_folder_result.total_result_count == 0
puts "Source Test Folder: " + $source_test_folder_formatted_id + "not found. Exiting."
exit
end
if target_test_folder_result.total_result_count == 0
puts "Target Test Folder: " + $target_test_folder_formatted_id + "not found. Target must exist before moving."
exit
end
source_test_folder = source_test_folder_result.first()
target_test_folder = target_test_folder_result.first()
# Populate full object for both Source and Target Test Folders
full_source_test_folder = source_test_folder.read
full_target_test_folder = target_test_folder.read
# Grab collection of Source Test Cases
source_test_cases = source_test_folder["TestCases"]
# Loop through Source Test Cases and Move to Target
source_test_cases.each do |source_test_case|
begin
test_case_to_update = source_test_case.read
source_test_case_formatted_id = test_case_to_update["FormattedID"]
target_project = full_target_test_folder["Project"]
target_project_full_object = target_project.read
target_project_name = target_project_full_object["Name"]
source_project = full_source_test_folder["Project"]
source_project_full_object = source_project.read
source_project_name = source_project_full_object["Name"]
puts "Source Project Name: #{source_project_name}"
puts "Target Project Name: #{target_project_name}"
# Test if the source project and target project are the same
source_target_proj_match = source_project_name.eql?(target_project_name)
# If the target Test Folder is in a different Project, we have to do some homework first:
# "un-Test Folder" the project
# Assign the Test Case to the Target Project
# Assign the Test Case to the Target Test Folder
if !source_target_proj_match then
fields = {}
fields["TestFolder"] = ""
test_case_updated = #rally.update(:testcase, test_case_to_update.ObjectID, fields) #by ObjectID
puts "Test Case #{source_test_case_formatted_id} successfully dissociated from: #{$source_test_folder_formatted_id}"
# Get full object on Target Project and assign Test Case to Target Project
fields = {}
fields["Project"] = target_project_full_object
test_case_updated = #rally.update(:testcase, test_case_to_update.ObjectID, fields) #by ObjectID
puts "Test Case #{source_test_case_formatted_id} successfully assigned to Project: #{target_project_name}"
end
# Change the Test Folder attribute on the Test Case
fields = {}
fields["TestFolder"] = target_test_folder
test_case_updated = #rally.update(:testcase, test_case_to_update.ObjectID, fields) #by ObjectID
puts "Test Case #{source_test_case_formatted_id} successfully moved to #{$target_test_folder_formatted_id}"
rescue => ex
puts "Test Case #{source_test_case_formatted_id} not updated due to error"
puts ex
end
end
end

Why must I use local path rather than 'svn://' with SVN bindings?

I'm using the Ruby SVN bindings built with SWIG. Here's a little tutorial.
When I do this
#repository = Svn::Repos.open('/path/to/repository')
I can access the repository fine. But when I do this
#repository = Svn::Repos.open('svn://localhost/some/path')
It fails with
/SourceCache/subversion/subversion-35/subversion/subversion/libsvn_subr/io.c:2710: 2: Can't open file 'svn://localhost/format': No such file or directory
When I do this from the command line, I do get output
svn ls svn://localhost/some/path
Any ideas why I can't use the svn:// protocol?
EDIT
Here's what I ended up doing, and it works.
require 'svn/ra'
class SvnWrapper
def initialize(repository_uri, repository_username, repository_password)
# Remove any trailing slashes from the path, as the SVN library will choke
# if it finds any.
#repository_uri = repository_uri.gsub(/[\/]+$/, '')
# Initialize repository session.
#context = Svn::Client::Context.new
#context.add_simple_prompt_provider(0) do |cred, realm, username, may_save|
cred.username = repository_username
cred.password = repository_password
cred.may_save = true
end
config = {}
callbacks = Svn::Ra::Callbacks.new(#context.auth_baton)
#session = Svn::Ra::Session.open(#repository_uri, config, callbacks)
end
def ls(relative_path, revision = nil)
relative_path = relative_path.gsub(/^[\/]+/, '').gsub(/[\/]+$/, '')
entries, properties = #session.dir(relative_path, revision)
return entries.keys.sort
end
def info(relative_path, revision = nil)
path = File.join(#repository_uri, relative_path)
data = {}
#context.info(path, revision) do |dummy, infoStruct|
# These values are enumerated at http://svn.collab.net/svn-doxygen/structsvn__info__t.html.
data['url'] = infoStruct.URL
data['revision'] = infoStruct.rev
data['kind'] = infoStruct.kind
data['repository_root_url'] = infoStruct.repos_root_url
data['repository_uuid'] = infoStruct.repos_UUID
data['last_changed_revision'] = infoStruct.last_changed_rev
data['last_changed_date'] = infoStruct.last_changed_date
data['last_changed_author'] = infoStruct.last_changed_author
data['lock'] = infoStruct.lock
end
return data
end
end
Enjoy.
The svn command is a client. It communicates with the Subversion server using several protocols (http(s)://, svn:// and file:///).
Repos.open is a repository function (much like svnadmin for instance). It operates directly on the database, and doesn't use a client protocol to communicate with the server.

Resources