I have a few python/selenium scripts that automate tedious work in browser-based apps (e.g., data entry from a CSV file).
I would like to share the scripts with my team members who are tech-savvy but don't know how to code.
How can I package up the scripts with all the dependencies so that "it just works" when downloaded?
Dependencies are:
Install python and add to PATH
Install selenium, pandas, autopylogger
Install selenium chromedrivers
...and a few other simple changes to variables/file paths in the scripts
My first instinct was to dockerize the environment, but because the automation need to use the user's Chrome profiles and docker runs on WSL I imagine that would cause issues. And setting up WSL + Docker Desktop isn't exactly simplifying things.
Jenkins, Replit, Maven, and other hosted options would have a similar issue using the user's Chrome profile (I think?)
I'm currently thinking I'll write a .BAT script that sets up the environment, but I haven't done that before and am hoping there's an easier way.
I would say,
1)Create a Jenkin job.
In Selenium, you can pass chrome user profile as an argument. They can save their chrome user profile from the chrome browser and pass it as an argument through jenkins.
Below is the sample code of how to pass custom chrome profiles:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = Options()
options.add_argument("user-data-
dir=C:\\Users\\AtechM_03\\AppData\\Local\\Google\\Chrome\\User Data\\Profile 2")
driver = webdriver.Chrome(executable_path=r'C:\path\to\chromedriver.exe',
chrome_options=options)
driver.get("https://www.google.co.in")
Let them store their profile in a accessible path, where it can be passed as an argument to Jenkins
Related
We're switching from a classic Desktop application project, where everything fitted nicely within Visual Studio, to an Angular Web Project, where everything seems to be scattered over different environments all over the place.
One of the changes is nuget packages, which makes me feel like we're back in the 80's, where everything needs to be done with commands in Command Prompt, instead of just using buttons and menus. I find this slightly annoying, as now you have to remember a long list of commands where previously you could just remember a button's location.
Is there a place where you can store a list of commands that you can browse in your Package Manager Console? For instance, if I forget the command that script my model into SQL Scripts, I can just browse the list, instead of looking for the email where a colleague told me what it was?
The Package Manager Console is a PowerShell session, so you could create a PowerShell script Module with everything in it.
And even load from your profile (for package manager specific use file: run $profile.CurrentUserCurrentHost in the console to get the path).
I have a fresh build server and need to setup jenkins there. So I created a first user on the Mac mini, and used that user to install Jenkins on the machine.
Jenkins then created another user (Jenkins) on the Mac Mini.
I thought that correct way to proceed is to login as that Jenkins user (with Admin rights) and prepare the build environment as this Jenkins user.
But I cant install Visual Studio for MAC, cant install SourceTree... bcs even though Jenkins has Admin rights, I am getting
"You do not have permissions to open the application 'Install Visual
Studio for Mac' contact your administrator..." message.
But as I said the Jenkins user is Admin user. So am I doing it wrong? What user should prepare all the build tools? And if it is supposed to be the first user I created (after booting the Mac mini for the 1st time), how do I let Jenkins use the apps then? When I tried doing this on my macbook, jenkins couldnt use some of the build tools, bcs apparently it did not have the permissions to do so.
So what is the correct way to set up a jenkins environment, so I can install whatever tools I need, and Jenkins can use these during builds/deploys?
Any help appreciated, all the guides about jenkins speak of jenkins user, but none gave me an idea what is the best way to set this all up.
My 2 cents.
I have not worked with MAC. So some of my assumptions may be wrong or suggestions may be not applicable to you.
I have set up Jenkins mostly on Linux and a couple of times on Windows. I used to install Jenkins using the RPM package which, upon installation, will create a 'jenkins' user and group.
For start/stop/restart, I always use 'sudo' since Jenkins is installed as a service.
Other build related tools like Java, Maven etc, I always make sure that Jenkins can access them and execute them. ie, I give execute privilege on those tools.
The JENKINS_HOME directory, I used to create a symbolic link from the default home directory to a file system with enough storage. I will do the same for Jenkins log file.
The benefit of having those Jenkins/tool files/directories as part of a custom defined directory structure is that I do not need to remember installation paths of each tool. ie, I have my Jenkins, maven, java, sonarQube everything under a directory I know. Even if I set up these tools in different servers, I will stick to the same directory structure.
My suggestions to you.
Setup build tools which are to be used/accessed from Jenkins, set them up with your 'first user' and give execute privilege to Jenkins user/group.
Create a directory 'ci' and link or place all tools inside. You can further have subdirectories for app - for binaries/installed files, data - for generated data like Jenkins_home, log - to keep log files of these tools.
ci/jenkins
ci/sourcetree
ci/apache-maven
Suggestion 2 is initially time-consuming but it will save a lot of your time as you use the tools on a daily basis.
I have to use d3 graph at my web page. I never worked with d3. That's why, I'm facing some problem using it. Basically, I've to work at dropbox folder. So, my clients can see the worked file locally at their browser. Consider, I put a d3 chart at file.html file and my folder structure is like this
D:\Projects\Dropbox (Company)\MyName\FolderName\file.html
But, I saw that to run and see/show d3 examples I need run web server or run a python server. So, I've downloaded Python 3.4.1 and installed by double click. It's installed at this directory: C:\Python34
After that, I tried to follow d3's documentation for installing python server. So, I opened my cmd and type:
python -m http.server 8888 &
But, it gives me this error:
So, my question is:
How can I install python web server?
After installing that web server, is it possible see the d3 chart via this link: file:///D:/Projects/Dropbox (Company)/MyName/FolderName/file.html or I've put my files inside htdocs and run via http://localhost/folderName/file.html (I don't want to put files inside htdocs. It'll be tough for my clients see the output of the files directly from their pc)?
If it can't be seen without putting inside htdocs folder, I may find solution for running d3.js locally without installing any additional software/server(though I've found this type of solution for some d3.js chart but not for all). Thanks in advance and please don't mind if it's a lame question. Basically, it's my first day working with d3.js and I'm only ameture level skilled with javascript and jQuery.
OK, browsers are designed with security in mind, by default they don't let scripts go and grab a file from anywhere for very good reasons. They allow you to grab a file from the server or through requests. So to share your work with your client you will either need to use a hosting service - I would recommend bl.ocks, design your visualisation so it doesn't require any external data or provide instructions on how to disable browser security. You can read more about this here, here and here.
On python, in many cases python is already installed on people machines, so running a server from python shouldn't be an issue. All you have to do (on a windows machine) is launch your command prompt navigate to your directory and start your python server. Then open a browser and navigate to the localhost. Please note that python needs to be set as an environmental variable (i.e. your system path), the python documentation might help you here.
Okay. I know that I am asking a hackneyed, overchewed question but I am posting this because my problem is not getting solved despite trying all the solutions provided on various links.
My constraint - I don't want to use a saved firefox profile to save the website's certificate.
Selenium server version - 2.0b3
Solutions tried -
1) http://seleniumhq.org/docs/05_selenium_rc.html#handling-https-and-security-popups
Browser launcher *firefox
Selenium started with trustAllSSLCertificates option.
Did not help. FF again asks to save the certificate.
2) http://blog.mogotest.com/2010/04/13/how-to-accept-self-signed-ssl-certificates-in-selenium/
Browser launcher *firefoxproxy
Selenium started with trustAllSSLCertificates option.
Did not help.
3) Selenium Testing HTTPs Trust All certificates working for FF but not IE
4) Besides that I tried creating my own cert_override.txt and populating the 5 fields as explained in https://developer.mozilla.org/En/Cert_override.txt but getting the value for the 5th field which is the certificate's serial number and the issuer name as a base64 encoded string is not straightforward as explained in http://tinyurl.com/ce4vn99.
5) Using the Remember Certificate Exception addon for Firefox http://sejq.blogspot.in/2009/01/remember-certificate-exception.html is yet another option but I have not been able to do a full silent install of it. When we drop the xpi file in the required directory, the next time Firefox starts, it will prompt saying that should it found the addon and should it go ahead and install. The addon won't just get reflected in the browser. If anyone can atleast help me with a FULL silent install of this, it would be great!
All I want is that this works in Firefox. I am not interested in IE and Google Chrome.
Any help in providing an answer and not just terming this as a duplicate would be greatly appreciated.
I would suggest an alternative to Selenium.
Have you tried Sahi? It's basically a very similar application, in that it allows scripted browser control, but for a lot of cases it seems to work better than Selenium.
From the Sahi FAQ page:
How does it compare to Selenium?
Sahi is easier to learn and use for testers. Sahi’s recorder works on all browsers. Sahi automatically waits for AJAX and page loads. It does not use XPaths, but uses more intuitive APIs like _in and _near. Sahi works well across frames, iframes, popup windows, sites with dynamic ids, https sites, 401 authentication sites etc. It has inbuilt HTML reports, and ability to run multiple scripts in parallel.
In my experience, this is accurate. There are certain types of browser event that Selenium struggles with, that Sahi handles much better.
I know that doesn't solve your problem with Selenium, and I know it's a pain if you've already written your tests in Selenium to re-do them in Sahi so it may not be the solution you're looking for, but if all else fails, it may be worth considering.
ps - you didn't mention anything about your development environment, but there is a PHP-based testing tool called Mink which integrates with both Sahi and Selenium -- ie you can write the a single test script and have it work in either. I don't know of any similar tool in other languages, but it might be of interest if it helps you migrate between the two.
So the solution turned out to be the 5th point mentioned above.
Here are the steps to solve the issue -
1) Download the Remember Certificate Exception xpi file from https://addons.mozilla.org/en-US/firefox/addon/remember-certificate-exception/
2) Unzip it to get a set of files. Keep them in a temporary folder. (temp_folder/extracted_files)
3) Open the install.rdf file and copy the content of em:id node present in the Description node.
4) Rename the temporary folder just created with this copied string.
5) Place this folder under the profile_name/extensions/ directory. Also delete the extensions.cache/.ini/.rdf files from your profile folder. These need to be deleted so as to prevent the user from being prompted that his profile has been changed and that there is a new addon to be installed. Only if the 3 files are deleted, it will be a fully silent install.
This can be automated using a script.
#!/bin/bash
mkdir /tmp/addon
cp -r ~/POC/remember_certificate_exception-1.0.0-fx.xpi /tmp/addon
unzip /tmp/addon/remember_certificate_exception-1.0.0-fx.xpi -d /tmp/addon
rm /tmp/addon/remember_certificate_exception-1.0.0-fx.xpi
folder_name=$(cat /tmp/addon/install.rdf | grep -m 1 "<em:id>" | cut -f2 -d ">" | cut -f1 -d "<")
mv /tmp/addon /tmp/$folder_name
cp -r /tmp/$folder_name ~/.mozilla/firefox/*.default/extensions
rm -f ~/.mozilla/firefox/*.default/extensions.
https://addons.mozilla.org/en-US/firefox/addon/skip-cert-error/
Use this addon. This will bypass the https errors. This is configurable as well
I've put together an example that shows how to use Selenium to test HTTPS endpoints -- without ignoring certificate validation errors. It shows how to setup the selenium clients with trusted root CAs that can be used to validate the server certificates presented by the HTTPS endpoints being tested. See https://github.com/JeNeSuisPasDave/Selenium-and-TLS
The example demonstrates Chrome and Firefox Selenium nodes. Chrome looks in the system trusted certificate store for root CA certs used for validation; Firefox looks in the cert8.db file within the current profile.
I think this approach is important because I think that instructing your test harness to ignore certificate errors is unnecessarily risky; doing so may eventually hide errors that you wish you had trapped before they got deployed to production.
I wrote an short post about this topic, https://develves.net/blogs/asd/2017-02-20-selenium-and-tls/, but you really just need the example repo linked above.
A certain test script of mine needs to be run by the "operations" team periodically. My script uses the following components -
1. TestNG
2. Excel (for the input specifications)
3. Selenium RC, ofcourse.
It currently runs in Eclipse.
Is there a way I can package and host it in an, ideally, web accessible location, that folks in operations can click on and review results?
Thanks.
I ended up writing an Ant script controlling execution.