I'm having a weird event occur, where my dashing dashboard's list widget is showing erroneous data. Here's the screenshot from my live Dashing widget
Erroneous Widget
Expected Output
What follows is the code for the widget:
Code in .erb
<li data-row="1" data-col="1" data-sizex="2" data-sizey="6">
<div data-id="facebook_insights" data-view="List" data-unordered="true" data-title="Facebook Insights: Weekly Post Views" data-moreinfo="Updated every 10 seconds"</div>
</li>
Code in job .rb
require 'mysql2'
social_count = Hash.new({ value: 0 })
time = Time.new()
date_time1 = Time.new(time.year, time.month, time.day-1)
...
SCHEDULER.every '10s' do
begin
db = Mysql.new(<HOST>,<USER>,<PASS>,<DBNAME>)
mysql1 = "SELECT <VAR> FROM <TABLE> WHERE <VAR> = '#{date_time1}' ORDER BY <VAR> DESC LIMIT 1"
...
result1 = db.query(mysql1)
...
rescue
ensure
db.close
end
result1.each do |row|
strrow1 = row[0]
$value1 = strrow1.to_i
end
...
social_count[0] = {label: "1:", value: $value1}
...
send_event('facebook_insights', { items: social_count.values })
end
What is really baffling, is that this code works for a similar widget using different data in the SQL query. Can anyone help me understand why?
I checked and re-checked the data and in my other, working code, I had my $value variables defined as $valueX with X being the number. I thought to myself "Maybe the variable names are getting confused due to them having the same name", so I changed my code to
Working Code
result1.each do |row|
strrow1 = row[0]
$variable1 = strrow1.to_i
end
...
social_count[0] = {label: "1:", value: $variable1}
Et Voila! Eureka! It worked. Not sure why it still got confused with the names, but from now on, my names will be unique!
Related
I am trying to create an app which will fetch the gitlab/github commit history of the user which i can just show in a side panel and it can be checked on or off depending on my criteria. I want to know if there is a way to fetch the current logged user's gitlab/github commit history. I tried to use the gem
https://github.com/libgit2/rugged
But couldn't find a way to implement my need. If anyone knows how to implement this it would be so much helpful. Thanks.
Update
I have now managed to get the user by using :
Gitlab.endpoint = 'https://gitlab.com/api/v4'
Gitlab.private_token = 'token'
g = Gitlab.client(
endpoint: 'https://gitlab.com/api/v4',
private_token: 'token',
httparty: {
headers: { 'Cookie' => 'gitlab_canary=true' }
}
)
By using the command g.user i am able to get the user but i need to get the commits of the user that he has done in gitlab.
Use this gitlab API GET /projects/:id/repository/commits to fetch all the commits on a repository gitlab api. Check the attached code for more details.
Basically this git log --author="user_name" command can give you git commit history for a specific user, you can even use email address just the first name or last name in the above command.
Once you have authenticated with gitlab you can run the following command from ruby.
cmd = 'git log --author="user_name"'
system(cmd)
Sample code form ruby to connect to gitlab using private token which is not ideal but just an example
require 'json'
require 'curb'
require 'net/http'
begin
def parseCoverageReport(report_text)
coverage_perc = report_text.match /All files\s+\|\s+(\d+\.?\d+).*\n/
if not coverage_perc then
coverage_perc = report_text.match /^TOTAL\s+\d+\s+\d+\s+(\d+)%$/
end
if coverage_perc then
#if we found coverage value in job trace
#puts "coverage_perc[1]: #{coverage_perc[1]}"
coverage_perc[1].to_i
end
end
gen_config = YAML.load_file("config/general.yml")
gitlab_config = YAML.load_file("config/gitlab.yml")
SCHEDULER.every gen_config[:job_trigger_interval], :first_in => 0 do |job|
table = {
title: "Projects",
hrows: Array.new.push({cols: [
{value: "Project name"},
{value: "Open Merge Requests"},
{value: "Code coverage"}
]}),
rows: Array.new
}
instances = gitlab_config['instances']
instances.each do |instance|
gitlab_url = gitlab_config['instances'][instance.first]['url']
# gitlab_token = gitlab_config['instances'][instance.first]['api_key']
gitlab_token = ENV[gitlab_config['instances'][instance.first]['api_key']]
red_threshold = gitlab_config['instances'][instance.first]['red_threshold']
orange_threshold = gitlab_config['instances'][instance.first]['orange_threshold']
cov_red_threshold = gitlab_config['instances'][instance.first]['cov_red_threshold']
cov_orange_threshold = gitlab_config['instances'][instance.first]['cov_orange_threshold']
projects = gitlab_config['instances'][instance.first]['projects']
projects.each do |name, project|
merge_reqs = JSON.parse(Curl.get("#{gitlab_url}/api/v4/projects/#{project['id']}/merge_requests?state=opened&private_token=#{gitlab_token}&per_page=200").body_str)
git_project = JSON.parse(Curl.get("#{gitlab_url}/api/v4/projects/#{project['id']}?private_token=#{gitlab_token}").body_str)
opened_mrs = merge_reqs.select { |merge_reqs| %w[opened].include? merge_reqs['state'] }
repo_name = git_project['name']
repo_url = git_project['web_url']
status = case
when opened_mrs.size >= red_threshold then 'danger'
when opened_mrs.size >= orange_threshold then 'warning'
else
'ok'
end
mrs_count = "#{opened_mrs.size}"
send_event("#{name}_mr", { current: mrs_count, status: status })
color = case
when opened_mrs.size >= red_threshold then 'red'
when opened_mrs.size >= orange_threshold then 'orange'
else
'green'
end
font_color = color == 'orange' ? 'black' : 'white'
cov_color = color
font_cov_color = 'white'
code_coverage = "---"
code_coverage_tag = "---"
cov_job_url = ''
jobs = JSON.parse(Curl.get("#{gitlab_url}/api/v4/projects/#{project['id']}/jobs?scope=success&private_token=#{gitlab_token}&per_page=30").body_str)
code_cov_job = jobs.find { |gitlab_job| !gitlab_job['coverage'].nil? }
if not code_cov_job then
#if no job has 'coverage' feature set up in Gitlab try to parse
#'coverage' from jobs trace manually
jobs.each do |job|
trace_report = Curl.get("#{gitlab_url}/api/v4/projects/#{project['id']}/jobs/#{job['id']}/trace?private_token=#{gitlab_token}").body_str
code_cov_percentage = parseCoverageReport(trace_report)
if code_cov_percentage then
code_cov_job = job
code_cov_job['coverage'] = code_cov_percentage
break
end
end
end
if code_cov_job then
#found code coverage data => process them
code_coverage = code_cov_job['coverage'].to_i
cov_job_url = code_cov_job['web_url'].to_s
#update code covergate SprintProgress widgets at the same job
widget_title = "code_coverage_progress_#{project['id']}"
send_event(widget_title, {
title: "Code Coverage - #{git_project['name']}",
sprintboard_url: cov_job_url,
min: 0,
max: 100,
value: code_coverage,
moreinfo: ''
})
cov_color = case
when code_coverage <= cov_red_threshold then 'red'
when code_coverage <= cov_orange_threshold then 'orange'
else
'green'
end
code_coverage = "#{code_coverage}%"
code_coverage_tag = "<a href='#{cov_job_url}' target='_blank'>#{code_coverage.to_s}</a>"
end
repo_name_a_tag = "<a href='#{repo_url}' target='_blank'>#{repo_name}</a>"
open_mrs_size = "<a href='#{repo_url}/merge_requests' target='_blank'>#{opened_mrs.size}</a>"
table[:rows].push({
cols: [
{ value: repo_name_a_tag, style: "color: #{font_color}; background-color: #{color}" },
{ value: open_mrs_size, style: "color: #{font_color}; background-color: #{color}" },
{ value: code_coverage_tag, style: "color: #{cov_color == 'orange' ? 'black' : 'white'}; background-color: #{cov_color}" }
]
})
end
end
send_event('open_merge_requests_table', table)
end
rescue Errno::ENOENT
puts "No config file found for gitlab - not starting the Gitlab job"
end
In the above ruby example please have a look at the following code snippet
merge_reqs = JSON.parse(Curl.get("#{gitlab_url}/api/v4/projects/#{project['id']}/merge_requests?state=opened&private_token=#{gitlab_token}&per_page=200").body_str)
git_project = JSON.parse(Curl.get("#{gitlab_url}/api/v4/projects/#{project['id']}?private_token=#{gitlab_token}").body_str)
opened_mrs = merge_reqs.select { |merge_reqs| %w[opened].include? merge_reqs['state'] }
repo_name = git_project['name']
repo_url = git_project['web_url']
In here what i am trying to do is connect to our gitlab instance using a private_token and then for a specific project id (which you can get it form the UI of gitlab) check for the open merge request. I also get the git_project from which i get the name and web_url (which was my use case).
For your use case you will have to get the project_id(for gitlab UI) and then use some appropriate method to get the commits.gitlab docs
I have a code which looks like this
require 'net/http'
base = 'www.uniprot.org'
tool = 'mapping'
params = {
'from' => 'ACC+ID', 'to' => 'P_ENTREZGENEID', 'format' => 'tab',
'query' => 'A0A0K3AVS5 A0A0K3AVV4 A0A0K3AW32 A0A0K3AWP0'
}
http = Net::HTTP.new base
$stderr.puts "Submitting...\n";
response = http.request_post '/' + tool + '/',
params.keys.map {|key| key + '=' + params[key]}.join('&')
loc = nil
while response.code == '302'
loc = response['Location']
response = http.request_get loc
end
while loc
wait = response['Retry-After'] or break
$stderr.puts "Waiting (#{wait})...\n";
sleep wait.to_i
response = http.request_get loc
end
response.value # raises http error if not 2xx
puts response.body
which gives me what I need. however, I have two questions
1- How to load a query list instead I parse it into the code ? lets say I save a txt file with all the query i want to the desktop of a mac
2- How to export the output ?
If I have
B2D6P1
G5EC52
B2FDA8-2
B2MZB1
B3CJ34
B3CKG1
B3GWA1
what #tadman showed gives me the answer
however, I have the following
B2D6P1
G5EC52;B2D6P4
B2FDA8-2;B2FDA8
B2MZB1;P18834
B3CJ34
B3CKG1
B3GWA1;Q8I7K5
and the answer is like below
B2D6P1 rmd-2
G5EC52 tlf-1
B2D6P4 tlf-1
B2FDA8 smc-3
B2MZB1 col-14
P18834 col-14
B3CJ34 gcn-1
B3CKG1 urm-1
B3GWA1 nono-1
Q8I7K5 nono-1
what I want is that if I have two entries in each row (separated with ;) leading to the similar output , it gives me only one , otherwise give me as many as they have for example in above example , my desire output is
B2D6P1 rmd-2
G5EC52;B2D6P4 tlf-1
B2FDA8-2;B2FDA8 smc-3
B2MZB1;P18834 col-14
B3CJ34 gcn-1
B3CKG1 urm-1
B3GWA1;Q8I7K5 nono-1
is this possible ?
Reading query data:
query = File.readlines('ids.txt').map(&:chomp).join(' ')
That way you can have them on separate lines, easier to edit, and they're space separated when submitted.
That makes your params look like:
params = {
'query' => query,
...
}
Writing data:
File.open('output.txt', 'w') do |f|
f.write(response.body)
end
That's all there is to it. If it's a string, or can be converted to a string, you can write it to a file.
I have some sample code I can execute for our Nexpose server and I need to do some mass asset tagging. Here is an example of the code.
nsc = Nexpose::Connection.new('your_nexpose_instance', 'username', 'password', 3780)
nsc.login
criterion = Nexpose::Tag::Criterion.new('IP_RANGE', 'IN', ['ip1', 'ip2'])
criteria = Nexpose::Tag::Criteria.new(criterion)
tag = Nexpose::Tag.new("tagname", Nexpose::Tag::Type::Generic::CUSTOM)
tag.search_criteria = criteria
tag.save(nsc)
I have a file called with the following data.
ip1,ip2,tagname
192.168.1.1,192.168.1.255,Workstations
How would I go about running a for loop and using the CSV to quickly process the above code? I have no experiance with Ruby and tried to follow some example but I'm confused at this point.
There's a CSV library in Ruby's standard lib collection that you can use.
Basic example based on your code example and data, not tested:
require 'csv'
nsc = Nexpose::Connection.new('your_nexpose_instance', 'username', 'password', 3780)
nsc.login
CSV.foreach("path/to/file.csv", headers: true) do |row|
criterion = Nexpose::Tag::Criterion.new('IP_RANGE', 'IN', [row['ip1'], row['ip2'])
criteria = Nexpose::Tag::Criteria.new(criterion)
tag = Nexpose::Tag.new(row['tagname'], Nexpose::Tag::Type::Generic::CUSTOM)
tag.search_criteria = criteria
tag.save(nsc)
end
I made a directory with input.csv and main.rb
input.csv
ip1,ip2,tagname
192.168.1.1,192.168.1.255,Workstations
main.rb
require "csv"
CSV.foreach("input.csv", headers: true) do |row|
puts "ip1: #{row['ip1']}"
puts "ip2: #{row['ip2']}"
puts "tagname: #{row['tagname']}"
end
the output is
ip1: 192.168.1.1
ip2: 192.168.1.255
tagname: Workstations
I hope this can help. If you have questions I'm here :)
If you just need to loop through each line of the file and fire that chunk of code for each line, you could do something like this:
file = Net::HTTP.get(URI(<whatever_your_file_name_is>))
index = 0
file.each_line do |line|
next if index == 0
index += 1
split_line = line.split(',')
ip1 = split_line[0]
ip2 = split_line[1]
tagname = split_line[2]
nsc = Nexpose::Connection.new('your_nexpose_instance', 'username', 'password', 3780)
nsc.login
criterion = Nexpose::Tag::Criterion.new('IP_RANGE', 'IN', [ip1, ip2])
criteria = Nexpose::Tag::Criteria.new(criterion)
tag = Nexpose::Tag.new(tagname, Nexpose::Tag::Type::Generic::CUSTOM)
tag.search_criteria = criteria
tag.save(nsc)
end
NOTE: This code example is assuming that the CSV file is stored remotely, not locally.
ALSO: In case you're wondering, the next if index == 0 is there to skip your header record.
UPDATE
To use this approach for a local file, you can use File.open() instead of Net::HTTP.get(), like so:
file = File.open(<whatever_your_file_name_is>).read
Two things to note:
Make sure you use the fully-qualified name of the file - i.e. ~/folder/folder/filename.csv instead of just filename.csv.
If the files you're going to be loading are enormous, this might not be an ideal approach because it's actually reading the whole file into memory. But considering your file only has 3 columns, you'd have to have an extreme number of rows in the file for this to be an issue.
Hi Im working on writing a script that pulls data out of a ticketing system. Once it pulls the data it analyze the content of it and if it the content match a specific criteria then it needs to build a data structure file that will be dump in the same server.
I was able to parse the data in JSON format, listed below is the content:
[{"id"=>10423,
"type"=>"Ticket",
"lastUpdated"=>"2014-11-04T10:58:47Z",
"shortSubject"=>"FOO STATUS UPDATE",
"shortDetail"=>"Reply to this message if all systems are functional..",
"displayClient"=>"No Client",
"updateFlagType"=>0,
"prettyLastUpdated"=>"54 minutes ago",
"latestNote"=>
{"id"=>16850,
"type"=>"TechNote",
"mobileListText"=>"<b>t. trust: </b> All Systems are OK",
"noteColor"=>"clear",
"noteClass"=>"bubble right"}},
{"id"=>10422,
"type"=>"Ticket",
"lastUpdated"=>"2014-11-04T10:54:07Z",
"shortSubject"=>"FOO STATUS UPDATE",
"shortDetail"=>"Reply to this message if all systems are functional..",
"displayClient"=>"No Client",
"updateFlagType"=>0,
"prettyLastUpdated"=>"58 minutes ago",
"latestNote"=>nil},
{"id"=>10421,
"type"=>"Ticket",
"lastUpdated"=>"2014-11-04T10:53:17Z",
"shortSubject"=>"FOO STATUS UPDATE",
"shortDetail"=>"Reply to this message if all systems are functional..",
"displayClient"=>"No Client",
"updateFlagType"=>0,
"prettyLastUpdated"=>"59 minutes ago",
"latestNote"=>nil}]
In the data above you can see that each ticket has an id, lastupdate, short Subject, short Detail and lastest note the value of the latest note will be nill if no one reply to the ticket but if someone does reply then the value mobileListText will have something.
So what I need to do pretty much is once I get this data the script will look for the subject that complies with "FOO STATUS UPDATE" if that value matches then looks for the content of the shortDetail matches "Reply to this message if all systems are functional.." and if this complies then looks for its latestNote, if latestNote is nill then it will create a log file specifiying date and time when it run, the id of the ticket with this state and a message saying, ticket has not being reply, but if the latest note has the value "mobileListText"=>"t. trust: All Systems are OK", then creates the follwing data structure:
{"LastUpdate":1415130257,"Service":[{"time":"11-04-2014 10:58:47 GMT","region:":"","id":"","description":"All Systems are OK","service":""},{"time":"11-04-2014 10:54:07 GMT","region:":"","id":"","description":"All Systems are OK","service":""},{"time":"11-04-2014 10:53:17 GMT","region:":"","id":"","description":"All Systems are OK","service":""}]}
Im able to have part of this however, based on the data above, only one ticket has All Systems are OK, meaining that only one of the tickets has being reply, and it only should write something like this:
{"LastUpdate":1415130257,"Service":[{"time":"11-04-2014 10:58:47 GMT","region:":"","id":"","description":"All Systems are OK","service":""}]}
But Instead repeats this only ticket that has being replied serveral times.
this my code so far:
require 'rubygems'
require 'json'
require 'net/http'
require 'highline/import'
require 'pp'
require 'logger'
#usersol='foo'
#passol= 'foo123'
#urlsol= "http://dev-webhelpdesk.foo.corp:8081/helpdesk/WebObjects/Helpdesk.woa/ra/Tickets?list=group&page=1&limit=#{#limit}&username=#{#usersol}&password=#{#passol}"
#limit = '25'
#log = #log= Logger.new( 'message_solar.log')
def ticket_data #looks for ticket data in solarwinds
resp = Net::HTTP.get_response(URI.parse(#urlsol))
url_output = resp.body
JSON.parse(url_output)
end
#CRONJOB THAT START ALL
#echo "Reply to this message if all systems are functional.." | mail -r noc#foo.com -s "FOO STATUS UPDATE:" noc-team#FOO.com >> /dev/null
# Looking for all the tickets with the following content
# ticket id, ticket subject and content
def search_allok(allok)
description = []
allok.each do |systems|
output1 = systems.has_key?'id'
if output1
systems.values_at('shortSubject').each do | subject |
output2 = subject.match(%r(TRUST STATUS UPDATE))
if output2
latestnote = systems.values_at('latestNote')
latestnote.each do |content|
if content
final = content.values_at('mobileListText')
final_ok = final[0].sub!(/^\<b\>.*\<\/b\>\s/, "")
systems_ok = final_ok.match(%r(All Systems are OK))
if systems_ok
ids = systems['id']
notify = {"LastUpdate" => Time.now.to_i, "Service" => []}
allok.each do |lastup|
reference = lastup.has_key? 'id'
if reference
timeid = lastup.values_at('lastUpdated')
timeid.each do |lines|
final=lines.split(/[-, T, Z]/)
notify["Service"] << { "time" => "#{final[1]}-#{final[2]}-#{final[0]} #{final[3]} GMT", "region:" => "", "id" => "#{ids}", "description" => "#{systems_ok}" , "service" => ''}
end
end
end
File.open("notify.json", "w") do |fileformatted|
fileformatted.puts (JSON.dump(notify))
end
else
time = Time.now
#log.info("#{time} - Ticket ID #{systems['id']} has not being updated")
end
else
#log.info("#{time} - Ticket ID #{systems['id']} has not being reply")
end
end
end
end
end
end
end
# If the content is there then it need to create
# the data structure including the lastupdated
# (time when it run the script), and the lastupdate for the ticket
# and the description All Systems OK
#This method below I added to the one above, but I was thinking on doing it separate but I encouter issues passing the information needed from above to below
def datastructure(format_file) #creates JSON file lastupdated of each ticket in the queue
notify = {"LastUpdate" => Time.now.to_i, "Service" => []}
format_file.each do |lastup|
reference = lastup.has_key? 'id'
if reference
timeid = lastup.values_at('lastUpdated')
timeid.each do |lines|
final=lines.split(/[-, T, Z]/)
notify["Service"] << { "time" => "#{final[1]}-#{final[2]}-#{final[0]} #{final[3]} GMT", "region:" => "", "id" => "", "description" => region , "service" => ''}
end
end
end
File.open("notify.json", "w") do |fileformatted|
fileformatted.puts (JSON.dump(notify))
end
end
#ticket_data
#datastructure(ticket_data)
search_allok(ticket_data)
The code you've written is basically doing a roundabout version of what is achieved by using ruby's map and select methods. See this article: Ruby Explained: Map, Select, and Other Enumerable Methods
I am doing an Ajax call, using Ruby and Sinatra. The query should return multiple rows, it only returns one though.
The ajax script is:
$(document).ready(function() {
$(".showmembers").click(function(e) {
e.preventDefault();
alert('script');
var short_id = $('#shortmembers').val();
console.log(short_id);
$.getJSON(
"/show",
{ 'id' : short_id },
function(res, status) {
console.log(res);
$('#result').html('');
$('#result').append('<input type=checkbox value=' + res["email"] + '>');
$('#result').append( res["first"] );
$('#result').append( res["last"] );
$('#result').append( res["email"] );
});
});
});
and the Ruby script is:
get '/show' do
id = params['id']
DB["select shortname, first, last, email from shortlists sh JOIN shortmembers sm ON sm.short_id = sh.list_id JOIN candidates ca ON ca.id = sm.candidate_id where sh.list_id = ?", id].each do |row|
#shortname = row[:shortname]
#first = row[:first]
#last = row[:last]
#email = row[:email]
puts #shortname
puts #first
puts #last
puts #email
halt 200, { shortname: #shortname, first: #first, last: #last, email: #email }.to_json
end
end
If I run the query directly in the terminal on postgres I get 9 rows returned but, as above on my website, it just returns the first row only.
What's the problem? No error in the console, just one record.
You have halt 200 inside your loop. This will cause Sinatra to terminate the request processing and return the result back up the stack.
To return a full set of results, you will need to do something like the following:
get '/show' do
id = params['id']
results = DB["select shortname, first, last, email from shortlists sh
JOIN shortmembers sm ON sm.short_id = sh.list_id
JOIN candidates ca ON ca.id = sm.candidate_id
where sh.list_id = ?", id].map do |row|
{
:short_name => row[:shortname],
:first=>row[:first],
:last=>row[:last],
:email=>row[:email]
}
end
halt 200, results.to_json
end
This will return the selected fields from each row as an array of hashes.
In fact, as I look at the above, the solution might even be as simple as:
get '/show' do
id = params['id']
results = DB["select shortname, first, last, email from shortlists sh
JOIN shortmembers sm ON sm.short_id = sh.list_id
JOIN candidates ca ON ca.id = sm.candidate_id
where sh.list_id = ?", id]
halt 200, results.to_json
end
since you don't seem to be selecting anything but the columns you desire in the first place.