Need assistance with creating a Git client-side 'commit-msg' hook - ruby

I have installed the 'ticket_status.rb' server-side hook on Assembla. Although this is exactly what I'm looking for (in theory), it does not flag until the developer attempts to push to the server. If they have made several commits before pushing, it becomes incredibly frustrating to go back through their history and edit any invalid commit messages.
I am looking to create a client-side hook that will reject a developer's commit if an open ticket in Assembla is not referenced in the commit message. I assume that since it is client-side, it will not be able to check if the ticket is open in the Assembla project space. However, if the hook could at least check that '#n' has been included in the commit message (where 0 < n < 10,000), it should catch the majority of invalid commit messages.
GitHub has provided sample code for a client-side 'commit-msg' hook. I would like assistance in modifying the code below to instead search for a ticket number (#n) in the commit message (or an open ticket in the Assembla project space, if possible):
#!/bin/sh
#
# An example hook script to check the commit log message.
# Called by "git commit" with one argument, the name of the file
# that has the commit message. The hook should exit with non-zero
# status after issuing an appropriate message if it wants to stop the
# commit. The hook is allowed to edit the commit message file.
#
# To enable this hook, rename this file to "commit-msg".
# Uncomment the below to add a Signed-off-by line to the message.
# Doing this in a hook is a bad idea in general, but the prepare-commit-msg
# hook is more suited to it.
#
# SOB=$(git var GIT_AUTHOR_IDENT | sed -n 's/^\(.*>\).*$/Signed-off-by: \1/p')
# grep -qs "^$SOB" "$1" || echo "$SOB" >> "$1"
# This example catches duplicate Signed-off-by lines.
test "" = "$(grep '^Signed-off-by: ' "$1" |
sort | uniq -c | sed -e '/^[ ]*1[ ]/d')" || {
echo >&2 Duplicate Signed-off-by lines.
exit 1
}
I have also provided the source code for the server-side hook that rejects the commit if it does not contain a valid open ticket number in the commit message (ticket_status.rb):
#!/usr/bin/env ruby
# -*- encoding : utf-8 -*-
#
# Reject a push to a branch if it has commits that do refer a ticket in open state
#
# ref = ARGV[0]
sha_start = ARGV[1]
sha_end = ARGV[2]
# HOOK PARAMS
space = 'space-wiki-name'
api_key = 'user-api-key'
api_secret = 'user-api-secret'
# HOOK START, end of params block
require "net/https"
require "uri"
begin
require "json"
rescue LoadError
require 'rubygems'
require 'json'
end
# Check referred tickets that are in open stage
class TicketValidator
API_URL = "https://api.assembla.com"
attr_accessor :space, :api_key, :api_secret
def initialize()
#ticket_statuses = []
#tickets = {}
end
def init
init_http
load_statuses
end
def check(sha, comment)
comment.to_s.scan(/#\d+/).each do |t|
ticket = t.tr('#', '')
# Do not check it twice
next if #tickets[ticket]
ticket_js = api_call "/v1/spaces/#{space}/tickets/#{ticket}.json"
error = nil
if ticket_js['error'].nil?
unless #ticket_statuses.include? ticket_js['status'].downcase
error = "Ticket #{t} is not open!"
end
else
error = ticket_js['error']
end
if error
#tickets[ticket] = {:error => error, :sha => sha}
else
#tickets[ticket] = :ok
end
end
end
def load_statuses
statuses = api_call "/v1/spaces/#{space}/tickets/statuses.json"
statuses.each do |status|
if status["state"] == 1 # open
#ticket_statuses << status["name"].downcase
end
end
end
def api_call(uri)
request = Net::HTTP::Get.new(uri,
{'Content-Type' => 'application/json',
'X-Api-Key' => api_key,
'X-Api-Secret' => api_secret})
result = #http.request(request)
JSON.parse(result.body)
end
def init_http
uri = URI.parse(API_URL)
#http = Net::HTTP.new(uri.host, uri.port)
#http.use_ssl = true
#http.verify_mode = OpenSSL::SSL::VERIFY_NONE
end
def show_decision!
#tickets.reject! {|_, value| value == :ok }
unless #tickets.empty?
puts "You have references to tickets in closed state"
#tickets.each do |ticket, details|
puts "\t#{details[:sha]} - ##{ticket} #{details[:error]}"
end
puts "Valid statuses: #{#ticket_statuses.join(', ')}"
exit 1
end
end
end
class Parser
def initialize(text, validator)
#text = text
#validator = validator
end
def parse
commit = nil
comment = nil
#validator.init
#text.to_s.split("\n").each do |line|
if line =~ /^commit: ([a-z0-9]+)$/i
new_commit = $1
if comment
#validator.check(commit, comment)
comment = nil
end
commit = new_commit
else
comment = comment.to_s + line + "\n"
end
end
# Check last commit
#validator.check(commit, comment) if comment
end
end
text = `git log --pretty='format:commit: %h%n%B' #{sha_start}..#{sha_end}`
#validator = TicketValidator.new
#validator.space = space
#validator.api_key = api_key
#validator.api_secret = api_secret
Parser.new(text, #validator).parse
#validator.show_decision!
Any help is much appreciated. Thanks

You can try this commit-msg validator. It's not ruby but you can easily configure it for your needs and you can even write your own Assembla reference to validate ticket numbers against their API. See the repo README for more details.
Here is a starting point for your custom reference, and its associated test file. I haven't tested it thoroughly but it should be pretty easy to change as you wish, since it's basically JavaScript.
lib/references/assembla.js
'use strict';
var exec = require('child_process').exec;
var https = require('https');
var util = require('util');
// HOOK PARAMS
var space = 'space-wiki-name';
var apiKey = 'user-api-key';
var apiSecret = 'user-api-secret';
function Ticket(ticket, match) {
this.allowInSubject = true;
this.match = match;
this._ticket = ticket;
}
Ticket.prototype.toString = function() {
return '#' + this._ticket;
}
Ticket.prototype.isValid = function(cb) {
var options = {
hostname: 'api.assembla.com',
path: util.format('/v1/spaces/%s/tickets/%s.json', space, this._ticket),
headers: {
'Content-Type' : 'application/json',
'X-Api-Key' : apiKey,
'X-Api-Secret' : apiSecret
}
};
https.get(options, function(res) {
if (res.statusCode === 404) {
return cb(null, false); // invalid
}
var body = '';
res.on('data', function(chunk) {
body += chunk.toString();
});
res.on('end', function () {
var response = body ? JSON.parse(body) : false;
if (res.statusCode < 300 && response) {
return cb(null, true); // valid?
}
console.error('warning: Reference check failed with status code %d',
res.statusCode,
response && response.message ? ('; reason: ' + response.message) : '');
cb(null, false); // request errored out?
});
});
}
// Fake class that requires the existence of a ticket # in every commit
function TicketRequired() {
Ticket.call(this);
this.error = new Error('Commit should include an Assembla ticket #');
}
util.inherits(TicketRequired, Ticket);
TicketRequired.prototype.isValid = function(cb) {
cb(null, false);
}
Ticket.parse = function(text) {
var instances = [];
var cb = function(match, ticket) {
instances.push( new Ticket(ticket, match) );
};
text.replace(/#(-?\d+)\b/gi, cb);
if (!instances.length) {
// maybe should skip merge commits here
instances.push(new TicketRequired());
}
return instances;
}
module.exports = Ticket;
test/references/assembla.js
'use strict';
var assert = require('assert');
var Ticket = require('../../lib/references/assembla');
describe('references/assembla', function() {
it('should validate correctly using the API', function(done) {
this.timeout(5000); // allow enough time
var tickets = Ticket.parse('Change functionality\n\nFixes #13 and #9999 (invalid)');
var ct = 0;
var checkDone = function() {
if (++ct == tickets.length) done();
};
var valid = [true, false];
valid.forEach(function(val, idx) {
tickets[idx].isValid(function(err, valid) {
assert.equal(valid, val, tickets[idx].toString());
checkDone();
});
});
});
it('should require a ticket #', function() {
var tickets = Ticket.parse('Commit message without any ticket ref #');
assert.equal(tickets.length, 1);
assert.equal(tickets[0].error.message, 'Commit should include an Assembla ticket #');
});
});

Related

Discord.js Breaking down Ban Command for Command Handler issues

I have a rather large index.js so iv been busy stripping it out into individual command files, i have successfully exported all my code into separate commands, all but ban. So far the ban code works as intended, but if a 18 digit user ID is entered, and they have left the server it should check and log it in a .json file but currently its not getting past if (isNaN(args[1])) it just plain wont accept anything, number and or letters all result in "You need to enter a vlaid #Member or UserID #" This is very frustrating as it took me hours to get working within my index.js
run: async (bot, message, args) => {
if (!message.member.hasPermission(["BAN_MEMBERS", "ADMINISTRATOR"])) return message.channel.send("You do not have permission to perform this command!")
const user1 = message.mentions.users.first();
var member = message.mentions.members.first()
if (member) {
const member = message.mentions.members.first()
let reason = args.slice(2).join(' '); // arguments should already be defined
var user = message.mentions.users.first();
member.ban({ reason: `${args.slice(2).join(' ')}` }).then(() => {
let uEmbed = new RichEmbed()
.setTitle('**' + `Sucessfully Banned ${user1.tag}!` + '**')
.setThumbnail('https://i.gyazo.com/8988806671312f358509cf0fd69341006.jpg')
.setImage('https://media3.giphy.com/media/H99r2HtnYs492/giphy.gif?cid=ecf05e47db8ad81dd0dbb6b132bb551add0955f9b92ba021&rid=giphy.gif')
.setColor(0x320b52)
.setTimestamp()
.setFooter('Requested by ' + message.author.tag, 'https://i.gyazo.com/8988806671312f358509cf0fd69341006.jpg')
message.channel.send(uEmbed);
}).catch(err => {
message.channel.send('I was unable to kick the member');
console.log(err);
});
} else {
let user = message.mentions.users.first(),
userID = user ? user.id : args[1]
if (isNaN(args[1])) return message.channel.send("You need to enter a vlaid #Member or UserID #");
if (args[1].length <= 17 || args[1].length >= 19) return message.channel.send("UserID # must be 18 Digits");
if (userID) {
let bannedIDs = require('./bannedIDs.json').ids || []
if (!bannedIDs.includes(userID)) bannedIDs.push(userID)
fs.writeFileSync('./bannedIDs.json', JSON.stringify({ ids: bannedIDs }))
let reason = args.slice(2).join(' ');
let uEmbed = new RichEmbed()
.setTitle('**' + `UserID #${args[1]}\n Will be Banned on Return!` + '**')
.setThumbnail('https://i.gyazo.com/8988806671312f358509cf0fd69341006.jpg')
.setImage('https://i.imgur.com/6Sh8csf.gif')
.setColor(0x320b52)
.setTimestamp()
.setFooter('Requested by ' + message.author.tag, 'https://i.gyazo.com/8988806671312f358509cf0fd69341006.jpg')
message.channel.send(uEmbed);
} else {
message.channel.send('Error')
}
}
}
}
Firstly, if (args[1].length <= 17 || args[1].length >= 19 can be reduced to if(args[1].length !== 18), I saw your other post and it suggests you got passed this bug, is this correct?

Post request for bulk API is giving status code of 406, How to resolve it?

I am using Elastic search 6.1 version
My data is appending correctly and I am adding '\n' at the end of the request.
My code is as follows:
def insert_in_bulk(self, filee, rtype):
U = urljoin(self.args.host, "/_bulk")
body = []
f = open(filee)
for line in f:
action = {
'index' :{
'_index' : self.args.index,
'_type' : rtype,
}
}
item = {
'word' : line.strip()
}
body.append(json.dumps(action))
body.append(json.dumps(item))
f.close()
body = '\n'.join(body)+'\n'
success = False
try:
r = requests.post(U, data=body)
self.log.info("after request")
if r.status_code == 200:
success = True
r = r.json()
self.log.info("inserted %s items of type = %s", self.args.index , rtype)
except (SystemExit, KeyboardInterrupt): raise
except:
self.log.exception("during bulk index")
if not success:
self.log.error("failed to index records of type = %s", rtype)
I am using the python to connect to elastic search.
I got the answer from this link
Bulk index document from JSON file into ElasticSearch
I have to pass the header to the request as application/x-ndjson.
Though it is quite some time question is asked, but i want to give a solution that has worked for me in most case,
def insert_in_bulk(self, filee, rtype):
U = urljoin(self.args.host, "/_bulk")
body = []
f = open(filee)
for line in f:
action = {
'index' :{
'_index' : self.args.index,
'_type' : rtype,
}
}
item = {
'word' : line.strip()
}
body.append(json.dumps(action))
body.append(json.dumps(item))
f.close()
payload = ""
for l in body:
payload = payload + f"{l} \n"
data = payload.encode('utf-8')
r = requests.post(U, data=data, headers={"Content-Type": "application/x-ndjson"})
print(r.text)

Unable to produce Proper Encryption Key in Ruby using HMAC

I am attempting to follow the documentation per Access Control and interrogating code like azure-documentdb-node SDK and I am unable to do so.
I get the following error: 401 Unauthorized: {"code":"Unauthorized","message":"The input authorization token can't serve the request. Please check that the expected payload is built as per the protocol, and check the key being used. Server used the following payload to sign: 'post\ndbs\n\n13 april 2015 18:21:05 gmt\n\n'\r\nActivityId: ...
My ruby code looks like the following:
require 'openssl'
require 'rest-client'
require 'base64'
require 'uri'
require 'json'
require 'time'
def get_databases url, master_key
time = Time.now.httpdate
authorization = get_master_auth_token "get", "", "dbs", time, master_key
header = { "authorization" => authorization, "x-ms-date" => time, "x-ms-version" => "2015-04-08" }
RestClient.get url, header
end
def get_master_auth_token verb, resource_id, resource_type, date, master_key
digest = OpenSSL::Digest::SHA256.new
key = Base64.decode64 master_key
text = verb + "\n" +
resource_type + "\n" +
resource_id + "\n" +
date + "\n" +
"\n"
hmac = OpenSSL::HMAC.digest digest, key, text.downcase
auth_string = "type=" + "master" + "&ver=" + "1.0" + "&sig=" + hmac
URI.escape auth_string
end
Thanks!
EDIT: After Ryan's advice and example I've simplified the code down to the following snippit that should be a match for the node code he has posted BUT it still fails in ruby:
def hard_coded_get_databases master_key, url
verb = "get"
resource_type = "dbs"
resource_id = ""
date = Time.now.httpdate
serv_version = '2014-08-21'
master_token = "master"
token_version = "1.0"
key = Base64.decode64 master_key
text = verb + "\n" + resource_type + "\n" + resource_id + "\n" + date + "\n\n"
body = text.downcase.force_encoding "utf-8"
signature = OpenSSL::HMAC.digest OpenSSL::Digest::SHA256.new, key, body
auth_token = URI.escape("type="+master_token + "&ver="+token_version + "&sig="+signature)
header = { "accept" => "application/json", "x-ms-version" => serv_version, "x-ms-date" => date, "authorization" => auth_token }
RestClient.get url, header
end
EDIT2: I believe I've isolated the problem to how I am doing the master key authentication.
Taking Ryan's example we can trim his node code down the following:
var crypto = require("crypto")
function encode_message(masterKey, message) {
var key = new Buffer(masterKey, "base64"); // encode/decode? base64 the masterKey
var body = new Buffer(message.toLowerCase(), "utf8"); // convert message to "utf8" and lower case
return crypto.createHmac("sha256", key).update(body).digest("base64"); // encrypt the message using key
}
If I call this node code I can produce the following key:
encode_message("blah", 'get\ncolls\n\nTue, 14 Apr 2015 13:34:22 GMT\n\n')
'IYlLuyZtVLx5ANkGMAxviDHgC/DJJXSj1gUGLvN0oM8='
If I produce the equivalent ruby code to create the authentication my ruby code looks like the following:
require 'base64'
require 'openssl'
def encode_message master_key, message
key = Base64.urlsafe_decode64 master_key
hmac = OpenSSL::HMAC.digest 'sha256', key, message
Base64.urlsafe_encode64 hmac
end
If I call this code I get the following:
2.2.1 :021 > encode_message("blah", "get\ncolls\n\nTue, 14 Apr 2015 13:34:22 GMT\n\n")
=> "N6BL3n4eSvYA8dIL1KzlTIvR3TcYpdqW2UNPtKWrjP8="
Clearly the 2 encoded auth tokens are not the same. (Ryan again thanks so much for the help to get this far).
I have found the answer. Thanks to Magnus Stahre ... he is the man for helping me figure it out.
It was the encoding as I thought and the trick is this:
def encode_message master_key, message
key = Base64.urlsafe_decode64 master_key
hmac = OpenSSL::HMAC.digest 'sha256', key, message.downcase
Base64.encode64(hmac).strip
end
I was downcasing in my code too early AND my Base64.encode64 was failing to strip away a newline character that ruby was adding on the end.
i'll start off by apologizing for my limited Ruby knowledge but let me try assist here;
in your get_master_auth_token function it appears you are decoding the key before using it. is this correct? if so, why?
here is a node.js sample that uses the master key, builds up the auth header value and does a simple http call to list the collections in a database
var crypto = require("crypto");
var https = require("https");
https.globalAgent.options.secureProtocol = "TLSv1_client_method";
var verb = 'get';
var resourceType = 'dbs'; //the resource you are trying to get. dbs, colls, docs etc.
var resourceId = ''; //the parent resource id. note: not the id, but the _rid. but for you, because you are trying to lookup list of databases there is no parent
var masterKey = '...'; //your masterkey
var date = new Date().toUTCString();
var auth = getAuthorizationTokenUsingMasterKey(verb, resourceId, resourceType, date, masterKey);
var options = {
hostname: '...', //your hostname (without https://)
port: 443,
path: '/dbs/',
method: 'GET',
headers: {
accept: 'application/json',
'x-ms-version': '2014-08-21',
'x-ms-date': date,
authorization: auth,
}
};
for (var i = 0; i < 1000; i++) {
var req = https.request(options, function (res) {
process.stdout.write(new Date().toUTCString() + " - statusCode: " + res.statusCode + "\n");
res.on('data', function (d) {
}).on('error', function (e) {
})
});
//console.log(req);
req.end();
}
function getAuthorizationTokenUsingMasterKey(verb, resourceId, resourceType, date, masterKey) {
var key = new Buffer(masterKey, "base64");
var text = (verb || "") + "\n" +
(resourceType || "") + "\n" +
(resourceId || "") + "\n" +
(date || "") + "\n" +
("") + "\n";
var body = new Buffer(text.toLowerCase(), "utf8");
var signature = crypto.createHmac("sha256", key).update(body).digest("base64");
var MasterToken = "master";
var TokenVersion = "1.0";
return encodeURIComponent("type=" + MasterToken + "&ver=" + TokenVersion + "&sig=" + signature);
}
In your example, the resourceId passed to the getAuthorizationTokenUsingMasterKey method should be "" and the resourceType should be "dbs" as you have it.
I did notice that in some cases you have to URI Encode the value, but I think you are doing that already as the very last line of the func.
the only difference I can spot in your code vs my code is that you appear to be decoding the master_key which I don't do.
what I would recommend you do is run this node sample and compare the values of the strings we have in body & signature to the your values. they need to match.

How can I stringify a BSON object inside of a MongoDB map function?

I have documents with field xyz containing
{ term: "puppies", page: { skip: 1, per_page: 20 } } // not useful as a composite key...
{ page: { skip: 1, per_page: 20 }, term: "puppies" } // different order, same contents
For the sake of determining the "top" values in xyz, I want to map them all to something like
emit('term="puppies",page={ skip: 1, per_page: 20 }', 1); // composite key
but I can't get the embedded objects into a meaningful strings:
emit('term="puppies",page=[object bson_object]', 1); // not useful
Any suggestions for a function to use instead of toString()?
# return the top <num> values of <field> based on a query <selector>
#
# example: top(10, :xyz, {}, {})
def top(num, field, selector, opts = {})
m = ::BSON::Code.new <<-EOS
function() {
var keys = [];
for (var key in this.#{field}) {
keys.push(key);
}
keys.sort ();
var sortedKeyValuePairs = [];
for (i in keys) {
var key = keys[i];
var value = this.#{field}[key];
if (value.constructor.name == 'String') {
var stringifiedValue = value;
} else if (value.constructor.name == 'bson_object') {
// this just says "[object bson_object]" which is not useful
var stringifiedValue = value.toString();
} else {
var stringifiedValue = value.toString();
}
sortedKeyValuePairs.push([key, stringifiedValue].join('='));
}
// hopefully we'll end up with something like
// emit("term=puppies,page={skip:1, per_page:20}")
// instead of
// emit("term=puppies,page=[object bson_object]")
emit(sortedKeyValuePairs.join(','), 1);
}
EOS
r = ::BSON::Code.new <<-EOS
function(k, vals) {
var sum=0;
for (var i in vals) sum += vals[i];
return sum;
}
EOS
docs = []
collection.map_reduce(m, r, opts.merge(:query => selector)).find({}, :limit => num, :sort => [['value', ::Mongo::DESCENDING]]).each do |doc|
docs.push doc
end
docs
end
Given that MongoDB uses SpiderMonkey as its internal JS engine, can't you use JSON.stringify (will work even if/when MongoDB switches to V8) or SpiderMonkey's non-standard toSource method?
(sorry, can't try it ATM to confirm it'd work)
toSource method will do the work, but it adds also brackets.
for a clean document use:
value.toSource().substring(1, value.toSource().length - 1)

Help re-write this Python into Ruby: AJAX POST, decode Base-64, and save

I'm following a phonegap tutorial and I do not know how to write this "def iphone_upload " action in ruby 1.9.2/rails 3.
http://wiki.phonegap.com/w/page/18270855/Image-Upload-using-JQuery-and-Python
function getPicture_Success(imageData)
{
var feedURL = APIPATH + "photos/iphone-upload/";
$.post(feedURL, {imageData:imageData}, function(data){
});
}
In Python (Django):
def iphone_upload(request):
import base64
data = base64.b64decode(request.POST.get("imageData"))
fileout = "/var/www/test.jpg"
f1 = open(fileout,'wb+')
f1.write(data)
f1.close()
def iphone_upload
#data = request.POST[:imageData].unpack("m")[0]
fileout = "/var/www/test.jpg"
File.open(fileout, 'w') {|f| f.write(#data) }
end

Resources