I'm fairly new to Nim, and I suspect I'm just doing something wrong here. I'm using Jester (for routing, etc) and Nimcrytpo (for hmac) but something isn't adding up. Here's how I'm attempting to verify a signature:
import jester
import dotenv
import os, strutils, times
import nimcrypto
const timestampHeader = "X-Slack-Request-Timestamp"
const slackSignatureHeader = "X-Slack-Signature"
const signatureVersion = "v0"
const signingSecret = os.getEnv("SLACK_SIGNING_SECRET")
proc isTimestampRecent(timestamp: int): bool =
abs(getTime().toUnix - timestamp) <= (60 * 5)
proc verifySignature*(request: Request): bool =
if (not request.headers.hasKey timestampHeader) or
(not request.headers.hasKey slackSignatureHeader):
return false
let timestamp = request.headers[timestampHeader].parseInt
if not timestamp.isTimestampRecent():
return false
let baseString = signatureVersion & ':' & $timestamp & ':' & $request.body
let mySignature = sha256.hmac(signingSecret, baseString)
let slackSignature = MDigest[256].fromHex(request.headers[slackSignatureHeader])
mySignature == slackSignature
A few things I'm running into:
The signature doesn't match, and I'm not really sure how to debug that. I'm definitely getting a valid request from Slack and following the instructions for verification here: https://api.slack.com/authentication/verifying-requests-from-slack#about, but it's incorrect.
I know I'm missing the v0= in the comparison, but I'm not quite sure how to do that with the time independent comparison (whether I should be skipping that part or not in the comparison, etc)
My best guess at this point is that somehow the Jester/Httpbeast request body isn't "raw" enough (though it's just plain json...?) or is somehow processed.
Any help or suggestions on how to debug would be greatly appreciated. Thank you in advance!
After fussing with this for a while, I found I was doing a number of things incorrectly! Hopefully this helps others:
The signingSecret is pulled from the env, so it shouldn't be a constant--I moved that into the proc itself defined with let instead.
The slack signature in the headers is prefixed with v0=, which makes it the wrong length for MDigest[256].fromHex(), so that was ending up as the null value (0000...) instead of what should've been.
Here's a working version, now in case anyone else should need one. Please let me know if you see anything that could be improved as well.
import jester
import dotenv
import os, strutils, times
import nimcrypto
const timestampHeader = "X-Slack-Request-Timestamp"
const slackSignatureHeader = "X-Slack-Signature"
const signatureVersion = "v0"
proc isTimestampRecent(timestamp: int): bool =
abs(getTime().toUnix - timestamp) <= (60 * 5)
proc verifySignature*(request: Request): bool =
let signingSecret = os.getEnv("SLACK_SIGNING_SECRET")
if (not request.headers.hasKey timestampHeader) or
(not request.headers.hasKey slackSignatureHeader):
return false
let timestamp = request.headers[timestampHeader].parseInt
if not timestamp.isTimestampRecent():
return false
let baseString = signatureVersion & ':' & $timestamp & ':' & $request.body
let mySignature = sha256.hmac(signingSecret, baseString)
var rawSlackSignature: string = $request.headers[slackSignatureHeader]
rawSlackSignature.removePrefix(signatureVersion & '=')
let slackSignature = MDigest[256].fromHex(rawSlackSignature)
mySignature == slackSignature
Related
import asyncio
import ib_insync as ibi
import symbol_list
import time
start = time.perf_counter()
stocklist = symbol_list.test
endDateTime = '20190328 09:30:00'
durationStr='1 D'
dataDirectory = './data/tmp'
class App:
async def run(self):
self.ib = ibi.IB()
with await self.ib.connectAsync():
contracts = [
ibi.Stock(symbol, 'SMART', 'USD')
for symbol in ['AAPL', 'TSLA', 'AMD', 'INTC']]
for contract in contracts:
# self.ib.reqMktData(contract)
bars = await self.ib.reqHistoricalDataAsync(contract,
endDateTime=endDateTime, durationStr=durationStr,
barSizeSetting='5 mins', whatToShow='MIDPOINT', useRTH=True)
df = ibi.util.df(bars)
df.to_csv(f"{dataDirectory}/{contract.symbol}.csv")
async for tickers in self.ib.pendingTickersEvent:
for ticker in tickers:
print(ticker)
def stop(self):
self.ib.disconnect()
app = App()
try:
asyncio.run(app.run())
except (KeyboardInterrupt, SystemExit):
app.stop()
endtime = (time.perf_counter() - start)/60
print(f"Process time: {endtime:,.2f} minutes")
Please help. I modified the example code from async-streaming-example. I didn't get any error message, but it just runs without giving me the shell prompt. And this code should only take less than a minute, if it runs properly. Essentially, instead of reqMktData, I want to use reqHistoricalDataAsync to get historical data, asynchronously. I've also looked at async execution with ib_insync, but I wasn't able to get that technique to work, either. Could you show me what I'm doing wrong? I welcome any async solutions. Thank you.
I tried creating a method in Postman and got really close but am having issues with the signature. We are trying to query the IP ranges for VPCs to add to a WAF rule, in order to allow traffic to a secure application.
Postman allows a pre-request script, in Javascript, and supports a handful of included JS libraries, including CryptoJS.
The code here creates exactly the request that Ali Cloud says needs to be signed. It signs with HMAC-SHA1 from CryptoJS and encodes to base 64.
All of the variables are included in the request parameters. I'm not sure what else it could be complaining about.
var dateIso = new Date().toISOString();
var randomString = function(length) {
var text = "";
var possible = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789";
for(var i = 0; i < length; i++) {
text += possible.charAt(Math.floor(Math.random() * possible.length));
}
return text;
}
var accesskeyid = "LTAI4GC7VEijsm5bV3zwcZxZ"
var action = "DescribePublicIPAddress"
var format = "XML"
var regionid = "cn-shanghai-eu13-a01"
var signaturemethod = "HMAC-SHA1"
var signaturenonce = randomString(16)
var signatureversion = "1.0"
var timestamp = dateIso.replace(/:/gi, "%253A")
var version = "2016-04-28"
pm.environment.set("AccessKeyId", accesskeyid)
pm.environment.set("Action", action)
pm.environment.set("Format", format)
pm.environment.set("RegionID", regionid)
pm.environment.set("SignatureMethod", signaturemethod)
pm.environment.set("SignatureNonce", signaturenonce)
pm.environment.set("SignatureVersion", signatureversion)
pm.environment.set("Timestamp", dateIso)
pm.environment.set("Version", version)
var request = "GET&%2F&" + "AccessKeyID%3D" + accesskeyid + "%26Action%3D" + action + "%26Format%3D" + format + "%26RegionID%3D" + regionid + "%26SignatureMethod%3D" + signaturemethod + "%26SignatureNonce%3D" + signaturenonce + "%26SignatureVersion%3D" + signatureversion + "%26Timestamp%3D" + timestamp + "%26Version%3D" + version
pm.environment.set("Request", request)
var hash = CryptoJS.HmacSHA1(request, "spwH5dNeEm4t4dlpqvYWVGgf7aEAxB&")
var base64 = CryptoJS.enc.Base64.stringify(hash)
var encodesig = encodeURIComponent(base64)
pm.environment.set("Signature", encodesig);
console.log(base64)
console.log(request)
The console output shows:
Signature: XbVi12iApzZ0rRgJLBv0ytJJ0LY=
Parameter string to be signed:
GET&%2F&AccessKeyID%3DLTAI4GC7VEijsm5bV3zwcZvC%26Action%3DDescribePublicIPAddress%26Format%3DXML%26RegionID%3Dcn-shanghai-eu13-a01%26SignatureMethod%3DHMAC-SHA1%26SignatureNonce%3DiP1QJtbasQNSOxVY%26SignatureVersion%3D1.0%26Timestamp%3D2020-06-01T15%253A38%253A12.266Z%26Version%3D2016-04-28
Request sent:
GET https://vpc.aliyuncs.com/?AccessKeyID=LTAI4GC7VEijsm5bV3zwcZvC&Action=DescribePublicIPAddress&Format=XML&RegionID=cn-shanghai-eu13-a01&SignatureMethod=HMAC-SHA1&SignatureNonce=iP1QJtbasQNSOxVY&SignatureVersion=1.0&Timestamp=2020-06-01T15:38:12.266Z&Version=2016-04-28&Signature=XbVi12iApzZ0rRgJLBv0ytJJ0LY%3D
Response Received:
<?xml version='1.0' encoding='UTF-8'?><Error><RequestId>B16D216F-56ED-4D16-9CEC-633C303F2B61</RequestId><HostId>vpc.aliyuncs.com</HostId><Code>IncompleteSignature</Code><Message>The request signature does not conform to Aliyun standards. server string to sign is:GET&%2F&AccessKeyID%3DLTAI4GC7VEijsm5bV3zwcZvC%26Action%3DDescribePublicIPAddress%26Format%3DXML%26RegionID%3Dcn-shanghai-eu13-a01%26SignatureMethod%3DHMAC-SHA1%26SignatureNonce%3DiP1QJtbasQNSOxVY%26SignatureVersion%3D1.0%26Timestamp%3D2020-06-01T15%253A38%253A12.266Z%26Version%3D2016-04-28</Message><Recommend><![CDATA[https://error-center.aliyun.com/status/search?Keyword=IncompleteSignature&source=PopGw]]></Recommend></Error>
When I check the "server string to sign" from the response and the parameter string that was signed in a compare, they are identical.
It looks like everything is built as needed but the signature is still barking. Guessing I missed something simple but haven't found it yet.
Note: The accesskeyID and key posted are for example purposes and not a real account so this code will not copy and paste to execute in Postman.
PS - I learned quite a bit from the other few threads on this topic, which is how I got to this point. akostadinov was super helpful on another thread.
I believe you have double encoded &. I have implemented other Alibaba Cloud REST APIs successfully. Could you please check this.
Following is the expected string to sign format:
GET&%2F&AccessKeyId%3Dtestid&Action%3DDescribeVpcs&Format%3DXML&
SignatureMethod%3DHMAC-SHA1&SignatureNonce%3D3ee8c1b8-83d3-44af-a94f-4e0ad82fd6cf&SignatureVersion%3D1.0&TimeStamp%3D2016-02-23T12%253A46%
253A24Z&Version%3D2014-05-15
A bit late to the party, but as this is the first result when googling for the IncompleteSignature error, I thought I might comment and hopefully save someone else the grief I have been through.
For me, the subtle detail that I missed in the official documentation here is that the key used for the signature requires an ampersand & to be added to the end, before being used.
As soon as I caught that, everything else worked perfectly.
macOS, swift3
Apple has an API Foundation > FileManager > setAttributes(_:ofItemAtPath:)
The declaration is
func setAttributes(_ attributes: [FileAttributeKey : Any], ofItemAtPath path: String) throws
It is for setting the creation date etc for a file.
I can handle the >ofItem path:String) throws< but the first part has me stumped.
The API says it can return 'true' but swift returns void. There is an attribute named 'creationDate'. What is the significance of the underscore '_'.
I think 'attributes' is a mutable dictionary
var myAttributesDictionary = [FileAttributeKey : Date]()
myAttributesDictionary[FileAttributeKey.creationDate] = myDateObject
let fm = FileManger()
let xxx = fm.setAttributes(myAttributesDictionary:[FileAttributeKey : creationDate], ofItemAtPath myPath)
I have tried many variations and now I am stumped and I do not know what is required. I cannot get the setAttributes line to compile
I spent some time looking for an answer before posting the question.
When I went looking for my questionI found an answer.
let mypath = "/path/to/file"
let myDateObject = NSDate() // NSDate() is todays date
let attributes = [FileAttributeKey.creationDate: myDateObject]
do {
try FileManager.default.setAttributes(attributes, ofItemAtPath: myPath)
}
catch
{
print(error)
}
I understand the error, yet do no understand it in the context of my code. This is in Python 3.4. The relevant bits of code (simplified somewhat for clarity):
from requests.adapters import HTTPAdapter
from urllib3.poolmanager import PoolManager
class SessionAdapter(HTTPAdapter):
def init_poolmanager(self, connections, maxsize, block=False):
self.poolmanager = PoolManager(num_pools=connections,
maxsize=maxsize,
block=block,
ssl_version=ssl.PROTOCOL_TLSv1,
cert_reqs = 'CERT_REQUIRED',
ca_certs = certifi.where(),
)
try:
app_session = requests.Session()
app_session.mount('https://', SessionAdapter())
app_response = app_session.post(
url = 'https://<FQD URL>'
, auth = (user, password)
, verify = True
)
# Code errors on the previous line and never executes the logger line
logger.debug('Status code: {}'.format(app_response.status_code))
if app_response.status_code == 401:
return 401
else:
return app_session
except:
logger.debug('Exception')
From sys.exc_info() I see:
", verify = True"): unorderable types: Retry() < int()
If the error were something like SessionAdapter() < int() it might make more sense. But I don't know where the Retry() check is being made.
Does the import of PoolManager need to be done differently for Python 3? I'm using version 1.7.1 of python-urllib3 on Ubuntu.
I've created an application, which is used to loop through the emails in an inbox and find all the undeliverable, mailbox full or delayed emails and generate a report.
The usual routine is to loop through all the emails in the inbox (up to a specified date).
If an email is undeliverable use regex to find the email. This works 95% of the time as this information is contained in the body of the Undelivered message (ReportItem).
So, my problem is I have a few emails which are returning blank emails to the report making it nigh on impossible to clean them or easily report that we have a problem with someone's email.
I have found that the information in the Internet Headers has who the mail was intended for, but cannot find anything on if it is possible to use an interop or some other object to obtain this information.
If anyone else has come across this problem and knows of a work around I would be very grateful.
Cheers
I was looking to automate an outlook mail box to move all undelivered emails and store the email address of the recipient of the undeliverable message in a list, so that I can later check if an entry of the list is present in an excel column and then remove it from the excel. I hope this helps !
I've found a Python solution for this problem. A python library that is used to connect to the outlook is win32com, so first we import all libraries that we will need:
import win32com.client
import re
import datetime as dt
from tqdm import tqdm
import time
import extract_msg
This is a good way to connect to a specific outlook account, if you have :
outlook = win32com.client.Dispatch("Outlook.Application").GetNamespace("MAPI")
accounts= win32com.client.Dispatch("Outlook.Application").Session.Accounts
Then create a loop that iterates through the whole outlook and gets to the specified mail account:
for account in accounts:
inbox = outlook.Folders(account.DeliveryStore.DisplayName)
if account.DeliveryStore.DisplayName == 'place_your_account_name_here':
for folder in inbox.Folders:
Find the folder in outlook you wish to check by folder name,
so if you would want to iterate through Inbox, type "Inbox" instead of "Folder_name"
if folder.__str__() == "Folder_name":
messages = folder.Items
messages.Sort('[ReceivedTime]', True)
if folder.Folders.Item('Undeliverable'):
undeliverable = folder.Folders.Item('Undeliverable')
list_of_undelivered_email_addresses = my_super_function(messages,undeliverable)
After we have reached the mail items and declared the undeliverable subfolder as "undeliverable", we specify the time period for which we want to do the below function:
def my_super_function(messages,undeliverable):
list_of_undelivered_email_addresses = []
last_n_days = dt.datetime.now() - dt.timedelta(days = 25)
messages = messages.Restrict("[ReceivedTime] >= '" +last_n_days.strftime('%m/%d/%Y %H:%M %p')+"'")
rl= list()
I have found that the msot popular times of undeliverable email addresses present some sort of an error, and below the error is the original version of the email I have sent. Most of them (with very few exceptions, have a line that says:
To: "Some_email_address" ....
This is why I used this regular expression to get read the whole line after my pattern (which is "To: "")
pattern = re.compile('To: ".*\n?',re.MULTILINE)
for counter, message in enumerate(messages):
It is very important that you save the email somewhere on your PC, because otherwise as soon as you read it's body, the email gets encrypted.
message.SaveAs("undeliverable_emails.msg")
f = r'specify_the_absolute_path_where_you_want_it_saved'
try:
msg = extract_msg.Message(f)
print(counter)
Search the saved msg body for the keyword Undeliverable:
if msg.body.find("undeliverable")!= -1 or msg.body.find("Undeliverable")!= -1 or msg.subject.find("Undeliverable")!= -1 or msg.subject.find("undeliverable")!= -1 or msg.body.find("wasn't found at")!= -1:
Save the actual email to a list, so you can move it to the undeliverables subfolder later
rl.append(message)
m = re.search(pattern, msg.body)
m = m[0]
mail_final = m.split('"')[1]
list_of_undelivered_email_addresses.append(mail_final)
list_of_undelivered_email_addresses=list(filter(None, list_of_undelivered_email_addresses))
else:
print('this email is not an undeliverable one')
except:
pass
Move all mails in the list to the undeliverables folder:
if len(rl) ==0:
pass
else:
for m in tqdm(rl):
m.Move(undeliverable)
return list_of_undelivered_email_addresses
Here is the full code:
import win32com.client
import re
import datetime as dt
from tqdm import tqdm #tqdm gives you the progress bar
import time
import extract_msg
outlook = win32com.client.Dispatch("Outlook.Application").GetNamespace("MAPI")
accounts= win32com.client.Dispatch("Outlook.Application").Session.Accounts
def my_super_function(messages,undeliverable):
list_of_undelivered_email_addresses = []
last_n_days = dt.datetime.now() - dt.timedelta(days = 25)
messages = messages.Restrict("[ReceivedTime] >= '" +last_n_days.strftime('%m/%d/%Y %H:%M %p')+"'")
rl= list()
pattern = re.compile('To: ".*\n?',re.MULTILINE)
for counter, message in enumerate(messages):
message.SaveAs("undeliverable_emails.msg")
f = r'some_absolute_path'
try:
msg = extract_msg.Message(f)
print(counter)
if msg.body.find("undeliverable")!= -1 or msg.body.find("Undeliverable")!= -1 or msg.subject.find("Undeliverable")!= -1 or msg.subject.find("undeliverable")!= -1 or msg.body.find("wasn't found at")!= -1:
rl.append(message)
m = re.search(pattern, msg.body)
m = m[0]
mail_final = m.split('"')[1]
list_of_undelivered_email_addresses.append(mail_final)
list_of_undelivered_email_addresses=list(filter(None, list_of_undelivered_email_addresses))
else:
print('else')
except:
pass
if len(rl) ==0:
pass
else:
for m in tqdm(rl):
m.Move(undeliverable)
return list_of_undelivered_email_addresses
for account in accounts:
inbox = outlook.Folders(account.DeliveryStore.DisplayName)
if account.DeliveryStore.DisplayName == 'desired_email_address':
for folder in inbox.Folders:
if folder.__str__() == "Inbox":
messages = folder.Items
messages.Sort('[ReceivedTime]', True)
if folder.Folders.Item('Undeliverable'):
undeliverable = folder.Folders.Item('Undeliverable')
list_of_undelivered_email_addresses = my_super_function(messages,undeliverable)
looks like what I want isnt part of the ReportItem properties.
The possible options are Extended IMAPI, CDO or Redemption
http://www.tech-archive.net/Archive/Outlook/microsoft.public.outlook.program_vba/2004-11/0084.html