I'm trying to setup a .py plugin that will save decoded Protobuf responses to file, but whatever I do, the result is always file in byte format (not decoded). I have also tried to do the same by using "w" in Mitmproxy - although on screen I saw decoded data, in the file it was encoded again.
Any thoughts how to do it correctly?
Sample code for now:
import mitmproxy
def response(flow):
# if flow.request.pretty_url.endswith("some-url.com/endpoint"):
if flow.request.pretty_url.endswith("some-url.com/endpoint"):
f = open("test.log","ab")
with decoded(flow.response)
f.write(flow.request.content)
f.write(flow.response.content)
Eh, I'm not sure this helps, but what happens if you don't open the file in binary mode
f = open("test.log","a")
?
Hy,
some basic things that I found.
Try replacing
f.write(flow.request.content)
with
f.write(flow.request.text)
I read it on this website
https://discourse.mitmproxy.org/t/modifying-https-response-body-not-working/645/3
Please read and try this to get the requests and responses assembled.
MITM Proxy, getting entire request and response string
Best of luck with your project.
I was able to find the way to do that. Seems mitmdump or mitmproxy wasn't able to save raw decoded Protobuf, so I used:
mitmdump -s decode_script.py
with the following script to save the decoded data to a file:
import mitmproxy
import subprocess
import time
def response(flow):
if flow.request.pretty_url.endswith("HERE/IS/SOME/API/PATH"):
protobuffedResponse=flow.response.content
(out, err) = subprocess.Popen(['protoc', '--decode_raw'], stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE).communicate(protobuffedResponse)
outStr = str(out, 'utf-8')
outStr = outStr.replace('\\"', '"')
timestr = time.strftime("%Y%m%d-%H%M%S")
with open("decoded_messages/" + timestr + ".decode_raw.log","w") as f:
f.write(outStr)
Related
I have been working on a project for a while now, and I just reached another big step! However, for some .txt files that my program creates, it will give me this message:
File was loaded in the wrong encoding: 'UTF-8'
Most of the .txt files are fine, but it gives me this error for others at the top (I can still read them). Here is my code:
from socket import *
import codecs
import subprocess
ipa = '192.168.1.' # These are the first 3 digits of the IP addresses that the program looks for.
def is_up(adr):
s = socket(AF_INET, SOCK_STREAM)
s.settimeout(0.01)
if not s.connect_ex((adr, 135)):
s.close()
return 1
else:
s.close()
def main():
for i in range(1, 256):
adr = ipa + str(i)
if is_up(adr):
with codecs.open("" + getfqdn(adr) + ".txt", "w+", 'utf-8-sig') as f:
subprocess.run('ipconfig | findstr /i "ipv4"', stdout=f, shell=True, check=True)
subprocess.run('wmic/node:'+adr+' product get name, version, vendor', stdout=f, shell=True, check=True)
main()
# Most code provided by Ashish Jain
Unfortunately I don't think I'm allowed to say exactly which files are giving me trouble, because I might be distributing information that someone can use for malicious intent.
Since your script only writes to files, there's no reason to open it in w+ mode, which enables reading. Opening the files in w mode should be enough.
Furthermore, the commands that your script runs must not be outputting in utf-8-sig-encoded text, and hence the error. In most cases outputting with default encoding by not specifying an encoding will suffice.
Lastly, you're missing a space between wmic and /node: in the second command you run.
I have been trying to read a jpeg file using PIL in python 3.4. I need to save this file into string format. Although some options are provided on this site but I have tried a few but it is not working. Following is my code snippet which i have found on this site only:-
from io import StringIO
fp = Image.open("images/login.jpg")
output = StringIO()
fp.save(output, format="JPEG")
contents = output.getvalue()
output.close()
But i am facing the following error :-
TypeError: string argument expected, got 'bytes'
Could you please suggest what I have done wrong and how to get this working?
In python 3 you should use a BytesIO,
whereas as read in python docs:
StringIO is a native in-memory unicode container
.
Thanks a lot for the hint. I Actually have a found a different way of reading the image file and storing in string object in python2.x . Here is the code. Please let me know if there is any disadvantage of using this.
imgText = open("images/login.jpg", 'rb')
imgTextStr = imgText.read()
imgText.close()
I want to embed a small python script inside a bash script so that I can send a json object to a socket.
I have tried the following code:
python -c "$(cat << 'EOF'
import socket
import json
data = {'ip':192.168.1.150', 'status':'up'}
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(('127.0.0.1', 13373))
s.send(json.dumps(data))
result = json.loads(s.recv(1024))
print result
s.close()
EOF
)"
and this:
python -c "
import socket
import json
data = {'ip':192.168.1.150', 'status':'up'}
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(('127.0.0.1', 13373))
s.send(json.dumps(data))
result = json.loads(s.recv(1024))
print result
s.close()
"
But I keep getting the following error:
data = {'ip':192.168.1.150', 'status':'up'}
^
SyntaxError: invalid syntax
I'm assuming this is because bash is interpreting this, not python. I've tested the code in a python script, and it works.
Also, do I need the -c option?
Apologies, I'm completely inexperienced in python, I've written some quite extensive bash scripting for the project I'm working on and need to send the output of these to sockets as json objects. Such a small snipped of embedded Python code seems by far the simplest answer, unless anyone has other suggestions.
Python version installed on CentOS Python 2.6.6
The problem that you're having that results in the SyntaxError is that you don't have an opening single quote on the IP value in the data dict. You have:
data = {'ip':192.168.1.150', 'status':'up'}
You need:
data = {'ip':'192.168.1.150', 'status':'up'}
I am working on porting a project in Windows over to OSX. I have overcome issues with VBA for OSX Word 2011 not allowing you to send POSTs to a server and have figured out how to return a string result from an external script. Now I have to insert an image in my Word file from a URL that is built using the return of my external script.
The current attempt is as follows, and works in Windows but crashes Word in OSX:
Selection.InlineShapes.AddPicture FileName:=File_Name, _
LinkToFile:=False, SaveWithDocument:=True
After doing some research, it looks like MS may have disabled this functionality in OSX as a "security risk". I still need to make it work. Does anybody know of a way within VBA for Office 2011 to make this work, or barring that a workaround? I am trying to avoid writing the image file to the disk if possible.
UPDATE: I have created a Python script for getting the image file from a URL, but I still do not know how to get this image from the Python script into VBA, and from there into the Word document at the location of the cursor. The important bits of the script are below. The image is read in as a PIL object and I can show it using img.show() just fine, but I am not sure what filetype this is or how to get VBA to accept it.
# Import the required libraries
from urllib2 import urlopen, URLError
from cStringIO import StringIO
from PIL import Image
# Send request to the server and receive response, with error handling!
try:
# Read the response and print to a file
result = StringIO(urlopen(args.webAddr + args.filename).read())
img = Image.open(result)
img.show()
except URLError, e:
if hasattr(e, 'reason'): # URL error case
# a tuple containing error code and text error message
print 'Error: Failed to reach a server.'
print 'Reason: ', e.reason
elif hasattr(e, 'code'): # HTTP error case
# HTTP error code, see section 10 of RFC 2616 for details
print 'Error: The server could not fulfill the request.'
print 'Error code: ', e.code
Note that in the above, args.webAddr and args.filename are passed to the script using the argparse library. This script works, and will show the image file that I expect. Any ideas on how to get that image into Word 2011 for OSX and insert it under the cursor?
Thanks a lot!
Edit: updated the link to the project since migrating to github.
Old question, but no answer, and I see the same crash here when the image is at an http URL. I think you can use the following workaround
Sub insertIncludePictureAndUnlink()
' Put your URL in here...
Const theImageURL As String = ""
Dim f As Word.Field
Dim r As Word.Range
Set f = Selection.Fields.Add(Range:=Selection.Range, Type:=wdFieldIncludePicture, Text:=Chr(34) & theImageURL & Chr(34), PreserveFormatting:=False)
Set r = f.Result
f.Unlink
Set f = Nothing
' should have an inlineshape in r
Debug.Print r.InlineShapes.Count
' so now you can do whatever you need, e.g....
r.Copy
Set r = Nothing
End Sub
I'm porting a large VBA project over from Windows to the new Mac Word 2011. It's actually going very well...almost all of the code is working.
My code needs to call scripts on my server. On Windows, I call the system function InternetOpenUrl to call a script and InternetReadFile to read the results returned by the script. For example, I call a script like:
"http://www.mysite.com/cgi-bin/myscript.pl?param1=Hello¶m2=World
and it returns a string like "Success"
What's the best way to do the equivalent on the Mac? Is using Applescript (via the vba MacScript function) the answer? I do that to display the file chooser dialog, but I can't find what the applescript to call an online script would look like. Or is there a better/faster way to do this?
Thanks in advance,
gary
You can try the URL Access Scripting library, which is a front-end for curl, or go to the script via a browser and reading the text through there.
I recently figured this out for making a call to a server to convert a user-defined LaTeX string to an image of the equation. The call is made through VBA via the MacScript command as:
command = "do shell script """ & pyPath & "python " & getURLpath & "getURL.py --formula '" _
& Latex_Str & "' --fontsize " & Font_Size & " " & WebAdd & """"
result = MacScript(command)
Which looks ugly, but this is just building the command do shell script /usr/bin/python {path to script}/getURL.py --formula '{LaTeX formula string}' --fontsize {int} {myurl} and passing it to the command. My Python script then uses argparse to parse the arguments sent to it, and urllib and urllib2 to handle sending the request to the server. The MacScript command read the stdout of my Python script and returns it as a string to result.
This guide on urllib2 should help you get the Python script up and running.
EDIT: Sorry, my answer was incomplete last time. The Python script I used to finish the job is below.
# Import the required libraries
from urllib import urlencode
from urllib2 import Request, urlopen, URLError, ProxyHandler, build_opener, install_opener
import argparse
# Set up our argument parser
parser = argparse.ArgumentParser(description='Sends LaTeX string to web server and returns meta data used by LaTeX in Word project')
parser.add_argument('webAddr', type=str, help='Web address of LaTeX in Word server')
parser.add_argument('--formula', metavar='FRML', type=str, help='A LaTeX formula string')
parser.add_argument('--fontsize', metavar='SIZE', type=int, default=10, help='Integer representing font size (can be 10, 11, or 12. Default 10)')
parser.add_argument('--proxServ', metavar='SERV', type=str, help='Web address of proxy server, i.e. http://proxy.server.com:80')
parser.add_argument('--proxType', metavar='TYPE', type=str, default='http', help='Type of proxy server, i.e. http')
# Get the arguments from the parser
args = parser.parse_args()
# Define formula string if input
if args.formula:
values = {'formula': str(args.fontsize) + '.' + args.formula} # generate formula from args
else:
values = {}
# Define proxy settings if proxy server is input.
if args.proxServ: # set up the proxy server support
proxySupport = ProxyHandler({args.proxType: args.proxServ})
opener = build_opener(proxySupport)
install_opener(opener)
# Set up the data object
data = urlencode(values)
data = data.encode('utf-8')
# Send request to the server and receive response, with error handling!
try:
req = Request(args.webAddr, data)
# Read the response and print to a file
response = urlopen(req)
print response.read()
except URLError, e:
if hasattr(e, 'reason'): # URL error case
# a tuple containing error code and text error message
print 'Error: Failed to reach a server.'
print 'Reason: ', e.reason
elif hasattr(e, 'code'): # HTTP error case
# HTTP error code, see section 10 of RFC 2616 for details
print 'Error: The server could not fulfill the request.'
print 'Error code: ', e.code