How to make a pdf signature visible (along with validity icons) - ruby

Initially the question was about making a pdf signature visible with Ruby. That i can do now with the code below.
My only problem now is with the validity icon. It shows up on FoxIt Reader, but not on Acrobat Reader (i'm using XI right now)
This is what I have right now
require 'openssl'
begin
require 'origami'
rescue LoadError
ORIGAMIDIR = "C:\RailsInstaller\Ruby1.9.3\lib\ruby\gems\1.9.1\gems\origami-1.2.4\lib"
$: << ORIGAMIDIR
require 'origami'
end
include Origami
def draw_signature(name, attr = {})
load! if #instructions.nil?
x, y = attr[:x], attr[:y]
#instructions << PDF::Instruction.new('q')
#instructions << PDF::Instruction.new('cm', 1, 0, 0, 1, x, y)
#instructions << PDF::Instruction.new('Do', name)
#instructions << PDF::Instruction.new('Q')
end
def SignPdf(inputF)
#inputfile = String.new(inputF)
outputF = #inputfile.insert(inputF.rindex("."),"_signed8")
certFile = "cert.pem"
rsakeyFile = "pk.pem"
passphrase = "mypass"
key4pem=File.read rsakeyFile
key = OpenSSL::PKey::RSA.new key4pem, passphrase
cert = OpenSSL::X509::Certificate.new(File.read certFile)
pdf = PDF.read(inputF)
page = pdf.get_page(1)
#signature image
imageobject = Origami::Graphics::ImageXObject.from_image_file('c:\rails_projects\RecibosOnline\app\assets\images\logo.jpg', 'jpg')
imageobject.Width = 200
imageobject.Height = 141
#formobject = Origami::Graphics::FormXObject.new
#formobject.write("Teste")
width = 300
height=141
x=201
y=186
signedby = "My Company"
location = "Portugal"#pdf.signature[pdf.signature.keys[5]]
contact = "mail#mail.com" #pdf.signature[pdf.signature.keys[6]]
reason = "Testing" #pdf.signature[pdf.signature.keys[7]]
date = Time.now
caption="Digitally Signed By: #{signedby}\nContact: #{contact}\nLocation: #{location}\nReason: #{reason}\nDate: #{date} "
n0 = Annotation::AppearanceStream.new
n0.Type=Origami::Name.new("XObject")
n0.BBox = [ 0, 0, 100, 100 ]
n0.Matrix = [ 1, 0, 0, 1, 0, 0 ]
n0.set_indirect(true)
n0.Resources = Resources.new
n0.Resources.ProcSet = [Origami::Name.new("PDF"),Origami::Name.new("Text"),Origami::Name.new("Text"),Origami::Name.new("ImageB"),Origami::Name.new("ImageC"),Origami::Name.new("ImageI")]
n0.draw_stream('% DSBlank')
#n0.write(Origami::Name.new("% DSBlank"))
n2 = Annotation::AppearanceStream.new
n2.Type = Origami::Name.new("XObject")
#n2.Subtype = Origami::Name.new("Form")
n2.Resources = Resources.new
n2.Resources.ProcSet = [Origami::Name.new("Text")]
n2.set_indirect(true)
n2.Matrix = [ 1, 0, 0, 1, 0, 0 ]
n2.BBox = [ 0, 0, width, height ]
n2.write(caption,:x => 40, :y => height-35, :size => 15)
#n2.draw_stream("q 0 0 198 48 re W n BT /Helv 7.645 Tf 0 g 0 40.135 Td (Digitally ) Tj 28.889 0 Td (signed ) Tj 24.652 0 Td (by ) Tj10.198 0 Td (James ) Tj 24.641 0 Td (Pravetz ) Tj-88.38 -9.174 Td (DN: ) Tj 15.29 0 Td (cn=James ) Tj37.18 0 Td (Pravetz, ) Tj 30.165 0 Td (c=CA, ) Tj 23.156 0 Td0 -9.174 Td (Reason: ) Tj 30.599 0 Td (I ) Tj 4.248 0 Td(have ) Tj 18.702 0 Td (reviewed ) Tj 32.719 0 Td(this ) Tj 14.02 0 Td (document ) Tj -100.288 -9.174 Td(Date: ) Tj 20.397 0 Td (2006.06.08 ) Tj 40.386 0 Td(13:39:35 ) Tj 31.883 0 Td (-07'00') Tj ET Q")
#Sets the root dictionary element
frm = Annotation::AppearanceStream.new
frm.set_indirect(true)
#frm.Type = Origami::Name.new("XObject")
#frm.Subtype = Origami::Name.new("Form")
frm.Resources = Resources.new
frm.Resources.ProcSet = [Origami::Name.new("PDF")]
frm.Resources.add_xobject(Origami::Name.new("n0"), n0)
frm.Resources.add_xobject(Origami::Name.new("n2"), n2)
frm.Matrix = [ 1, 0, 0, 1, 0, 0 ]
frm.BBox = [ 0, 0, width, height ]
frm.draw_stream('q 1 0 0 1 0 0 cm /n0 Do Q')
frm.draw_stream('q 1 0 0 1 0 0 cm /n2 Do Q')
xo17 = Annotation::Widget::Signature.new
#xo17 = Annotation::Widget::Signature.new
xo17.Rect = Rectangle[ llx: x, lly: y, urx: x+width, ury: y+height ]
#xo17.Resources = Resources.new
#xo17.Resources.add_xobject(Origami::Name.new("FRM"), frm)
xo17.F = Annotation::Flags::PRINT #sets the print mode on
xo17.H = Annotation::Widget::Highlight::INVERT
streamN = Annotation::AppearanceStream.new #.setFilter(:FlateDecode)
streamN.set_indirect(true)
streamN.BBox = [ 0, 0, width, height ]
streamN.Resources = Resources.new
streamN.Resources.ProcSet = [Origami::Name.new("PDF")]
streamN.Resources.add_xobject(Origami::Name.new("FRM"), frm)
streamN.Subtype = nil
#cs = ContentStream.new('q 1 0 0 1 0 0 cm /FRM Do Q',streamN)
streamN.draw_stream('q 1 0 0 1 0 0 cm /FRM Do Q')
#streamN.draw_image(Origami::Name.new("FRM"),{x:x,y:y})
#streamN.write(caption,:x => 40, :y => height-35, :size => 15)
xo17.set_normal_appearance(streamN)
# page.add_xobject(Origami::Name.new("FRM"),frm)
# page.add_xobject(Origami::Name.new("n0"),n0)
# page.add_xobject(Origami::Name.new("n2"),n2)
page.add_annot(xo17)
# Sign the PDF with the specified keys
pdf.sign(cert, key,
:method => 'adbe.pkcs7.sha1',
:annotation => xo17,
:location => location,
:contact => contact,
:reason => reason
)
# Save the resulting file
pdf.save(outputF)
end
SignPdf("Sample.pdf")
EDIT 2
Following mkl intervention, i went a little further into the source...
I noticed the xobject.rb file, with the following code for drawing an image
def draw_image(name, attr = {})
load! if #instructions.nil?
x, y = attr[:x], attr[:y]
#instructions << PDF::Instruction.new('q')
#instructions << PDF::Instruction.new('cm', 300, 0, 0, 300, x, y)
#instructions << PDF::Instruction.new('Do', name)
#instructions << PDF::Instruction.new('Q')
end
I have then created the method below on that same file
def draw_stream(name)
load! if #instructions.nil?
#instructions << PDF::Instruction.new(name)
end
which allows me to control the stream from its input.
I also went to ISO32000:2008, and iterated a bit thorugh my code, but got nowhere further.
Here is my currently generated pdf file , and the part of it that relates to signatures:
16 0 obj
<<
/Rect [ 201 186 501 327 ]
/F 4
/H /I
/AP <<
/N 18 0 R
>>
/P 1 0 R
/V 17 0 R
/T (undef28504)
/Subtype /Widget
/FT /Sig
/DA (/F1 10 Tf 0 g)
>>
endobj
17 0 obj
<<
/Type /Sig
/SubFilter /adbe.pkcs7.sha1
/Reason <54657374696E67>
/Prop_Build <<
/Filter <<
/Name /Adobe.PPKMS
/R 131101
/Date (2013-04-03 14:18:41 +0100)
>>
/SigQ <<
/Preview false
/R 131101
>>
/PubSec <<
/NonEFontNoWarn false
/Date (2013-04-03 14:18:41 +0100)
/R 131101
>>
/App <<
/TrustedMode false
/OS [ /Win ]
/R 458752
/Name /Exchange-Pro
>>
>>
/M (D:20130403131841Z00'00)
/Location <506F72747567616C>
/Filter /Adobe.PPKMS
/Contents <308207AE06092A864886F70D010702A082079F3082079B020101310B300906052B0E03021A0500302306092A864886F70D010701A0160414BBA1B7480DDB1B75B136B4DFA2A4C120EC7972DFA0820597308205933082047BA0030201020204426F938C300D06092A864886F70D0101050500303E310B300906035504061302707431153013060355040A130C4D554C5449434552542D4341311830160603550403130F4D554C5449434552542D4341203032301E170D3133303332303137303134375A170D3134303332303136343733365A3081A5310B300906035504061302505431153013060355040A130C4D554C5449434552542D434131163014060355040B130D4345525449504F52202D20524131123010060355040B1309436F72706F726174653120301E060355040B13174553435249544120494E54454C4947454E5445204C444131183016060355040B130F576562204170706C69636174696F6E311730150603550403130E52454349424F53204F4E4C494E4530819F300D06092A864886F70D010101050003818D0030818902818100ACCEA4069031B578896C450D77C83C13DD75C668BF2762368DB0A0B1591B1800E58C1ACDBB25DA15501AF931F9F6BAE0F8F64C481686E959DE005AEC82B33D0D32665037D18BF36BF77C4D56620C42FBBA92D3FF397F9D8377FEABBA93478FCEF009216483D346ACCC76CA10C9B8653B80B8F6EB49BD93897A43DF062D0D9ED30203010001A38202B3308202AF300B0603551D0F0404030203F8303806082B06010505070101042C302A302806082B06010505073001861C687474703A2F2F6F6373702E6D756C7469636572742E636F6D2F63613081E00603551D200481D83081D5304D06092B06010401B03C0A023040303E06082B060105050702011632687474703A2F2F7777772E6D756C7469636572742E636F6D2F6370732F6D756C7469636572742D63612D6370732E68746D6C308183060B2B06010401B03C0A0288063074307206082B0601050507020230661E640068007400740070003A002F002F007700770077002E006D0075006C007400690063006500720074002E0063006F006D002F00630070002F006D0075006C007400690063006500720074002D00630061002D0031003000330030002E00680074006D006C301106096086480186F84201010404030204B030200603551D11041930178115696E666F4072656369626F736F6E6C696E652E7074308201010603551D1F0481F93081F630819AA08197A08194862F687474703A2F2F7777772E6D756C7469636572742E636F6D2F63612F6D756C7469636572742D63612D30322E63726C86616C6461703A2F2F6C6461702E6D756C7469636572742E636F6D2F636E3D4D554C5449434552542D434125323030322C6F3D4D554C5449434552542D43412C633D50543F63657274696669636174655265766F636174696F6E4C6973743F626173653057A055A053A451304F310B300906035504061302707431153013060355040A130C4D554C5449434552542D4341311830160603550403130F4D554C5449434552542D4341203032310F300D0603550403130643524C323935301F0603551D230418301680141DC3B988A518BE60A72CA663CA662AFC0C27C1BD301D0603551D0E0416041406D81F7236619EEB17369C299E2D74FFD038301930090603551D1304023000300D06092A864886F70D0101050500038201010041511FCD5C7561981B0D54326B57F7B87C435AACB7A2962808768313892AB1407237E957577B875414DE810A41323F9E0A768E9AC45C0F66AE093C32C11453C6463F856FEF62E27821134DD09F75208000040E89A814E6309623C5D04163C03CFDE33153908AC3DFCA5B1C66C3DCB89644A3030FE794D50BD255D396535A7AF267C3A538147B9371D05F23D8CA481AEBC7D7A7D97C2E7FB5AB49C4E4554E4800648BC76B1AF5612A1D9361D172084E70690928A911FC209833C506219CF186B6507B599E4C0F42F3BF23C2B8F0A078D01D9BF5FD4746D97253EE4FE8F472489B3DC26F7072C6587218825BB3061074B9C240F8928D36FEFC0F708897752CD946318201C7308201C30201013046303E310B300906035504061302707431153013060355040A130C4D554C5449434552542D4341311830160603550403130F4D554C5449434552542D43412030320204426F938C300906052B0E03021A0500A081D8301806092A864886F70D010903310B06092A864886F70D010701301C06092A864886F70D010905310F170D3133303430333133313834315A302306092A864886F70D010904311604142948A25663C43F9571225B0F12B9097DFE7149DB307906092A864886F70D01090F316C306A300B060960864801650304012A300B0609608648016503040116300B0609608648016503040102300A06082A864886F70D0307300E06082A864886F70D030202020080300D06082A864886F70D0302020140300706052B0E030207300D06082A864886F70D0302020128300D06092A864886F70D010101050004818021F8A3A104E8B6EC1D9026CA9A822C4357EEBAABD5D87422BE6199802F45E83E9D2417711CE26DCB28A8246ACE119B128956394A8FA641346E5F3CBBDD84ED7E36B2F947DD5D92EC1C2C078A1630A729773F17027FC3B64C18844C26F7FA18A7B4FC9DE334741235E178280ED25CD8840A83A91FCE240646EB84117E348BF82E0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000>
/ContactInfo <6D61696C406D61696C2E636F6D>
/ByteRange [ 0 12880 17078 2366 ]
>>
endobj
18 0 obj
<<
/BBox [ 0 0 300 141 ]
/Resources <<
/ProcSet [ /PDF ]
/XObject <<
/FRM 19 0 R
>>
/Font <<
/F1 <<
/Type /Font
/Subtype /Type1
/BaseFont /Helvetica
/Name /F1
>>
>>
>>
/Length 27
/Subtype /Form
>>stream
q 1 0 0 1 0 0 cm /FRM Do Q
endstream
endobj
19 0 obj
<<
/Resources <<
/ProcSet [ /PDF ]
/XObject <<
/n0 20 0 R
/n2 21 0 R
>>
/Font <<
/F1 <<
/Type /Font
/Subtype /Type1
/BaseFont /Helvetica
/Name /F1
>>
>>
>>
/Matrix [ 1 0 0 1 0 0 ]
/BBox [ 0 0 300 141 ]
/Length 52
/Subtype /Form
>>stream
q 1 0 0 1 0 0 cm /n0 Do Q
q 1 0 0 1 0 0 cm /n2 Do Q
endstream
endobj
20 0 obj
<<
/Type /XObject
/BBox [ 0 0 100 100 ]
/Matrix [ 1 0 0 1 0 0 ]
/Resources <<
/ProcSet [ /PDF /Text /Text /ImageB /ImageC /ImageI ]
/Font <<
/F1 <<
/Type /Font
/Subtype /Type1
/BaseFont /Helvetica
/Name /F1
>>
>>
>>
/Length 10
/Subtype /Form
>>stream
% DSBlank
endstream
endobj
21 0 obj
<<
/Type /XObject
/Resources <<
/ProcSet [ /Text ]
/Font <<
/F1 <<
/Type /Font
/Subtype /Type1
/BaseFont /Helvetica
/Name /F1
>>
>>
>>
/Matrix [ 1 0 0 1 0 0 ]
/BBox [ 0 0 300 141 ]
/Length 176
/Subtype /Form
>>stream
BT
/F1 15 Tf
40 106 Td
20 TL
(Digitally Signed By: My Company) Tj
(Contact: mail#mail.com) '
(Location: Portugal) '
(Reason: Testing) '
(Date: 2013-04-03 14:18:41 +0100 ) '
ET
endstream
endobj
22 0 obj
<<
/Fields [ 16 0 R ]
/SigFlags 3
>>
endobj
What am I missing?

Sometimes, a man just need to know how to actually understand what he reads over and over again...
The ppkappearances file had it all there. I didn't even went as far as I expected i would have with complying with the dictionary structure they recommend there.
I just needed to add the n1 and n3 layers, and the stream content could even be just "% DS Blank". the question mark string stream you find was the one i took from PdfSignatureAppearance.cs file from iText solution. If I use that one Foxit Reader presents the question mark (even if the signature is valid). For Adobe it doesn't matter i use that or DSBlank.
require 'openssl'
begin
require 'origami'
rescue LoadError
ORIGAMIDIR = "C:\RailsInstaller\Ruby1.9.3\lib\ruby\gems\1.9.1\gems\origami-1.2.4\lib"
$: << ORIGAMIDIR
require 'origami'
end
include Origami
def draw_signature(name, attr = {})
load! if #instructions.nil?
x, y = attr[:x], attr[:y]
#instructions << PDF::Instruction.new('q')
#instructions << PDF::Instruction.new('cm', 1, 0, 0, 1, x, y)
#instructions << PDF::Instruction.new('Do', name)
#instructions << PDF::Instruction.new('Q')
end
def SignPdf(inputF)
#inputfile = String.new(inputF)
outputF = #inputfile.insert(inputF.rindex("."),"_signed8")
certFile = "cert.pem"
rsakeyFile = "pk.pem"
passphrase = "mypass"
key4pem=File.read rsakeyFile
key = OpenSSL::PKey::RSA.new key4pem, passphrase
cert = OpenSSL::X509::Certificate.new(File.read certFile)
pdf = PDF.read(inputF)
page = pdf.get_page(1)
#signature image
imageobject = Origami::Graphics::ImageXObject.from_image_file('c:\rails_projects\RecibosOnline\app\assets\images\logo.jpg', 'jpg')
imageobject.Width = 200
imageobject.Height = 141
#formobject = Origami::Graphics::FormXObject.new
#formobject.write("Teste")
width = 300
height=141
x=201
y=186
signedby = "My Company"
location = "Portugal"#pdf.signature[pdf.signature.keys[5]]
contact = "mail#mail.com" #pdf.signature[pdf.signature.keys[6]]
reason = "Testing" #pdf.signature[pdf.signature.keys[7]]
date = Time.now
caption="Digitally Signed By: #{signedby}\nContact: #{contact}\nLocation: #{location}\nReason: #{reason}\nDate: #{date} "
#DSBlankstream= "% DSBLank"
questionMark = "% DSUnknown\n" +
"q\n" +
"1 G\n" +
"1 g\n" +
"0.1 0 0 0.1 9 0 cm\n" +
"0 J 0 j 4 M []0 d\n" +
"1 i \n" +
"0 g\n" +
"313 292 m\n" +
"313 404 325 453 432 529 c\n" +
"478 561 504 597 504 645 c\n" +
"504 736 440 760 391 760 c\n" +
"286 760 271 681 265 626 c\n" +
"265 625 l\n" +
"100 625 l\n" +
"100 828 253 898 381 898 c\n" +
"451 898 679 878 679 650 c\n" +
"679 555 628 499 538 435 c\n" +
"488 399 467 376 467 292 c\n" +
"313 292 l\n" +
"h\n" +
"308 214 170 -164 re\n" +
"f\n" +
"0.44 G\n" +
"1.2 w\n" +
"1 1 0.4 rg\n" +
"287 318 m\n" +
"287 430 299 479 406 555 c\n" +
"451 587 478 623 478 671 c\n" +
"478 762 414 786 365 786 c\n" +
"260 786 245 707 239 652 c\n" +
"239 651 l\n" +
"74 651 l\n" +
"74 854 227 924 355 924 c\n" +
"425 924 653 904 653 676 c\n" +
"653 581 602 525 512 461 c\n" +
"462 425 441 402 441 318 c\n" +
"287 318 l\n" +
"h\n" +
"282 240 170 -164 re\n" +
"B\n" +
"Q\n";
n0 = Annotation::AppearanceStream.new
n0.Type=Origami::Name.new("XObject")
n0.BBox = [ 0, 0, 100, 100 ]
n0.Matrix = [ 1, 0, 0, 1, 0, 0 ]
n0.set_indirect(true)
n0.Resources = Resources.new
n0.Resources.ProcSet = [Origami::Name.new("PDF"),Origami::Name.new("Text"),Origami::Name.new("Text"),Origami::Name.new("ImageB"),Origami::Name.new("ImageC"),Origami::Name.new("ImageI")]
n0.draw_stream('% DSBlank')
#n0.write(Origami::Name.new("% DSBlank"))
n1 = Annotation::AppearanceStream.new
n1.Type=Origami::Name.new("XObject")
n1.BBox = [ 0, 0, 100, 100 ]
n1.Matrix = [ 1, 0, 0, 1, 0, 0 ]
n1.set_indirect(true)
n1.Resources = Resources.new
n1.Resources.ProcSet = [Origami::Name.new("PDF"),Origami::Name.new("Text"),Origami::Name.new("Text"),Origami::Name.new("ImageB"),Origami::Name.new("ImageC"),Origami::Name.new("ImageI")]
n1.draw_stream('% DSBlank')
n3 = Annotation::AppearanceStream.new
n3.Type=Origami::Name.new("XObject")
n3.BBox = [ 0, 0, 100, 100 ]
n3.Matrix = [ 1, 0, 0, 1, 0, 0 ]
n3.set_indirect(true)
n3.Resources = Resources.new
n3.Resources.ProcSet = [Origami::Name.new("PDF"),Origami::Name.new("Text"),Origami::Name.new("Text"),Origami::Name.new("ImageB"),Origami::Name.new("ImageC"),Origami::Name.new("ImageI")]
n3.draw_stream('% DSBlank')
n2 = Annotation::AppearanceStream.new
n2.Type = Origami::Name.new("XObject")
#n2.Subtype = Origami::Name.new("Form")
n2.Resources = Resources.new
n2.Resources.ProcSet = [Origami::Name.new("Text")]
n2.set_indirect(true)
n2.Matrix = [ 1, 0, 0, 1, 0, 0 ]
n2.BBox = [ 0, 0, width, height ]
n2.write(caption,:x => 40, :y => height-35, :size => 15)
#n2.draw_stream("q 0 0 198 48 re W n BT /Helv 7.645 Tf 0 g 0 40.135 Td (Digitally ) Tj 28.889 0 Td (signed ) Tj 24.652 0 Td (by ) Tj10.198 0 Td (James ) Tj 24.641 0 Td (Pravetz ) Tj-88.38 -9.174 Td (DN: ) Tj 15.29 0 Td (cn=James ) Tj37.18 0 Td (Pravetz, ) Tj 30.165 0 Td (c=CA, ) Tj 23.156 0 Td0 -9.174 Td (Reason: ) Tj 30.599 0 Td (I ) Tj 4.248 0 Td(have ) Tj 18.702 0 Td (reviewed ) Tj 32.719 0 Td(this ) Tj 14.02 0 Td (document ) Tj -100.288 -9.174 Td(Date: ) Tj 20.397 0 Td (2006.06.08 ) Tj 40.386 0 Td(13:39:35 ) Tj 31.883 0 Td (-07'00') Tj ET Q")
##Sets the root dictionary element
frm = Annotation::AppearanceStream.new
frm.set_indirect(true)
#frm.Type = Origami::Name.new("XObject")
#frm.Subtype = Origami::Name.new("Form")
frm.Resources = Resources.new
frm.Resources.ProcSet = [Origami::Name.new("PDF")]
frm.Resources.add_xobject(Origami::Name.new("n0"), n0)
frm.Resources.add_xobject(Origami::Name.new("n1"), n1)
frm.Resources.add_xobject(Origami::Name.new("n2"), n2)
frm.Resources.add_xobject(Origami::Name.new("n3"), n3)
frm.Matrix = [ 1, 0, 0, 1, 0, 0 ]
frm.BBox = [ 0, 0, width, height ]
frm.draw_stream('q 1 0 0 1 0 0 cm /n0 Do Q')
frm.draw_stream('q 1 0 0 1 0 0 cm /n1 Do Q')
frm.draw_stream('q 1 0 0 1 0 0 cm /n2 Do Q')
frm.draw_stream('q 1 0 0 1 0 0 cm /n3 Do Q')
xo17 = Annotation::Widget::Signature.new
#xo17 = Annotation::Widget::Signature.new
xo17.Rect = Rectangle[ llx: x, lly: y, urx: x+width, ury: y+height ]
#xo17.Resources = Resources.new
#xo17.Resources.add_xobject(Origami::Name.new("FRM"), frm)
xo17.F = Annotation::Flags::PRINT #sets the print mode on
xo17.H = Annotation::Widget::Highlight::INVERT
streamN = Annotation::AppearanceStream.new #.setFilter(:FlateDecode)
streamN.set_indirect(true)
streamN.BBox = [ 0, 0, width, height ]
streamN.Resources = Resources.new
streamN.Resources.ProcSet = [Origami::Name.new("PDF")]
streamN.Resources.add_xobject(Origami::Name.new("FRM"), frm)
streamN.Subtype = nil
#cs = ContentStream.new('q 1 0 0 1 0 0 cm /FRM Do Q',streamN)
streamN.draw_stream('q 1 0 0 1 0 0 cm /FRM Do Q')
#streamN.draw_image(Origami::Name.new("FRM"),{x:x,y:y})
#streamN.write(caption,:x => 40, :y => height-35, :size => 15)
xo17.set_normal_appearance(streamN)
# page.add_xobject(Origami::Name.new("FRM"),frm)
# page.add_xobject(Origami::Name.new("n0"),n0)
# page.add_xobject(Origami::Name.new("n2"),n2)
page.add_annot(xo17)
# Sign the PDF with the specified keys
pdf.sign(cert, key,
:method => 'adbe.pkcs7.sha1',
:annotation => xo17,
:location => location,
:contact => contact,
:reason => reason
)
# Save the resulting file
pdf.save(outputF)
end
SignPdf("Sample.pdf")
What a fight!
I feel like saying :
In the end, the winners are the ones that you will find standing in the middle of the battle field. Sadly looking around them into the battle field, and wondering why it had to be like that. Only if they knew how to read...then nothing of that would have happened. At the same time though, if it wouldn't have happened, they wouldn't know they didn't know how to read with the right focus.

Related

Eigen Dynamic-sized matrix verticaly/horizontaly Linspaced?

I'm currently working on a project where i need to be able to create matrix such as :
MatrixXi lin_spaced_horizontaly =
0 1 2 3 4 ... ncols
0 1 2 3 4 ... ncols
0 1 2 3 4 ... ncols
MatrixXi lin_spaced_verticaly =
0 0 0
1 1 1
2 2 2
3 3 3
4 4 4
. . .
nrows nrows nrows
Currently i am trying things like that :
Eigen::VectorXi v_lin_vec = Eigen::VectorXi::LinSpaced(nrows_, 0, nrows_).transpose
Eigen::MatrixXi v_lin_matrix (nrows_, ncols_);
for (auto i = 0; i<ncols_; i++)
v_lin_matrix << v_lin_vec;
Eigen::VectorXi h_lin_vec = Eigen::VectorXi::LinSpaced(ncols_, 0, ncols_)
Eigen::MatrixXi h_lin_matrix (nrows_, ncols_);
for (auto i = 0; i<ncols_; i++)
h_lin_matrix << h_lin_vec;
And I am getting results such as :
v_lin_matrix
-------------
0 0 0 0 0
1 0 0 0 0
2 0 0 0 0
3 0 0 0 0
4 0 0 0 0
h_lin_matrix
-------------
0 0 0 0 0
1 0 0 0 0
2 0 0 0 0
3 0 0 0 0
4 0 0 0 0
Thanks in advance !
You can use .rowwise().replicate(ncols_) and .colwise().replicate(nrows_) like so:
Eigen::MatrixXi v_lin_matrix = Eigen::VectorXi::LinSpaced(nrows_, 0, nrows_)
.rowwise().replicate(ncols_);
Eigen::MatrixXi h_lin_matrix = Eigen::RowVectorXi::LinSpaced(ncols_, 0, ncols_)
.colwise().replicate(nrows_);
Or alternatively:
Eigen::MatrixXi v_lin_matrix = Eigen::VectorXi::LinSpaced(nrows_, 0, nrows_)
.replicate(1,ncols_);
Eigen::MatrixXi h_lin_matrix = Eigen::RowVectorXi::LinSpaced(ncols_, 0, ncols_)
.replicate(nrows_,1);
Regarding your use of <<: This is meant to be used in combination with the , operator to initialize a matrix in a single expression, like this:
Eigen::MatrixXi A(4,ncols_);
A << row0, row1, row2, row3;
What you wrote will assert at runtime (if you compile without -DNDEBUG, which I strongly recommend until your code is sufficiently tested!)

Dompdf not working in Symfony and displays characters

I already installed the dompdf in composer but I cannot render a pdf file and it shows some characters on the browser. Is there something i missed out on the code?
%PDF-1.3 1 0 obj << /Type /Catalog /Outlines 2 0 R /Pages 3 0 R >> endobj 2 0 obj << /Type /Outlines /Count 0 >> endobj 3 0 obj << /Type /Pages /Kids [6 0 R ] /Count 1 /Resources << /ProcSet 4 0 R /Font << /F1 8 0 R >> >> /MediaBox [0.000 0.000 595.280 841.890] >> endobj 4 0 obj [/PDF /Text ] endobj 5 0 obj << /Producer (��dompdf + CPDF) /CreationDate (D:20191015175221+02'00') /ModDate (D:20191015175221+02'00') >> endobj 6 0 obj << /Type /Page /MediaBox [0.000 0.000 595.280 841.890] /Parent 3 0 R /Contents 7 0 R >> endobj 7 0 obj << /Filter /FlateDecode /Length 67 >> stream x��2�300P#&�ҹ�B�M���-L�L�,BR����B��5R�5cB�\C�'3 endstream endobj 8 0 obj << /Type /Font /Subtype /Type1 /Name /F1 /BaseFont /Times-Roman /Encoding /WinAnsiEncoding >> endobj xref 0 9 0000000000 65535 f 0000000009 00000 n 0000000074 00000 n 0000000120 00000 n 0000000274 00000 n 0000000303 00000 n 0000000462 00000 n 0000000565 00000 n 0000000703 00000 n trailer << /Size 9 /Root 1 0 R /Info 5 0 R /ID[<728657938b76cb1e658d7f5ccfa3c466><728657938b76cb1e658d7f5ccfa3c466>] >> startxref 812 %%EOF
use Dompdf\Dompdf; use Dompdf\Options;
/**
* #Route("/add")
*/
public function pdf()
{
$pdfOptions = new Options();
$pdfOptions->set('defaultFont', 'Arial');
$dompdf = new Dompdf($pdfOptions);
$dompdf->loadHtml('Hello world');
$dompdf->setPaper('A4', 'portrait');
$dompdf->render();
$dompdf->stream("mypdf.pdf", [
"Attachment" => false
]);
}
I had the same problem. I fixed this by adding exit(0); after $dompdf->stream();
Same problem here !
Find my way around by adding ob_get_clean(); just before $dompdf->stream('name.pdf');
$html = 'Test';
$dompdf = new Dompdf();
$dompdf->loadHtml($html);
$dompdf->setPaper('A4');
$dompdf->render();
ob_get_clean();
$dompdf->stream('name.pdf');

Python3 : unpack requires a bytes object of length 117

I am trying to run script for astronomical algorithm VSOP2013.
but when I running this script, it displays error in line 178
how to solve it?
What's wrong with unpack function?
FYI, the original script is python2, i am using python3
a = self.fmt.unpack(terms.encode())
error: unpack requires a bytes object of length 117
This is my full python3 script, adapted from the original python2 version from http://domenicomustara.blogspot.co.id
# -*- coding: utf-8 -*-
import gmpy2 as gmp
import struct
import ctypes
gmp.get_context().precision=200
def cal2jul(year, month, day, hour=0, minute =0, second=0):
month2 = month
year2 = year
if month2 <= 2:
year2 -= 1
month2 += 12
if (year*10000 + month*100 + day) > 15821015:
a = int(year2/100)
b = 2 - a + int(a/4)
else:
a = 0
b = 0
if year < 0:
c = int((365.25 * year2)-0.75)
else:
c = int(365.25 * year2)
d = int(30.6001 *(month2 + 1))
return b + c + d + day + hour / 24.0 + minute / 1440.0 + second / 86400.0 + 1720994.5
class VSOP2013():
def __init__(self, t, planet, precision=1e-7):
# calculate millennia from J2000
self.JD = t
self.t = gmp.div((t - cal2jul(2000,1,1,12)), 365250.0)
# predefine powers of self.t
self.power = []; self.power.append(gmp.mpfr(1.0)); self.power.append(self.t)
for i in range(2,21):
t = self.power[-1]
self.power.append(gmp.mul(self.t,t))
# choose planet file in a dict
self.planet = planet
self.planets = {'Mercury':'VSOP2013p1.dat',
'Venus' :'VSOP2013p2.dat',
'EMB' :'VSOP2013p3.dat',
'Mars' :'VSOP2013p4.dat',
'Jupiter':'VSOP2013p5.dat',
'Saturn' :'VSOP2013p6.dat',
'Uranus' :'VSOP2013p7.dat',
'Neptune':'VSOP2013p8.dat',
'Pluto' :'VSOP2013p9.dat'}
# VSOP2013 routines precision
self.precision = precision
# lambda coefficients
# l(1,13) : linear part of the mean longitudes of the planets (radian).
# l(14): argument derived from TOP2013 and used for Pluto (radian).
# l(15,17) : linear part of Delaunay lunar arguments D, F, l (radian).
self.l = (
(gmp.mpfr(4.402608631669), gmp.mpfr(26087.90314068555)),
(gmp.mpfr(3.176134461576), gmp.mpfr(10213.28554743445)),
(gmp.mpfr(1.753470369433), gmp.mpfr(6283.075850353215)),
(gmp.mpfr(6.203500014141), gmp.mpfr(3340.612434145457)),
(gmp.mpfr(4.091360003050), gmp.mpfr(1731.170452721855)),
(gmp.mpfr(1.713740719173), gmp.mpfr(1704.450855027201)),
(gmp.mpfr(5.598641292287), gmp.mpfr(1428.948917844273)),
(gmp.mpfr(2.805136360408), gmp.mpfr(1364.756513629990)),
(gmp.mpfr(2.326989734620), gmp.mpfr(1361.923207632842)),
(gmp.mpfr(0.599546107035), gmp.mpfr(529.6909615623250)),
(gmp.mpfr(0.874018510107), gmp.mpfr(213.2990861084880)),
(gmp.mpfr(5.481225395663), gmp.mpfr(74.78165903077800)),
(gmp.mpfr(5.311897933164), gmp.mpfr(38.13297222612500)),
(gmp.mpfr(0.000000000000), gmp.mpfr(0.3595362285049309)),
(gmp.mpfr(5.198466400630), gmp.mpfr(77713.7714481804)),
(gmp.mpfr(1.627905136020), gmp.mpfr(84334.6615717837)),
(gmp.mpfr(2.355555638750), gmp.mpfr(83286.9142477147)))
# planetary frequencies in longitude
self.freqpla = {'Mercury' : gmp.mpfr(0.2608790314068555e5),
'Venus' : gmp.mpfr(0.1021328554743445e5),
'EMB' : gmp.mpfr(0.6283075850353215e4),
'Mars' : gmp.mpfr(0.3340612434145457e4),
'Jupiter' : gmp.mpfr(0.5296909615623250e3),
'Saturn' : gmp.mpfr(0.2132990861084880e3),
'Uranus' : gmp.mpfr(0.7478165903077800e2),
'Neptune' : gmp.mpfr(0.3813297222612500e2),
'Pluto' : gmp.mpfr(0.2533566020437000e2)}
# target variables
self.ax = gmp.mpfr(0.0) # major semiaxis
self.ml = gmp.mpfr(0.0) # mean longitude
self.kp = gmp.mpfr(0.0) # e*cos(perielium longitude)
self.hp = gmp.mpfr(0.0) # e*sin(perielium longitude)
self.qa = gmp.mpfr(0.0) # sin(inclination/2)*cos(ascending node longitude)
self.pa = gmp.mpfr(0.0) # sin(inclination/2)*cos(ascending node longitude)
self.tg_var = {'A':self.ax, 'L':self.ml, 'K':self.kp,
'H':self.hp, 'Q':self.qa, 'P':self.pa }
# eps = (23.d0+26.d0/60.d0+21.41136d0/3600.d0)*dgrad
self.eps = gmp.mpfr((23.0+26.0/60.0+21.411360/3600.0)*gmp.const_pi()/180.0)
self.phi = gmp.mpfr(-0.051880 * gmp.const_pi() / 180.0 / 3600.0)
self.ceps = gmp.cos(self.eps)
self.seps = gmp.sin(self.eps)
self.cphi = gmp.cos(self.phi)
self.sphi = gmp.sin(self.phi)
# rotation of ecliptic -> equatorial rect coords
self.rot = [[self.cphi, -self.sphi*self.ceps, self.sphi*self.seps],
[self.sphi, self.cphi*self.ceps, -self.cphi*self.seps],
[0.0, self.seps, self.ceps ]]
self.fmt = struct.Struct("""6s 3s 3s 3s 3s x 3s 3s 3s 3s 3s x 4s 4s 4s 4s x
6s x 3s 3s 3s 20s x 3s 20s x 3s x""")
self.gmp_ = {
'Mercury' : gmp.mpfr(4.9125474514508118699e-11),
'Venus' : gmp.mpfr(7.2434524861627027000e-10),
'EMB' : gmp.mpfr(8.9970116036316091182e-10),
'Mars' : gmp.mpfr(9.5495351057792580598e-11),
'Jupiter' : gmp.mpfr(2.8253458420837780000e-07),
'Saturn' : gmp.mpfr(8.4597151856806587398e-08),
'Uranus' : gmp.mpfr(1.2920249167819693900e-08),
'Neptune' : gmp.mpfr(1.5243589007842762800e-08),
'Pluto' : gmp.mpfr(2.1886997654259696800e-12)}
self.gmsol = gmp.mpfr(2.9591220836841438269e-04)
self.rgm = gmp.sqrt(self.gmp_[self.planet]+self.gmsol)
# run calculus routine
self.calc()
def __str__(self):
vsop_out = "{:3.13} {:3.13} {:3.13} {:3.13} {:3.13} {:3.13}\n".format(
self.tg_var['A'],
self.tg_var['L'],
self.tg_var['K'],
self.tg_var['H'],
self.tg_var['Q'],
self.tg_var['P'])
vsop_out += "{:3.13} {:3.13} {:3.13} {:3.13} {:3.13} {:3.13}\n".format(
self.ecl[0],
self.ecl[1],
self.ecl[2],
self.ecl[3],
self.ecl[4],
self.ecl[5])
vsop_out += "{:3.13} {:3.13} {:3.13} {:3.13} {:3.13} {:3.13}\n".format(
self.equat[0],
self.equat[1],
self.equat[2],
self.equat[3],
self.equat[4],
self.equat[5])
return vsop_out
def calc(self):
with open(self.planets[self.planet]) as file_in:
terms = []
b = '*'
while b != '':
b = file_in.readline()
if b != '':
if b[:5] == ' VSOP':
header = b.split()
#print header[3], header[7], header[8], self.t**int(header[3])
no_terms = int(header[4])
for i in range(no_terms):
#6x,4i3,1x,5i3,1x,4i4,1x,i6,1x,3i3,2a24
terms = file_in.readline()
# print('terms',terms)
a = self.fmt.unpack(terms.encode())
S = gmp.mul(gmp.mpfr(a[18]),gmp.exp10(int(a[19])))
C = gmp.mul(gmp.mpfr(a[20]),gmp.exp10(int(a[21])))
if gmp.sqrt(S*S+C*C) < self.precision:
break
aa = 0.0; bb = 0.0;
for j in range(1,18):
aa += gmp.mul(gmp.mpfr(a[j]), self.l[j-1][0])
bb += gmp.mul(gmp.mpfr(a[j]), self.l[j-1][1])
arg = aa + bb * self.t
power = int(header[3])
comp = self.power[power] * (S * gmp.sin(arg) + C * gmp.cos(arg))
if header[7] == 'L' and power == 1 and int(a[0]) == 1:
pass
else:
self.tg_var[header[7]] += comp
self.tg_var['L'] = self.tg_var['L'] + self.t * self.freqpla[self.planet]
self.tg_var['L'] = self.tg_var['L'] % (2 * gmp.const_pi())
if self.tg_var['L'] < 0:
self.tg_var['L'] += 2*gmp.const_pi()
print ("Julian date {}".format(self.JD))
file_in.close()
##print self.tg_var
#### def ELLXYZ(self):
xa = self.tg_var['A']
xl = self.tg_var['L']
xk = self.tg_var['K']
xh = self.tg_var['H']
xq = self.tg_var['Q']
xp = self.tg_var['P']
# Computation
xfi = gmp.sqrt(1.0 -xk * xk - xh * xh)
xki = gmp.sqrt(1.0 -xq * xq - xp * xp)
u = 1.0/(1.0 + xfi)
z = complex(xk, xh)
ex = abs(z)
ex2 = ex * ex
ex3 = ex2 * ex
z1 = z.conjugate()
#
gl = xl % (2*gmp.const_pi())
gm = gl - gmp.atan2(xh, xk)
e = gl + (ex - 0.1250 * ex3) * gmp.sin(gm)
e += 0.50 * ex2 * gmp.sin(2.0 * gm)
e += 0.3750 * ex3 * gmp.sin(3.0 * gm)
#
while True:
z2 = complex(0.0, e)
zteta = gmp.exp(z2)
z3 = z1 * zteta
dl = gl - e + z3.imag
rsa = 1.0 - z3.real
e = e + dl / rsa
if abs(dl) < 1e-15:
break
#
z1 = u * z * z3.imag
z2 = gmp.mpc(z1.imag, -z1.real)
zto = (-z+zteta+z2)/rsa
xcw = zto.real
xsw = zto.imag
xm = xp * xcw - xq * xsw
xr = xa * rsa
#
self.ecl = []; self.equ = {}
self.ecl.append(xr * (xcw -2.0 *xp * xm))
self.ecl.append(xr * (xsw +2.0 *xq * xm))
self.ecl.append(-2.0 * xr * xki * xm)
#
xms = xa *(xh + xsw) / xfi
xmc = xa *(xk + xcw) / xfi
xn = self.rgm / xa ** (1.50)
#
self.ecl.append( xn *((2.0 * xp * xp - 1.0) * xms + 2.0 * xp * xq * xmc))
self.ecl.append( xn *((1.0 -2.0 * xq * xq) * xmc -2.0 * xp * xq * xms))
self.ecl.append( 2.0 * xn * xki * (xp * xms + xq * xmc))
# Equatorial rectangular coordinates and velocity
#
#
# --- Computation ------------------------------------------------------
#
self.equat = [0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
for i in range(3):
for j in range(3):
self.equat[i] = self.equat[i] + self.rot[i][j] * self.ecl[j]
self.equat[i+3] = self.equat[i+3] + self.rot[i][j] * self.ecl[j+3]
if __name__ == '__main__':
for planet in ('Mercury', 'Venus', 'EMB', 'Mars', 'Jupiter',
'Saturn', 'Uranus', 'Neptune', 'Pluto'):
print ("PLANETARY EPHEMERIS VSOP2013 "+ planet + "(TDB)\n"+"""
1/ Elliptic Elements: a (au), lambda (radian), k, h, q, p - Dynamical Frame J2000
2/ Ecliptic Heliocentric Coordinates: X,Y,Z (au) X',Y',Z' (au/d) - Dynamical Frame J2000
3/ Equatorial Heliocentric Coordinates: X,Y,Z (au) X',Y',Z' (au/d) - ICRS Frame J2000
""")
init_date = cal2jul(1890,6,26,12)
set_date = init_date
while set_date < init_date + 41000:
v = VSOP2013(set_date, planet)
print (v)
set_date += 4000
Respond comment from Mr.barrycarter,
this is a bit of the result when i print(terms.encode()):
b' 2 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0.6684459764580090 -07 0.3603178002233933 -06\n'
b' 3 0 2 -4 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.2383757728520679 -07 0.9861749707454420 -07\n'
b' 4 0 4 -6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 -0.2193692495097233 -07 -0.8959173003201546 -07\n'
b' 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.0000000000000000 +00 0.1017891898227051 -03\n'
b' 2 0 3 -5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.4236543085970792 -07 -0.8775084424897674 -08\n'
b' 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.0000000000000000 +00 0.4702795245810685 -04\n'
b' 2 0 3 -5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 -0.5710471800210820 -09 -0.1800837750117577 -08\n'
b' 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.0000000000000000 +00 -0.5421827377115325 -06\n'
b' 2 0 3 -5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 -0.7074507338012408 -10 0.1742474656298139 -10\n'
b' 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.0000000000000000 +00 -0.2508633795522544 -07\n'
b' 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.0000000000000000 +00 0.4575014479216901 -09\n'
b' 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.0000000000000000 +00 0.5208591612817609 -11\n'
b' 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.0000000000000000 +00 -0.1737141639583644 -12'

How to crop a multipage pdf using ghostscript with an array of page specific cropboxes

This post is a follow-up of "How do I crop pages 3&4 in a multipage pdf using ghostscript", but it changes the input to an array of data.
The quest: I have a pdf file (a set of drawings) where all pages have the same size and I want to crop some pages in one way and others differently. The following screenshot shows how I generated the data below for cropping:spreadsheet The "left,bottom,right,top" are to be handed over to the postscript /CropBox [934 350 3318 2034] command. Pagenumbers are just consecutive numbers, so they may not rally be needed.
page#,left,bottom,right,top
1 0 0 4252 2384
2 0 0 4252 2384
3 0 0 4252 2384
4 0 0 4252 2384
5 934 350 3318 2034
6 934 350 3318 2034
7 441 0 3811 2384
8 441 0 3811 2384
With the solution in the above mentioned question I was able to crop a specific page in an multipage pdf and it probably is a good starting point for a solution to this question. I just didn't figure it out on my own.
The relevant postscript code which tried to use as a base to solve this problem is (thanks to KenS):
<<
/EndPage {
0 eq {
pop /Page# where {
/Page# get
3 eq {
(page 3) == flush
[/CropBox [0 0 1612 1792] /PAGE pdfmark
true
}
{
(not page 3) == flush
[/CropBox [500 500 612 792] /PAGE pdfmark
true
} ifelse
}{
true
} ifelse
}
{
false
}
ifelse
}
>> setpagedevice
I guess we need some test numbers for "realistic" page sizes for my crazy page size dictionary... just for some fun testing.
/MyCrazyPageSizeDictionary begin
/PageSizeArray [
[0 0 595 842] % original A4 portrait
[87 123 508 719] % cut to A5
[149 210 446 631] % cut to A6
[192 272 403 570] % cut to A7
[223 316 372 526] % cut to A8
] def
I'd suggest you place the cropping information for each page in an array, and then add each array to an enclosing array. The problem is likely to be retaining the information.
The best way to do this, probably, is to create the array of page information as a named object in a specific dictionary. If you don't create your own dictionary, then userdict will be used instead.
Then in your EndPage procedure you simply pull the relevant index of the enclosing array, which gives you an array of crop sizes:
So, for example;
%!
/MyCrazyPageSizeDictionary 1 dict def
/MyCrazyPageSizeDictionary begin
/PageSizeArray [
[ 0 0 4252 2384]
[ 0 0 4252 2384]
[ 0 0 4252 2384]
[ 0 0 4252 2384]
[ 934 350 3318 2034]
[ 934 350 3318 2034]
[ 441 0 3811 2384]
[ 441 0 3811 2384]
] def
end
<<
/EndPage {
0 eq {
pop /Page# where {
/Page# get % stack - pagenum
/MyCrazyPageSizeDictionary /PageSizeArray get % stack - pagenum [[]]
exch % stack - [[]] pagenum
get % stack - []
[ /CropBox % stack - [] [ /CropBox
3 -1 roll % stack - [ /CropBox []
/Page pdfmark
true
}{
true
} ifelse
}
{
false
}
ifelse
}
>> setpagedevice
If you put that in a file (named eg crop.ps) then run your PDF file through Ghostscript but put 'crop.ps' as one of the input files before your PDF file:
gs <options....> crop.ps input.pdf
Then it should do what you want. With the caveat that I haven't tested this program in any way......
[Edit, added the corrected program]
/MyCrazyPageSizeDictionary 1 dict def
MyCrazyPageSizeDictionary begin
/PageSizeArray [
[ 0 0 4252 2384]
[ 0 0 4252 2384]
[ 0 0 4252 2384]
[ 0 0 4252 2384]
[ 934 350 3318 2034]
[ 934 350 3318 2034]
[ 441 0 3811 2384]
[ 441 0 3811 2384]
] def
end
<<
/EndPage {
0 eq {
pop /Page# where {
/Page# get % stack - pagenum
1 sub % array index is 0 based, page numbers start at 1
MyCrazyPageSizeDictionary /PageSizeArray get % stack - pagenum [[]]
exch % stack - [[]] pagenum
1 index length mod % get array, find length, clamp page number to length
get % stack - []
[ /CropBox % stack - [] [ /CropBox
3 -1 roll % stack - [ /CropBox []
/PAGE pdfmark
true
}{
true
} ifelse
}
{
false
}
ifelse
}
>> setpagedevice

Converting SFrames into input dataset Sframes

I have a pretty bad way to convert my input logs to the input dataset.
I have an SFrame sf with the following format:
user_id int
timestamp datetime.datetime
action int
reasoncode str
action column takes up 9 values ranging from 1 to 9.
So, every user_id can perform more than 1 action, more than once.
I am trying to obtain all unique user_id from sf and create an op_sf in the following manner:
y = 225
def calc_class(a,x):
diffd = a['timestamp'].apply(lambda x: (dte - x).days)
g = 0
b = 0
for i in diffd:
if i > y:
g += 1
else:
b += 1
if b>= x:
return 4
elif b!= 0:
return 3
elif g>= 0:
return 2
else:
return 1
l1 = []
ids = z['user_id'].unique()
for idd in ids:
temp = sf[sf['user_id']== idd]
zero1 = temp[temp['action'] == 1]
zero2 = temp[temp['action'] == 2]
zero3 = temp[temp['action'] == 3]
zero4 = temp[temp['action'] == 4]
zero5 = temp[temp['action'] == 5]
zero6 = temp[temp['action'] == 6]
zero7 = temp[temp['action'] == 7]
zeroh8 = temp[temp['reasoncode'] == 'xyz']
zero9 = temp[temp['reasoncode'] == 'abc']
/* I'm getting clas1 to clas9 from function calc_class for each action
clas1 to clas9 are 4 integers ranging from 1 to 4
*/
clas1 = calc_class(zero1,2)
clas2 = calc_class(zero2,2)
clas3 = calc_class(zero3,2)
clas4 = calc_class(zero4,2)
clas5 = calc_class(zero5,2)
clas6 = calc_class(zero6,2)
clas7 = calc_class(zero7,2)
clas8 = calc_class(zero8,2)
clas9 = calc_class(zero9,2)
l1.append([idd,clas1,clas2,clas3,clas4,clas5*(-1),clas6*(-1),clas7*(-1),clas8*(-1),clas9])
I wanted to know if this is the fastest way of doing this. Specifically if it is possible to do the same thing without generating the zero1 to zero9 SFrames.
An example sf:
user_id timestamp action reasoncode
574 23/09/15 12:43 1 None
574 23/09/15 11:15 2 None
574 06/10/15 11:20 2 None
574 06/10/15 11:21 3 None
588 04/11/15 10:00 1 None
588 05/11/15 10:00 1 None
555 15/12/15 13:00 1 None
585 22/12/15 17:30 1 None
585 15/01/16 07:44 7 xyz
588 06/01/16 08:10 7 abc
l1 corresponding to the above sf:
574 1 2 2 0 0 0 0 0 0
588 3 0 0 0 0 0 0 0 3
555 3 0 0 0 0 0 0 0 0
585 3 0 0 0 0 0 0 3 0
I think your logic is relatively complex, but it's still more efficient to use column-wise operations on the whole dataset, rather than extracting the subset of rows for each user. The key tools are SFrame.groupby, SFrame.apply, SFrame.unstack, and SFrame.unpack. API docs here:
https://dato.com/products/create/docs/generated/graphlab.SFrame.html
Here's a solution that uses slightly simpler data than your example and slightly simpler logic to code the old vs. new actions.
# Set up and make the data
import graphlab as gl
import datetime as dt
sf = gl.SFrame({'user': [574, 574, 574, 588, 588, 588],
'timestamp': [dt.datetime(2015, 9, 23), dt.datetime(2015, 9, 23),
dt.datetime(2015, 10, 6), dt.datetime(2015, 11, 4),
dt.datetime(2015, 11, 5), dt.datetime(2016, 1, 6)],
'action': [1, 2, 3, 1, 1, 7]})
# Count old vs. new actions.
sf['days_elapsed'] = (dt.datetime.today() - sf['timestamp']) / (3600 * 24)
sf['old_threshold'] = sf['days_elapsed'] > 225
aggregator = {'total_count': gl.aggregate.COUNT('user'),
'old_count': gl.aggregate.SUM('old_threshold')}
grp = sf.groupby(['user', 'action'], aggregator)
# Code the actions according to old vs. new. Use your own logic here.
grp['action_code'] = grp.apply(
lambda x: 2 if x['total_count'] > x['old_count'] else 1)
grp = grp[['user', 'action', 'action_code']]
# Reshape the results into columns.
sf_new = (grp.unstack(['action', 'action_code'], new_column_name='action_code')
.unpack('action_code'))
# Fill in zeros for entries with no actions.
for c in sf_new.column_names():
sf_new[c] = sf_new[c].fillna(0)
print sf_new
+------+---------------+---------------+---------------+---------------+
| user | action_code.1 | action_code.2 | action_code.3 | action_code.7 |
+------+---------------+---------------+---------------+---------------+
| 588 | 2 | 0 | 0 | 2 |
| 574 | 1 | 1 | 1 | 0 |
+------+---------------+---------------+---------------+---------------+
[2 rows x 5 columns]

Resources