I have a Python3 function that combine two bytes, one use bytes.fromhex() method, and the other use to_bytes() method:
from datatime import datetime
def bytes_add() -> bytes:
bytes_a = bytes.fromhex('6812')
bytes_b = datetime.now().month.to_bytes(1, byteorder='little', signed=False)
return bytes_a + bytes_b
Is it possible to write a same function as above in Raku?(if so, How to control byteorder and signed params?)
as for byteorder, say convert number 1024 to bytes in Python:
(1024).to_bytes(2, byteorder='little') # Output: b'\x00\x04', byte 00 is before byte 04
as a contrast, convert number 1024 to Buf or Blob in Raku:
buf16.new(1024) # Output: Buf[uint16]:0x<0400>, byte 00 is after byte 04
is there any way to get Buf[uint16]:0x<0004> in the above example in Raku?
Update:
inspired by codesections, I try to figure out a solution similar to codesections's answer:
sub bytes_add() {
my $bytes_a = pack("H*", '6812');
my $bytes_b = buf16.new(DateTime.now.month);
$bytes_a ~ $bytes_b;
}
But still don't know how to use byteorder.
Is it possible to write a same function as above in Raku?
Yes. I'm not 100% sure I understand the overall goal of the function you provided, but a literal/line-by-line translation is certainly possible. If you would like to elaborate on the goal, it may also be possible to achieve the same goal in an easier/more idiomatic way.
Here's the line-by-line translation:
sub bytes-add(--> Blob) {
my $bytes-a = Blob(<68 12>);
my $bytes-b = Blob(DateTime.now.month);
Blob(|$bytes-a, |$bytes-b)
}
The output of bytes-add is printed by default using its hexadecimal representation (Blob:0x<44 0C 09>). If you'd like to print it more like Python prints its byte literals, you can do so with bytes-add».chr.raku, which prints as ("D", "\x[C]", "\t").
if so, How to control byteorder?
Because the code above constructs the Blob from a List, you can simply .reverse the list to use the opposite order.
I am able to read my files from the FTP location if I specify the exact filename. My problem is that I'm trying to automate this process where I have to read these files week over week and the filename changes randomly. There is no specific pattern to it, so it can't be predetermined.
Is there a way in SAS where I can read the name of all the files present at an FTP location and give the user a dialog box with this information, for them to enter the filename they want to read.
In the SAS Display Manager interface you can use a data step WINDOW or macro %WINDOW statements to define a simple picker, and DISPLAY or %DISPLAY to raise it. The simple picker is really simple, no scrollers or other modern adornments.
An FTP folder listing is retrieved using the filename FTP engine option LS
Sample code:
/**/
* location of FTP folder;
filename folder ftp
user = 'anonymous'
host = 'ftp.cdc.gov'
cd = '/pub/Health_Statistics/NCHS/Publications/ICD9-CM/2011'
ls
;
* retrieve listing;
data files;
infile folder;
input;
order + 1;
fileinfo = _infile_;
run;
/**/
%macro picker(
/* Dynamically build a %WINDOW definition, display it and return the last selected item */
name=, /* Name of window */
title=, /* First line text */
data=, /* data set containing items */
order=order, /* variable for ordering items in the picker*/
item=, /* variable to pick a value of */
result= /* name of macro variable that will contain the picked item, must pre-exist in caller scope */
);
%* field definitions will look like
%* #2 #2 field<i> 1 color=blue attr=rev_video " &filename<i>" ;
%local n i row field_def;
proc sql noprint;
select count(*) into :n trimmed from &data;
%do i = 1 %to &n;
%local field&i item&i;
%end;
select
&order, &item
into
:order1-, :item1-
from
&data;
quit;
%do i = 1 %to &N;
%let field_def = &field_def #%eval(&i+1) #2 field&i 1 color=blue attr=rev_video " &&item&i";
%end;
%WINDOW PICKER rows=30 columns=80
#1 #1 "&title. - Mark an item and Press F3"
&field_def
;
%display PICKER;
%do i = 1 %to &N;
%if %length(&&field&i) %then %let &result=&&item&i;
%end;
%mend;
%let selected=;
%picker(name=PICKER, title=Pick a file, data=files, item=fileinfo, result=selected);
%put &=selected;
More sophisticated pickers can be built using SAS/AF. Other possibilities include Stored Process prompt dialogs, SAS Studio snippets, or a SAS server page.
Time ago, I did move some files from a directory and copied to a new one,
while copying I did rename each one. For doing so , I used bash.
Something like this
for i in 8 12 16 20 ; do;
for j in 0.00 0.01 ; do ;
cp tera.dat new/tera$i$j.dat
...
Into each tera$i$j.dat, there is numeric data in 2 columns.
I would like to plot the data as a histogram, and save each image (if possible in a loop) for each file tera$i$j.dat . But unfortunately, I do not have much idea how to do this.
I managed to load the data in several tables using de = readtable.(filter(r"tera", readdir()),separator =' ', header= false,;.
But I can not create a plot for each data :(
So far I managed to read the file, and create a histogram plot for 1 file per time, how do I do it in a loop ? This is the code I am using.
using Plots StatPlots, Distributions, DataFrames, PlotlyJS, LaTeXStrings;
theme(:sand);
chp = pwd();
pe = readtable("tera120.00.dat",separator =' ', header= false)
gr()
histogram(pe[2], nbins=1000)
...
Thanks in advance :)
Untested, but it should be something like this:
using StatPlots, DataFrames
theme(:sand)
gr()
for i in [8, 12, 16, 20], j in [0, 0.01]
fn = #sprintf "tera%02d.%02d" i j
pe = readtable(fn*".dat", separator =' ', header = false)
p = histogram(pe[2], bins = 1000);
savefig(p, fn*".png")
end
Note that when you plot with StatPlots, you don't need using Plots or using PlotlyJS, I also don't see why you need Distributions or LaTeXStrings.
I would like to use fortran to read ultraviolet radiation data that has been produced by the Japan Aerospace Exploration Agency. This data is at a daily and monthly temporal resolution from 2000-2010 at a ~5 km spatial resolution. This question is worth answering as the data could be useful for a number of environment/health projects and is freely available, with proper acknowledgement of source and sharing of preprint of any subsequent publications, from:
ftp://suzaku.eorc.jaxa.jp/pub/GLI/glical/Global_05km/monthly/uvb/
There is a readme file available, which provides instructions on how to read data using fortran as follows:
Instructions for _le files
Header
Read header (size= pixel size *2byte):
character head*14400
read(10,rec=1) head
read(head,'(2i6,2f8.2,f8.4,2e12.5,a1,a8,a1,a40)')
& npixel,nline,lon_min,lat_max,reso,slope,offset,',',
& para,',',outfile
Read data (e.g., fortran77)
parameter(nl=7200, ml=3601)
... open file by "unformatted", "recl=nl*2(byte)" (,"bytereclen")
integer*2 i2buf(nl,ml)
do m=1,ml
read(10,rec=1+m) (i2buf(n,m), n=1,nl)
do n=1,nl
par=i2buf(n,m)*slope+offset
write(6,*) 'PAR[Ein/m^2/day]=',par
enddo
enddo
slope values
par__le : daily PAR [Ein/m^2/day] = DN * 0.01
dpar_le : direct PAR = DN * 0.01
swr__le : daily mean shortwave radiation [W/m^2] = DN * 0.01
tip__le : transmittance of instantaneous PAR at noon = DN * 0.0001
uva__le : daily mean UVA [W/m^2] = DN * 0.001
uvb__le : daily mean UVB [W/m^2] = DN * 0.0001
rpar_le : PAR-range surface reflectance (TOP of canopy/solid surfaces) = DN * 0.0001 (monthly data only)
error values
-1 as signed short integer (int16)
65535 as unsigned short integer (uint16)
Progress so far
I have downloaded and installed gfortran successfully on mac OSX. I have downloaded a test file (MOD02SSH_A20000224Av6_v601_7200_3601_uvb__le.gz) and decompressed it. I have created a program file:
PROGRAM readuvr
IMPLICIT NONE
!some code
END PROGRAM
I will then type the following into the command line to create an executable and run it to extract the data.
gfortran -o executable
./executable
As a complete beginner to fortran, my question is: how can I use the instructions provided to build a program that can read the data and output it into a text file?
Well, that file expands to 51,868,800 bytes. The comments imply the header is 14,400 bytes, which leaves 51,854,400 bytes of actual data payload.
There seem to be 7200 lines of data, so that means there are 7202 bytes per line. There seem to be 2 bytes (16-bit samples) so if we assume 2 bytes/sample, that means there are 3601 samples per line, which matches the ml=3601.
So basically, you need to read 14,400 bytes of header, then 7200 lines of data, each line consisting of 3601 values, each of those being 2 bytes wide...
Actually, if you are that unfamiliar with FORTRAN, you may like to extract the data with Perl which is already installed and available on OS X anyway. I have started a VERY SIMPLISTIC Perl program that reads the dat and prints the first 2 values on each line:
#!/usr/bin/perl
use strict;
use warnings;
# Read 14,400 bytes of header
my $buffer;
my $nBytes = 14400;
my $bytesRead = read (STDIN, $buffer, $nBytes) ;
my ($npixel,$nline,$lon_min,$lat_max,$reso,$slope,$offset,$junk)=split(' ',$buffer);
print "npixel:$npixel\n";
print "nline:$nline\n";
print "lon_min:$lon_min\n";
print "lat_max:$lat_max\n";
print "reso:$reso\n";
print "slope:$slope\n";
$offset =~ s/,.*//; # strip trailing comma and junk
print "offset:$offset\n";
# Read actual lines of data
my $line;
for(my $m=1;$m<=$nline;$m++){
read(STDIN,$line,$npixel*2);
my $x=$npixel*2;
my #values=unpack("S$x",$line);
printf "Line: %d",$m;
for(my $j=0;$j<2;$j++){
printf ",%f",$values[$j]*$slope+$offset;
}
printf "\n"; # newline
}
Save it as go.pl and then in the Terminal, type the following once to make it executable
chmod +x go.pl
and then run it like this
./go.pl < MOD02SSH_A20000224Av6_v601_7200_3601_uvb__le
Sample output extract:
npixel:7200
nline:3601
lon_min:0.00
lat_max:90.00
reso:0.0500
slope:0.10000E-03
offset:0.00000E+00
...
...
Line: 3306,0.099800,0.099800
Line: 3307,0.099900,0.099900
Line: 3308,0.099400,0.074200
Line: 3309,0.098900,0.098900
Line: 3310,0.098400,0.098400
Line: 3311,0.074300,0.074200
Line: 3312,0.071300,0.071200
fortran (f2003 or so) solution. (The linked instructions are awful by the way )
implicit none
character*80 para,outfile
character(len=:),allocatable::header,infile
integer npixel,nline,blen,i
c note kind=2 is not standard. This needs to be a 2-byte integer.
integer(kind=2),allocatable :: data(:,:)
real lon_min,lat_max,reso,slope,off
c header is plain text, so first open formatted and
c directly read header data
infile='MOD02SSH_A20000224Av6_v601_7200_3601_uvb__le'
open(10,file=infile)
read(10,*)npixel,nline,lon_min,lat_max,reso,slope,off,
$ para,outfile
close(10)
write(*,*)npixel,nline,lon_min,lat_max,reso,slope,off,
$ trim(para),' ',trim(outfile)
blen=2*npixel
allocate(character(len=blen)::header)
allocate(data(npixel,nline))
if( sizeof(data(1,1)).ne.2 )then
write(*,*)'error kind=2 did not give a 2 byte integer'
stop
endif
c now close and reopen for binary read.
c direct access approach:
open(20,file=infile,access='direct',recl=blen/4)
c note the granularity of the recl= specifier is not standard.
c ifort uses 4 bytes. (note this will break if npixel is not even )
read(20,rec=1)header
write(*,*)trim(header)
do i=1,nline
read(20,rec=i+1)data(:,i)
enddo
c note streams if available is simpler: (we don't need to know rec len )
c open(20,file=infile,access='stream')
c read(20)header,data
end
This is not actually validated because I don't have known file content to compare against.
I have saved data in format of blob using powerbuilder ole control in oracle.
Now we want to convert these blob to files,
files are of different format(PDF, JPG,EXCEL,TEXT,DOC)
There are more than 1 Million files so it is not easy to do manually open and save using olecontrol.
Can we do it through script auto saving of blob to file in powerbuilder
Yes, it is possible:
Write a cursor in PowerBuilder embedded SQL to get for each record in your blob table the key and the and file extension (if you have those). The syntax for that kind of thing looks like this:
Long ll_Key
String ls_Ext
DECLARE GetBlobCursor CURSOR FOR
SELECT blob_key,
blob_extension
FROM blob_table ;
/* need to loop here while SQLCA.SQLCode is good */
FETCH GetBlobCursor
INTO :ll_Key,
:ls_Ext ;
Use a SELECTBLOB embedded SQL statement to get the blob data into a PowerBuilder BLOB variable:
Blob lblob_File
SELECTBLOB fileblob
INTO :lblob_File
FROM blobtable
WHERE blob_key = :ll_Key ;
Use FileOpen and FileWrite to write the blob with a valid file name and extension:
Long ll_Loops, ll_Step
Int li_File
String ls_Path
ls_Path = "<where do you want me?>." + String(ll_Key) + "." + ls_Ext
li_File = FileOpen(ls_Path, StreamMode!, Write!, LockWrite!, Append!)
If li_File > 0 Then
// Determine how many times to call FileWrite
ll_FileLen = Len(lblob_File)
If ll_FileLen > 32765 Then
If Mod(ll_FileLen, 32765) = 0 Then
ll_Loops = ll_FileLen/32765
Else
ll_Loops = (ll_FileLen/32765) + 1
End If
Else
ll_Loops = 1
End If
For ll_Step = 1 To ll_Loops
FileWrite(li_File,BlobMid(lblob_File,((ll_Step - 1)*32765) + 1, 32765))
Next
Else
//log the error, or handle
End If
FileClose(li_File)
Hope that gets you started.