Append data at end of existing data with SORT - sorting

I want the below code to work for variable length data also in input. So, that I can append double quote at its end. I don't want to use COBOL. Is it possible?
ZZZZ.VVVV.FILE.CORR1 contains following records:-
ABCDEFG_AAAAAA_BBB0001.csv
ABCDEFG_AAAAAA_BBB0002.csv
ABCDEFG_AAAAAA_BBB0003.csv
Output records in file ZZZZ.VVVV.FILE.CORR2 by following code is:
r_file1="ABCDEFG_AAAAAA_BBB0001.csv"
r_file2="ABCDEFG_AAAAAA_BBB0002.csv"
r_file3="ABCDEFG_AAAAAA_BBB0003.csv"
JCL code is:
//STEPSRT5 EXEC PGM=SORT
//SYSOUT DD SYSOUT=*
//SORTIN DD DSN=ZZZZ.VVVV.FILE.CORR1, <ID>
// DISP=SHR
//SORTOUT DD DSN=ZZZZ.VVVV.FILE.CORR2, <OD>
// DCB=(RECFM=FB,LRECL=80,BLKSIZE=0),
// SPACE=(27920,(29,6),RLSE),
// DISP=(NEW,CATLG,DELETE)
//SYSIN DD *
SORT FIELDS=COPY
OUTREC FIELDS=(1:C'r_file',7:SEQNUM,1,ZD,8:C'="',10:1,26,36:C'"',44X)
/*

SORT FIELDS=COPY
INREC BUILD=(C'r_file',SEQNUM,1,ZD,C'="',5,44,80:X)
OUTREC FINDREP=(STARTPOS=14,IN=C' ',OUT=C'"',DO=1)
This is not JCL by the way, They are SORT Control Cards. You find all the details in the manual for your site's SORT product.
INREC/OUTREC/OUTFIL can contain one only of BUILD, OVERLAY, IFTHEN or one or more IFTHENs. Two ways to do it since you want two functions used unconditionally is like the above, or with two IFTHEN=(WHEN=NONE with the function-use part of the IFTHEN.
Your records are not variable. They are fixed-length (or your existing code would not work) likely 80 bytes (if not 80, change the 80:X appropriately).
Bear in mind that you will get duplicate sequence-numbers with more than 10 input records (and the tenth is going to give you r_file0).
You could also look at SQZ (squeeze) since you have no embedded blanks in your data.

Try the following:
//STEP01 EXEC PGM=SORT
//SYSOUT DD SYSOUT=*
//SORTOUT DD SYSOUT=*
//SYSIN DD *
SORT FIELDS=COPY
INREC BUILD=(C'r_file',SEQNUM,3,CSF,C'="',1,68,C'"')
OUTREC BUILD=(1,80,SQZ=(SHIFT=LEFT))
/*
//SORTIN DD *
ABCDEFG_AAAAAA_BBB0001.csv
ABCDEFG_AAAAAA_BBB0002.csv
ABCDEFG_AAAAAA_BBB0003.csv
ABCDEFG_AAAAAA_0702.csv
ABCDEFG_AAAAAA_AAAYZ0702.csv
ABCDEFG_AAAAAA_BBB.csv
ABCDEFG_AAAAAA_XXXXXXXAMY0702.csv
ABCDEFG_AAAAAA_YZ0702.csv
ABCDEFG_AAAAAA_C0702.csv
ABCDEFG_AAAAAA_0702.csv
ABCDEFG_AAAAAA_702.csv
ABCDEFG_AAAAAA_02.csv
ABCDEFG_AAAAAA_2.csv
/*
//SORTMSG DD SYSOUT=*
//
SORTOUT will contain:
r_file1="ABCDEFG_AAAAAA_BBB0001.csv"
r_file2="ABCDEFG_AAAAAA_BBB0002.csv"
r_file3="ABCDEFG_AAAAAA_BBB0003.csv"
r_file4="ABCDEFG_AAAAAA_0702.csv"
r_file5="ABCDEFG_AAAAAA_AAAYZ0702.csv"
r_file6="ABCDEFG_AAAAAA_BBB.csv"
r_file7="ABCDEFG_AAAAAA_XXXXXXXAMY0702.csv"
r_file8="ABCDEFG_AAAAAA_YZ0702.csv"
r_file9="ABCDEFG_AAAAAA_C0702.csv"
r_file10="ABCDEFG_AAAAAA_0702.csv"
r_file11="ABCDEFG_AAAAAA_702.csv"
r_file12="ABCDEFG_AAAAAA_02.csv"
r_file13="ABCDEFG_AAAAAA_2.csv"
INREC builds a record something like:
r_file 10="ABCDEFG_AAAAAA_0702.csv "
The trick is to use CFS for the sequence number so that they are created with leading spaces, that way we do not get messed up trying to trim leading zeros as would required with ZD (SQZ with PREBLANK=C'0' to get rid of leading zeros just doesn't work - it gets rid of all zeros, including imbeded and trailing)
Next OUTREC squeezes the spaces out of the record leaving the file name as you require it to be presented.
Note: This solution assumes that your file names do not contain imbedded spaces. If they do, there is more tinkering to be done.

Related

JCL SORT SQZ shift left using build

I have 2 input files. I merged them using JOINKEY. The first input file countains a record count (lenght of 10 with leading zeroes).
The second file, is just comma separated fields between double quote (ex ''xxxx'',''yyyy'',etc).
The problem I got is I want to get rid of the leading zeroes from the records count of the first input file.
My count is 0000000012, I will like to have 12.
I don't understand how to handle it with SQZ function since it will be a variable lenght. How can we handle it in the OUTREC BUILD since it will not be a fix length of 10 anymore but a variable length if I use SQZ function?
Thanks for your help.
Here is the JCL:
//SORT100 EXEC PGM=SORT
//*
//SORTJNF1 DD *
Counter written records: 0000000012
//SORTJNF2 DD
DSN=INPUT.WITHOUT.COUNT,DISP=SHR
//SORTOUT DD DSN=OUTPUT.WITH.COUNT,
// DISP=
(,CATLG),RECFM=FB,LRECL=130,DATACLAS=DCPRXTP
//SYSIN DD *
SORT FIELDS=COPY
JOINKEYS FILE=F1,FIELDS=(131,8,A),SORTED,NOSEQCK
JOINKEYS FILE=F2,FIELDS=(131,8,A),SORTED,NOSEQCK
REFORMAT FIELDS=(F2:1,80,F1:30,10)
OUTREC BUILD=(1,80,81,10,C'," "',36X)
//JNF1CNTL DD *
INREC OVERLAY=(131:SEQNUM,8,ZD)
//JNF2CNTL DD *
INREC OVERLAY=(131:SEQNUM,8,ZD)
//*
//SYSOUT DD SYSOUT=*
//

mask the middle number using the JCL

i have requrement like this. could you please help any one help me?
i have 16 digit length fileld in my input file. if more than 13 digit length then first 4 digit as it is and last 4 digits as it is and mask the remaining the number
if less than 13 then keep the number as it is.
example
IP file
0000359681243354
0000359681243354
810176733
OP
00003̶5̶9̶6̶8̶1̶2̶4̶3354
00003̶5̶9̶6̶8̶1̶2̶4̶3354
810176733
Edit 1:
As per #NicC's suggestions in the Comments section of this answer, you may try the following code:
INREC IFTHEN=(WHEN=INIT,BUILD=(1,16)),
IFTHEN=(WHEN=(13,1,CH,NE,X'40',AND,13,1,CH,NE,X'00'),OVERLAY=(5:8C'-'))
SORT FIELDS=COPY
Alternatively, you may also try the following code.
Firstly, we use OUTFIL with FTOV,VLTRIM=C' ' to convert the input records to VB with an RDW. This is helpful because we can access the RDW to get the length of data upto first nonblank character. But, how to access the RDW? See next step.
Secondly, we build a record in T2 file, with actual data, followed by the length of the data (RDW is in 1,2,BI. Subtract 4 from 1,2,BI to get the length of data).
Lastly, use INREC IFTHEN to check if the length of the data is GE 13 or not. Format data as required.
Note: Following code needs to be changed for DD names, position of the 16 digit length field and other input fields in the record.
//STEP1 EXEC PGM=ICETOOL
//TOOLMSG DD SYSOUT=*
//DFSMSG DD SYSOUT=*
//IN DD DSN=... input file (FB)
//TEMP1 DD DSN=&&TEMP1,UNIT=SYSDA,SPACE=(CYL,(5,5)),DISP=(,PASS)
//TEMP2 DD DSN=&&TEMP2,UNIT=SYSDA,SPACE=(CYL,(5,5)),DISP=(,PASS)
//OUT DD DSN=... output file (FB)
//TOOLIN DD *
COPY FROM(IN) USING(CTL1)
COPY FROM(TEMP1) TO(TEMP2) USING(CTL2)
COPY FROM(TEMP2) TO(OUT) USING(CTL3)
/*
//CTL1CNTL DD *
OUTFIL FNAMES=TEMP1,FTOV,VLTRIM=C' '
/*
//CTL2CNTL DD *
OUTFIL FNAMES=TEMP2,VTOF,
BUILD=(5,16,X,1,2,BI,SUB,+4,TO=ZD,LENGTH=2)
/*
//CTL3CNTL DD *
INREC IFTHEN=(WHEN=(18,2,ZD,GE,13),BUILD=(1:1,4,5:8C'-',13:13,4),
IFTHEN=(WHEN=(18,2,ZD,LT,13),BUILD=(1:1,6))
SORT FIELDS=COPY

JOINKEYS REFORMAT field for a VB file for F2 is not working

I am trying to right and left outer join these two RECFM VB files but
I don't get anything from the F2 file.
//STEP2000 EXEC PGM=SORT
//* JOIN
//*
//SYSOUT DD SYSOUT=*
//*
//SORTJNF1 DD DSN=YXX122.TEMP.EXPORT.TYPEN,
// DISP=SHR
//*
//SORTJNF2 DD DSN=YXX122.TEMP.EXPORT.TYPEC,
// DISP=SHR
//*
//SORTOUT DD DSN=YXX122.DYXX122.EXPORT.XSUM,
// DISP=(NEW,CATLG,DELETE),
// UNIT=(DEV,2),
// SPACE=(CYL,(150,20),RLSE),
// DCB=(RECFM=VB,LRECL=304,BLKSIZE=0)
//*
//SYSIN DD *
SORT FIELDS=COPY
JOINKEYS FILES=F1,
FIELDS=(13,4,A,18,5,A,17,1,A,23,1,A,33,8,A,41,4,A)
JOINKEYS FILES=F2,
FIELDS=(13,4,A,18,5,A,17,1,A,23,1,A,33,8,A,41,4,A)
JOIN UNPAIRED,F1,F2
REFORMAT FIELDS=(F1:5,300)
OUTFIL FTOV
//
The problem is I can't find how the REFORMAT FIELDS the F2 file.
I tried with REFORMAT FIELDS=(F1:5,300,F2:5,300) but the outfile was with a length of 600.
I will like to know how to have both file F1 and F2 in my SORTOUT file with a VB length 304.
Any idea on how to fix this problem?
It turns out you have DFSORT, not SyncSORT which makes things simpler, as you can definitely use the Match Marker ? in the REFORMAT statement. Up-to-date SyncSORT may have the Match Marker as an undocumented feature.
Putting all the unmatched records on one OUTFIL may be confusing (you won't know which input they have come from).
This conceptualises your join (where the Output is the joined data, and b represents blank).
F1
A
C
E
F2
B
C
F
Output
Ab
bB
Eb
bF
So if you want B and F you need to specify some data from F2. You also need to identify the "blanks" so that you know which part of the REFORMAT record currently has data in (DFSORT has a Match Marker for this, SyncSORT does not).
For that you need to identify one byte which can never be blank in the record. If that is not possible, one byte which can never be another given value (which you specify on FILL= on the REFORMAT). Failing that, two or more bytes with the same characteristics. As a final fail-safe you can check the entire part of the REFORMAT record from one file or the other for blank.
Since you want V-type output, you could make your REFORMAT record variable:
REFORMAT FIELDS=(F1:1,4,?,F1:5,300,F2:5)
And use VLTRIM on OUTFIL.
Or fixed:
REFORMAT FIELDS=(F1:5,300,F2:5,300)
And use FTOV with VLTRIM on OUTFIL.
Then you need some code, which tests the byte/bytes/partofdata you have chosen for being space/thevalueyouhavechosen and uses BUILD to create a record which contains the data you want (plus trailing blanks/values which will be killed by the VLTRIM).
IFTHEN=(WHEN=(logicalexpression),
BUILD=(1,4,5,300)),
IFTHEN=(WHEN=NONE,
BUILD=(1,4,305,300))
Or
IFTHEN=(WHEN=(logicalexpression),
BUILD=(1,300)),
IFTHEN=(WHEN=NONE,
BUILD=(301,300))
Here's some code which does what you want. Probably. I can't test it with SyncSORT.
Data:
F1
A 11111111111111111111111111111111111
C 2222222222222222222222
E 3
F2
B 4444444444444444
C 55555555555555555555555555
F 6666666666666
Code:
OPTION COPY
JOINKEYS F1=INA,FIELDS=(5,1,A),SORTED,NOSEQCK
JOINKEYS F2=INB,FIELDS=(5,1,A),SORTED,NOSEQCK
JOIN UNPAIRED,F1,F2,ONLY
REFORMAT FIELDS=(F1:1,4,F1:5,76,F2:5)
OUTFIL FNAMES=EXT,VLTRIM=C' ',
IFTHEN=(WHEN=(81,1,CH,EQ,C'2'),
BUILD=(1,4,82)),
IFTHEN=(WHEN=NONE,
BUILD=(1,4,5,76))
The Match Marker, ?, will be set to 1 for unmatched F2, 2 for unmatched F2 and B for matched records (which you won't get, because of the ONLY on the JOIN statement).
This presumes your data is already in sequence. Remove the SORTED,NOSEQCK for data which is not in sequence.
I've used an LRECL of 80 and a simple key and some simple data.
Output:
For EXT:
A 11111111111111111111111111111111111
B 4444444444444444
E 3
F 6666666666666
SORTOUT would show the unchanged REFORMAT record. That is for you to see how it works. You can remove the FNAMES=EXT or remove the SORTOUT from the JCL when you understand everything.
The F1:1,4 ensures that the REFORMAT record is variable-length. The 5,300 should use blank-padding for shorter records. That's why you need the VLTRIM later. The F2:5 says "file two, position five, to the end of the file two record".
If your data can have genuine trailing blanks, you'll have to use FILL= and VLTRIM= for the same character.
IFTHEN=(WHEN=(logicalexpression) processing finishes when an IFTHEN is true. So the combination in the code is effectively an IF/ELSE.
See also this, Compare two files and write it to "match" and "nomatch" files and Sync sort, Unpaired records of File1 have spaces for no records in F2 file. Can we replace those specific column's spaces by ZEROS? for further examples.

Sed function (Bash) - Changing way of commented lines

I have such task to do but I have no idea how to write it with sed function.
I have to change the way on commenting in a file from:
//something6
//something4
//something5
//something3
//something2
to
/*something6
* something4
* something5
* something3
* something2*/
from
//something6
//something4
//something5
//something3
//something2
to
/*something6
something4
something5
something3
something2*/
from
/*something6
* something4
* something5
* something3
* something2*/
to
//something6
//something4
//something5
//something3
//something2
from
/*something6
something4
something5
something3
something2*/
to
//something6
//something4
//something5
//something3
//something2
Those 4 patterns must be made by sed function (I guess but not sure about that).
Tried doing it but without luck. I can replace single words to other ones but how to change the way of commenting? No clue. Would be very gratefull for help and assisstance.
Given that the task is:
Please write a script that allows to change style of comments in source files for example : /* .... */ goes to // .... The style of comment is an argument of the script.
I have tried to use just typical:
sed -i 's/'"$lookingfor"'/'"$changing"'/g' $filename
In this context, either $lookingfor or $changing or both will contain slashes, so that simple formulation doesn't work, as you correctly observe.
The conversion of // comments to /* comments is easy as long as you know that you can choose an arbitrary character to separate the sections of the s/// command, such as %. So, for example, you could use:
sed -i.bak -e 's%// *\(.*\)%/*\1 */%'
This looks for a double-slash followed by zero or more spaces and anything and converts it to /* anything */.
The conversion of /* comments is much harder. There are two cases to be concerned about:
/* A single line comment */
/*
** A multiline comment
*/
That's before you get into:
/* OK */ "/* OK */" /* Really?! */
which is a single line containing two comments and a string containing text that outside a string would look like a comment. This I am studiously ignoring! Or, more accurately, I am studiously deciding that it will be OK when converted to:
// OK */ "/* OK */" /* Really?!
which isn't the same at all, but serves you right for writing convoluted C in the first place.
You can deal with the first case with something like:
sed -e '\%/\*\(.*\)\*/% { s%%//\1%; n; }'
I have the grouping braces and the n command in there so that single line comments don't also match the second case:
-e '\%/\*%,\%\*/% {
\%/\*% { s%/\*\(.*\)%//\1%; n; }
\%\*/% { s%\(.*\)\*/%//\1%; n; }
s%^\( *\)%\1//%
}'
The first line selects a range of lines between one matching /* and the next matching */. The \% tells sed to use the % instead of / as the search delimiter. There are three operations within the outer grouping { … }:
Convert /*anything into //anything and start on the next line.
Convert anything*/ into //anything and start on the next line.
Convert any other line so that it preserves leading blanks but puts // after them.
This is still ridiculously easy to subvert if the comments are maliciously formed. For example:
/* a comment */ int x = 0;
is mapped to:
// a comment int x = 0;
Fixing problems like that, and the example with a string, is something I'd not even start trying in sed. And that's before you get onto the legal but implausible C comments, like:
/\
\
* comment
*\
\
/
/\
/\
noisiness \
commentary \
continued
Which contains just two comments (but does contain two comments!). And before you decide to deal with trigraphs (??/ is a backslash). Etc.
So, a moderate approximation to a C to C++ comment conversion is:
sed -e '\%/\*\(.*\)\*/% { s%%//\1%; n; }' \
-e '\%/\*%,\%\*/% {
\%/\*% { s%/\*\(.*\)%//\1%; n; }
\%\*/% { s%\(.*\)\*/%//\1%; n; }
s%^\( *\)%\1//%
}' \
-i.bak "$#"
I'm assuming you aren't using a C shell; if you are, you need more backslashes at the ends of the lines in the script so that the multi-line single-quoted sed command is treated correctly.

Automatically increment filename VideoWriter MATLAB

I have MATLAB set to record three webcams at the same time. I want to capture and save each feed to a file and automatically increment it the file name, it will be replaced by experiment_0001.avi, followed by experiment_0002.avi, etc.
My code looks like this at the moment
set(vid1,'LoggingMode','disk');
set(vid2,'LoggingMode','disk');
avi1 = VideoWriter('X:\ABC\Data Collection\Presentations\Correct\ExperimentA_002.AVI');
avi2 = VideoWriter('X:\ABC\Data Collection\Presentations\Correct\ExperimentB_002.AVI');
set(vid1,'DiskLogger',avi1);
set(vid2,'DiskLogger',avi2);
and I am incrementing the 002 each time.
Any thoughts on how to implement this efficiently?
Thanks.
dont forget matlab has some roots to C programming language. That means things like sprintf will work
so since you are printing out an integer value zero padded to 3 spaces you would need something like this sprintf('%03d',n) then % means there is a value to print that isn't text. 0 means zero pad on the left, 3 means pad to 3 digits, d means the number itself is an integer
just use sprintf in place of a string. the s means String print formatted. so it will output a string. here is an idea of what you might do
set(vid1,'LoggingMode','disk');
set(vid2,'LoggingMode','disk');
for (n=1:2:max_num_captures)
avi1 = VideoWriter(sprintf('X:\ABC\Data Collection\Presentations\Correct\ExperimentA_%03d.AVI',n));
avi2 = VideoWriter(sprintf('X:\ABC\Data Collection\Presentations\Correct\ExperimentB_002.AVI',n));
set(vid1,'DiskLogger',avi1);
set(vid2,'DiskLogger',avi2);
end

Resources