I am learning mainframe programming (TSO, JCL, COBOL...) and I came across a problem I can't seem to solve, yet.
I have a file with records that contain 3 fields and I want to use 'outfil' to include only certain records in the output file.
My JCL is the following:
000001 //C8112J1 JOB (12345678),C8112,MSGCLASS=H,
000002 // MSGLEVEL=(1,1),NOTIFY=C8112
(...)
000007 //DSNDEL EXEC PGM=IDCAMS
000008 //SYSPRINT DD SYSOUT=*
000009 //SYSIN DD *
000010 DELETE C8112.DATA.FX.SORT
000011 SET MAXCC = 0
000012 /*
(...)
000014 //* --------- S O R T -----------
000020 //MYSORT EXEC PGM=SORT,COND=(0,NE)
000021 //SORTIN DD DSN=C8112.DATA.FX,DISP=SHR
000022 //SORTOUT DD DSN=C8112.DATA.FX.SORT,
000023 // SPACE=(TRK,(100,100),RLSE),
000024 // DISP=(NEW,CATLG,DELETE),
000025 // DCB=(LRECL=61,RECFM=FB)
000026 //SYSIN DD *
000027 SORT FIELDS=COPY
000028 //* SORT FIELDS=(1,18,CH,A) OU SORT=COPY
000029 OUTFIL INCLUDE=(1,18,CH,EQ,C'ANDRADE,LUISA')
...
000031 //* OUTREC=(39,41,1,18,19,38)
000032 END
000033 //SYSOUT DD SYSOUT=*
000034 /*
and the output file:
000001 LUIS,CARLOS AV.PAULO VI RMR
000002 DIAS,PATTY RUA PEDRO CALMON LX
000003 ANDRADE,LUISA AV. DA LIBERDADE LX (1)
000004 SILVA,TONI PALACIO VALMOR LX
000005 ANDRADE,LUISA AV. DA BOAVISTA POT (2)
000006 CISCO,MANUEL QUINTA DA AVELEDA TC
Wasn't I supposed to get only the records (1) and (2)?! By the way, I also tried the 'outrec' to change the bytes organization but it didn't work too.
Can anyone give me a hint on what I am doing wrong?
You can use a single * as a comment delimiter in DFSORT rather than //* - which, as has already been pointed out terminates the SYSIN just as if you'd coded /. (You indeed have a / later on, I notice.)
Try the following:
000020 //MYSORT EXEC PGM=SORT,COND=(0,NE)
000021 //SORTIN DD DSN=C8112.DATA.FX,DISP=SHR
000022 //SORTOUT DD DSN=C8112.DATA.FX.SORT,
000023 // SPACE=(TRK,(100,100),RLSE),
000024 // DISP=(NEW,CATLG,DELETE),
000025 // DCB=(LRECL=61,RECFM=FB)
000026 //SYSIN DD *
000027 SORT FIELDS=COPY
000029 OUTFIL INCLUDE=(1,18,CH,EQ,C'ANDRADE,LUISA')
...
000032 END
000031 /*
000033 //SYSOUT DD SYSOUT=*
000034 //*
SORT SYSIN does not allow for "comments". Those lines you thought might have been comments really are't. They terminate the SYSIN DD statements. Basically I just got rid of:
//* SORT FIELDS=(1,18,CH,A) OU SORT=COPY
and
//* OUTREC=(39,41,1,18,19,38)
then it all works fine. Remember, a JCL comment is not a SORT comment! The stuff after a DD * are data, not JCL source statements so JCL type commenting conventions do not work here. The DD * is typically terminated by a line starting with: /*.
Related
How to write JCL to send an email but the content(Data) should be picked from another PDS/member. If anyone can let me know the JCL for what I need will be helpful.**
Here is an old example but it sounds like what you are looking for. It uses IEBGENER to send an e-mail. (I didn't write the content so complain to IBM if you don't like it.)
This is from Knowledge Center
//BATSMTP JOB (userid,nn),MSGCLASS=B,PRTY=12,MSGLEVEL=(2,1)
//*
//* Store message in a PDS
//*
//PUTMSG EXEC PGM=IEBGENER
//SYSIN DD DUMMY
//SYSUT1 DD *
HELO YOURMVS
MAIL FROM:<CAROL#YOURMVS>
RCPT TO:<msgs#rsch.our.edu>
RCPT TO:<alice#ai.our.edu>
DATA
Date: Thur, 26 Mar 92 21:48:57 EST
From: Carol <CAROL#YOURMVS>
To: <msgs#rsch.your.edu>
Cc: <alice#ai.your.edu> Subject: update
Mike: Cindy stubbed her toe. Bobby went to
baseball camp. Marsha made the cheerleading team.
Jan got glasses. Peter has an identity crisis.
Greg made dates with 3 girls and couldn't
remember their names.
.
QUIT
/*
//SYSUT2 DD DSN=MYPDS.OF.MESSAGES(MSGID1),DISP=SHR
//*
//SYSPRINT DD SYSOUT=A
//*
//* Send Message from placed in PDS in prior step
//*
//SENDMSG EXEC PGM=IEBGENER
//SYSIN DD DUMMY
//SYSUT1 DD DSN=MYPDS.OF.MESSAGES(MSGID1),DISP=SHR
//*
//SYSUT2 DD SYSOUT=(B,smtp)
//* | v
//* v SMTP address space name for external writer
//* SYSOUT class
//SYSPRINT DD SYSOUT=A
Concatenate your PDS member with your other stuff in SYSUT1 e.g.
//SYSUT1 DD *
your stuff here
// DD DSM=your.pds(member),DISP=SHR
`You may need other stuff after your member - just concatenate more DD *.
Remember that your PDS data must be LRECL=80
Concatenation works fine if all source data has the same DCB. If not, use SORT to create a temporary VB file with LRECL = your longest LRECL + 4. E.g., if you need to merge 80-byte PDS data with 133-byte reports, use SORT to copy everything to a VB file with LRECL=137 and send that to the sysout.
If anyone is still reading, I'd like to find out how to set up a BCC: recipient. Apparently, only TO: and CC: are supported.
Sample JCL for building the VB file:
000007 //BUILD PROC
000008 //SORT EXEC PGM=SORT
000009 //SORTOUT DD DISP=(MOD,PASS),DSN=&&EMAIL,UNIT=SYSDA,
000010 // SPACE=(CYL,(1,1)),DCB=(RECFM=VB,LRECL=430,BLKSIZE=27998)
000011 //*
000012 //SYSIN DD *
000013 OPTION COPY
000014 OUTFIL FNAMES=SORTOUT,FTOV
000015 //SORTWK01 DD SPACE=(CYL,(5,5),RLSE),UNIT=SYSDA
000016 //SORTWK02 DD SPACE=(CYL,(5,5),RLSE),UNIT=SYSDA
000017 //SORTWK03 DD SPACE=(CYL,(5,5),RLSE),UNIT=SYSDA
000018 //SORTWK04 DD SPACE=(CYL,(5,5),RLSE),UNIT=SYSDA
000019 //SORTWK05 DD SPACE=(CYL,(5,5),RLSE),UNIT=SYSDA
000020 //SYSPRINT DD SYSOUT=*
000021 //SORTMSG DD SYSOUT=*
000022 //SYSDBOUT DD SYSOUT=*
000023 //SYSUDUMP DD SYSOUT=*
000024 //SYSOUT DD SYSOUT=*
000025 //*
000026 // PEND
000027 //*
000028 //HEADER EXEC BUILD --------------------------------------------
000029 //SORT.SORTOUT DD DISP=(,PASS)
000030 //SORT.SORTIN DD *
000031 HELO MVSOSA.CAPITALONE.COM
000032 MAIL FROM:<{from address}>
000033 RCPT TO:<{to address}>
000034 DATA
000035 FROM: <{from address}>
000036 TO:<{to address>
000037 SUBJECT: {subject}
000038 MIME-Version: 1.0
000039 Content-type: multipart/mixed; boundary="=_boundary"
000040
000041 These comments don't show up in the email.
000042
000043 Note: After each boundary, one or more records describes the
000044 content that follows. After the last "Content-..." record
000045 in each section, be sure to have a blank line.
000046
000047
000049
000050 See also https://www.ibm.com/support/pages/
000051 outbound-email-attachments-using-smtp-or-cssmtp-zos
000052
000053 --=_boundary
000054 Content-Type: text/plain;
000055
000056 Good day.
000057 Please find attached the reports you wanted.
000058
000059 Regards -- Me
000060
000061 --=_boundary
000062 Content-Type: application;
000063 Content-Disposition: attachment; filename={report name}.txt
000064 Content-Transfer-Encoding: 8bit;
000065
000066 /*
000067 //STATRPT EXEC BUILD
000068 //SORT.SORTIN DD DISP=SHR,DSN={file name with report}
000069 //*
000070 //*
000071 //*-------------------------------------------------------------*
000072 //EMAILSTP EXEC PGM=IEBGENER
000073 //SYSPRINT DD SYSOUT=*
000074 //SYSIN DD DUMMY
000075 //SYSUT2 DD SYSOUT=(B,SMTP)
000076 //SYSUT1 DD DISP=(OLD,PASS),DSN=&&EMAIL
000077 //*
000078 //*==============================================================
000079 //*
I get the following output when I disassembled a simple ARM binary file using the command "arm-linux-gnueabihf-objdump -d a.out"
00008480 <_start>:
8480: f04f 0b00 mov.w fp, #0
8484: f04f 0e00 mov.w lr, #0
8488: bc02 pop {r1}
848a: 466a mov r2, sp
What do different columns represent here? For example, 8480 and f04f 0b00 (from the 2nd line of code)
The first column is the address of the code in memory. 0x8480 means the memory address of this piece of code is 0x8480.
The second column is the hex representation of the code. f04f 0b00 means from memory address 0x8480(included) to 0x8484(excluded), there are 4 bytes of code, f0, 4f, 0b, 00.
The remaining is the disassembled code. They are disassembled from the code in second column.
I'd like to do the equivalent of what I can do in C via:
pread(fdesc, tgtbuf, size, file_offset);
or the combination:
lseek(fd, file_offset, SEEK_SET);
read(fd, tgtbuf, size)
as a shell command.
For some sizes/offsets, one can use:
dd if=file bs=size skip=$((file_offset/size)) count=1
That works ... but only if file_offset is divisible by size. Which isn't sufficient for my usecase, unfortunately.
The device I'm attempting to read from is "blocked" in 8-byte units for read but allows (requires) byte offsets for seek. dd always reads in units of bs/ibs but also always seeks in these units, which in my case is mutually exclusive.
I know I can do this via perl/python/C/... - but is there a way to do this from a simple shell script ?
EDIT: Since it was suggested to use dd bs=1 count=8 ... here - NO THIS DOES NOT WORK. strace it and you'll see that this does:
$ strace -e lseek,read dd if=/dev/zero bs=1 skip=1234 count=8
[ ... ]
lseek(0, 1234, SEEK_CUR) = 0
read(0, "\0", 1) = 1
read(0, "\0", 1) = 1
read(0, "\0", 1) = 1
read(0, "\0", 1) = 1
read(0, "\0", 1) = 1
read(0, "\0", 1) = 1
read(0, "\0", 1) = 1
read(0, "\0", 1) = 1
Which is not what I need - it must be a single read().
Edit2:
The device (/dev/cpu/<ID>/msr) I'm trying to read from is strange in the sense that the offset is treated as an index number, but you'll always have to read eight bytes else the driver gives EINVAL on read.
But every index returns a different 8-byte value, so you cannot "reconstruct" reading from offset x+1 by reading x and x+8 and extracting the bytes. This is highly unusual ... but it's the way /dev/cpu/<ID>/msr works in Linux.
bs=size : size can be 1 byte, or more ... (it is an indication to dd on how to access the device, but for a file you can use whatever you need.. it's usually more efficient reading blocks of larger sizes, though)
try:
dd if=file bs=1 skip=whateveryouneed count=8 #to read 8 bytes starting at whateveryouneed
if (contrary to what you seem to state in the question) you can only seek to multiple of 8 (and read 8 bytes from there):
dd if=file bs=8 skip=X count=1 #to read 8 bytes starting at whateveryouneed
#X being: whateveryouneed / 8 (ex: echo "4000 / 8" | bc )
(as I say in my comment, I really have troubles to imagine a device that allows you to seek anywhere, and force you to read 8 bytes from anywhere, if anywhere is not also a multiple of 8 ... but, hey, anything is possible ^^ If so, you'll need another tool than dd, i'm afraid)
If it really is so weird: extract 2 blocks of 8 bytes around the adress you need, and then extract the exact part you need from it :
blockoffset=$(($address/8))
blockstart=$(($blockoffset*8))
shift=$(($address - $blockstart))
if [ "$shift" -eq 0 ]
dd if=file bs=8 skip=$blockoffset count=1 > final
else
dd if=file bs=8 skip=$blockoffset count=2 > bigger #we read 2 blocks from blockoffset
dd if=bigger bs=1 skip=$shift count=8 > final
fi
Given the amount of trouble you've gone to already, just put the C code into an executable program. Or get really ambitious and make the program into a bash extension with "enable -f pread.so pread"
http://www.gnu.org/software/bash/manual/html_node/Bash-Builtins.html
likely over the top. A separate program is easier.
I'm trying to inspect a kernel module that utilizes usb, and so from the module itself I'm writing a message to ftrace using trace_printk; and then I wanted to inspect when does a USB Bulk Out URB Submit appear in the system after that.
The problem is that on my Ubuntu Lucid 11.04 (kernel 2.6.38-16), there are only local and global clocks in ftrace - and although their resolution is the same (microseconds) as the timestamps by usbmon, their values differ significantly.
So not knowing any better (as I couldn't find anywhere else talking about this), what I did was attempt to redirect usbmon to trace_marker, using cat:
# ... activate ftrace here ...
usbpid=$(sudo bash -c 'cat /sys/kernel/debug/usb/usbmon/2u > /sys/kernel/debug/tracing/trace_marker & echo $!')
sleep 3 # do test, etc.
sudo kill $usbpid
# ... deactivate ftrace here...
... and then, when I read from /sys/kernel/debug/tracing/trace, I get a log with problematic timestamps (see below). So what I'd like to know is:
Is there a way to make usbmon have it's messages appear directly in /debug/tracing/trace, instead of in /debug/usb/usbmon/2u ? (not that I can see, but I'd like to have this confirmed)
If not, is there a better way to "directly" redirect output of /sys/kernel/debug/usb/usbmon/2u without any possible overhead/buffering issues of cat and/or shell redirection?
If not, is there some sort of an algorithm, where I could use the extra usbmon timestamp, to somehow "correct" the position of these events in the kernel timestamp domain? (see example below)
Here is a brief example snippet of a /sys/kernel/debug/tracing/trace log I got:
<idle>-0 [000] 44989.403572: my_kernel_function: 1 00 00 64 1 64 5
<...>-29765 [000] 44989.403918: my_kernel_function: 1 00 00 64 2 128 2
<...>-29787 [000] 44989.404202: 0: f1f47280 3237249791 S Bo:2:002:2 -115 64 = 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
<...>-29787 [000] 44989.404234: 0: f1f47080 3237250139 S Bo:2:002:2 -115 64 = 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
<idle>-0 [000] 44989.404358: my_kernel_function: 1 00 00 64 3 192 4
<...>-29787 [000] 44989.404402: 0: f1f47c00 3237250515 S Bo:2:002:2 -115 64 = 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
So when the kernel timestamp is 44989.404202, the usbmon timestamp is 3237.249791 (= 3237249791/1e6); neither the seconds nor the microseconds part match. To make it a bit easier on the eyes, here's the same snippet with only time information remaining:
(1) 44989.403572 MYF 0
(2) 44989.403918 MYF 0.000346
(3) 44989.404202 USB | 0 3237.249791 0
(4) 44989.404234 USB | 0.000032 3237.250139 0.000348
(5) 44989.404358 MYF 0.000440 | |
(6) 44989.404402 USB 0.000168 3237.250515 0.000376
So judging by the kernel timestamps, 32 μs expired between event (3) and event (4) - but judging by the usbmon timestamps, 348 μs expired between the same events! Whom to trust now?!
Now, if we assume that the usbmon timestamps are more correct for those messages, given they got "printed" before they ended up in the ftrace buffer to begin with - we could assume that the first usb message (3) may have been scheduled right after (1) executed, but something preempted it - and so the second USB message (4) triggered the "printout" (or rather, "entry") of both (3) and (4) in the ftrace buffer (which is why their kernel timestamps are so clcse together?)
So, if I assume (4) is the more correct one, I can try push (3) back for 348 μs:
(1) 44989.403572 MYF 0
(3) 44989.403854 USB | 0 3237.249791 0
(2) 44989.403918 MYF 0.000346 | |
(4) 44989.404234 USB | 0.000380 3237.250139 0.000348
(5) 44989.404358 MYF 0.000440 | |
(6) 44989.404402 USB 0.000168 3237.250515 0.000376
... and that sort of looks better (also USB now fires 282 μs, 316 μs, and 44 μs after MYF) - for first and second MYF/USB pairs (if that is indeed how they behave); but then the third step doesn't really match, and so on... Cannot really think of an algorithm to help me adjust the USB events position according to the data in the usbmon timestamp...
While the best approach for redirecting usbmon output to ftrace is still an open question, I got an answer about correlating their timestamps from this thread:
Using both usbmon and ftrace? [linux-usb mailing list]
You can call the following
subroutine to get a usbmon-style timestamp value, which can then be
added to an ftrace message or simply printed in the kernel log:
#include <linux/time.h>
static unsigned usbmon_timestamp(void)
{
struct timeval tval;
unsigned stamp;
do_gettimeofday(&tval);
stamp = tval.tv_sec & 0xFFF;
stamp = stamp * 1000000 + tval.tv_usec;
return stamp;
}
For example,
pr_info("The usbmon time is: %u\n", usbmon_timestamp());
Is there a parameter for sort programme to exclude (ignore) the first line of a file from sorting in jcl.
Thanks,
If your Sort level is up-to-date, you can use DATASORT. Borrowed from an answer by Frank Yaeger, via google.
//S1 EXEC PGM=ICETOOL
//TOOLMSG DD SYSOUT=*
//DFSMSG DD SYSOUT=*
//IN DD *
FIRST
AAAA
CCCC
DDDD
FFFF
GGGG
//OUT DD SYSOUT=*
//TOOLIN DD *
DATASORT FROM(IN) TO(OUT) FIRST USING(CTL1)
/*
//CTL1CNTL DD *
SORT FIELDS=(1,4,CH,A)
/*
You can try the following:
//SYSIN DD *
SORT FIELDS=...
SKIPREC=1
/*
You may have to do a sort copy and in a subsequent JCL step do the sort.
Try using the ICETOOL SUBSET operator. Here is a really simple example:
//STEP01 EXEC PGM=ICETOOL
//TOOLMSG DD SYSOUT=*
//DFSMSG DD SYSOUT=*
//IN1 DD * -- Unsorted input data...
FIRST LINE
5
6
7
1
4
/*
//TOOLIN DD * -- ICETOOL commands
SUBSET FROM(IN1) TO(OUT1) REMOVE INPUT HEADER
/*
//OUT1 DD SYSOUT=* -- Sorted output goes here
Upon completion OUT1 contains:
1
4
5
6
7
which are the data from IN1, sorted, missing the first input line.
The DFSORT/ICETOOL manual can be found here and the
ICETOOL SUBSET operator is
documented here
edit
Based on your comment to Gilbert, I suggest using a second job step to
IDCAMS REPRO (copy) the first record from the original input file and then concatenate it to the ICETOOL output. The JCL is relatively straight forward.