we are sending data trough MQ from a z/OS/CICS system to an AS400.
Original encoding of the message is CCSID 500 with a MQSTR Format. The client application is getting the message with the CONVERT option and CCSID 819.
Data is almost converted correctly except for the end of line caracter. Any idea?
The z/OS is sending 0D (CR) as end of line caracter. If they had 0D+0A (CR+LF), CCSID automatically change from 500 to 437, and the end of line still ain't right on the client side.
Have already you tried only "0A" (LF)? Have you tried without an expliciet CONVERT? We've never had any issues with this part of the conversion and our machine talks with windows, AIX, Linux and so. Almost all the time we do rely on automatic conversions.
Also, you may want to check the USER of your job that picks the MQ message from the Queue. Most of the time the USER uses the system attributes (as defined in the appropriate SYSVAL). Maybe this userprofile has been configured very specific and different from the SYSVAL which stops the iSeries in doing the correct automatic conversion.
As a simple test, FTP your data to the iSeries. That will also force an automatic conversion that uses the system tables (WRKTBL). That may show an error in that configuration.
We had a problem with the CCSID of the queue manager not being set correctly and the wrong conversion table used.
Related
I am trying to validate a json using ValidateRecord Processor via Avroschemaregistry. I need to store validation error message into a sql table, so i tried to capture the error message in attribute but i am unable to capture the error message in attribute, any idea how to do it
After your ValidateRecord Processor, you can choose to route flow files which are 'invalid' to a separte log and route them to your sql table, you can do the same if they 'fail'. I am assuming from the 'error message' you mean the 'bullentin' which would occur when the Processor can neither validate or invalidate the flow file based on your schema.
A potential solution to this is to use the SiteToSiteBulletinReportingTask
Screenshot of SiteToSiteBulletinReportingTask
You can build a dataflow to receive these bulletin events, manipulate them as you want and store them in a location of your choice for your auditing needs.
From the sounds of it, the SiteToSiteBulletinReportingTask should be able to achieve what you want. To implement this, add a iteToSiteBulletinReportingTask to the 'Reporting Tasks' in the NiFi Settings: Reporting Tasks in NiFi Settings
You can name your input port and have it flow towards your SQL store and you should have what you're after.
You need to allow NiFi nodes to receive data via site-to-site on the input port and you also need to grant the correct permissions on the root process group so the nodes are able to see the component, view and modify the data.
Side note: I would usually log everything, and have all failures and invalid route to log files, which I put to store, e.g. HBase/SQL. One suggestion I've seen is configure the logging subsystem to additionally send specific error categories to your destination of choice (e.g. active notification vs passive parsing of logs). NiFi is leveraging a very flexible logback system (an evolution of log4j). The best part - changes to the $NIFI_HOME/conf/logback.xml configuration file do not require an instance restart, will be picked up within 30 seconds or less.
I have a clarion application running in Win Server 2016 talking to a sybase DB, over the past few weeks we find the application gets frozen for different users at a given time. However the user can leave the session as such and start a new one and that works good. The users are known to use multiple instances of the same application in one remote server or on multiple servers. Having said that, I wanted to get more information on the freezeup and looked through the application event logs in the system where I see explorer.exe crashes but these correlate to the time of occurrence of the issue at certain times but not always, checked the DB transaction logs from Sybase and I do not find any crashes, errors or stuck connections. Having said that, since I have exhausted all possible options I am reaching out to you guys to know if there are any other places that i can look for to gather more information.
I would love to know of any application / tools that we could use to gather logs of a frozen clarion application on windows. Also good to know if anyone has faced such a situation and where and how have you guys looked into the issue.
Thanks in advance for your help in this.
Clarion's runtime library and database drivers expect a persistent connection. Disconnects that are normal with remote ODBC can cause a problem (including app hangs) unless you test for them at the ABC file mgr level and reconnect, or use similar steps to test and recover.
If you're looking for specifics about what's going on between the driver and the SQL backend, I suggest using Clarion's database driver trace facilities. From the help topic: "Logging Driver I/O for debugging":
To view the trace details in debugview, name the target trace file "DEBUG:"
Logging opens the named logfile for exclusive access. If the file exists, the new log data is appended to the file.
On Demand Logging
For on-demand logging you can use property syntax within your program to conditionally turn various levels of logging on and off. The logging is effective for the target table and any view for which the target table is the primary table.
file{PROP:Profile}=Pathname !Turns Clarion I/O logging on
file{PROP:Profile}="DEBUG:" !Turns Clarion I/O logging on and
!sends output via OutputDebugString()
!(viewable via debugview, etc)
file{PROP:Profile}='' !Turns Clarion I/O logging off
PathName = file{PROP:Profile} !Queries the name of the log file
file{PROP:Log}=string !Writes the string to the log file
file{PROP:Log}="DEBUG:" !Writes the string to the log file
file{PROP:Details}=1 !Turns Record Buffer logging on
fFile{PROP:Details}=0 !Turns Record Buffer logging off
where Pathname is the full pathname or the filename of the log file to create. If you do not specify a path, the driver writes the log file to the current directory.
You can also accomplish on demand logging with a SEND() command and the LOGFILE driver string. See LOGFILE for more information.
Example I use frequently, which was based on the help above:
SYSTEM{PROP:DriverTracing} = '1'
CRMNotes{PROP:TraceFile} = 'DEBUG:'
CRMNotes{PROP:Details}=1
CRMNotes{PROP:Profile}= 'DEBUG:'
CRMNotes{PROP:LogSQL} = 1
I'm working on an application that connects to IBM MQ 8.0 using the C API, calling the MQCONNX function. I run several instances of this application at the same time, and when I open MQ Explorer to list application connections for the queue manager, I see an entry for each connection. However, the entries all have the same "app name" (the name of the executable), making it difficult to distinguish which instances are or are not connected.
Is it possible to change the value that ends up being displayed as the "app name" in MQ Explorer? I'd like to pass an instance identifier from my application, but looking through the documentation for MQCONNX none of the options seem to apply.
In the IBM MQ v8.0 Knowledge Center page "DISPLAY CONN" under APPLTAG it shows how MQ determines the value. You did not state what OS your client app is running on, the information for Unix and Windows is:
UNIX process
Note
On HP-UX if the process name exceeds 14 characters, only the first 14 characters are shown.
On Linux and Solaris, if the process name exceeds 15 characters, only the first 15 characters are shown.
On AIX®, if the process name exceeds 28 characters, only the first 28 characters are shown.
Windows process
Note
This consists of the full program path and executable file name. If it is more than 28 characters long, only the last 28 characters are shown.
As mentioned by #Attila Repasi you can change the ApplName in the MQMD of each message using Set All Context, this does not impact the "app name" displayed in MQ Explorer.
One work around I thought of if you are running on Unix is to create separate symlinks to your program each with different names. If you then run each copy using the different symlink names, this should then reflect as the "app name" displayed by MQ Explorer.
A second work around that I tested on Linux that works without the need for symlinks is setting the process name in your program. I first tried to write over the argv[0], but I found while this changes the process name in a ps output it does not change the APPLTAG value that MQ displays. The working way that I found in Linux is below, this must be called prior to MQCONN.
char *process_name = "samplename\0";
prctl(PR_SET_NAME,process_name,NULL,NULL,NULL);
#Fusspawn's answer to "How to name a thread in Linux? [duplicate]" helped with the above syntax.
Note while testing this on Unix operating system MQI clients I found that on Linux and Solaris that the process name is limited to 15 characters by the OS. Only on AIX was RAPPLTAG able to display a full 28 characters. Windows MQI clients are also able to display the full 28 characters. The IBM Knowledge center only says that HP-UX is limited to 14 characters but indicates that on other Unix platforms it is limited to 28 characters, this appears to be incorrect with Linux and Solaris having a 15 character limit.
Helpful answers:
Linux process name size limit: Thread name longer than 15 chars?
Solaris process name size limit: psinfo_t solaris does not contain full process name in its field
UPDATE: The IBM KC page has been updated based on the feedback I provided, it now states the correct process name limits for Linux and Solaris.
In the interest of providing information to people who may be using IBM MQ classes for Java API the "DISPLAY CONN" page in the Knowledge Center does NOT note that a IBM MQ classes for Java API client can set this value. This is noted on IBM MQ Knowledge Center page "Setting up the IBM MQ environment for IBM MQ classes for Java" under Identifying a connection to the queue manager by setting an application name. This has only been available since v7.5.
Application names are limited to 28 characters and longer names are truncated to fit. If an application name is not specified, a default is provided. The default name is based on the invoking (main) class, but if this information is not available, the text WebSphere MQ Client for Java is used.
...
To set an application name in the properties hash table that is passed to the MQQueueManager constructor, add the name to the properties hash table with a key of MQConstants.APPNAME_PROPERTY.
IBM MQ classes for JMS API client can also set this value. This is noted on IBM MQ Knowledge Center page "Properties of IBM MQ classes for JMS objects > APPLICATIONNAME". This has only been available since v7.5.
Applicable Objects
ConnectionFactory, QueueConnectionFactory, TopicConnectionFactory,
XAConnectionFactory, XAQueueConnectionFactory,
XATopicConnectionFactory
JMS administration tool long name: APPLICATIONNAME
JMS administration tool short name: APPNAME
Programmatic access
Setters/getters
MQConnectionFactory.setAppName()
MQConnectionFactory.getAppName()
Values
Any valid string that is no longer than 28 characters. Longer names
are adjusted to fit by removing leading package names, if necessary.
For example, if the invoking class is com.example.MainApp, the full
name is used, but if the invoking class is
com.example.dictionaryAndThesaurus.multilingual.mainApp, the name
multilingual.mainApp is used, because it is the longest combination of
class name and rightmost package name that fits into the available
length.
If the class name itself is more than 28 characters long, it is
truncated to fit. For example,
com.example.mainApplicationForSecondTestCase becomes
mainApplicationForSecondTest.
You can change the application name on messages you send, using the MQOO_SET_ALL_CONTEXT and MQPMO_SET_ALL_CONTEXT options.
But I don't think you can change the application name displayed on the channel status.
However you could run different instances of your application under different OS users. The user name is also displayed among the status information, which could be used for your purpose if you don't override the MCA user id.
(Or you could do something similar by using SSL connections, and using the SSL cert information displayed to distinguish your app instances.)
While I agree with Attila that you "could" use the MQOO_SET_ALL_CONTEXT and MQPMO_SET_ALL_CONTEXT options, I strongly suggest you not do it.
If you do use those options then you need to understand the consequences. The following MQMD fields of each message will be left blank and/or null and it is up to the application to populate them:
AccountingToken
ApplIdentityData
ApplOriginData
PutApplType
PutApplName
PutDate
PutTime
UserIdentifier
So, how much effort are you going to go through to see something in MQ Explorer which is pointless otherwise?
Also, to use those options, the UserId (or group) requires "+set" MQ OAM permission for the queue it is opening.
I'm trying to upload a file from linux to z/OS via ftp (with filezilla). The upload fails with this error:
550-SVC99 RETURN CODE=4 S99INFO=0 S99ERROR=38668 HEX=970C S99ERSN code X'000042CE'.
550 Unable to create data set TESTUSER.UNIX.USERLIB.SOME for STOR command.
Does anyone know how to interpret this message?
SVC99, also known as DYNALLOC, is the dynamic allocation routine on z/OS. Documentation is voluminous, see the IBM Documentation for z/OS and look under z/OS MVS -> z/OS MVS Programming: Authorized Assembler Services -> Dynamic Allocation -> Requesting Dynamic Allocation Functions -> Interpreting Error Reason Codes from DYNALLOC. For z/OS 2.2 that path should take you here.
Meaning: Severe SMS VTOC service error.
Application programmer action:
Check field S99ERSN for a non-zero value. A non-zero value in S99ERSN
is a reason code. See SMS reason code (S99ERSN) for the possible
reason codes. If the system displayed message IKJ56893I, SMS messages
for the error follow IKJ56893I. If the system displayed message
IKJ56894I, no messages were returned. Request the message processing
option of dynamic allocation to obtain related messages and resubmit
the request. Because SMS creates entries in the logrec data set, if
you still cannot obtain messages, locate the logrec data set entry for
the error that corresponds to the reason code. If you need further
assistance contact your system programmer.
Corresponding Messages: IKJ56893I (messages were displayed) or
IKJ56894I (messages not displayed).
Following the link, your S99ERSN code is documented as...
42CE (17102)
Meaning: While trying to define a non-VSAM data set, a catalog error or
exceptional condition caused the allocation attempt to fail.
Application Programmer Action: Examine the return code and reason
code in accompanying message IGD17102I in z/OS MVS System Messages,
Vol 8 (IEF-IGD).
So there should be an accompanying message IGD17102I in the system log with even more information. Looking at the documentation, I'd contact your support staff as this doesn't look like something you can fix yourself.
I have connected remotely to a QMgr via MQ Explorer on Windows. The MQ server version is 7.5.0.1. I can put messages in SYSTEM.ADMIN.COMMAND.EVENT from MQ Explorer successfully and when I dump SYSTEM.ADMIN.COMMAND.EVENT, I can see my messages. As long as I know, I should be able to run PCF commands and MQSC commands via this channel. So, I put DISPLAY QMGR ALL message inside this queue and I can successfully see this message on MQ Server. My question is how can I run this message remotely via this channel? Thanks.
IBM Doc indicates that I should be able to receive the command result in SYSTEM.MQSC.REPLY.QUEUE. But I can not browse this queue from client MQ Explorer. The queue type for this queue is Model.
Couple of problems here.
First, you are using the wrong queue. The command server listens on SYSTEM.ADMIN.COMMAND.QUEUE. The queue to which you are sending messages, SYSTEM.ADMIN.COMMAND.EVENT is the queue to which the QMgr puts event messages after executing commands, provided of course that command events are enabled.
The second problem, as Jason mentions, is that the runmqsc processor takes human-readable script and converts it into commands the QMgr can understand. Passing textual commands directly to the command server won't work.
Typically we do what you want by passing the commands to runmqsc directly such as...
echo DISPLAY QMGR ALL | runmqsc MYQMGRNAME
If you require the ability to do this as a client, then you want to either download SupportPac MO72, or head over to MQ Gem and pick up a copy of MQSCX. Either of these will accept the command above on a local queue manager, and both can also be supplied with MQ Channel params and connect to a remote QMgr.
In addition to this basic functionality, the MQSCX product also has its own internal script parsing and execution. Suppose, for instance, that you want to do something depending on the command level of the QMgr.
Using runmqsc you could issue the command above, filter the resulting 2-column output through grep, awk, or similar, then capture the final output into a variable. You might need to do this multiple times to capture multiple values, invoking a new runmqsc each time and parsing the output in your script. You must then generate the string for the actual command you wanted to run when you started all this, and pass it to another invocation of runmqsc.
Alternatively, MQSCX lets you issue the DISPLAY command, then reference the resulting values directly by name. For example, you can pass MQSCX a couple lines of script telling it to inquire on the QMgr and then take a conditional action based on the command version, all without ever having to drop back into shell, bat or Perl script.
Full disclosure, I do not work for or get a commission from MQ Gem. I just don't like to beat my head against the wall writing 100 lines of code where 2 will do. If you do any amount of MQSC scripting, the ROI on MQSCX is measured in minutes. And it happens to be 100% on-topic as an answer to this question.
The command server doesnt process textual messages, it processes PCF messages. You need to build a message in PCF format and it can be processed. See http://www-01.ibm.com/support/knowledgecenter/SSFKSJ_8.0.0/com.ibm.mq.adm.doc/q019980_.htm
Ideally you would use real PCF format but there is a PCF format where you can send MQSC commands ('escaped' PCF - see here http://www-01.ibm.com/support/knowledgecenter/SSFKSJ_8.0.0/com.ibm.mq.ref.adm.doc/q087230_.htm?lang=en)