I think title is confusing. I try to explain it in detail.
I have a big xml file that contains a lot of messages for queue. The structure looks like: <'objects'><...>.....<...><'/objects'>. It's an array of objects tags and one item of objects is a message. I want to put this file that should be broken into a separate messages into queue(mq) by using rfhutil.
I know it's possible but I'm confused about delimiter. The whole menu looks puzzled.
Is anyone using rfhutil and mq? Maybe you have a guide for that. I couldn't find any information about rfhutil.
I hope I explained well.there is a pic of rfhutil menu. load Q
"Load Q" option is used to send all messages from file which was previously created with "Save Q" option. It will not properly parse your custom xml. Even if you format xml in a way that is parsable by custom delimiter, xml still contains only message payload and lacks message headers.
However rfhutil is part of lagerger IH03 SupportPack and there you will find:
additional software to send multiple custom messages like mqput
a manual how to do it (ih03.doc)
Simplest "out of head" solution would be to split xml in multiple smaller files and than send them all with one mqput2 command.
Related
I have been asked to create a system for allowing third party companies to dump data into several of our tables. These third parties provide csv files on a periodic basis, and after doing some research it seemed like Oracle themselves had a standard tool for doing so, "sqlldr". I've since gotten it working to an acceptable degree, and we have a job scheduled to run that script once a day.
But one of the third parties supplies really dirty data, of the sort where I can't expect it to always load every row/record (looking like up to about 8% will fail). My boss asked me to forward "all output" from the first few tests to him, and like a moron I also sent the log file.
He has asked that this "report" be modified to include those exceptions that aren't unique constraints along with the line in the input file that caused the exception.
This means that I need data from the log file, but also from the (I believe) reject file in a single document. Rather than write a convoluted shell script to combine those two, does SQL*Loader itself allow any customization that might achieve the same thing? I've read through the Oracle documentation and haven't found anything that suggests this, but I've also learned not to trust it entirely either.
Is this possible? Ideally, the solution would allow me to add values to the reject file that don't exist in the original input file, but I'm also interested in any customization of the log file or reject file.
No.
I was going to stop there, but you can define the name of the log file, which might help with issue. Most automation with SQL*Loader involves wrapping it within shell scripts; aka "roll your own."
What is the procedure to define new external collectors in bosun using scollector.
Can we write python or shell scripts to collect data?
The documentation around this is not quite up to date. You can do it as described in http://godoc.org/bosun.org/cmd/scollector#hdr-External_Collectors , but we also support JSON output which is better.
Either way, you write something and put it in the external collectors directory, followed by a frequency directory, and then an executable script or binary. Something like:
<external_collectors_dir>/<freq_sec>/foo.sh.
If the directory frequency is zero 0, then the the script is expected to be continuously running, and you put a sleep inside the code (This is my preferred method for external collectors). The scripts outputs the telnet format, or the undocumented JSON format to stdout. Scollector picks it up, and queues that information for sending.
I created an issue to get this documented not long ago https://github.com/bosun-monitor/bosun/issues/1225. Until one of us gets around to that, here is the PR that added JSON https://github.com/bosun-monitor/bosun/commit/fced1642fd260bf6afa8cba169d84c60f2e23e92
Adding to what Kyle said, you can take a look at some existing external collectors to see what they output. here is one written in java that one of our colleagues wrote to monitor jvm stuff. It uses the text format, which is simply:
metricname timestamp value tag1=foo tag2=bar
If you want to use the JSON format, here is an example from one of our collectors:
{"metric":"exceptional.exceptions.count","timestamp":1438788720,"value":0,"tags":{"application":"AdServer","machine":"ny-web03","source":"NY_Status"}}
And you can also send metadata:
{"Metric":"exceptional.exceptions.count","Name":"rate","Value":"counter"}
{"Metric":"exceptional.exceptions.count","Name":"unit","Value":"errors"}
{"Metric":"exceptional.exceptions.count","Name":"desc","Value":"The number of exceptions thrown per second by applications and machines. Data is queried from multiple sources. See status instances for details on exceptions."}`
Or send error messages to stderror:
2015/08/05 15:32:00 lookup OR-SQL03: no such host
Is there a tool which can take 1000 Seperate HL7 Messages and combine them into a single document for 7edit? I need to run a test, and if I can do one document and choose send all, it will be better than me running it manually for each of these 1000 messages.
Yes, There exist a way to combine those messages in a single file. You can do that using any integration engine, I will take Mirth in this case.
Follow these steps in sequential order
Download Mirth Connect from here using the .exe installer (in case you don't have it).
Setup your account and do initial configuration on your local system.
Create a Channel called Appending Channel, put Source inbound and outbound connector as HL7v2.x.
Go to Source Tab, Put Connector type as File Reader. Give the location of the directory where your messages will reside(D:\x\read in my case). Make sure you have the directory shared
You can make Delete file after read as a Yes, which will prune the files after they are read from this location.If you do a NO, then specify where you want to move those files to.
Put Process Batch files as a No.
Go to Destinations tab, create a Destination called as Appender and make it a File Writer type.
Give the directory(D:\x\Output in my case) where your final file will be placed.Give the file name as final.txt.
Choose Append on the file exists tab.
In Template, Drag Raw Data from the list on the right hand side, and put it here or else what you can do is type ${message.rawData} in the template section.
Save Channel and Deploy it.
Place all your messages in the read folder (mentioned above), and wait for Mirth to poll the folder (default setting is 1000 ms).
Once that is done, go to final.txt to see all the messages appended in the same file.
The downside is that even though this process is 100 percent working, the message thus appended will not be seperated by any means. So it will look like below
|2688684|||||||||||||||||||||||||199912271408||||||002376853MSH|^~\&|EPIC|EPICADT|
^ End of first message
You don't need any tool for that. 7edit is able to read multi-message files. You just need to append each message into one single text file like this (two ADT messages):
MSH|^~\&|SystemA|CompanyA|SystemB|CompanyB|20121116122025||ADT^A01|101|T|2.5||||||UNICODE UTF-8
EVN|A01|20130823080958
PID|||1000||Lastname^Firstname
PV1||I
MSH|^~\&|SystemA|CompanyA|SystemB|CompanyB|20121116122026||ADT^A01|102|T|2.5||||||UNICODE UTF-8
EVN|A01|20130823080958
PID|||1000||Lastname^Firstname
PV1||I
Open this file with 7edit and you will see this (multiple messages):
Now you can send all messages at once by pressing on Send and then select All Messages:
It is that simple - no other tool necessary (just to make the append in one file maybe)
You could also try to use HL7Browser (www.nule.org), a tool that is similar to 7Edit, with less features but free.
You should be able to open many single HL7 messages files, HL7Browser will cache them in its viewer and should allow you to save them all to a single file.
Hope helps
Davide
if you have multiple HL7 files in one folder and want to combine them into 1 hl7 file, you can do following:
create a batch file in this folder named combine.cmd
write following into this batch file
del combined.hl7
for %%f in (*.hl7) do type "%%f" >> combined.hl
move combined.hl combined.hl7
run this batch file
result: all hl7 files in this folder are combined into a single file called "combined.hl7"
Is there a way to detect the file name of a file when a user prints from any application, somwhere in the printer events? I am looking for a windows api where I can determine what file is being printed.
Print jibs are opened with names but typically this doesn't mean the filename - the names which are displayed in the print queue are accessible by querying the printer driver directly I believe.
The printer driver does not have any external API for finding the filename. I assume you would be looking at creating some kind of systray agent sort of app that would be monitoring print queues for jobs being sent. If so -
Refer to http://support.microsoft.com/kb/196805 for a microsoft tool that allows you to monitor print queues for print job status
Refer to http://msdn.microsoft.com/en-us/library/ff562742%28v=vs.85%29.aspx which explains how Windows supports printer change notifications. You can create an app that registers for this and get handle to the Print Queue. Once you get a handle you can call the GetPrinter and GetJob APIs to get access to the JOB_INFO_2 structure. The pDocumentName in the JOB_INFO_2 structure is the name of the file
Hope this helps. If so, please vote a +1 :)
No, there is not, at least not reliably. Keep in mind that there may not be a file name at all. For example, I could open an application like notepad, type some stuff, and print. What file am I printing? None.
In the above scenario most applications will provide some sort of default file name like "untitled", and sometimes you can find that name by parsing the print job's name. For example, if you call GetJob, the JOB_INFO_1 struct's pDocument member will contain a pointer to the print job's name, and that name will often contain the file name.
However, every application formats it differently, and some don't provide it at all. So the answer is you can find the file name perhaps 75% of the time with a lot of effort, but there simply is no 100% solution.
Using Google protobuf, I am saving my serialized messaged data to a file - in each file there are several messages. We have both C++ and Python versions of the code, so I need to use protobuf functions that are available in both languages. I have experimented with using SerializeToArray and SerializeAsString and there seems to be the following unfortunate conditions:
SerializeToArray: As suggested in one answer, the best way to use this is to prefix each message with it's data size. This would work great for C++, but in Python it doesn't look like this is possible - am I wrong?
SerializeAsString: This generates a serialized string equivalent to it's binary counterpart - which I can save to a file, but what happens if one of the characters in the serialization result is \n - how do we find line endings, or the ending of messages for that matter?
Update:
Please allow me to rephrase slightly. As I understand it, I cannot write binary data in C++ because then our Python application cannot read the data, since it can only parse string serialized messages. Should I then instead use SerializeAsString in both C++ and Python? If yes, then is it best practice to store such data in a text file rather than a binary file? My gut feeling is binary, but as you can see this doesn't look like an option.
We have had great success base64 encoding the messages, and using a simple \n to separate messages. This will ofcoirse depend a lot on your use - we need to store the messages in "log" files. It naturally has overhead encoding/decoding this - but this has not even remotely been an issue for us.
The advantage of keeping these messages as line separated text has so far been invaluable for maintenance and debugging. Figure out how many messages are in a file ? wc -l . Find the Nth message - head ... | tail. Figure out what's wrong with a record on a remote system you need to access through 2 VPNs and a citrix solution ? copy paste the message and mail it to the programmer.
The best practice for concatenating messages in this way is to prepend each message with its size. That way you read in the size (try a 32bit int or something), then read that number of bytes into a buffer and deserialize it. Then read the next size, etc. etc.
The same goes for writing, you first write out the size of the message, then the message itself.
See Streaming Multiple Messages on the protobuf documentation for more information.
Protobuf is a binary format, so reading and writing should be done as binary, not text.
If you don't want binary format, you should consider using something other than protobuf (there are lots of textual data formats, such as XML, JSON, CSV); just using text abstractions is not enough.