I'm attempting to extract the text from the MSGDATA column (HUGEBLOB) in the MAXINTMSGTRK table:
I've tried the options outlined here: How to query hugeblob data:
select
msg.*,
utl_raw.cast_to_varchar2(dbms_lob.substr(msgdata,1000,1)) msgdata_expanded,
dbms_lob.substr(msgdata, 1000,1) msgdata_expanded_2
from
maxintmsgtrk msg
where
rownum = 1
However, the output is not text:
How can I extract text from MSGDATA column?
It's is possible to do it using Automation script, uncompress data using psdi.iface.jms.MessageUtil class.
from psdi.iface.jms import MessageUtil
...
msgdata_blob = maxintmsgtrkMbo.getBytes("msgdata")
byteArray = MessageUtil.uncompressMessage(msgdata_blob, maxintmsgtrkMbo.getLong("msglength"))
msgdata_clob = ""
for symb1 in byteArray:
msgdata_clob = msgdata_clob + chr(symb1)
It sounds like it's not possible because the value is compressed:
Starting in Maximo 7.6, the messages written by the Message Tracking
application are stored in the database. They are no longer written as
xml files as in previous versions.
Customers have asked how to search and view MSGDATA data from the
MAXINTMSGTRK table.
It is not possible to search or retrieve the data in the maxintmsgtrk
table in 7.6.using SQL. The BLOB field is stored compressed.
MIF 7.6 Message tracking changes
Related
TLDR
I have a column of serialized protobufs in a table in Clickhouse, and I would like to flatten those protobufs into another table via a materialized view. How do I format the materialized view to do this?
Details
I have a RabbitMQ queue which serves messages to my Clickhouse server. The message outer.proto consists of a service name and a serialized protobuf message payload:
//outer.proto
syntax = "proto2"
message WrappedMessage {
required string svc_id = 1;
required bytes msg = 2;
}
This service name and payload are then stored in Clickhouse as such:
CREATE TABLE raw_records_rmq
(
svc_id String NOT NULL
, msg String NOT NULL
) ENGINE = RabbitMQ SETTINGS
--skip rabbitmq settings, this part works
rabbitmq_format='ProtobufSingle'
rabbitmq_schema = 'outer:WrappedMessage'
;
CREATE TABLE raw_records
(
svc_id String NOT NULL
, msg String NOT NULL
) ENGINE = MergeTree ORDER BY tuple()
;
CREATE MATERIALIZED VIEW raw_mv
TO raw_records AS
SELECT * FROM raw_records_rmq
This process works as expected, with the serialized message stored in raw_records.msg. The message is defined as such:
//inner.proto
syntax = "proto2"
message Person {
optional uint32 id = 1;
optional string name = 2;
optional bool arb_bool = 3;
}
I would now like to query the contents of the stored message; to simplify this, I create a destination table:
CREATE TABLE people
(
id UInt32
, name String
, arb_bool UInt8
) ENGINE = MergeTree ORDER BY tuple()
But this is where my success stops. My attempts so far have been to query the column as a subquery and then attempt to parse the results as protobufs using Clickhouse FORMAT and SETTINGS, as described in their documentation:
CREATE MATERIALIZED VIEW mv
TO people AS
SELECT * FROM (SELECT proto FROM raw_records)
FORMAT ProtobufSingle
SETTINGS format_schema='inner:Person'
However, this fails to unpack the protobuf message. Changing from a Materialized View to a standard View shows that Clickhouse is returning only the single column specified in the subquery with the entire protobuf message as each result.
Any advice on how to properly format this materialized view, or alternatives for processing protobufs-inside-protobufs would be greatly appreciated!
You can't convert a single column into multiple values.
You can use the protobuf format but it will be the entire protobuf message: https://clickhouse.com/docs/en/interfaces/formats/#protobuf
I am trying to fetch xml data of a Sales Order record but I can't find a way to do it.
The data appears when we add '&xml=t' to the url of a record.
I want to fetch all the data in XML format into a variable.
Found the answer.
Just use record.load and store it in a variable.
var objRecord = record.load({
type: record.Type.SALES_ORDER,
id: 157,
isDynamic: true,
});
Logging this varialbe won't help, as there is character limit of 3999 characters. You can store it in File Cabinet.
I have to ingest data in hive table from hdfs but I don't know how to choose correct data type for the data mentioned below:-
$34740$#$Disrupt Worldwide LLC$#$40425$#$null$#$13$#$6$#$317903$#$null$#$Scott Bodily$#$+$#$null$#$10$#$0$#$1$#$0$#$disruptcentral.com$#$null$#$null$#$1$#$null$#$null$#$null$#$Scott Bodily$#$1220DB56-56D7-E411-80D6-005056A451E3$#$true$
$34741$#$The Top Tipster Leagues Limited$#$35605$#$null$#$13$#$7$#$317902$#$null$#$AM Support Team$#$+447886 027371$#$null$#$1$#$1$#$1$#$0$#$www.toptipsterleagues.com, www.toptipsterleagues.co.uk, http://test.toptipsterleague.com$#$Jamil Johnson$#$Cheng Liem Li$#$1$#$0.70$#$1.50$#$1.30$#$Bono van Nijnatten$#$0B758BF9-F1D6-E411-80D7-005056A44C5C$#$true$
Refer this link for different data types,
Click here
Other than all the numeric and decimal fields you can use STRING data type. For the numeric fields based on the range and precision you can use INT or DECIMAL.
Using string and varchar or any other string data types will read null in your data as string i.e "null" for handling nul you should mention the table properties like below,
ALTER TABLE tablename SET
SERDEPROPERTIES ('serialization.null.format' = 'null');
Let me know if anything needed on this.
I am facing a problem in deleting large data from parse.com
Firstly i filtered the data using filter but it displays me only at max 100 rows and then i have to select this 100 rows and delete , and then again select and delete next 100.
Is there any way i can delete all data matching the filter,
something like
DELETE FROM Tablename WHERE fieldname LIKE '%foo%'
or is it possible to execute query in parse.com
or is there a way to deleted it using shell script and parse somehow (any package might help me)
If you want to do this programmatically, you can create a query to get all the objects and then delete them. Here is an example using swift for iOS:
var query = PFQuery(className: TABLENAME)
query.whereKey(fieldname, equals: "%foo%")
query.findObjectsInBackgroundWithBlock(
{(objects: [AnyObject]!, error: NSError!) -> Void in
for object in objects {
object.deleteInBackground()
}
})
The documentation for parse in any of its supported languages can be found here: https://parse.com/docs/
Currently I'm working on legacy application, that uses crystal report engine. I have to get value of database fields programmatically. As I've assumed, I need proper event for getting next code to work:
Report.Database.Tables(1).Fields(1).Value
But the value is always empty in DownloadStarted/Finished event handlers. What I'm doing wrong and is it at least possible?
I think that if you want to get value of your table fields in program the best way is that you get the field name from report and then connect to your table directly and use report field names as the table columns name
i do it in c# i hope it can help you in vb6 too:
string name = report2.Database.Tables[1].Fields[1].Name;
string[] names = name.Split('.');
and then add your database to your program and use names like this:
DataTable dt = new DataTable();
string[] value = dt.Columns[names[1]];
if you just need your tables values, you can use my last answer, but if you need value of database fields in crystal report, i mean something like formula field ,this code can help you:
CRAXDRT.FormulaFieldDefinitions definitions = report2.FormulaFields;
string formulaText = "IF " + report2.Database.Tables[1].Fields[3].Name
+ " > 10 THEN" + report2.Database.Tables[1].Fields[2].Name;
definitions.Add("Test", formulaText);
report2.Sections[1].AddFieldObject(definitions[1], 0, 0);