I am trying to set up Multi Instacnce MQ. I am using MQ v7.5. I am following the below steps:
Installed MQ in Server A and Server B
Configured NFS in Server A(acting as server) and in Server B (acting as client) in a folder MQ.
Created a queue manager in server A using the below command crtmqm -ld /MQ/logs -md /MQ/qmrgs -q QM1
Got the queue manager information using dspmqinf -o command QM1
In Server B used addmqinf <queue info from server A>
Start the queue manager in Server A using strmqm -x QM1 and its started successfully.
Start the queue manager in Server B using strmqm -x QM1 and its failed to start.
I got the below FDC:
+-----------------------------------------------------------------------------+
| |
| WebSphere MQ First Failure Symptom Report |
| ========================================= |
| |
| Date/Time :- Mon November 10 2014 15:23:29 IST |
| UTC Time :- 1415613209.614999 |
| UTC Time Offset :- 330 (IST) |
| Host Name :- InBlrBnc60 |
| Operating System :- Linux 2.6.32-71.el6.x86_64 |
| PIDS :- 5724H7230 |
| LVLS :- 7.5.0.0 |
| Product Long Name :- WebSphere MQ for Linux (x86-64 platform) |
| Vendor :- IBM |
| Installation Path :- /opt/mqm |
| Installation Name :- Installation1 (1) |
| Probe Id :- XY509007 |
| Application Name :- MQM |
| Component :- xcsOpenFileLock |
| SCCS Info :- /build/p000_P/src/lib/cs/unix/amqxnflx.c, |
| Line Number :- 428 |
| Build Date :- Jun 4 2012 |
| Build Level :- p000-L120604 |
| Build Type :- IKAP - (Production) |
| Effective UserID :- 525 (mqm) |
| Real UserID :- 525 (mqm) |
| Program Name :- crtmqm |
| Addressing mode :- 64-bit |
| LANG :- en_US.UTF-8 |
| Process :- 16729 |
| Process(Thread) :- 16729 |
| Thread :- 1 |
| ThreadingModel :- PosixThreads |
| QueueManager :- MQ1 |
| UserApp :- FALSE |
| ConnId(2) QM :- 2 |
| Last HQC :- 2.0.0-5133648 |
| Last HSHMEMB :- 2.1.1-22456 |
| Major Errorcode :- xecF_E_UNEXPECTED_SYSTEM_RC |
| Minor Errorcode :- OK |
| Probe Type :- MSGAMQ6119 |
| Probe Severity :- 2 |
| Probe Description :- AMQ6119: An internal WebSphere MQ error has occurred |
| ('37 - No locks available' from fcntl.) |
| FDCSequenceNumber :- 0 |
| Arith1 :- 37 (0x25) |
| **Comment1 :- '37 - No locks available' from fcntl.**
I tried browsing about 37 - No locks available' from fcntl, but not able to get why this error is coming. I have verified my system config with su mqm -c /opt/mqm/bin/mqconfig and it satisfies the MQ recommended value. I have also verified my NFS set up is working fine.
Related
We develop a spring-boot application which is deployed on OpenShift 3. The application should be scalable to at least two pods. But we use internal caches and other "global" data (some lists, some maps...) which should be the same (i.e. shared) for all the pods.
Is there a way to achieve such data sharing by a) a service, which is embedded inside the spring-boot application itself (this implies that each pod needs to find/know each other) or does it b) need in every case a standalone (potentially also scalable) cache service?
a)
|---- Application ----|
| |
| |-------------| |
| | Pod 1 | * | |
| |----------^--| |
| | |
| |----------v--| |
| | Pod 2 | * | |
| |----------^--| |
| | |
| |----------v--| |
| | Pod n | * | |
| |-------------| |
| |
|----------------------
* "embedded cache service"
b)
|---- Application ----|
| |
| |-------------| |
| | Pod 1 | |-----\
| |-------------| | \
| | | \
| |-------------| | \ |-----------------------|
| | Pod 2 | |-----------| Cache Service/Cluster |
| |-------------| | / |-----------------------|
| | | /
| |-------------| | /
| | Pod n | |------/
| |-------------| |
| |
|----------------------
Typically, if we would use memcached or redis I think b) would be the only solution. But how is it with Hazlecast?
With Hazelcast, you can both use a & b.
For scenario a, assuming you're using k8s on OpenShift, you can use Hazelcast Kubernetes discovery plugin so that pods deployed in the same k8s cluster discover themselves & form a cluster: https://github.com/hazelcast/hazelcast-kubernetes
For scenario b, Hazelcast has an OpenShift image as well, which requires an Enterprise subscription: https://github.com/hazelcast/hazelcast-openshift. If you need open-source version, you can use Hazelcast Helm Chart to deploy data cluster separately: https://github.com/helm/charts/tree/master/stable/hazelcast
I have a massive chunk of files in an extremely fast SAN disk that I like to do Hive query on them.
An obvious option is to copy all files into HDFS by using a command like this:
hadoop dfs -copyFromLocal /path/to/file/on/filesystem /path/to/input/on/hdfs
However, I don't want to create a second copy of my files, just to be to Hive query in them.
Is there any way to point an HDFS folder into a local folder, such that Hadoop sees it as an actual HDFS folder? The files keep adding to the SAN disk, so Hadoop needs to see the new files as they are being added.
This is similar to Azure's HDInsight approach that you copy your files into a blob storage and HDInsight's Hadoop sees them through HDFS.
For playing around with small files using the local file system might be fine, but I wouldn't do it for any other purpose.
Putting a file in an HDFS means that it is being split to blocks which are replicated and distributed.
This gives you later on both performance and availability.
Locations of [external] tables can be directed to the local file system using file:///.
Whether it works smoothly or you'll start getting all kinds of error, that's to be seen.
Please note that for the demo I'm doing here a little trick to direct the location to a specific file, but your basic use will probably be for directories.
Demo
create external table etc_passwd
(
Username string
,Password string
,User_ID int
,Group_ID int
,User_ID_Info string
,Home_directory string
,shell_command string
)
row format delimited
fields terminated by ':'
stored as textfile
location 'file:///etc'
;
alter table etc_passwd set location 'file:///etc/passwd'
;
select * from etc_passwd limit 10
;
+----------+----------+---------+----------+--------------+-----------------+----------------+
| username | password | user_id | group_id | user_id_info | home_directory | shell_command |
+----------+----------+---------+----------+--------------+-----------------+----------------+
| root | x | 0 | 0 | root | /root | /bin/bash |
| bin | x | 1 | 1 | bin | /bin | /sbin/nologin |
| daemon | x | 2 | 2 | daemon | /sbin | /sbin/nologin |
| adm | x | 3 | 4 | adm | /var/adm | /sbin/nologin |
| lp | x | 4 | 7 | lp | /var/spool/lpd | /sbin/nologin |
| sync | x | 5 | 0 | sync | /sbin | /bin/sync |
| shutdown | x | 6 | 0 | shutdown | /sbin | /sbin/shutdown |
| halt | x | 7 | 0 | halt | /sbin | /sbin/halt |
| mail | x | 8 | 12 | mail | /var/spool/mail | /sbin/nologin |
| uucp | x | 10 | 14 | uucp | /var/spool/uucp | /sbin/nologin |
+----------+----------+---------+----------+--------------+-----------------+----------------+
You can mount your hdfs path into local folder, for example with hdfs mount
Please follow this for more info
But if you want speed, it isn't an option
I am trying to senfd message to a queue using Websphere MQ but Getting following error while sending message
-FFDC called in uninitialized Trace module
Following is my code:
MQQueueConnectionFactory cf=new MQQueueConnectionFactory();
cf.setHostName("localhost");
cf.setPort(1414);
cf.setTransportType(1);
cf.setQueueManager("LocalManager");
cf.setChannel("xyz");
MQQueueConnection connection = (MQQueueConnection) cf.createQueueConnection();
MQQueueSession session = (MQQueueSession) connection.createQueueSession(false, Session.AUTO_ACKNOWLEDGE);
MQQueue queue = (MQQueue) session.createQueue("queue:///inputQ");
MQQueueSender sender = (MQQueueSender) session.createSender(queue);
MQQueueReceiver receiver = (MQQueueReceiver) session.createReceiver(queue);
System.out.println("creating message");
JMSTextMessage message = (JMSTextMessage) session.createTextMessage("Test message");
connection.start();
sender.send(message);
System.out.println("message sent");
Error In console which I am not aware of-
FFDC called in uninitialized Trace module
FDCTitle
Product :- ProductName
Date/Time :- Mon Apr 04 17:49:55 IST 2016
System time :- 1459772395537
Operating System :- Windows 7
UserID :- xyz
Java Vendor :- Oracle Corporation
Java Version :- 21.0-b17
Data
----
| connectionTypeName :- com.ibm.msg.client.wmq
| exception :- ExceptionDepth is 2
| exception :-
| | Cause:1 :- java.lang.NullPointerException
| | StackTrace:1 :- java.lang.NullPointerException
| | at com.ibm.msg.client.jms.JmsFactoryFactory.getInstance(JmsFactoryFactory.java:180)
| | at com.ibm.mq.jms.MQConnectionFactory.initialiseMQConnectionFactory(MQConnectionFactory.java:3314)
| | at com.ibm.mq.jms.MQConnectionFactory.<init>(MQConnectionFactory.java:283)
| | at com.ibm.mq.jms.MQQueueConnectionFactory.<init>(MQQueueConnectionFactory.java:81)
| | at MQHeader.main(MQHeader.java:20)
| |
No Components Found
Stack trace
-----------
Stack trace to show the location of the FFST call
| FFST Location :- java.lang.Exception
| at com.ibm.msg.client.commonservices.trace.Trace.getCurrentPosition(Trace.java:1972)
| at com.ibm.msg.client.commonservices.trace.Trace.createFFSTString(Trace.java:1911)
| at com.ibm.msg.client.commonservices.trace.Trace.ffstInternal(Trace.java:1800)
| at com.ibm.msg.client.commonservices.trace.Trace.ffst(Trace.java:1673)
| at com.ibm.msg.client.jms.JmsFactoryFactory.getInstance(JmsFactoryFactory.java:214)
| at com.ibm.mq.jms.MQConnectionFactory.initialiseMQConnectionFactory(MQConnectionFactory.java:3314)
| at com.ibm.mq.jms.MQConnectionFactory.<init>(MQConnectionFactory.java:283)
| at com.ibm.mq.jms.MQQueueConnectionFactory.<init>(MQQueueConnectionFactory.java:81)
| at MQHeader.main(MQHeader.java:20)
|
WorkQueueMananger Contents
--------------------------
| Maintain ThreadPool size :- false
| Maximum ThreadPool size :- -1
| ThreadPool inactive timeout :- 0
| unavailable - :- com.ibm.msg.client.commonservices.CSIException: JMSCS0002
Runtime properties
------------------
| Available processors :- 4
| Free memory in bytes (now) :- 125076032
| Max memory in bytes :- 1886519296
| Total memory in bytes (now) :- 127074304
Component Manager Contents
--------------------------
Common Services Components:
Messaging Provider Components:
FFDC called in uninitialized Trace module
FDCTitle
Product :- ProductName
Date/Time :- Mon Apr 04 17:49:55 IST 2016
System time :- 1459772395584
Operating System :- Windows 7
UserID :- xyz
Java Vendor :- Oracle Corporation
Java Version :- 21.0-b17
Source Class :- com.ibm.mq.jms.MQQueueConnectionFactory
Source Method :- readObject()
ProbeID :- XF001003
Thread :- name=main priority=5 group=main ccl=sun.misc.Launcher$AppClassLoader#37af78ce
Data
----
Data :- none
No Components Found
Stack trace
-----------
Stack trace to show the location of the FFST call
| FFST Location :- java.lang.Exception
| at com.ibm.msg.client.commonservices.trace.Trace.getCurrentPosition(Trace.java:1972)
| at com.ibm.msg.client.commonservices.trace.Trace.createFFSTString(Trace.java:1911)
| at com.ibm.msg.client.commonservices.trace.Trace.ffstInternal(Trace.java:1800)
| at com.ibm.msg.client.commonservices.trace.Trace.ffst(Trace.java:1624)
| at com.ibm.mq.jms.MQConnectionFactory.initialiseMQConnectionFactory(MQConnectionFactory.java:3324)
| at com.ibm.mq.jms.MQConnectionFactory.<init>(MQConnectionFactory.java:283)
| at com.ibm.mq.jms.MQQueueConnectionFactory.<init>(MQQueueConnectionFactory.java:81)
| at MQHeader.main(MQHeader.java:20)
|
WorkQueueMananger Contents
--------------------------
| Maintain ThreadPool size :- false
| Maximum ThreadPool size :- -1
| ThreadPool inactive timeout :- 0
| unavailable - :- com.ibm.msg.client.commonservices.CSIException: JMSCS0002
Runtime properties
------------------
| Available processors :- 4
| Free memory in bytes (now) :- 124412624
| Max memory in bytes :- 1886519296
| Total memory in bytes (now) :- 127074304
Component Manager Contents
--------------------------
Common Services Components:
Messaging Provider Components:
FFST Object Properties
----------------------
All currently set properties
| XMSC_CONNECTION_TYPE :- 1
| XMSC_CONNECTION_TYPE_NAME :- com.ibm.msg.client.wmq
FFDC called in uninitialized Trace module
FDCTitle
Product :- ProductName
Date/Time :- Mon Apr 04 17:49:55 IST 2016
System time :- 1459772395589
Operating System :- Windows 7
UserID :- xyz
Java Vendor :- Oracle Corporation
Java Version :- 21.0-b17
Source Class :- com.ibm.msg.client.commonservices.nls.PINLSServices
Source Method :- createException(String,HashMap)
ProbeID :- XC003001
Thread :- name=main priority=5 group=main ccl=sun.misc.Launcher$AppClassLoader#37af78ce
Data
----
Data :- none
No Components Found
Stack trace
-----------
Stack trace to show the location of the FFST call
| FFST Location :- java.lang.Exception
| at com.ibm.msg.client.commonservices.trace.Trace.getCurrentPosition(Trace.java:1972)
| at com.ibm.msg.client.commonservices.trace.Trace.createFFSTString(Trace.java:1911)
| at com.ibm.msg.client.commonservices.trace.Trace.ffstInternal(Trace.java:1800)
| at com.ibm.msg.client.commonservices.trace.Trace.ffst(Trace.java:1624)
| at com.ibm.msg.client.commonservices.nls.PINLSServices.createException(PINLSServices.java:121)
| at com.ibm.msg.client.commonservices.nls.NLSServices.createException(NLSServices.java:233)
| at com.ibm.msg.client.jms.internal.JmsErrorUtils.createException(JmsErrorUtils.java:109)
| at com.ibm.msg.client.jms.internal.JmsFactoryFactoryImpl.getInstance(JmsFactoryFactoryImpl.java:172)
| at com.ibm.msg.client.jms.admin.JmsConnectionFactoryImpl.setProviderFactory(JmsConnectionFactoryImpl.java:168)
| at com.ibm.mq.jms.MQConnectionFactory.<init>(MQConnectionFactory.java:293)
| at com.ibm.mq.jms.MQQueueConnectionFactory.<init>(MQQueueConnectionFactory.java:81)
| at MQHeader.main(MQHeader.java:20)
|
WorkQueueMananger Contents
--------------------------
| Maintain ThreadPool size :- false
| Maximum ThreadPool size :- -1
| ThreadPool inactive timeout :- 0
| unavailable - :- com.ibm.msg.client.commonservices.CSIException: JMSCS0002
Runtime properties
------------------
| Available processors :- 4
| Free memory in bytes (now) :- 124412624
| Max memory in bytes :- 1886519296
| Total memory in bytes (now) :- 127074304
Component Manager Contents
--------------------------
Common Services Components:
Messaging Provider Components:
unavailable - :-
com.ibm.msg.client.commonservices.CSIException: JMSCS0002
It appears you have not included all of the required MQ JAR files. You are missing at least com.ibm.mq.jmqi.jar MQ JAR file.
I've got the following problem:
There is a Postgres database which I need to get data from, via a Nagios Linux distribution.
My intention is to make a resulting SELECT be saved to a .txt, that would be sent via email to me using MUTT.
Until now, I've done:
#!/bin/sh
psql -d roaming -U thdroaming -o saida.txt << EOF
\d
\pset border 2
SELECT central, imsi, mapver, camel, nrrg, plmn, inoper, natms, cba, cbaz, stall, ownms, imsi_translation, forbrat FROM vw_erros_mgisp_totalizador
EOF
My problem is:
The .txt "saida.txt" is bringing me info about the database, as follows:
Lista de relações
Esquema | Nome | Tipo | Dono
---------+----------------------------------+-----------+------------
public | apns | tabela | jmsilva
public | config_imsis_centrais | tabela | thdroaming
public | config_imsis_sgsn | tabela | postgres
(3 Registers)
+---------+---------+----------+---------+---------+--------+------------+-------+---------+----------+-------+-------+------------------+-----------+
| central | imsi | mapver | camel | nrrg | plmn | inoper | natms | cba | cbaz | stall | ownms | imsi_translation | forbrat |
+---------+---------+----------+---------+---------+--------+------------+-------+---------+----------+-------+-------+------------------+-----------+
| MCTA02 | 20210 | | | | | INOPER-127 | | | | | | | |
| MCTA02 | 20404 | | | | | INOPER-127 | | | | | | | |
| MCTA02 | 20408 | | | | | INOPER-127 | | | | | | | |
| MCTA02 | 20412 | | | | | INOPER-127 | | | | | | | |
.
.
.
How could I make the first table not to be imported to the .txt?
Remove the '\d' portion of the script which causing listing the tables in the DB you see at the top of your output. So your script will become:
#!/bin/sh
psql -d roaming -U thdroaming -o saida.txt << EOF
\pset border 2
SELECT central, imsi, mapver, camel, nrrg, plmn, inoper, natms, cba, cbaz, stall, ownms, imsi_translation, forbrat FROM vw_erros_mgisp_totalizador
EOF
To get the output to appear CSV formatted in a file named /tmp/output.csv do you can do the following:
#!/bin/sh
psql -d roaming -U thdroaming -o saida.txt << EOF
\pset border 2
COPY (SELECT central, imsi, mapver, camel, nrrg, plmn, inoper, natms, cba, cbaz, stall, ownms, imsi_translation, forbrat FROM vw_erros_mgisp_totalizador) TO '/tmp/output.csv' WITH (FORMAT CSV)
EOF
Question
How to communicate with another program (for instance, a windows service one) through environment variables (not system or user ones)?
What do we have
Well, I have the following scheme for a data logger:
------------------------- --------------------------------
| the things to measure | | the things that do something |
------------------------- --------------------------------
| ^
| sensors | switches
V |
-------------------------------------------------------------------
| dedicated hardware |
-------------------------------------------------------------------
| ^
| | serial communication
V |
--------------- -------------
| Windows | ------------------------------------> | user |
| service | <------------------------------------ | interface |
--------------- udp communication -------------
|^ keyboard
V| and screen
--------
| user |
--------
On current development:
windows service is always running when Windows is running
user can open and close user interface (of course :p)
windows service acquires data from sensors
user interface automatic requests data to windows service every 100ms and shows it to user via udp communication through some implemented protocol (we call it GetData() command and response to it)
user can send some other commands to change the data to acquire through implemented protocol (we call it SetSensors() command and response to it)
Both user interface and windows service are developed on Borland C+ Builder 6 and use NMUDP component, from FastNet tab, for UDP communication.
What we are thinking to do
Because of some buffer issues and to free udp channel only for sending SetSensors()command and response to it, we are considering that instead of using GetData():
Windows service would get data from sensors and put them on environment variables
the user interface would read them to show to user
Scheme after doing what we are thinking
------------------------- --------------------------------
| the things to measure | | the things that do something |
------------------------- --------------------------------
| ^
| sensors | switches
V |
-------------------------------------------------------------------
| dedicated hardware |
-------------------------------------------------------------------
| ^
| | serial communication
V |
--------------- -------------
| | ------------------------------------> | |
| | environment variables | |
| | (get data from sensors) | |
| Windows | | user |
| service | | interface |
| | | |
| | ------------------------------------> | |
| | <------------------------------------ | |
--------------- udp communication -------------
(send commands to service) |^ keyboard
V| and screen
--------
| user |
--------
Any way to do that?
We would not use system and user environment variables, because it writes on Windows Registry, i.e., it will save to hard drive and it gets more slow...
As #HansPassant said, I cannot do that directly. Although I saw some ways to do that via memory mapped file, it is so easy only to add one more udp communication channel through other port. So:
------------------------- --------------------------------
| the things to measure | | the things that do something |
------------------------- --------------------------------
| ^
| sensors | switches
V |
-------------------------------------------------------------------
| dedicated hardware |
-------------------------------------------------------------------
| ^
| | serial communication
V |
--------------- -------------
| | ------------------------------------> | |
| | udp communication (port 3) | |
| | (get data from sensors) | |
| Windows | | user |
| service | | interface |
| | (port 1) | |
| | ------------------------------------> | |
| | <------------------------------------ | |
--------------- udp communication (port 2) -------------
(send commands to service) |^ keyboard
V| and screen
--------
| user |
--------
If someone provide a better solution, I'll mark it as solution in future.