Today i need help with the speech recognition Pocketsphinx which i use in Freeswitch. So there is a demo "pizza demo" which does not work because the programm doesn't "hear" me.
I tried another example with an lua script. And also here the Pocketsphinx does not "hear" me.
So maybe somebody knows whats not working. Because i don't implement anything, i don't know which code i can paste here. So if you need some code or configurations let me know.
My idea: maybe i must set which .dic file the pocketsphinx must use. I hope somebody can help me.
EDIT://
2014-10-14 15:13:08.923330 [NOTICE] switch_channel.c:1055 New Channel sofia/internal/1001#myip [326a4157-aa80-48d2-bd7e-db8d8afd525b]
2014-10-14 15:13:09.042378 [INFO] mod_dialplan_xml.c:558 Processing me <1001>->74992 in context default
2014-10-14 15:13:09.042378 [CRIT] mod_dptools.c:1628 WARNING WARNING WARNING WARNING WARNING WARNING WARNING WARNING WARNING
2014-10-14 15:13:09.042378 [CRIT] mod_dptools.c:1628 Open /usr/local/freeswitch/conf/vars.xml and change the default_password.
2014-10-14 15:13:09.042378 [CRIT] mod_dptools.c:1628 Once changed type 'reloadxml' at the console.
2014-10-14 15:13:09.042378 [CRIT] mod_dptools.c:1628 WARNING WARNING WARNING WARNING WARNING WARNING WARNING WARNING WARNING
2014-10-14 15:13:19.932900 [INFO] switch_core_media.c:5162 Activating RTCP PORT 4077
2014-10-14 15:13:19.932900 [NOTICE] sofia_media.c:92 Pre-Answer sofia/internal/1001#myip!
2014-10-14 15:13:19.943925 [NOTICE] fssession.cpp:1167 Channel [sofia/internal/1001#myip] has been answered
INFO: cmd_ln.c(691): Parsing command line:
\
-samprate 8000 \
-hmm /usr/local/freeswitch/grammar/model/communicator \
-jsgf /usr/local/freeswitch/grammar/pizza_order.gram \
-lw 6.5 \
-dict /usr/local/freeswitch/grammar/default.dic \
-frate 50 \
-silprob 0.005
Current configuration:
[NAME] [DEFLT] [VALUE]
-agc none none
-agcthresh 2.0 2.000000e+00
-alpha 0.97 9.700000e-01
-ascale 20.0 2.000000e+01
-aw 1 1
-backtrace no no
-beam 1e-48 1.000000e-48
-bestpath yes yes
-bestpathlw 9.5 9.500000e+00
-bghist no no
-ceplen 13 13
-cmn current current
-cmninit 8.0 8.0
-compallsen no no
-debug 0
-dict /usr/local/freeswitch/grammar/default.dic
-dictcase no no
-dither no no
-doublebw no no
-ds 1 1
-fdict
-feat 1s_c_d_dd 1s_c_d_dd
-featparams
-fillprob 1e-8 1.000000e-08
-frate 100 50
-fsg
-fsgusealtpron yes yes
-fsgusefiller yes yes
-fwdflat yes yes
-fwdflatbeam 1e-64 1.000000e-64
-fwdflatefwid 4 4
-fwdflatlw 8.5 8.500000e+00
-fwdflatsfwin 25 25
-fwdflatwbeam 7e-29 7.000000e-29
-fwdtree yes yes
-hmm /usr/local/freeswitch/grammar/model/communicator
-input_endian little little
-jsgf /usr/local/freeswitch/grammar/pizza_order.gram
-kdmaxbbi -1 -1
-kdmaxdepth 0 0
-kdtree
-latsize 5000 5000
-lda
-ldadim 0 0
-lextreedump 0 0
-lifter 0 0
-lm
-lmctl
-lmname default default
-logbase 1.0001 1.000100e+00
-logfn
-logspec no no
-lowerf 133.33334 1.333333e+02
-lpbeam 1e-40 1.000000e-40
-lponlybeam 7e-29 7.000000e-29
-lw 6.5 6.500000e+00
-maxhmmpf -1 -1
-maxnewoov 20 20
-maxwpf -1 -1
-mdef
-mean
-mfclogdir
-min_endfr 0 0
-mixw
-mixwfloor 0.0000001 1.000000e-07
-mllr
-mmap yes yes
-ncep 13 13
-nfft 512 512
-nfilt 40 40
-nwpen 1.0 1.000000e+00
-pbeam 1e-48 1.000000e-48
-pip 1.0 1.000000e+00
-pl_beam 1e-10 1.000000e-10
-pl_pbeam 1e-5 1.000000e-05
-pl_window 0 0
-rawlogdir
-remove_dc no no
-round_filters yes yes
-samprate 16000 8.000000e+03
-seed -1 -1
-sendump
-senlogdir
-senmgau
-silprob 0.005 5.000000e-03
-smoothspec no no
-svspec
-tmat
-tmatfloor 0.0001 1.000000e-04
-topn 4 4
-topn_beam 0 0
-toprule
-transform legacy legacy
-unit_area yes yes
-upperf 6855.4976 6.855498e+03
-usewdphones no no
-uw 1.0 1.000000e+00
-var
-varfloor 0.0001 1.000000e-04
-varnorm no no
-verbose no no
-warp_params
-warp_type inverse_linear inverse_linear
-wbeam 7e-29 7.000000e-29
-wip 0.65 6.500000e-01
-wlen 0.025625 2.562500e-02
INFO: cmd_ln.c(691): Parsing command line:
\
-alpha 0.97 \
-dither yes \
-doublebw no \
-nfilt 31 \
-ncep 13 \
-lowerf 200 \
-upperf 3500 \
-nfft 256 \
-wlen 0.0256 \
-transform legacy \
-feat s2_4x \
-agc none \
-cmn current \
-varnorm no
Current configuration:
[NAME] [DEFLT] [VALUE]
-agc none none
-agcthresh 2.0 2.000000e+00
-alpha 0.97 9.700000e-01
-ceplen 13 13
-cmn current current
-cmninit 8.0 8.0
-dither no yes
-doublebw no no
-feat 1s_c_d_dd s2_4x
-frate 100 50
-input_endian little little
-lda
-ldadim 0 0
-lifter 0 0
-logspec no no
-lowerf 133.33334 2.000000e+02
-ncep 13 13
-nfft 512 256
-nfilt 40 31
-remove_dc no no
-round_filters yes yes
-samprate 16000 8.000000e+03
-seed -1 -1
-smoothspec no no
-svspec
-transform legacy legacy
-unit_area yes yes
-upperf 6855.4976 3.500000e+03
-varnorm no no
-verbose no no
-warp_params
-warp_type inverse_linear inverse_linear
-wlen 0.025625 2.560000e-02
INFO: acmod.c(246): Parsed model-specific feature parameters from /usr/local/freeswitch/grammar/model/communicator/feat.params
INFO: fe_interface.c(299): You are using the internal mechanism to generate the seed.
INFO: feat.c(713): Initializing feature stream to type: 's2_4x', ceplen=13, CMN='current', VARNORM='no', AGC='none'
INFO: cmn.c(142): mean[0]= 12.00, mean[1..12]= 0.0
INFO: mdef.c(517): Reading model definition: /usr/local/freeswitch/grammar/model/communicator/mdef
INFO: bin_mdef.c(179): Allocating 104160 * 8 bytes (813 KiB) for CD tree
INFO: tmat.c(205): Reading HMM transition probability matrices: /usr/local/freeswitch/grammar/model/communicator/transition_matrices
INFO: acmod.c(121): Attempting to use SCHMM computation module
INFO: ms_gauden.c(198): Reading mixture gaussian parameter: /usr/local/freeswitch/grammar/model/communicator/means
INFO: ms_gauden.c(292): 1 codebook, 4 feature, size:
INFO: ms_gauden.c(294): 256x12
INFO: ms_gauden.c(294): 256x24
INFO: ms_gauden.c(294): 256x3
INFO: ms_gauden.c(294): 256x12
INFO: ms_gauden.c(198): Reading mixture gaussian parameter: /usr/local/freeswitch/grammar/model/communicator/variances
INFO: ms_gauden.c(292): 1 codebook, 4 feature, size:
INFO: ms_gauden.c(294): 256x12
INFO: ms_gauden.c(294): 256x24
INFO: ms_gauden.c(294): 256x3
INFO: ms_gauden.c(294): 256x12
INFO: ms_gauden.c(354): 59 variance values floored
INFO: s2_semi_mgau.c(903): Loading senones from dump file /usr/local/freeswitch/grammar/model/communicator/sendump
INFO: s2_semi_mgau.c(927): BEGIN FILE FORMAT DESCRIPTION
INFO: s2_semi_mgau.c(990): Rows: 256, Columns: 6256
INFO: s2_semi_mgau.c(1022): Using memory-mapped I/O for senones
INFO: s2_semi_mgau.c(1296): Maximum top-N: 4 Top-N beams: 0 0 0 0
INFO: dict.c(317): Allocating 137549 * 32 bytes (4298 KiB) for word entries
INFO: dict.c(332): Reading main dictionary: /usr/local/freeswitch/grammar/default.dic
INFO: dict.c(211): Allocated 1010 KiB for strings, 1664 KiB for phones
INFO: dict.c(335): 133436 words read
INFO: dict.c(341): Reading filler dictionary: /usr/local/freeswitch/grammar/model/communicator/noisedict
INFO: dict.c(211): Allocated 0 KiB for strings, 0 KiB for phones
INFO: dict.c(344): 17 words read
INFO: dict2pid.c(396): Building PID tables for dictionary
INFO: dict2pid.c(404): Allocating 51^3 * 2 bytes (259 KiB) for word-initial triphones
INFO: dict2pid.c(131): Allocated 62832 bytes (61 KiB) for word-final triphones
INFO: dict2pid.c(195): Allocated 62832 bytes (61 KiB) for single-phone word triphones
INFO: fsg_search.c(145): FSG(beam: -1080, pbeam: -1080, wbeam: -634; wip: -26, pip: 0)
INFO: jsgf.c(581): Defined rule: <pizza_order.g00000>
INFO: jsgf.c(581): Defined rule: PUBLIC <pizza_order.delivery>
INFO: fsg_model.c(215): Computing transitive closure for null transitions
INFO: fsg_model.c(270): 9 null transitions added
INFO: fsg_model.c(421): Adding silence transitions for <sil> to FSG
INFO: fsg_model.c(441): Added 8 silence word transitions
INFO: fsg_model.c(421): Adding silence transitions for ++AE++ to FSG
INFO: fsg_model.c(441): Added 8 silence word transitions
INFO: fsg_model.c(421): Adding silence transitions for ++AH++ to FSG
INFO: fsg_model.c(441): Added 8 silence word transitions
INFO: fsg_model.c(421): Adding silence transitions for ++BACKGROUND++ to FSG
INFO: fsg_model.c(441): Added 8 silence word transitions
INFO: fsg_model.c(421): Adding silence transitions for ++BREATH++ to FSG
INFO: fsg_model.c(441): Added 8 silence word transitions
INFO: fsg_model.c(421): Adding silence transitions for ++COUGH++ to FSG
INFO: fsg_model.c(441): Added 8 silence word transitions
INFO: fsg_model.c(421): Adding silence transitions for ++EH++ to FSG
INFO: fsg_model.c(441): Added 8 silence word transitions
INFO: fsg_model.c(421): Adding silence transitions for ++ER++ to FSG
INFO: fsg_model.c(441): Added 8 silence word transitions
INFO: fsg_model.c(421): Adding silence transitions for ++LAUGH++ to FSG
INFO: fsg_model.c(441): Added 8 silence word transitions
INFO: fsg_model.c(421): Adding silence transitions for ++MM++ to FSG
INFO: fsg_model.c(441): Added 8 silence word transitions
INFO: fsg_model.c(421): Adding silence transitions for ++MUMBLE++ to FSG
INFO: fsg_model.c(441): Added 8 silence word transitions
INFO: fsg_model.c(421): Adding silence transitions for ++NOISE++ to FSG
INFO: fsg_model.c(441): Added 8 silence word transitions
INFO: fsg_model.c(421): Adding silence transitions for ++OH++ to FSG
INFO: fsg_model.c(441): Added 8 silence word transitions
INFO: fsg_model.c(421): Adding silence transitions for ++SMACK++ to FSG
INFO: fsg_model.c(441): Added 8 silence word transitions
INFO: fsg_model.c(421): Adding silence transitions for ++UH++ to FSG
INFO: fsg_model.c(441): Added 8 silence word transitions
INFO: fsg_model.c(421): Adding silence transitions for ++UH_NOISE++ to FSG
INFO: fsg_model.c(441): Added 8 silence word transitions
INFO: fsg_model.c(421): Adding silence transitions for ++UM++ to FSG
INFO: fsg_model.c(441): Added 8 silence word transitions
INFO: fsg_model.c(421): Adding silence transitions for ++UM_NOISE++ to FSG
INFO: fsg_model.c(441): Added 8 silence word transitions
INFO: fsg_search.c(366): Added 0 alternate word transitions
INFO: fsg_lextree.c(108): Allocated 832 bytes (0 KiB) for left and right context phones
INFO: fsg_lextree.c(253): 213 HMM nodes in lextree (199 leaves)
INFO: fsg_lextree.c(255): Allocated 27264 bytes (26 KiB) for all lextree nodes
INFO: fsg_lextree.c(258): Allocated 25472 bytes (24 KiB) for lextree leafnodes
2014-10-14 15:13:25.442814 [NOTICE] switch_rtp.c:5132 Receiving an RTCP packet[2014-14-09 13:13:25.442953] SSRC[1123956418]RTT[0.001266] A[2683662693] - DLSR[22111] - LSR[2683640499]
INFO: cmn_prior.c(121): cmn_prior_update: from < 8.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 >
INFO: cmn_prior.c(139): cmn_prior_update: to < 7.58 0.08 -0.24 -0.08 -0.24 -0.18 -0.21 -0.15 -0.06 -0.18 -0.08 -0.11 -0.11 >
INFO: fsg_search.c(1032): 86 frames, 1666 HMMs (19/fr), 6967 senones (81/fr), 886 history entries (10/fr)
INFO: fsg_search.c(1417): Start node <sil>.0:22:85
INFO: fsg_search.c(1417): Start node <sil>.0:22:55
INFO: fsg_search.c(1417): Start node <sil>.0:22:85
INFO: fsg_search.c(1417): Start node <sil>.0:22:55
INFO: fsg_search.c(1417): Start node <sil>.0:22:85
INFO: fsg_search.c(1417): Start node takeout.0:21:33
INFO: fsg_search.c(1417): Start node pickup.0:19:71
INFO: fsg_search.c(1456): End node <sil>.56:58:85 (-1076)
INFO: fsg_search.c(1456): End node <sil>.56:58:85 (-1076)
INFO: fsg_search.c(1456): End node <sil>.56:58:85 (-1076)
INFO: fsg_search.c(1456): End node <sil>.26:28:85 (-1180)
INFO: fsg_search.c(1456): End node <sil>.0:22:85 (-6201)
INFO: fsg_search.c(1456): End node <sil>.0:22:85 (-6201)
INFO: fsg_search.c(1456): End node <sil>.0:22:85 (-6201)
INFO: fsg_search.c(1680): lattice start node <s>.0 end node </s>.86
INFO: ps_lattice.c(1365): Normalizer P(O) = alpha(</s>:86:86) = -333411
INFO: ps_lattice.c(1403): Joint P(O,S) = -333414 P(S|O) = -3
2014-10-14 15:13:28.822614 [WARNING] mod_pocketsphinx.c:348 Lost the text, never mind....
2014-10-14 15:13:30.922352 [NOTICE] switch_rtp.c:5132 Receiving an RTCP packet[2014-14-09 13:13:30.922476] SSRC[1123956418]RTT[0.001648] A[2684021799] - DLSR[53573] - LSR[2683968118]
2014-10-14 15:13:36.403317 [NOTICE] switch_rtp.c:5132 Receiving an RTCP packet[2014-14-09 13:13:36.403451] SSRC[1123956418]RTT[0.002731] A[2684381000] - DLSR[85028] - LSR[2684295793]
INFO: fsg_search.c(1032): 149 frames, 1750 HMMs (11/fr), 8700 senones (58/fr), 1006 history entries (6/fr)
INFO: fsg_search.c(1417): Start node <sil>.0:2:90
INFO: fsg_search.c(1417): Start node <sil>.0:2:90
INFO: fsg_search.c(1456): End node <sil>.122:124:148 (-955)
INFO: fsg_search.c(1456): End node <sil>.122:124:148 (-955)
INFO: fsg_search.c(1456): End node <sil>.122:124:148 (-955)
INFO: fsg_search.c(1456): End node pickup.87:107:148 (-4233)
INFO: fsg_search.c(1680): lattice start node <s>.0 end node </s>.149
INFO: ps_lattice.c(1365): Normalizer P(O) = alpha(</s>:149:149) = -927641
INFO: ps_lattice.c(1403): Joint P(O,S) = -927641 P(S|O) = 0
2014-10-14 15:13:41.883453 [NOTICE] switch_rtp.c:5132 Receiving an RTCP packet[2014-14-09 13:13:41.883618] SSRC[1123956418]RTT[0.002487] A[2684740148] - DLSR[116488] - LSR[2684623497]
2014-10-14 15:13:44.732381 [NOTICE] sofia.c:952 Hangup sofia/internal/1001#myip [CS_EXECUTE] [NORMAL_CLEARING]
2014-10-14 15:13:44.732381 [ERR] SpeechTools.jm:368 Exception: Session is not active! (near: " rv = this.asr.session.collectInput(this.asr.onInput, this.asr, 500);")
INFO: fsg_search.c(1032): 33 frames, 377 HMMs (11/fr), 1733 senones (52/fr), 275 history entries (8/fr)
2014-10-14 15:13:44.802526 [INFO] mod_pocketsphinx.c:257 Port Closed.
2014-10-14 15:13:44.823711 [NOTICE] switch_core_session.c:1633 Session 25 (sofia/internal/1001#myip) Ended
2014-10-14 15:13:44.823711 [NOTICE] switch_core_session.c:1637 Close Channel sofia/internal/1001#myip [CS_DESTROY]
EDIT 2:
I find out that the speech recognition works and it detect my speech. So the problem is that in SpeechTools.jm the result from the xml can not be load and is undefined.
body = body.replace(/<\?.*?\?>/g, '');
console_log("debug", "----XML:\n" + body + "\n");
xml = new XML("<xml>" + body + "</xml>");
result = xml.result; //undefined
and my output from console_log
<result grammar="pizza_order">
<interpretation grammar="pizza_order" confidence="100">
<input mode="speech">pickup</input>
</interpretation>
</result>
Okay, the speech recognition works the whole time (see edit). Real problem is that the whole script (SpeechTools.jm) is not working. They switched from mozilla javascript engine to google v8 without editing the script. However fixing the script is an javascript problem and has nothing to do with this question anymore.
Related
I try to run a rpc node, after download headers, it show some warn. It didn't start download block. Anyone can help me?
Mar 27 00:06:43.791 INFO neard: Version: 1.25.0, Build: crates-0.12.0-31-g9b3d6ba55, Latest Protocol: 52
Mar 27 00:06:43.796 INFO near: Opening store database at "/home/aurora/.near/mainnet/data"
Mar 27 00:06:44.033 INFO near_network::peer_manager::peer_manager_actor: Bandwidth stats total_bandwidth_used_by_all_peers=0 total_msg_received_count=0 max_max_record_num_messages_in_progress=0
Mar 27 00:06:54.078 INFO stats: #62165908 Waiting for peers 1 peer ⬇ 0.1kiB/s ⬆ 3 B/s 0.00 bps 0 gas/s
Mar 27 00:07:04.080 INFO stats: #62165908 Waiting for peers 2 peers ⬇ 507.6kiB/s ⬆ 62.4kiB/s 0.00 bps 0 gas/s CPU: 30%, Mem: 145.0 MiB
Mar 27 00:07:14.082 INFO stats: #62165908 Waiting for peers 2 peers ⬇ 1015.8kiB/s ⬆ 616.2kiB/s 0.00 bps 0 gas/s CPU: 156%, Mem: 206.6 MiB
Mar 27 00:07:24.085 INFO stats: #62165908 Downloading headers 100.00% (468) 4 peers ⬇ 1018.6kiB/s ⬆ 2.0MiB/s 0.00 bps 0 gas/s CPU: 14%, Mem: 314.0 MiB
Mar 27 00:07:34.088 INFO stats: #62165908 Downloading headers 100.00% (473) 5 peers ⬇ 1.5MiB/s ⬆ 2.0MiB/s 0.00 bps 0 gas/s CPU: 38%, Mem: 366.0 MiB
Mar 27 00:07:44.091 INFO stats: #62165908 Downloading headers 100.00% (484) 5 peers ⬇ 2.0MiB/s ⬆ 2.5MiB/s 0.00 bps 0 gas/s CPU: 7%, Mem: 378.2 MiB
Mar 27 00:07:44.362 WARN near_network::peer_manager::peer_manager_actor: Peer bandwidth exceeded threshold peer_id=ed25519:DCiEjHES1eRwj8zYbU5EdFyWQRa8zrjq7hhrBmb3Seop bandwidth_used=31287035 msg_received_count=124
Mar 27 00:07:44.362 WARN near_network::peer_manager::peer_manager_actor: Peer bandwidth exceeded threshold peer_id=ed25519:5DNVteGRxgUv4WSpJu4337aQud5P9m8TN1uAwAGzTjFP bandwidth_used=31235365 msg_received_count=123
Mar 27 00:07:44.362 WARN near_network::peer_manager::peer_manager_actor: Peer bandwidth exceeded threshold peer_id=ed25519:EVqGxpKoP3rxqMruuTmkQrhZnVE3um8XF2gwap4HhVqd bandwidth_used=31395111 msg_received_count=248
Mar 27 00:07:44.362 WARN near_network::peer_manager::peer_manager_actor: Peer bandwidth exceeded threshold peer_id=ed25519:CzYbBGrPx3XrdJeoNHs1YAz7TY3garm1RhZqruJXGPY5 bandwidth_used=34275095 msg_received_count=383
Mar 27 00:07:44.362 WARN near_network::peer_manager::peer_manager_actor: Peer bandwidth exceeded threshold peer_id=ed25519:GVWThRijD1ZyqTf6pNvS4BBYBd9gCB2P1Qjfi9DAMi7G bandwidth_used=10089910 msg_received_count=2
Mar 27 00:07:44.362 INFO near_network::peer_manager::peer_manager_actor: Bandwidth stats total_bandwidth_used_by_all_peers=138282516 total_msg_received_count=880 max_max_record_num_messages_in_progress=52
Mar 27 00:08:44.364 INFO near_network::peer_manager::peer_manager_actor: Bandwidth stats total_bandwidth_used_by_all_peers=16194710 total_msg_received_count=1826 max_max_record_num_messages_in_progress=53
Mar 27 00:09:44.365 INFO near_network::peer_manager::peer_manager_actor: Bandwidth stats total_bandwidth_used_by_all_peers=6119474 total_msg_received_count=1653 max_max_record_num_messages_in_progress=85
Mar 27 00:10:44.367 INFO near_network::peer_manager::peer_manager_actor: Bandwidth stats total_bandwidth_used_by_all_peers=10920258 total_msg_received_count=1615 max_max_record_num_messages_in_progress=110
Mar 27 00:11:44.368 INFO near_network::peer_manager::peer_manager_actor: Bandwidth stats total_bandwidth_used_by_all_peers=4072216 total_msg_received_count=1857 max_max_record_num_messages_in_progress=169
Mar 27 00:12:44.369 INFO near_network::peer_manager::peer_manager_actor: Bandwidth stats total_bandwidth_used_by_all_peers=1962496 total_msg_received_count=1741 max_max_record_num_messages_in_progress=249
Mar 27 00:13:44.370 INFO near_network::peer_manager::peer_manager_actor: Bandwidth stats total_bandwidth_used_by_all_peers=2022483 total_msg_received_count=1689 max_max_record_num_messages_in_progress=290
Mar 27 00:14:44.373 INFO near_network::peer_manager::peer_manager_actor: Bandwidth stats total_bandwidth_used_by_all_peers=1901363 total_msg_received_count=1710 max_max_record_num_messages_in_progress=332
Mar 27 00:15:44.374 INFO near_network::peer_manager::peer_manager_actor: Bandwidth stats total_bandwidth_used_by_all_peers=1901830 total_msg_received_count=1635 max_max_record_num_messages_in_progress=372
Mar 27 00:16:44.376 INFO near_network::peer_manager::peer_manager_actor: Bandwidth stats total_bandwidth_used_by_all_peers=1944505 total_msg_received_count=1753 max_max_record_num_messages_in_progress=414```
I have a command -
some command | egrep -i "($query_value)"
which returns lines like -
> 2018-01-12 23:17:05.097 Info: Deleted 6 MB at 1129 MB/sec
> /Meters/00002c97 2018-01-12 23:17:05.104 Info: Deleted 6 MB at 864
> MB/sec /Meters/00002c99 2018-01-12 23:26:00.751 Info: Saving
> /Meters/00002c9a 2018-01-12 23:26:01.235 Info: Saved 6 MB at 12 MB/sec
> to /Meters/00002c9a 2018-01-12 23:26:01.268 Info: Merging 12 MB from
> /Meters/00002c9b and /Meters/00002c9a to /Meters/00002c9d,
> timestamp=15158205609400820 2018-01-12 23:26:04.248 Info: Deleted 9 MB
> at 3898 MB/sec /Meters/00002c9b 2018-01-12 23:26:04.250 Info: Deleted
> 6 MB at 2973 MB/sec /Meters/00002c9a 2018-01-12 23:34:01.420 Info:
> Saving /Meters/00002c9c 2018-01-12 23:34:02.192 Info: Saved 6 MB in 1
> sec at 8 MB/sec to /Meters/00002c9c 2018-01-12 23:34:02.248 Info:
> Merging 44 MB from /Meters/00002c98, /Meters/00002c9d, and
> /Meters/00002c9c to /Meters/00002c9f, timestamp=15158210419400730
> 2018-01-12 23:34:07.173 Info: Deleted 35 MB at 9431 MB/sec
> /Meters/00002c98 2018-01-12 23:34:07.175 Info: Deleted 12 MB at 6633
> MB/sec /Meters/00002c9d 2018-01-12 23:34:07.177 Info: Deleted 6 MB at
> 2869 MB/sec /Meters/00002c9c 2018-01-12 23:42:00.842 Info: Saving
> /Meters/00002c9e 2018-01-12 23:42:01.323 Info: Saved 6 MB at 12 MB/sec
> to /Meters/00002c9e 2018-01-12 23:51:00.951 Info: Saving
> /Meters/00002ca0 2018-01-12 23:51:01.680 Info: Saved 6 MB in 1 sec at
> 8 MB/sec to /Meters/00002ca0 2018-01-12 23:51:01.684 Info: Merging 10
> MB from /Meters/00002c9e and /Meters/00002ca0 to /Meters/00002ca2,
> timestamp=15158220605927080 2018-01-12 23:51:05.083 Info: Deleted 6 MB
> at 2127 MB/sec /Meters/00002c9e 2018-01-12 23:51:05.085 Info: Deleted
> 6 MB at 2933 MB/sec /Meters/00002ca0 2018-01-12 23:58:01.110 Info:
> Saving /Meters/00002ca1 2018-01-12 23:58:02.011 Info: Saved 6 MB in 1
> sec at 7 MB/sec to /Meters/00002ca1 2018-01-12 23:58:02.017 Info:
> Merging 12 MB from /Meters/00002ca2 and /Meters/00002ca1 to
> /Meters/00002ca4, timestamp=15158224804450160 2018-01-12 23:58:05.096
> Info: Deleted 9 MB at 419 MB/sec /Meters/00002ca2 2018-01-12
> 23:58:05.105 Info: Deleted 6 MB at 668 MB/sec /Meters/00002ca1
but further I want to exclude lines that contains patterns like merging*mb*from|merged*mb*from. I don't want lines like -
Merging 12 MB from or Merged 12 MB from .
I tried using grep's -v option but seems like it doesn't understand case sensitivity and 'AND', 'OR' operations.
Following sed may help you in same.
sed -n '/Merging.*MB.*from/d;/Merged.*MB.*from/d;p' Input_file
OR
sed -n '/[mM]erging.*[mM][bB].*from/d;/Merged.*MB.*from/d;p' Input_file
EDIT: Adding explanation too here for above code.
sed -n ' ##-n will STOP printing by default for any line of Input_file till we print it explicitly.
/Merging.*MB.*from/d; ##Looking for string Merging.*MB.*from and using d will skip it and will NOT print it.
/Merged.*MB.*from/d; ##Looking for string Merged.*MB.*from and using d will skip it and will not print it.
p ##mentioning p will print the lines apart from lines which have above mentioned string in them
' Input_file ##Mentioning Input_file name here.
Try grep -E (extended regex) -i (ignore case) -v (invert match):
grep -E -i -v 'merg(ing|ed).*mb.*from'
If you think "ignore case" is slow, then:
grep -E -v '[Mm]erg(ing|ed).*[Mm][Bb].*[Ff]rom'
I have a simple program:
#include <iostream>
int main() {
std::cout << "Starting" << std::endl;
while (1) {
}
return 0;
}
I compile and run it:
clang -o test test.cpp
bash$ ./test
Starting
In another terminal, I examine it and kill it:
bash$ top
Processes: 284 total, 3 running, 9 stuck, 272 sleeping, 1505 threads 14:45:49
Load Avg: 1.86, 1.81, 2.06 CPU usage: 13.57% user, 0.96% sys, 85.45% idle
SharedLibs: 21M resident, 12M data, 0B linkedit.
MemRegions: 87372 total, 4974M resident, 86M private, 1203M shared.
PhysMem: 15G used (1887M wired), 1450M unused.
VM: 729G vsize, 1064M framework vsize, 3299196(0) swapins, 3711511(0) swapouts.
Networks: packets: 2686338/648M in, 2068186/355M out.
Disks: 1141818/44G read, 1926370/100G written.
PID COMMAND %CPU TIME #TH #WQ #PORT MEM PURG CMPRS PGRP PPID STATE
30161 test 100.3 02:12.71 1/1 0 9 312K 0B 0B 29470 28181 running
66064 Terminal 2.7 01:29.69 9 3 257 53M 0B 4580K 66064 1 sleeping
bash$ ls -al /cores/
total 917024
drwxrwxr-t# 3 root admin 102 May 1 14:54 .
drwxr-xr-x 33 root wheel 1190 Apr 14 09:13 ..
bash$ killall -SIGSEGV test
bash$ ls -al /cores/
total 1818048
drwxrwxr-t# 4 root admin 136 May 1 14:55 .
drwxr-xr-x 33 root wheel 1190 Apr 14 09:13 ..
-r-------- 1 stebro admin 469516288 May 1 14:54 core.30161
vmmap says:
==== Summary for process 50745
ReadOnly portion of Libraries: Total=316K resident=280K(89%) swapped_out_or_unallocated=36K(11%)
Writable regions: Total=40.3M written=16K(0%) resident=12.1M(30%) swapped_out=14.4M(36%) unallocated=28.3M(70%)
REGION TYPE VIRTUAL
=========== =======
Kernel Alloc Once 4K
MALLOC 9388K see MALLOC ZONE table below
MALLOC (admin) 24K
STACK GUARD 56.0M
Stack 8192K
VM_ALLOCATE 4K
__DATA 680K
__LINKEDIT 70.9M
__TEXT 5960K
shared memory 4K
=========== =======
TOTAL 150.6M
VIRTUAL ALLOCATION BYTES
MALLOC ZONE SIZE COUNT ALLOCATED % FULL
=========== ======= ========= ========= ======
DefaultMallocZone_0x104a0e000 9216K 357 27K 0%
Why is my core file 469 MB?
Your core dump includes the complete state of memory as well as symbol information for the frameworks and libraries that your application was running with at the time. That's a lot of data. For more information you might check out the core dumps section of technical note 2124
I need the real-time phoneme recognition from the microphone on Windows 8 Desktop. So I followed http://cmusphinx.sourceforge.net/wiki/phonemerecognition and built pocketsphinx_continuous from the subversion source in VS2013. Running it in the command line as Administrator:
D:\_SPHINX\cmusphinx-code-13103-trunk\pocketsphinx\bin\Release\Win32>pocketsphinx_continuous.exe -infile ../../../test/data/goforward.raw -hmm ../../../model/en-us/en-us -allphone ../../../model/en-us/en-us-phone.lm.bin -backtrace yes -beam 1e-20 -pbeam 1e-20 -lw 2.0
INFO: pocketsphinx.c(145): Parsed model-specific feature parameters from ../../../model/en-us/en-us/feat.params
Current configuration:
[NAME] [DEFLT] [VALUE]
-agc none none
-agcthresh 2.0 2.000000e+000
-allphone ../../../model/en-us/en-us-phone.lm.bin
-allphone_ci no no
-alpha 0.97 9.700000e-001
-ascale 20.0 2.000000e+001
-aw 1 1
-backtrace no yes
-beam 1e-48 1.000000e-020
-bestpath yes yes
-bestpathlw 9.5 9.500000e+000
-ceplen 13 13
-cmn current current
-cmninit 8.0 40,3,-1
-compallsen no no
-debug 0
-dict
-dictcase no no
-dither no no
-doublebw no no
-ds 1 1
-fdict ../../../model/en-us/en-us/noisedict
-feat 1s_c_d_dd 1s_c_d_dd
-featparams ../../../model/en-us/en-us/feat.params
-fillprob 1e-8 1.000000e-008
-frate 100 100
-fsg
-fsgusealtpron yes yes
-fsgusefiller yes yes
-fwdflat yes yes
-fwdflatbeam 1e-64 1.000000e-064
-fwdflatefwid 4 4
-fwdflatlw 8.5 8.500000e+000
-fwdflatsfwin 25 25
-fwdflatwbeam 7e-29 7.000000e-029
-fwdtree yes yes
-hmm ../../../model/en-us/en-us
-input_endian little little
-jsgf
-keyphrase
-kws
-kws_delay 10 10
-kws_plp 1e-1 1.000000e-001
-kws_threshold 1 1.000000e+000
-latsize 5000 5000
-lda
-ldadim 0 0
-lifter 0 22
-lm
-lmctl
-lmname
-logbase 1.0001 1.000100e+000
-logfn
-logspec no no
-lowerf 133.33334 1.300000e+002
-lpbeam 1e-40 1.000000e-040
-lponlybeam 7e-29 7.000000e-029
-lw 6.5 2.000000e+000
-maxhmmpf 30000 30000
-maxwpf -1 -1
-mdef ../../../model/en-us/en-us/mdef
-mean ../../../model/en-us/en-us/means
-mfclogdir
-min_endfr 0 0
-mixw
-mixwfloor 0.0000001 1.000000e-007
-mllr
-mmap yes yes
-ncep 13 13
-nfft 512 512
-nfilt 40 25
-nwpen 1.0 1.000000e+000
-pbeam 1e-48 1.000000e-020
-pip 1.0 1.000000e+000
-pl_beam 1e-10 1.000000e-010
-pl_pbeam 1e-10 1.000000e-010
-pl_pip 1.0 1.000000e+000
-pl_weight 3.0 3.000000e+000
-pl_window 5 5
-rawlogdir
-remove_dc no no
-remove_noise yes yes
-remove_silence yes yes
-round_filters yes yes
-samprate 16000 1.600000e+004
-seed -1 -1
-sendump ../../../model/en-us/en-us/sendump
-senlogdir
-senmgau
-silprob 0.005 5.000000e-003
-smoothspec no no
-svspec 0-12/13-25/26-38
-tmat ../../../model/en-us/en-us/transition_matrices
-tmatfloor 0.0001 1.000000e-004
-topn 4 4
-topn_beam 0 0
-toprule
-transform legacy dct
-unit_area yes yes
-upperf 6855.4976 6.800000e+003
-uw 1.0 1.000000e+000
-vad_postspeech 50 50
-vad_prespeech 20 20
-vad_startspeech 10 10
-vad_threshold 2.0 2.000000e+000
-var ../../../model/en-us/en-us/variances
-varfloor 0.0001 1.000000e-004
-varnorm no no
-verbose no no
-warp_params
-warp_type inverse_linear inverse_linear
-wbeam 7e-29 7.000000e-029
-wip 0.65 6.500000e-001
-wlen 0.025625 2.562500e-002
INFO: feat.c(715): Initializing feature stream to type: '1s_c_d_dd', ceplen=13, CMN='current', VARNORM='no', AGC='none'
INFO: cmn.c(143): mean[0]= 12.00, mean[1..12]= 0.0
INFO: acmod.c(164): Using subvector specification 0-12/13-25/26-38
INFO: mdef.c(518): Reading model definition: ../../../model/en-us/en-us/mdef
INFO: mdef.c(531): Found byte-order mark BMDF, assuming this is a binary mdef file
INFO: bin_mdef.c(336): Reading binary model definition: ../../../model/en-us/en-us/mdef
INFO: bin_mdef.c(516): 42 CI-phone, 137053 CD-phone, 3 emitstate/phone, 126 CI-sen, 5126 Sen, 29324 Sen-Seq
INFO: tmat.c(206): Reading HMM transition probability matrices: ../../../model/en-us/en-us/transition_matrices
at the last INFO line Windows 8 throws this error:
Is anything wrong with PocketSphinx debug output, or my command-line options? Or it is a pure Windows problem? I noticed this folder: /bin/Release/Win32. My Windows 8 is 64bit on Intel NUC. Sphinxbase.dll was compiled from subversion in Debug mode, while PacketSphinx had only Release mode.
Also I read somewhere that phonemes timing information is available - how to get it?
ADDITION: following Nikolay's advice, with these parameters, I eliminated errors, but got no phonemes:
D:\_SPHINX\pocketsphinx\bin\Debug>pocketsphinx_continuous.exe -infile ../../test/data/goforward.raw -hmm ../../model/en-us/en-us -allphone ../../model/en-us/en-us.lm.dmp -backtrace yes -beam 1e-20 -pbeam 1e-20 -lw 2.0 -debug 3 -verbose yes
INFO: cmd_ln.c(697): Parsing command line:
pocketsphinx_continuous.exe \
-infile ../../test/data/goforward.raw \
-hmm ../../model/en-us/en-us \
-allphone ../../model/en-us/en-us.lm.dmp \
-backtrace yes \
-beam 1e-20 \
-pbeam 1e-20 \
-lw 2.0 \
-debug 3 \
-verbose yes
. . . .
INFO: acmod.c(252): Parsed model-specific feature parameters from ../../model/en-us/en-us/feat.params
INFO: fe_interface.c(177): Current FE Parameters:
INFO: fe_interface.c(178): Sampling Rate: 16000.000000
INFO: fe_interface.c(179): Frame Size: 410
INFO: fe_interface.c(180): Frame Shift: 160
INFO: fe_interface.c(181): FFT Size: 512
INFO: fe_interface.c(183): Lower Frequency: 130
INFO: fe_interface.c(185): Upper Frequency: 6800
INFO: fe_interface.c(186): Number of filters: 25
INFO: fe_interface.c(187): Number of Overflow Samps: 0
INFO: fe_interface.c(188): Start Utt Status: 0
INFO: fe_interface.c(190): Will not remove DC offset at frame level
INFO: fe_interface.c(196): Will not add dither to audio
INFO: fe_interface.c(200): Will apply sine-curve liftering, period 22
INFO: fe_interface.c(203): Will normalize filters to unit area
INFO: fe_interface.c(205): Will round filter frequencies to DFT points
INFO: fe_interface.c(207): Will not use double bandwidth in mel filter
INFO: feat.c(715): Initializing feature stream to type: '1s_c_d_dd', ceplen=13, CMN='current', VARNORM='no', AGC='none'
INFO: cmn.c(143): mean[0]= 12.00, mean[1..12]= 0.0
INFO: acmod.c(171): Using subvector specification 0-12/13-25/26-38
INFO: mdef.c(518): Reading model definition: ../../model/en-us/en-us/mdef
INFO: mdef.c(531): Found byte-order mark BMDF, assuming this is a binary mdef file
INFO: bin_mdef.c(336): Reading binary model definition: ../../model/en-us/en-us/mdef
INFO: bin_mdef.c(516): 42 CI-phone, 137053 CD-phone, 3 emitstate/phone, 126 CI-sen, 5126 Sen, 29324 Sen-Seq
INFO: tmat.c(206): Reading HMM transition probability matrices: ../../model/en-us/en-us/transition_matrices
INFO: acmod.c(124): Attempting to use PTM computation module
INFO: ms_gauden.c(198): Reading mixture gaussian parameter: ../../model/en-us/en-us/means
INFO: ms_gauden.c(292): 42 codebook, 3 feature, size:
INFO: ms_gauden.c(294): 128x13
INFO: ms_gauden.c(294): 128x13
INFO: ms_gauden.c(294): 128x13
INFO: ms_gauden.c(198): Reading mixture gaussian parameter: ../../model/en-us/en-us/variances
INFO: ms_gauden.c(292): 42 codebook, 3 feature, size:
INFO: ms_gauden.c(294): 128x13
INFO: ms_gauden.c(294): 128x13
INFO: ms_gauden.c(294): 128x13
INFO: ms_gauden.c(354): 222 variance values floored
INFO: ptm_mgau.c(476): Loading senones from dump file ../../model/en-us/en-us/sendump
INFO: ptm_mgau.c(500): BEGIN FILE FORMAT DESCRIPTION
INFO: ptm_mgau.c(563): Rows: 128, Columns: 5126
INFO: ptm_mgau.c(595): Using memory-mapped I/O for senones
INFO: ptm_mgau.c(835): Maximum top-N: 4
INFO: phone_loop_search.c(115): State beam -225 Phone exit beam -225 Insertion penalty 0
INFO: dict.c(320): Allocating 4101 * 20 bytes (80 KiB) for word entries
INFO: dict.c(342): Reading filler dictionary: ../../model/en-us/en-us/noisedict
INFO: dict.c(213): Allocated 0 KiB for strings, 0 KiB for phones
INFO: dict.c(345): 5 words read
INFO: dict2pid.c(396): Building PID tables for dictionary
INFO: dict2pid.c(406): Allocating 42^3 * 2 bytes (144 KiB) for word-initial triphones
INFO: dict2pid.c(132): Allocated 21336 bytes (20 KiB) for word-final triphones
INFO: dict2pid.c(196): Allocated 21336 bytes (20 KiB) for single-phone word triphones
INFO: ngram_model_arpa.c(77): No \data\ mark in LM file
INFO: ngram_model_dmp.c(142): Will use memory-mapped I/O for LM file
INFO: ngram_model_dmp.c(196): ngrams 1=19794, 2=1377200, 3=3178194
INFO: ngram_model_dmp.c(242): 19794 = LM.unigrams(+trailer) read
INFO: ngram_model_dmp.c(288): 1377200 = LM.bigrams(+trailer) read
INFO: ngram_model_dmp.c(314): 3178194 = LM.trigrams read
INFO: ngram_model_dmp.c(339): 57155 = LM.prob2 entries read
INFO: ngram_model_dmp.c(359): 10935 = LM.bo_wt2 entries read
INFO: ngram_model_dmp.c(379): 34843 = LM.prob3 entries read
INFO: ngram_model_dmp.c(407): 2690 = LM.tseg_base entries read
INFO: ngram_model_dmp.c(463): 19794 = ascii word strings read
INFO: allphone_search.c(239): Building PHMM net of 137095 phones
INFO: allphone_search.c(312): 29324 nodes, 1958591 links
INFO: allphone_search.c(611): Allphone(beam: -450, pbeam: -450)
INFO: continuous.c(299): pocketsphinx_continuous.exe COMPILED ON: Aug 23 2015, AT: 14:00:33
INFO: cmn_prior.c(131): cmn_prior_update: from < 40.00 3.00 -1.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 >
INFO: cmn_prior.c(149): cmn_prior_update: to < 44.50 -4.13 0.15 6.94 4.06 -5.38 -2.56 -3.13 -6.12 -1.20 -7.44 -2.25 0.48 >
INFO: allphone_search.c(852): 214 frames, 214 HMMs (1/fr), 642 senones (3/fr), 214 history entries (1/fr)
INFO: allphone_search.c(865): allphone 0.61 CPU 0.283 xRT
INFO: allphone_search.c(867): allphone 0.62 wall 0.290 xRT
INFO: allphone_search.c(911): Hyp: SIL
INFO: pocketsphinx.c(1133): SIL (-858993460)
word start end pprob ascr lscr lback
SIL 51 264 1.000 -1627 0 0
INFO: allphone_search.c(911): Hyp: SIL
SIL
INFO: cmn_prior.c(131): cmn_prior_update: from < 44.50 -4.13 0.15 6.94 4.06 -5.38 -2.56 -3.13 -6.12 -1.20 -7.44 -2.25 0.48 >
INFO: cmn_prior.c(149): cmn_prior_update: to < 44.50 -4.13 0.15 6.94 4.06 -5.38 -2.56 -3.13 -6.12 -1.20 -7.44 -2.25 0.48 >
INFO: allphone_search.c(852): 0 frames, 0 HMMs (0/fr), 0 senones (0/fr), 0 history entries (0/fr)
INFO: allphone_search.c(651): TOTAL fwdflat 0.61 CPU 0.285 xRT
INFO: allphone_search.c(654): TOTAL fwdflat 0.64 wall 0.298 xRT
What is the correct set of command-line parameters, to get phonemes output?
Problems:
More and more data nodes become bad health in Cloudera Manager.
Clue1:
no any task or job, just an idle data node here,
top
-bash-4.1$ top
top - 18:27:22 up 4:59, 3 users, load average: 4.55, 3.52, 3.18
Tasks: 139 total, 1 running, 137 sleeping, 1 stopped, 0 zombie
Cpu(s): 14.8%us, 85.2%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 7932720k total, 1243372k used, 6689348k free, 52244k buffers
Swap: 6160376k total, 0k used, 6160376k free, 267228k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
13766 root 20 0 2664m 21m 7048 S 85.4 0.3 190:34.75 java
17688 root 20 0 2664m 19m 7048 S 75.5 0.3 1:05.97 java
12765 root 20 0 2859m 21m 7140 S 36.9 0.3 133:25.46 java
2909 mapred 20 0 1894m 113m 14m S 1.0 1.5 2:55.26 java
1850 root 20 0 1469m 62m 4436 S 0.7 0.8 2:54.53 python
1332 root 20 0 50000 3000 2424 S 0.3 0.0 0:12.04 vmtoolsd
2683 hbase 20 0 1927m 152m 18m S 0.3 2.0 0:36.64 java
Clue2:
-bash-4.1$ ps -ef|grep 13766
root 13766 1850 99 16:01 ? 03:12:54 java -classpath /usr/share/cmf/lib/agent-4.6.3.jar com.cloudera.cmon.agent.DnsTest
Clue3:
in cloudera-scm-agent.log,
[30/Aug/2013 16:01:58 +0000] 1850 Monitor-HostMonitor throttling_logger ERROR Timeout with args ['java', '-classpath', '/usr/share/cmf/lib/agent-4.6.3.jar', 'com.cloudera.cmon.agent.DnsTest']
None
[30/Aug/2013 16:01:58 +0000] 1850 Monitor-HostMonitor throttling_logger ERROR Failed to collect java-based DNS names
Traceback (most recent call last):
File "/usr/lib64/cmf/agent/src/cmf/monitor/host/dns_names.py", line 53, in collect
result, stdout, stderr = self._subprocess_with_timeout(args, self._poll_timeout)
File "/usr/lib64/cmf/agent/src/cmf/monitor/host/dns_names.py", line 42, in _subprocess_with_timeout
return SubprocessTimeout().subprocess_with_timeout(args, timeout)
File "/usr/lib64/cmf/agent/src/cmf/monitor/host/subprocess_timeout.py", line 70, in subprocess_with_timeout
raise Exception("timeout with args %s" % args)
Exception: timeout with args ['java', '-classpath', '/usr/share/cmf/lib/agent-4.6.3.jar', 'com.cloudera.cmon.agent.DnsTest']
"cloudera-scm-agent.log" line 30357 of 30357 --100%-- col 1
Backgrouds:
if I restart all nodes, then everythings are OK, but after half and hour or more, bad health is coming one by one.
Version: Cloudera Standard 4.6.3 (#192 built by jenkins on 20130812-1221 git: fa61cf8559fbefeb5af7f223fd02164d1a0adfdb)
I added all nodes in /etc/hosts
the installed CDH is 4.3.1.
in fact, these nodes are VMs with fixed IP address.
Any suggestions?
BTW, where can I download source code of com.cloudera.cmon.agent.DnsTest?