near indexer does not add anything to the database - nearprotocol
I've tried to run https://github.com/near/near-indexer-for-explorer
No firewall, IP accessable (tested right now).
With empty data, it waits for peers forever.
With data from the run started some days ago
./target/release/indexer-explorer --home-dir ../.near/mainnet run --store-genesis --stream-while-syncing --allow-missing-relations-in-first-blocks 1000 sync-from-latest
It does something
Nov 01 18:42:23.293 INFO indexer_for_explorer: AccessKeys from genesis were added/updated successful.
Nov 01 18:42:33.188 INFO stats: # 9820210 Waiting for peers 1/1/40 peers ⬇ 0 B/s ⬆ 0 B/s 0.00 bps 0 gas/s CPU: 0%, Mem: 0 B
Nov 01 18:42:43.190 INFO stats: # 9820210 Downloading headers 68.72% (13074549) 3/3/40 peers ⬇ 149.3kiB/s ⬆ 6.0kiB/s 0.00 bps 0 gas/s CPU: 23%, Mem: 510.7 MiB
Nov 01 18:42:53.192 INFO stats: # 9820210 Downloading headers 68.72% (13074559) 2/2/40 peers ⬇ 299.4kiB/s ⬆ 297.5kiB/s 0.00 bps 0 gas/s CPU: 40%, Mem: 621.3 MiB
Nov 01 18:43:03.194 INFO stats: # 9820210 Downloading headers 68.72% (13074569) 1/1/40 peers ⬇ 150.1kiB/s ⬆ 148.9kiB/s 0.00 bps 0 gas/s CPU: 42%, Mem: 520.7 MiB
Nov 01 18:43:13.196 INFO stats: # 9820210 Downloading headers 68.72% (13074578) 2/2/40 peers ⬇ 150.3kiB/s ⬆ 148.8kiB/s 0.00 bps 0 gas/s CPU: 10%, Mem: 631.6 MiB
Nov 01 18:43:23.198 INFO stats: # 9820210 Downloading headers 68.72% (13074590) 2/1/40 peers ⬇ 294.1kiB/s ⬆ 297.6kiB/s 0.00 bps 0 gas/s CPU: 14%, Mem: 601.5 MiB
Nov 01 18:43:33.200 INFO stats: # 9820210 Downloading headers 68.72% (13074598) 1/1/40 peers ⬇ 149.4kiB/s ⬆ 148.8kiB/s 0.00 bps 0 gas/s CPU: 2%, Mem: 602.9 MiB
Nov 01 18:43:43.203 INFO stats: # 9820210 EPnLgE7iEq9s7yTkos96M3cWymH5avBAPm3qx3NXqR8H -/4 2/2/40 peers ⬇ 150.0kiB/s ⬆ 148.8kiB/s 0.00 bps 0 gas/s CPU: 9%, Mem: 657.0 MiB
Nov 01 18:43:53.209 INFO stats: # 9820210 Downloading headers 68.72% (13074608) 1/1/40 peers ⬇ 150.5kiB/s ⬆ 148.8kiB/s 0.00 bps 0 gas/s CPU: 3%, Mem: 661.0 MiB
Nov 01 18:44:03.212 INFO stats: # 9820210 EPnLgE7iEq9s7yTkos96M3cWymH5avBAPm3qx3NXqR8H -/4 1/1/40 peers ⬇ 148.6kiB/s ⬆ 148.8kiB/s 0.00 bps 0 gas/s CPU: 4%, Mem: 664.8 MiB
Nov 01 18:44:13.213 INFO stats: # 9820210 EPnLgE7iEq9s7yTkos96M3cWymH5avBAPm3qx3NXqR8H -/4 0/0/40 peers ⬇ 0 B/s ⬆ 0 B/s 0.00 bps 0 gas/s CPU: 2%, Mem: 664.8 MiB
Nov 01 18:44:23.215 INFO stats: # 9820210 EPnLgE7iEq9s7yTkos96M3cWymH5avBAPm3qx3NXqR8H -/4 0/0/40 peers ⬇ 0 B/s ⬆ 0 B/s 0.00 bps 0 gas/s CPU: 1%, Mem: 666.8 MiB
Nov 01 18:44:33.217 INFO stats: # 9820210 Downloading headers 68.72% (13074655) 1/1/40 peers ⬇ 150.0kiB/s ⬆ 148.8kiB/s 0.00 bps 0 gas/s CPU: 11%, Mem: 614.7 MiB
Nov 01 18:44:43.219 INFO stats: # 9820210 EPnLgE7iEq9s7yTkos96M3cWymH5avBAPm3qx3NXqR8H -/4 0/0/40 peers ⬇ 0 B/s ⬆ 0 B/s 0.00 bps 0 gas/s CPU: 1%, Mem: 614.9 MiB
Nov 01 18:44:53.224 INFO stats: # 9820210 EPnLgE7iEq9s7yTkos96M3cWymH5avBAPm3qx3NXqR8H -/4 0/0/40 peers ⬇ 0 B/s ⬆ 0 B/s 0.00 bps 0 gas/s CPU: 1%, Mem: 614.9 MiB
Nov 01 18:45:03.227 INFO stats: # 9820210 EPnLgE7iEq9s7yTkos96M3cWymH5avBAPm3qx3NXqR8H -/4 0/0/40 peers ⬇ 0 B/s ⬆ 0 B/s 0.00 bps 0 gas/s CPU: 1%, Mem: 616.4 MiB
Nov 01 18:45:13.232 INFO stats: # 9820210 EPnLgE7iEq9s7yTkos96M3cWymH5avBAPm3qx3NXqR8H -/4 0/0/40 peers ⬇ 0 B/s ⬆ 0 B/s 0.00 bps 0 gas/s CPU: 1%, Mem: 616.4 MiB
But nothing get added to database.
What am I doing wrong?
Your concern about the indexing part (no data in the database) will get resolved once the node reaches “syncing blocks” stage. Currently, your node is still only at “syncing block headers” stage. To speed up this process, start from a backup: https://docs.near.org/docs/develop/node/validator/running-a-node#starting-a-node-from-backup
As to the fact that the node dropped off the p2p network, I have no clues why that could have happened. I recommend you start with running a simple neard node and report any issues there before you get to the Indexer (which is just an extension to nearcore and thus you can use the same home & data folder)
First of all, indexer requires full archive mode. Links to 5-epoch are misleading. They are not usable for indexer.
Second (may save lots of time for downloading), indexer requires AVX extension to run. If your CPU does not AVX, don't bother building nearcore. That should be mentioned in docs. nearcore depends on some wasm, wasm requires AVX to run. Indexer will run for some time and fail miserably without AVX.
Related
Cannot download blocks
I try to run a rpc node, after download headers, it show some warn. It didn't start download block. Anyone can help me? Mar 27 00:06:43.791 INFO neard: Version: 1.25.0, Build: crates-0.12.0-31-g9b3d6ba55, Latest Protocol: 52 Mar 27 00:06:43.796 INFO near: Opening store database at "/home/aurora/.near/mainnet/data" Mar 27 00:06:44.033 INFO near_network::peer_manager::peer_manager_actor: Bandwidth stats total_bandwidth_used_by_all_peers=0 total_msg_received_count=0 max_max_record_num_messages_in_progress=0 Mar 27 00:06:54.078 INFO stats: #62165908 Waiting for peers 1 peer ⬇ 0.1kiB/s ⬆ 3 B/s 0.00 bps 0 gas/s Mar 27 00:07:04.080 INFO stats: #62165908 Waiting for peers 2 peers ⬇ 507.6kiB/s ⬆ 62.4kiB/s 0.00 bps 0 gas/s CPU: 30%, Mem: 145.0 MiB Mar 27 00:07:14.082 INFO stats: #62165908 Waiting for peers 2 peers ⬇ 1015.8kiB/s ⬆ 616.2kiB/s 0.00 bps 0 gas/s CPU: 156%, Mem: 206.6 MiB Mar 27 00:07:24.085 INFO stats: #62165908 Downloading headers 100.00% (468) 4 peers ⬇ 1018.6kiB/s ⬆ 2.0MiB/s 0.00 bps 0 gas/s CPU: 14%, Mem: 314.0 MiB Mar 27 00:07:34.088 INFO stats: #62165908 Downloading headers 100.00% (473) 5 peers ⬇ 1.5MiB/s ⬆ 2.0MiB/s 0.00 bps 0 gas/s CPU: 38%, Mem: 366.0 MiB Mar 27 00:07:44.091 INFO stats: #62165908 Downloading headers 100.00% (484) 5 peers ⬇ 2.0MiB/s ⬆ 2.5MiB/s 0.00 bps 0 gas/s CPU: 7%, Mem: 378.2 MiB Mar 27 00:07:44.362 WARN near_network::peer_manager::peer_manager_actor: Peer bandwidth exceeded threshold peer_id=ed25519:DCiEjHES1eRwj8zYbU5EdFyWQRa8zrjq7hhrBmb3Seop bandwidth_used=31287035 msg_received_count=124 Mar 27 00:07:44.362 WARN near_network::peer_manager::peer_manager_actor: Peer bandwidth exceeded threshold peer_id=ed25519:5DNVteGRxgUv4WSpJu4337aQud5P9m8TN1uAwAGzTjFP bandwidth_used=31235365 msg_received_count=123 Mar 27 00:07:44.362 WARN near_network::peer_manager::peer_manager_actor: Peer bandwidth exceeded threshold peer_id=ed25519:EVqGxpKoP3rxqMruuTmkQrhZnVE3um8XF2gwap4HhVqd bandwidth_used=31395111 msg_received_count=248 Mar 27 00:07:44.362 WARN near_network::peer_manager::peer_manager_actor: Peer bandwidth exceeded threshold peer_id=ed25519:CzYbBGrPx3XrdJeoNHs1YAz7TY3garm1RhZqruJXGPY5 bandwidth_used=34275095 msg_received_count=383 Mar 27 00:07:44.362 WARN near_network::peer_manager::peer_manager_actor: Peer bandwidth exceeded threshold peer_id=ed25519:GVWThRijD1ZyqTf6pNvS4BBYBd9gCB2P1Qjfi9DAMi7G bandwidth_used=10089910 msg_received_count=2 Mar 27 00:07:44.362 INFO near_network::peer_manager::peer_manager_actor: Bandwidth stats total_bandwidth_used_by_all_peers=138282516 total_msg_received_count=880 max_max_record_num_messages_in_progress=52 Mar 27 00:08:44.364 INFO near_network::peer_manager::peer_manager_actor: Bandwidth stats total_bandwidth_used_by_all_peers=16194710 total_msg_received_count=1826 max_max_record_num_messages_in_progress=53 Mar 27 00:09:44.365 INFO near_network::peer_manager::peer_manager_actor: Bandwidth stats total_bandwidth_used_by_all_peers=6119474 total_msg_received_count=1653 max_max_record_num_messages_in_progress=85 Mar 27 00:10:44.367 INFO near_network::peer_manager::peer_manager_actor: Bandwidth stats total_bandwidth_used_by_all_peers=10920258 total_msg_received_count=1615 max_max_record_num_messages_in_progress=110 Mar 27 00:11:44.368 INFO near_network::peer_manager::peer_manager_actor: Bandwidth stats total_bandwidth_used_by_all_peers=4072216 total_msg_received_count=1857 max_max_record_num_messages_in_progress=169 Mar 27 00:12:44.369 INFO near_network::peer_manager::peer_manager_actor: Bandwidth stats total_bandwidth_used_by_all_peers=1962496 total_msg_received_count=1741 max_max_record_num_messages_in_progress=249 Mar 27 00:13:44.370 INFO near_network::peer_manager::peer_manager_actor: Bandwidth stats total_bandwidth_used_by_all_peers=2022483 total_msg_received_count=1689 max_max_record_num_messages_in_progress=290 Mar 27 00:14:44.373 INFO near_network::peer_manager::peer_manager_actor: Bandwidth stats total_bandwidth_used_by_all_peers=1901363 total_msg_received_count=1710 max_max_record_num_messages_in_progress=332 Mar 27 00:15:44.374 INFO near_network::peer_manager::peer_manager_actor: Bandwidth stats total_bandwidth_used_by_all_peers=1901830 total_msg_received_count=1635 max_max_record_num_messages_in_progress=372 Mar 27 00:16:44.376 INFO near_network::peer_manager::peer_manager_actor: Bandwidth stats total_bandwidth_used_by_all_peers=1944505 total_msg_received_count=1753 max_max_record_num_messages_in_progress=414```
(NEAR protocol) How to get the Validator Node up
I am trying to run a validator node using the https://docs.near.org/docs/develop/node/validator/deploy-on-mainnet instruction. I have successfully deployed mainnet Staking Pool with the following command (2nd step of the instruction):
near call poolv1.near create_staking_pool '{"staking_pool_id":"<name_of_pool>", "owner_id":"<wallet_name>.near", "stake_public_key":"ed25519:3QohztWwCktk3j3MBiCuGaB6vXxeqjUasLan6ChSnduh", "reward_fee_fraction": {"numerator": 3, "denominator": 100}}' --account_id <wallet_name>.near --amount 30 --gas 300000000000000
The transaction
https://explorer.mainnet.near.org/transactions/93xQC8UozL6toVddkPk14qiExdRZMt3gJqCfHz9BBNpV
But after starting NEAR node, the database synchronization does not start (3nd step of the instruction).
target/release/neard run
The operating system listens on ports 3030 and 24567. Both ports are open in the Firewall.
I needed to add boot_nodes to config.json. rm ~/.near/config.json wget -O ~/.near/config.json https://s3-us-west-1.amazonaws.com/build.nearprotocol.com/nearcore-deploy/mainnet/config.json After that, the node started Nov 01 12:08:09.253 INFO stats: #51601485 EgEtVtQuGmvAkpRdjMb6rRwc7qjh2wYEKtmoWuqTDxf5 -/60 33/27/40 peers ⬇ 314.7kiB/s ⬆ 252.9kiB/s 0.80 bps 100.56 Tgas/s CPU: 26%, Mem: 6.0 GiB Nov 01 12:08:19.256 INFO stats: #51601494 73BhJpUURFQU36Npgi5VajgJxPNdfuF3TJKFHneeA3Pd -/60 33/27/40 peers ⬇ 318.9kiB/s ⬆ 253.2kiB/s 0.90 bps 109.87 Tgas/s CPU: 24%, Mem: 6.0 GiB Nov 01 12:08:29.299 INFO stats: #51601503 4o7iuNUaypY6KQphhrEYoGU5ZRECyPwyM7Vge8AVnsYq -/60 33/27/40 peers ⬇ 313.7kiB/s ⬆ 247.1kiB/s 0.90 bps 116.37 Tgas/s CPU: 28%, Mem: 6.0 GiB Nov 01 12:08:39.303 INFO stats: #51601511 GxQePJAQBrirSNwt7uraNeWqXGFZG1YkpKLJfX4Wd9Pb -/60 33/27/40 peers ⬇ 313.6kiB/s ⬆ 247.9kiB/s 0.80 bps 78.89 Tgas/s CPU: 22%, Mem: 6.0 GiB Nov 01 12:08:49.306 INFO stats: #51601520 9HzEv22KpcGmip83s9T3FqmKB9qbH6Mu1Cdh8cULf9BN -/60 33/27/40 peers ⬇ 319.0kiB/s ⬆ 250.8kiB/s 0.90 bps 98.36 Tgas/s CPU: 25%, Mem: 6.0 GiB Nov 01 12:08:59.309 INFO stats: #51601529 7NaEtQW8qp1jtZ3CbzRFcjzjjFx4DLtT9bUCc1CVRCB8 -/60 33/27/40 peers ⬇ 318.7kiB/s ⬆ 250.9kiB/s 0.90 bps 115.87 Tgas/s CPU: 24%, Mem: 6.0 GiB
Hadoop: Diagnose long running job
I need help with diagnosing why a particular Job in Jobtracker is long-running and workarounds for improving it. Here is an excerpt of the job in question (please pardon the formatting): Hadoop job_201901281553_38848 User: mapred Job-ACLs: All users are allowed Job Setup: Successful Status: Running Started at: Fri Feb 01 12:39:05 CST 2019 Running for: 3hrs, 23mins, 58sec Job Cleanup: Pending Kind % Complete Num Tasks Pending Running Complete Killed Failed/Killed Task Attempts map 100.00% 1177 0 0 1177 0 0 / 0 reduce 95.20% 12 0 2 10 0 0 / 0 Counter Map Reduce Total File System Counters FILE: Number of bytes read 1,144,088,621 1,642,723,691 2,786,812,312 FILE: Number of bytes written 3,156,884,366 1,669,567,665 4,826,452,031 FILE: Number of read operations 0 0 0 FILE: Number of large read operations 0 0 0 FILE: Number of write operations 0 0 0 HDFS: Number of bytes read 11,418,749,621 0 11,418,749,621 HDFS: Number of bytes written 0 8,259,932,078 8,259,932,078 HDFS: Number of read operations 2,365 5 2,370 HDFS: Number of large read operations 0 0 0 HDFS: Number of write operations 0 12 12 Job Counters Launched map tasks 0 0 1,177 Launched reduce tasks 0 0 12 Data-local map tasks 0 0 1,020 Rack-local map tasks 0 0 157 Total time spent by all maps in occupied slots (ms) 0 0 4,379,522 Total time spent by all reduces in occupied slots (ms) 0 0 81,115,664 Map-Reduce Framework Map input records 77,266,616 0 77,266,616 Map output records 77,266,616 0 77,266,616 Map output bytes 11,442,228,060 0 11,442,228,060 Input split bytes 177,727 0 177,727 Combine input records 0 0 0 Combine output records 0 0 0 Reduce input groups 0 37,799,412 37,799,412 Reduce shuffle bytes 0 1,853,727,946 1,853,727,946 Reduce input records 0 76,428,913 76,428,913 Reduce output records 0 48,958,874 48,958,874 Spilled Records 112,586,947 62,608,254 175,195,201 CPU time spent (ms) 2,461,980 14,831,230 17,293,210 Physical memory (bytes) snapshot 366,933,626,880 9,982,947,328 376,916,574,208 Virtual memory (bytes) snapshot 2,219,448,848,384 23,215,755,264 2,242,664,603,648 Total committed heap usage (bytes) 1,211,341,733,888 8,609,333,248 1,219,951,067,136 AcsReducer ColumnDeletesOnTable- 0 3,284,862 3,284,862 ColumnDeletesOnTable- 0 3,285,695 3,285,695 ColumnDeletesOnTable- 0 3,284,862 3,284,862 ColumnDeletesOnTable- 0 129,653 129,653 ColumnDeletesOnTable- 0 129,653 129,653 ColumnDeletesOnTable- 0 129,653 129,653 ColumnDeletesOnTable- 0 129,653 129,653 ColumnDeletesOnTable- 0 517,641 517,641 ColumnDeletesOnTable- 0 23,786 23,786 ColumnDeletesOnTable- 0 594,872 594,872 ColumnDeletesOnTable- 0 597,739 597,739 ColumnDeletesOnTable- 0 595,665 595,665 ColumnDeletesOnTable- 0 36,101,345 36,101,345 ColumnDeletesOnTable- 0 11,791 11,791 ColumnDeletesOnTable- 0 11,898 11,898 ColumnDeletesOnTable-0 176 176 RowDeletesOnTable- 0 224,044 224,044 RowDeletesOnTable- 0 224,045 224,045 RowDeletesOnTable- 0 224,044 224,044 RowDeletesOnTable- 0 17,425 17,425 RowDeletesOnTable- 0 17,425 17,425 RowDeletesOnTable- 0 17,425 17,425 RowDeletesOnTable- 0 17,425 17,425 RowDeletesOnTable- 0 459,890 459,890 RowDeletesOnTable- 0 23,786 23,786 RowDeletesOnTable- 0 105,910 105,910 RowDeletesOnTable- 0 107,829 107,829 RowDeletesOnTable- 0 105,909 105,909 RowDeletesOnTable- 0 36,101,345 36,101,345 RowDeletesOnTable- 0 11,353 11,353 RowDeletesOnTable- 0 11,459 11,459 RowDeletesOnTable- 0 168 168 WholeRowDeletesOnTable- 0 129,930 129,930 deleteRowsCount 0 37,799,410 37,799,410 deleteRowsMicros 0 104,579,855,042 104,579,855,042 emitCount 0 48,958,874 48,958,874 emitMicros 0 201,996,180 201,996,180 rollupValuesCount 0 37,799,412 37,799,412 rollupValuesMicros 0 234,085,342 234,085,342 As you can see its been running almost 3.5 hours now. There were 1177 Map tasks and they complete some time ago. The Reduce phase is incomplete at 95%. So I drill into the 'reduce' link and it takes me to the tasklist. If I drill into the first incomplete task, here it is: Job job_201901281553_38848 All Task Attempts Task Attempts Machine Status Progress Start Time Shuffle Finished Sort Finished Finish Time Errors Task Logs Counters Actions attempt_201901281553_38848_r_000000_0 RUNNING 70.81% 2/1/2019 12:39 1-Feb-2019 12:39:59 (18sec) 1-Feb-2019 12:40:01 (2sec) Last 4KB 60 Last 8KB All From there I can see the machine/datanode running the task so i ssh into it and I look at the log (filtering on just the task in question) from datanode $/var/log/hadoop-0.20-mapreduce/hadoop-mapred-tasktracker-.log 2019-02-01 12:39:40,836 INFO org.apache.hadoop.mapred.TaskTracker: LaunchTaskAction (registerTask): attempt_201901281553_38848_r_000000_0 task's state:UNASSIGNED 2019-02-01 12:39:40,838 INFO org.apache.hadoop.mapred.TaskTracker: Trying to launch : attempt_201901281553_38848_r_000000_0 which needs 1 slots 2019-02-01 12:39:40,838 INFO org.apache.hadoop.mapred.TaskTracker: In TaskLauncher, current free slots : 21 and trying to launch attempt_201901281553_38848_r_000000_0 which needs 1 slots 2019-02-01 12:39:40,925 INFO org.apache.hadoop.mapred.TaskController: Writing commands to /disk12/mapreduce/tmp-map-data/ttprivate/taskTracker/mapred/jobcache/job_201901281553_38848/attempt_201901281553_38848_r_000000_0/taskjvm.sh 2019-02-01 12:39:41,904 INFO org.apache.hadoop.mapred.TaskTracker: JVM with ID: jvm_201901281553_38848_r_-819481850 given task: attempt_201901281553_38848_r_000000_0 2019-02-01 12:39:49,011 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201901281553_38848_r_000000_0 0.09402435% reduce > copy (332 of 1177 at 23.66 MB/s) > 2019-02-01 12:39:56,250 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201901281553_38848_r_000000_0 0.25233644% reduce > copy (891 of 1177 at 12.31 MB/s) > 2019-02-01 12:39:59,206 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201901281553_38848_r_000000_0 0.25233644% reduce > copy (891 of 1177 at 12.31 MB/s) > 2019-02-01 12:39:59,350 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201901281553_38848_r_000000_0 0.33333334% reduce > sort 2019-02-01 12:40:01,599 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201901281553_38848_r_000000_0 0.33333334% reduce > sort 2019-02-01 12:40:02,469 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201901281553_38848_r_000000_0 0.6667039% reduce > reduce 2019-02-01 12:40:05,565 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201901281553_38848_r_000000_0 0.6667039% reduce > reduce 2019-02-01 12:40:11,666 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201901281553_38848_r_000000_0 0.6668788% reduce > reduce 2019-02-01 12:40:14,755 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201901281553_38848_r_000000_0 0.66691136% reduce > reduce 2019-02-01 12:40:17,838 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201901281553_38848_r_000000_0 0.6670001% reduce > reduce 2019-02-01 12:40:20,930 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201901281553_38848_r_000000_0 0.6671631% reduce > reduce 2019-02-01 12:40:24,016 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201901281553_38848_r_000000_0 0.6672566% reduce > reduce .. and these lines repeat in this manner for hours.. ^ so it appears the shuffle/sort phase went very quick but after that, its just the reduce phase crawling, the percentage is slowly increasing but takes hours before the task completes. 1) So that looks like the bottleneck here- am I correct in identifying the cause of my long-running job is because this task (and many tasks like it) is taking a very long time on the reduce phase for the task? 2) if so, what are my options for speeding it up? Load appears to be reasonably low on the datanode assigned that task, as well as its iowait: top - 15:20:03 up 124 days, 1:04, 1 user, load average: 3.85, 5.64, 5.96 Tasks: 1095 total, 2 running, 1092 sleeping, 0 stopped, 1 zombie Cpu(s): 3.8%us, 1.5%sy, 0.9%ni, 93.6%id, 0.2%wa, 0.0%hi, 0.0%si, 0.0%st Mem: 503.498G total, 495.180G used, 8517.543M free, 5397.789M buffers Swap: 2046.996M total, 0.000k used, 2046.996M free, 432.468G cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 82236 hbase 20 0 16.9g 16g 17m S 136.9 3.3 26049:16 java 30143 root 39 19 743m 621m 13m R 82.3 0.1 1782:06 clamscan 62024 mapred 20 0 2240m 1.0g 24m S 75.1 0.2 1:21.28 java 36367 mapred 20 0 1913m 848m 24m S 11.2 0.2 22:56.98 java 36567 mapred 20 0 1898m 825m 24m S 9.5 0.2 22:23.32 java 36333 mapred 20 0 1879m 880m 24m S 8.2 0.2 22:44.28 java 36374 mapred 20 0 1890m 831m 24m S 6.9 0.2 23:15.65 java and a snippet of iostat -xm 4: avg-cpu: %user %nice %system %iowait %steal %idle 2.15 0.92 0.30 0.17 0.00 96.46 Device: rrqm/s wrqm/s r/s w/s rMB/s wMB/s avgrq-sz avgqu-sz await r_await w_await svctm %util sda 0.00 350.25 0.00 30.00 0.00 1.49 101.67 0.02 0.71 0.00 0.71 0.04 0.12 sdb 0.00 2.75 0.00 6.00 0.00 0.03 11.67 0.00 0.00 0.00 0.00 0.00 0.00 sdd 0.00 9.75 0.00 1.25 0.00 0.04 70.40 0.00 0.00 0.00 0.00 0.00 0.00 sdf 0.00 6.50 0.00 0.75 0.00 0.03 77.33 0.00 0.00 0.00 0.00 0.00 0.00 sdg 0.00 5.75 0.00 0.50 0.00 0.02 100.00 0.00 0.00 0.00 0.00 0.00 0.00 sdc 0.00 8.00 0.00 0.75 0.00 0.03 93.33 0.00 0.00 0.00 0.00 0.00 0.00 sdh 0.00 6.25 0.00 0.50 0.00 0.03 108.00 0.00 0.00 0.00 0.00 0.00 0.00 sdi 0.00 3.75 93.25 0.50 9.03 0.02 197.57 0.32 3.18 3.20 0.00 1.95 18.30 sdj 0.00 3.50 0.00 0.50 0.00 0.02 64.00 0.00 0.00 0.00 0.00 0.00 0.00 sdk 0.00 7.00 0.00 0.75 0.00 0.03 82.67 0.00 0.33 0.00 0.33 0.33 0.03 sdl 0.00 6.75 0.00 0.75 0.00 0.03 80.00 0.00 0.00 0.00 0.00 0.00 0.00 sdm 0.00 7.75 0.00 5.75 0.00 0.05 18.78 0.00 0.04 0.00 0.04 0.04 0.03 #<machine>:~$ df -h Filesystem Size Used Avail Use% Mounted on /dev/sda3 40G 5.9G 32G 16% / tmpfs 252G 0 252G 0% /dev/shm /dev/sda1 488M 113M 350M 25% /boot /dev/sda8 57G 460M 54G 1% /tmp /dev/sda7 9.8G 1.1G 8.2G 12% /var /dev/sda5 40G 17G 21G 45% /var/log /dev/sda6 30G 4.4G 24G 16% /var/log/audit.d /dev/sdb1 7.2T 3.3T 3.6T 48% /disk1 /dev/sdc1 7.2T 3.3T 3.6T 49% /disk2 /dev/sdd1 7.2T 3.3T 3.6T 48% /disk3 /dev/sde1 7.2T 3.3T 3.6T 48% /disk4 /dev/sdf1 7.2T 3.3T 3.6T 48% /disk5 /dev/sdi1 7.2T 3.3T 3.6T 48% /disk6 /dev/sdg1 7.2T 3.3T 3.6T 48% /disk7 /dev/sdh1 7.2T 3.3T 3.6T 48% /disk8 /dev/sdj1 7.2T 3.3T 3.6T 48% /disk9 /dev/sdk1 7.2T 3.3T 3.6T 48% /disk10 /dev/sdm1 7.2T 3.3T 3.6T 48% /disk11 /dev/sdl1 7.2T 3.3T 3.6T 48% /disk12 This is version Hadoop 2.0.0-cdh4.3.0. Its highly-available with 3 zookeeper nodes, 2 namenodes, and 35 datanodes. YARN is not installed. Using hbase, oozie. Jobs mainly come in via Hive and HUE. Each datanode has 2 physical cpus, each with 22 cores. Hyperthreading is enabled. If you need more information, please let me know. My guess is maybe I need more reducers, there are mapred-site.xml settings that need tuning, perhaps the input data from map phase is too large, or Hive query needs written better. Im fairly new administrator to Hadoop, any detailed advice is great. Thanks!
Bash - Parsing output of apt-get install for progress bar
I'm looking for a way to write in a file the percentage and the speed of the download of apt-get (all the line in fact) Is it possible? I've seen this question and the response but I wonder if there was a solution in bash, or if you can tell me the name of this type of output? Thanks
With the help of BroSlow, I've tried with this script: #! /bin/bash exec >/home/guillaume/work/apt-install-firefox.log 2>&1 unbuffer apt-get install --reinstall firefox The output is what I would like: Lecture des listes de paquets… 0% Lecture des listes de paquets… 100% Lecture des listes de paquets... Fait Construction de l'arbre des dépendances… 0% Construction de l'arbre des dépendances… 0% Construction de l'arbre des dépendances… 50% Construction de l'arbre des dépendances… 50% Construction de l'arbre des dépendances Lecture des informations d'état… 0% Lecture des informations d'état… 0% Lecture des informations d'état... Fait 0 mis à jour, 0 nouvellement installés, 1 réinstallés, 0 à enlever et 0 non mis à jour. Il est nécessaire de prendre 35,8 Mo dans les archives. Après cette opération, 0 o d'espace disque supplémentaires seront utilisés. 0% [En cours] Réception de : 1 http://bouyguestelecom.ubuntu.lafibre.info/ubuntu/ trusty-updates/main firefox amd64 33.0+build2-0ubuntu0.14.04.1 [35,8 MB] 0% [1 firefox 6 959 B/35,8 MB 0%] 1% [1 firefox 198 kB/35,8 MB 1%] 1% [1 firefox 351 kB/35,8 MB 1%] 1% [1 firefox 528 kB/35,8 MB 1%] 2% [1 firefox 751 kB/35,8 MB 2%] 3% [1 firefox 990 kB/35,8 MB 3%] 4% [1 firefox 1 300 kB/35,8 MB 4%] 4% [1 firefox 1 552 kB/35,8 MB 4%] 5% [1 firefox 1 794 kB/35,8 MB 5%] 6% [1 firefox 2 096 kB/35,8 MB 6%] 7% [1 firefox 2 396 kB/35,8 MB 7%] 8% [1 firefox 2 799 kB/35,8 MB 8%] 9% [1 firefox 3 171 kB/35,8 MB 9%] 511 kB/s 1min 3s 10% [1 firefox 3 575 kB/35,8 MB 10%] 511 kB/s 1min 3s 11% [1 firefox 3 865 kB/35,8 MB 11%] 511 kB/s 1min 2s 12% [1 firefox 4 275 kB/35,8 MB 12%] 511 kB/s 1min 1s 13% [1 firefox 4 706 kB/35,8 MB 13%] 511 kB/s 60s 15% [1 firefox 5 214 kB/35,8 MB 15%] 511 kB/s 59s 16% [1 firefox 5 622 kB/35,8 MB 16%] 511 kB/s 59s Need the package expect-dev
Unknown Memory leak in android
adb shell dumpsys meminfo of my package shows the following and my native allocated size increases and finally causing mobile restart. Is that any memory ? How can i fix that?? native dalvik other total limit bitmap nativeBmp size: 445456 5955 N/A 451411 32768 N/A N/A allocated: 445024 3726 N/A 448750 N/A 10948 1912 free: 43 2229 N/A 2272 N/A N/A N/A (Pss): 132631 870 300292 433793 N/A N/A N/A (shared dirty): 2532 1656 5552 9740 N/A N/A N/A (priv dirty): 132396 708 298960 432064 N/A N/A N/A Objects Views: 0 ViewRoots: 0 AppContexts: 0 Activities: 0 Assets: 6 AssetManagers: 6 Local Binders: 5 Proxy Binders: 14 Death Recipients: 1 OpenSSL Sockets: 0 SQL heap: 0 MEMORY_USED: 0 PAGECACHE_OVERFLOW: 0 MALLOC_SIZE: 50 Asset Allocations zip:/data/app/com.outlook.screens-2.apk:/resources.arsc: 25K zip:/data/app/com.outlook.screens-2.apk:/assets/font/RobotoCondensedRegular. ttf: 156K zip:/data/app/com.outlook.screens-2.apk:/assets/font/RobotoCondensedBold.ttf : 158K zip:/data/app/com.outlook.screens-2.apk:/assets/font/RobotoCondensedLight.tt f: 157K Uptime: 190161845 Realtime now=432619753