So, I had just started out with Elasticsearch on my local machine.
I have started 5 instances of Elasticsearch nodes. (simple ./bin/elasticsearch)
curl -s 'localhost:9200/_cat/nodes?v' gives:
host ip heap.percent ram.percent load node.role master name
127.0.0.1 127.0.0.1 5 99 3.13 d m Shirow Ishihara
127.0.0.1 127.0.0.1 7 100 3.13 d m Madame Web
127.0.0.1 127.0.0.1 5 100 3.13 d m Anthropomorpho
127.0.0.1 127.0.0.1 5 100 3.13 d m Paste-Pot Pete
127.0.0.1 127.0.0.1 2 100 3.13 d * Mephisto
My index has 2 primary shards and 5 replicas (total 10 replicas).
I had read that ES automatically scales horizontally and assigns/moves shards to new nodes. However, still all the 10 replicas are unassigned and both the 2 primary shards are in the same node.
curl -s 'localhost:9200/_cat/allocation?v' gives:
shards disk.indices disk.used disk.avail disk.total disk.percent host ip node
0 0b 105.5gb 6.2gb 111.8gb 94 127.0.0.1 127.0.0.1 Shirow Ishihara
0 0b 105.5gb 6.2gb 111.8gb 94 127.0.0.1 127.0.0.1 Paste-Pot Pete
2 318b 105.5gb 6.2gb 111.8gb 94 127.0.0.1 127.0.0.1 Mephisto
0 0b 105.5gb 6.2gb 111.8gb 94 127.0.0.1 127.0.0.1 Anthropomorpho
0 0b 105.5gb 6.2gb 111.8gb 94 127.0.0.1 127.0.0.1 Madame Web
10 UNASSIGNED
You have too few available disk space an ES is actually trying to move away some shards. But all your nodes are on the same machine so there is nowhere else where to move them and they stay unassigned. The used disk space is more than 90% of the total disk size and ES is hitting the the high watermark.
Read here more about this.
Related
I am getting ElasticsearchStatusWarning saying that the cluster state is yellow. Upon running the health check API, I see below
curl -X GET http://localhost:9200/_cluster/health/
{"cluster_name":"my-elasticsearch","status":"yellow","timed_out":false,"number_of_nodes":8,"number_of_data_nodes":3,"active_primary_shards":220,"active_shards":438,"relocating_shards":0,"initializing_shards":2,"unassigned_shards":0,"delayed_unassigned_shards":0,"number_of_pending_tasks":0,"number_of_in_flight_fetch":0,"task_max_waiting_in_queue_millis":0,"active_shards_percent_as_number":99.54545454545455}
initializing_shards is 2. So, I further run the below call
curl -X GET
http://localhost:9200/_cat/shards?h=index,shard,prirep,state,unassigned.reason
|grep INIT
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 33457 100 33457 0 graph_vertex_24_18549 0 r INITIALIZING ALLOCATION_FAILED
0 79609 0 --:--:-- --:--:-- --:--:-- 79659
curl -X GET http://localhost:9200/_cat/shards/graph_vertex_24_18549
graph_vertex_24_18549 0 p STARTED 8373375 8.4gb IP1 elasticsearch-data-1
graph_vertex_24_18549 0 r INITIALIZING IP2 elasticsearch-data-2
And rerunning the same command in few mins, shows now it's being initialized in elasticsearch-data-0. See below
graph_vertex_24_18549 0 p STARTED 8373375 8.4gb IP1 elasticsearch-data-1
graph_vertex_24_18549 0 r INITIALIZING IP0 elasticsearch-data-0
If i rerun it again in few mins, I can see it's again being initialized in elasticsearch-data-2 again. But it never gets STARTED.
curl -X GET http://localhost:9200/_cat/allocation?v
shards disk.indices disk.used disk.avail disk.total disk.percent host ip node
147 162.2gb 183.8gb 308.1gb 492gb 37 IP1 IP1 elasticsearch-data-2
146 217.3gb 234.2gb 257.7gb 492gb 47 IP2 IP2 elasticsearch-data-1
147 216.6gb 231.2gb 260.7gb 492gb 47 IP3 IP3 elasticsearch-data-0
curl -X GET http://localhost:9200/_cat/nodes?v
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
IP1 7 77 20 4.17 4.57 4.88 mi - elasticsearch-master-2
IP2 72 59 7 2.59 2.38 2.19 i - elasticsearch-5f4bd5b88f-4lvxz
IP3 57 49 3 0.75 1.13 1.09 di - elasticsearch-data-2
IP4 63 57 21 2.69 3.58 4.11 di - elasticsearch-data-0
IP5 5 59 7 2.59 2.38 2.19 mi - elasticsearch-master-0
IP6 69 53 13 4.67 4.60 4.66 di - elasticsearch-data-1
IP7 8 70 14 2.86 3.20 3.09 mi * elasticsearch-master-1
IP8 30 77 20 4.17 4.57 4.88 i - elasticsearch-5f4bd5b88f-wnrl4
curl -s -XGET http://localhost:9200/_cluster/allocation/explain -d '{
"index": "graph_vertex_24_18549", "shard":
0, "primary": false }' -H 'Content-type: application/json'
{"index":"graph_vertex_24_18549","shard":0,"primary":false,"current_state":"initializing","unassigned_info":{"reason":"ALLOCATION_FAILED","at":"2020-11-04T08:21:45.756Z","failed_allocation_attempts":1,"details":"failed shard on node [1XEXS92jTK-wwanNgQrxsA]: failed to perform indices:data/write/bulk[s] on replica [graph_vertex_24_18549][0], node[1XEXS92jTK-wwanNgQrxsA], [R], s[STARTED], a[id=RnTOlfQuQkOumVuw_NeuTw], failure RemoteTransportException[[elasticsearch-data-2][IP:9300][indices:data/write/bulk[s][r]]]; nested: CircuitBreakingException[[parent] Data too large, data for [<transport_request>] would be [4322682690/4gb], which is larger than the limit of [4005632409/3.7gb], real usage: [3646987112/3.3gb], new bytes reserved: [675695578/644.3mb]]; ","last_allocation_status":"no_attempt"},"current_node":{"id":"o_9jyrmOSca9T12J4bY0Nw","name":"elasticsearch-data-0","transport_address":"IP:9300"},"explanation":"the shard is in the process of initializing on node [elasticsearch-data-0], wait until initialization has completed"}
Thing is I was earlier getting alerted for Unassigned Shards due to the same exception as above - "CircuitBreakingException[[parent] Data too large, data for [<transport_request>] would be [4322682690/4gb], which is larger than the limit of [4005632409/3.7gb]"
But back then heap was only 2G. I increased it to 4G. And now I am seeing same error, but this time with respect to Initialising shards instead of Unallocated shards.
How can I remediate this?
Using Episerver in VS 2017
My site runs in IIS Express. When I start the project from VS in Chrome, I get the default IIS Screen (The blue screen with IIS info) - so IIS starts, but the site does not launch. If I reload the browser, the site launches.
Here is my site definition in the applicationhost.config:
<site name="MySite" id="3">
<application path="/" applicationPool="Clr4IntegratedAppPool">
<virtualDirectory path="/" physicalPath="C:\Projects\mysite-episerver\src\MySite" />
</application>
<bindings>
<binding protocol="http" bindingInformation="*:80:mysite.mydomain.com" />
</bindings>
</site>
Update
Looking at the log:
Software: Microsoft Internet Information Services 10.0
Version: 1.0
Date: 2018-05-25 18:09:24
Fields: date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) cs(Referer) sc-status sc-substatus sc-win32-status time-taken
2018-05-25 18:09:24 127.0.0.1 GET / - 80 - 127.0.0.1 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/66.0.3359.181+Safari/537.36 - 302 0 0 13991
2018-05-25 18:09:31 127.0.0.1 GET /personal/ - 80 - 127.0.0.1 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/66.0.3359.181+Safari/537.36 - 200 0 0 6389
2018-05-25 18:09:31 127.0.0.1 GET /dist/styles/main.css v=636627702363674333 80 - 127.0.0.1 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/66.0.3359.181+Safari/537.36 http://mysite.mydomain.com/personal/ 200 0 0 37
2018-05-25 18:09:31 127.0.0.1 GET /dist/fonts/stylesheet.css - 80 - 127.0.0.1 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/66.0.3359.181+Safari/537.36 http://mysite.mydomain.com/personal/ 200 0 0 29
2018-05-25 18:09:31 127.0.0.1 GET /EPiServer.Forms/DataSubmit/GetFormInitScript formGuid=6bd570d3-3e91-447f-903e-566e9d9f9468&formLanguage=en 80 - 127.0.0.1 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/66.0.3359.181+Safari/537.36 http://mysite.mydomain.com/personal/ 200 0 0 171
2018-05-25 18:09:31 127.0.0.1 GET /WebResource.axd d=TzRGlCddaaqtz0Im2nSJhBlH3P1Fze7Czb4HaEn0ZkkpUFjLB2tREFUTizpsCAfxQQOQDZCZqUiBZLAo7l7EhsGTwsuUFnRXRabRz6n3XcvVK2Bd0VQe-ZDaVHtSCoc3paD6qdAAJwaIPHOv9FJMVOt6_eqfG_NNb1zepx_9qUk1&t=636601600300000000 80 - 127.0.0.1 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/66.0.3359.181+Safari/537.36 http://mysite.mydomain.com/personal/ 200 0 0 81
2018-05-25 18:09:31 127.0.0.1 GET /dist/scripts/sticky-header.js v=636627702363754120 80 - 127.0.0.1 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/66.0.3359.181+Safari/537.36 http://mysite.mydomain.com/personal/ 200 0 0 130
2018-05-25 18:09:31 127.0.0.1 GET /dist/scripts/main.js v=636627702363724200 80 - 127.0.0.1 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/66.0.3359.181+Safari/537.36 http://mysite.mydomain.com/personal/ 200 0 0 149
2018-05-25 18:09:31 127.0.0.1 GET /dist/scripts/forms.js v=636627702363704257 80 - 127.0.0.1 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/66.0.3359.181+Safari/537.36 http://mysite.mydomain.com/personal/ 200 0 0 165
2018-05-25 18:09:31 127.0.0.1 GET /siteassets/settings/logo_mysiteusa.svg - 80 - 127.0.0.1 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/66.0.3359.181+Safari/537.36 http://mysite.mydomain.com/personal/ 200 0 0 171
2018-05-25 18:09:31 127.0.0.1 GET /dist/images/profile.svg - 80 - 127.0.0.1 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/66.0.3359.181+Safari/537.36 http://mysite.mydomain.com/dist/styles/main.css?v=636627702363674333 200 0 0 185
2018-05-25 18:09:31 127.0.0.1 GET /siteassets/image-blocks/785x650.png - 80 - 127.0.0.1 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/66.0.3359.181+Safari/537.36 http://mysite.mydomain.com/personal/ 200 0 0 181
2018-05-25 18:09:31 127.0.0.1 GET /siteassets/heros/180800277.jpg - 80 - 127.0.0.1 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/66.0.3359.181+Safari/537.36 http://mysite.mydomain.com/personal/ 200 0 0 294
2018-05-25 18:09:31 127.0.0.1 GET /dist/images/BG-Pattern.png - 80 - 127.0.0.1 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/66.0.3359.181+Safari/537.36 http://mysite.mydomain.com/dist/styles/main.css?v=636627702363674333 200 0 0 70
2018-05-25 18:09:31 127.0.0.1 GET /siteassets/image-and-text-blocks/bbb.png - 80 - 127.0.0.1 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/66.0.3359.181+Safari/537.36 http://mysite.mydomain.com/personal/ 200 0 0 217
2018-05-25 18:09:31 127.0.0.1 GET /dist/images/chevron.svg - 80 - 127.0.0.1 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/66.0.3359.181+Safari/537.36 http://mysite.mydomain.com/dist/styles/main.css?v=636627702363674333 200 0 0 65
2018-05-25 18:09:31 127.0.0.1 GET /siteassets/image-and-text-blocks/bestmysite.png - 80 - 127.0.0.1 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/66.0.3359.181+Safari/537.36 http://mysite.mydomain.com/personal/ 200 0 0 255
2018-05-25 18:09:31 127.0.0.1 GET /siteassets/settings/logo_ehl.svg - 80 - 127.0.0.1 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/66.0.3359.181+Safari/537.36 http://mysite.mydomain.com/personal/ 200 0 0 253
2018-05-25 18:09:31 127.0.0.1 GET /dist/images/jumplink.svg - 80 - 127.0.0.1 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/66.0.3359.181+Safari/537.36 http://mysite.mydomain.com/dist/styles/main.css?v=636627702363674333 200 0 0 89
2018-05-25 18:09:31 127.0.0.1 GET /siteassets/settings/footer-mountains.png - 80 - 127.0.0.1 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/66.0.3359.181+Safari/537.36 http://mysite.mydomain.com/personal/ 200 0 0 112
2018-05-25 18:09:31 127.0.0.1 GET /dist/images/footer_linkedin.svg - 80 - 127.0.0.1 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/66.0.3359.181+Safari/537.36 http://mysite.mydomain.com/dist/styles/main.css?v=636627702363674333 200 0 0 38
2018-05-25 18:09:31 127.0.0.1 GET /dist/images/icon_quotation.png - 80 - 127.0.0.1 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/66.0.3359.181+Safari/537.36 http://mysite.mydomain.com/dist/styles/main.css?v=636627702363674333 200 0 0 37
2018-05-25 18:09:31 127.0.0.1 GET /dist/images/footer_twitter.svg - 80 - 127.0.0.1 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/66.0.3359.181+Safari/537.36 http://mysite.mydomain.com/dist/styles/main.css?v=636627702363674333 200 0 0 76
2018-05-25 18:09:31 127.0.0.1 GET /dist/images/doublearrow.svg - 80 - 127.0.0.1 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/66.0.3359.181+Safari/537.36 http://mysite.mydomain.com/dist/styles/main.css?v=636627702363674333 200 0 0 173
2018-05-25 18:09:31 127.0.0.1 GET /dist/images/footer_facebook.svg - 80 - 127.0.0.1 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/66.0.3359.181+Safari/537.36 http://mysite.mydomain.com/dist/styles/main.css?v=636627702363674333 200 0 0 68
2018-05-25 18:09:31 127.0.0.1 GET /dist/images/mobilephone.svg - 80 - 127.0.0.1 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/66.0.3359.181+Safari/537.36 http://mysite.mydomain.com/dist/styles/main.css?v=636627702363674333 200 0 0 43
2018-05-25 18:09:31 127.0.0.1 GET /dist/images/email.svg - 80 - 127.0.0.1 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/66.0.3359.181+Safari/537.36 http://mysite.mydomain.com/dist/styles/main.css?v=636627702363674333 200 0 0 115
2018-05-25 18:09:31 127.0.0.1 GET /dist/images/footer_youtube.svg - 80 - 127.0.0.1 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/66.0.3359.181+Safari/537.36 http://mysite.mydomain.com/dist/styles/main.css?v=636627702363674333 200 0 0 200
2018-05-25 18:09:31 127.0.0.1 GET /dist/images/doublearrow-white.svg - 80 - 127.0.0.1 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/66.0.3359.181+Safari/537.36 http://mysite.mydomain.com/dist/styles/main.css?v=636627702363674333 200 0 0 105
2018-05-25 18:09:31 127.0.0.1 GET /dist/images/worldwide.svg - 80 - 127.0.0.1 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/66.0.3359.181+Safari/537.36 http://mysite.mydomain.com/dist/styles/main.css?v=636627702363674333 200 0 0 115
2018-05-25 18:09:31 127.0.0.1 GET /util/LiveMonitor/Tracking/Index contentId=34&languageId=en&r=0.6407196012207004&referrer= 80 - 127.0.0.1 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/66.0.3359.181+Safari/537.36 http://mysite.mydomain.com/personal/ 200 0 0 131
2018-05-25 18:09:32 127.0.0.1 GET /util/LiveMonitor/Tracking/Index contentId=34&languageId=en&r=0.5590579783850214&referrer= 80 - 127.0.0.1 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/66.0.3359.181+Safari/537.36 http://mysite.mydomain.com/personal/ 200 0 0 565
2018-05-25 18:09:36 127.0.0.1 GET /login/ - 80 - 127.0.0.1 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/66.0.3359.181+Safari/537.36 - 200 0 0 223
2018-05-25 18:09:36 127.0.0.1 GET /util/LiveMonitor/Tracking/Index contentId=271&languageId=en&r=0.29442535680181825&referrer= 80 - 127.0.0.1 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/66.0.3359.181+Safari/537.36 http://mysite.mydomain.com/login/ 200 0 0 55
2018-05-25 18:09:36 127.0.0.1 GET /util/LiveMonitor/Tracking/Index contentId=271&languageId=en&r=0.21276262992917028&referrer= 80 - 127.0.0.1 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/66.0.3359.181+Safari/537.36 http://mysite.mydomain.com/login/ 200 0 0 578
2018-05-25 18:09:52 127.0.0.1 POST /login/post/ - 80 - 127.0.0.1 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/66.0.3359.181+Safari/537.36 http://mysite.mydomain.com/login/ 302 0 0 443
2018-05-25 18:09:52 127.0.0.1 GET /login/post/mysiteusa/cms/ - 80 admin 127.0.0.1 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/66.0.3359.181+Safari/537.36 http://mysite.mydomain.com/login/ 404 0 0 506
2018-05-25 18:09:52 127.0.0.1 GET /util/LiveMonitor/Tracking/Index contentId=9&languageId=en&r=0.2522424054704595&referrer=http%3A//mysite.mydomain.com/login/ 80 admin 127.0.0.1 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/66.0.3359.181+Safari/537.36 http://mysite.mydomain.com/login/post/%2Fmysiteusa%2Fcms%2F 200 0 0 37
2018-05-25 18:09:52 127.0.0.1 GET /util/LiveMonitor/Tracking/Index contentId=9&languageId=en&r=0.9305181288066644&referrer=http%3A//mysite.mydomain.com/login/ 80 admin 127.0.0.1 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/66.0.3359.181+Safari/537.36 http://mysite.mydomain.com/login/post/%2Fmysiteusa%2Fcms%2F 200 0 0 42
2018-05-25 18:09:52 127.0.0.1 GET /epiutil/images/quicknav/quicknav-arrow.png - 80 admin 127.0.0.1 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/66.0.3359.181+Safari/537.36 http://mysite.mydomain.com/epiutil/styles/quicknavigator.css 200 0 0 57
<WebProjectProperties>
<UseIIS>True</UseIIS>
<AutoAssignPort>True</AutoAssignPort>
<DevelopmentServerPort>0</DevelopmentServerPort>
<DevelopmentServerVPath>/</DevelopmentServerVPath>
<IISUrl>http://goalaskausa.mydelphic.com</IISUrl>
<NTLMAuthentication>False</NTLMAuthentication>
<UseCustomServer>False</UseCustomServer>
<CustomServerUrl>
</CustomServerUrl>
<SaveServerSettingsInUserFile>False</SaveServerSettingsInUserFile>
</WebProjectProperties>
UseIIS keeps getting reset to true. I think whats happening is that its hitting IIS, not IIS express. Then when I refresh it hits IIS express.
I am trying to configure ntp server on centos 7 instead of ip address its showing INIT while executing ntpq -p command.
my ntp server configuration file.
/etc/ntp.conf
driftfile /var/lib/ntp/drift
restrict default nomodify notrap nopeer noquery
restrict 127.0.0.1
restrict ::1
# Hosts on local network are less restricted.
Allow 10.0.3.0/24 network clients to syncronize time with this server
restrict 10.0.3.0 mask 255.255.255.0 nomodify notrap
# Use public servers from the pool.ntp.org project.
# Please consider joining the pool (http://www.pool.ntp.org/join.html).
server 2.in.pool.ntp.org
server 1.asia.pool.ntp.org
server 3.asia.pool.ntp.org
includefile /etc/ntp/crypto/pw
keys /etc/ntp/keys
disable monitor
I have have following outputs while executing ntpq -q command
[centos#ip-10-0-3-53 etc]$ ntpq -p
remote refid st t when poll reach delay offset jitter
==============================================================================
ntp.your.org .INIT. 16 u - 128 0 0.000 0.000 0.000
ns3.weiszhostin .INIT. 16 u - 128 0 0.000 0.000 0.000
resolver2.skyfi .INIT. 16 u - 128 0 0.000 0.000 0.000
ntp2.wiktel.com .INIT. 16 u - 128 0 0.000 0.000 0.000
[spryiq#ip-10-0-3-53 etc]$
Port 123 is listening correctly
[centos#ip-10-0-3-53 etc]$ netstat -plnu | grep 123
(Not all processes could be identified, non-owned process info
will not be shown, you would have to be root to see it all.)
udp 0 0 10.0.3.53:123 0.0.0.0:* -
udp 0 0 127.0.0.1:123 0.0.0.0:* -
udp 0 0 0.0.0.0:123 0.0.0.0:* -
udp6 0 0 fe80::10c9:ecff:fe9:123 :::* -
udp6 0 0 ::1:123 :::* -
udp6 0 0 :::123 :::* -
[centos#ip-10-0-3-53 etc]$ ntpstat
timeout
[centos#ip-10-0-3-53 etc]$ ntpq
ntpq> as
ind assid status conf reach auth condition last_event cnt
===========================================================
1 1255 8011 yes no none reject mobilize 1
2 1256 8011 yes no none reject mobilize 1
3 1257 8011 yes no none reject mobilize 1
4 1258 8011 yes no none reject mobilize 1
I have configured iptables correctly.instead of INIT it should display ip addresses How can i troubleshoot this problem??
So I'm having what I believe is a common issue with Cisco AnyConnect and Vagrant/VirtualBox, i'm using host-only networking so that I can stand up a virtual server at an ip such as 10.10.10.11 and then I can map this in my /etc/hosts to :
# IP to hostname mapping
10.10.10.10 vagrant.dev
And then I can pull a webpage from curl vagrant.dev for example. Real standard stuff.
Things go awry when I connect to my work VPN which makes this nice little dev box kind of useless as I can no longer hit it by hostname or ip any longer. I can still do a vagrant ssh to the box but there is no other way to hit it.
I tried taking a look at the routing table and it appears that my pre-vbox route entry (netstat -rn):
Internet:
Destination Gateway Flags Refs Use Netif Expire
PreVPN
10.10.10/24 link#11 UCS 1 0 vboxnet
10.10.10.11 8:0:27:96:91:84 UHLWIi 1 70 vboxnet 1197
PostVPN
10.10.10/24 link#19 UCS 1 0 utun1
10.10.10.11 link#19 UHW3I 0 4 utun1 62
Route Flag Description
|PRE|POST| Flag Description |
|---|----|-----------------------------------------------|
| U | U | Route usable |
| H | H | Host entry (net otherwise) |
| L | | Valid protocol to link address translation |
| W | W | Route was generated as a result of cloning |
| | 3 | Protocol specified route flag #3 |
| I | I | Route is assocaited with an interface scope |
| i | | Route is a holding reference to the interface |
Question
Is it possible to restore the routes changed by VPN in order to make my vagrant connections work again? And if-so how?
Other Data
Full Route Table (PRE) (IP4 only)
Internet:
Destination Gateway Flags Refs Use Netif Expire
default 192.168.2.1 UGSc 23 1 en0
10.10.10/24 link#11 UCS 1 0 vboxnet
10.10.10.11 8:0:27:96:91:84 UHLWIi 1 70 vboxnet 1197
127 127.0.0.1 UCS 0 7 lo0
127.0.0.1 127.0.0.1 UH 11 128344 lo0
169.254 link#4 UCS 0 0 en0
192.168.2 link#4 UCS 0 0 en0
192.168.2.1 68:7f:74:81:8b:b0 UHLSr 27 126 en0
192.168.2.106/32 link#4 UCS 0 0 en0
Full Route Table (POST) (IP4 only)
Internet:
Destination Gateway Flags Refs Use Netif Expire
default link#19 UCS 9 0 utun1
default 192.168.2.1 UGScI 3 0 en0
10.10.10/24 link#19 UCS 1 0 utun1
10.10.10.11 link#19 UHW3I 0 4 utun1 62
17.158.10.25 link#19 UHW3I 0 1 utun1 62
17.158.10.42 link#19 UHW3I 0 20 utun1 62
17.158.10.46 link#19 UHW3I 0 1 utun1 62
127 127.0.0.1 UCS 0 7 lo0
127.0.0.1 127.0.0.1 UH 15 128507 lo0
128.29.154.114 link#19 UHWIi 31 117 utun1
129.83.20.9 link#19 UHW3I 0 20 utun1 61
129.83.26.209 link#19 UHWIi 2 2 utun1
129.83.100.38 link#19 UHW3I 0 46 utun1 62
169.254 link#19 UCS 0 0 utun1
172.31.160/19 link#19 UCS 0 0 utun1
172.31.163.172/32 127.0.0.1 UGSc 3 67 lo0
192.80.55.9/32 192.168.2.1 UGSc 1 0 en0
192.168.2 link#19 UCS 0 0 utun1
192.168.2.1 68:7f:74:81:8b:b0 UHLSr 7 8 en0
192.168.2.1/32 link#19 UCS 0 0 utun1
192.168.2.106/32 link#4 UCS 0 0 en0
216.58.217.132 link#19 UHW3I 0 2 utun1 59
239.255.255.250 link#19 UHmW3I 0 17 utun1 62
Components
OSX
Cisco Any Connect
VirtualBox 5.0.8
Unfortunately there is no fix for the issue, only work-around/help I could think of is after disconnecting the VPN, you could use "route -n add" to establish the connectivity back - at least by this way u could save whole reboot of machines.
Ex:
sudo route -n add -net 192.168.56.0/24 -interface vboxnet0
(Try this after disconnecting from VPN - This method wont work when VPN is on)
In your case, something like, "sudo route -n add -net 10.10.10.0/24 -interface vboxnet0" - depends on your subnet obviously !
Also please see Ticket #14293
When facing this exact same problem what worked for us is to change the address of the Vagrant box to 11.0.0.100
I have a two nodes elasticsearch cluster with logstash and kibana. The cluster was on green status until I have started the logstash the elasticsearch cluster went to yellow status.
running curl -XGET http://localhost:9200/_cat/shards?pretty
will show the following
.kibana 0 p STARTED 2 8.2kb 172.17.0.1 Ereshkigal
.kibana 0 r UNASSIGNED
logstash-2015.10.18 4 r STARTED 69 101.9kb 172.17.0.1 Ereshkigal
logstash-2015.10.18 4 p STARTED 69 101.7kb 172.17.0.2 Hargen the Measurer
logstash-2015.10.18 0 r STARTED 62 65.7kb 172.17.0.1 Ereshkigal
logstash-2015.10.18 0 p STARTED 62 89.4kb 172.17.0.2 Hargen the Measurer
logstash-2015.10.18 3 p STARTED 76 48.1kb 172.17.0.1 Ereshkigal
logstash-2015.10.18 3 r UNASSIGNED
logstash-2015.10.18 1 p STARTED 74 78.8kb 172.17.0.1 Ereshkigal
logstash-2015.10.18 1 r UNASSIGNED
logstash-2015.10.18 2 r STARTED 79 56.8kb 172.17.0.1 Ereshkigal
logstash-2015.10.18 2 p STARTED 79 65.1kb 172.17.0.2 Hargen the Measurer
logstash-2015.10.19 4 p STARTED 7 43.4kb 172.17.0.1 Ereshkigal
logstash-2015.10.19 4 r UNASSIGNED
logstash-2015.10.19 0 r STARTED 7 50.8kb 172.17.0.1 Ereshkigal
logstash-2015.10.19 0 p STARTED 7 58.3kb 172.17.0.2 Hargen the Measurer
logstash-2015.10.19 3 r STARTED 9 67.4kb 172.17.0.1 Ereshkigal
logstash-2015.10.19 3 p STARTED 9 67.3kb 172.17.0.2 Hargen the Measurer
logstash-2015.10.19 1 r STARTED 12 76.4kb 172.17.0.1 Ereshkigal
logstash-2015.10.19 1 p STARTED 12 13.8kb 172.17.0.2 Hargen the Measurer
logstash-2015.10.19 2 p STARTED 13 78kb 172.17.0.1 Ereshkigal
logstash-2015.10.19 2 r UNASSIGNED
How can I get the cluster back in green status?
Yellow means you have unassigned shards, as shown by your output.
Make sure your shard allocation is enabled:
curl -XPUT localhost:9200/_cluster/settings -d '{
"transient" : {
"cluster.routing.allocation.enable" : "all"
}
}'
You can also try forcing the shard onto a node:
https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-reroute.html