How to Backup & Restore ElasticSearch - elasticsearch

I have been trying the following methods to Backup & Restore an ElasticSearch cluster from one server to another with little success. I have not used a backup process thus far and I would like to move my whole ElasticSearch cluster from a small 2GB cluster to 15GB cluster. I used the following methods.
Using taskrabit/elasticsearch-dump - I was successfully able to export the complete database to backup.json file, however when restoring the backup.json, it gave me the following output. After researching the output further I understood that the bulk input of the plug-in was not fully developed.
./bin/elasticdump --all=true --input=/home/user/backup.json --output=http://192.168.0.213:9200/ --type=data
Thu, 09 Feb 2017 06:43:29 GMT | starting dump
Thu, 09 Feb 2017 06:43:29 GMT | got 61 objects from source file (offset: 0)
Thu, 09 Feb 2017 06:43:29 GMT | sent 61 objects to destination elasticsearch, wrote 0
Thu, 09 Feb 2017 06:43:29 GMT | got 0 objects from source file (offset: 61)
Thu, 09 Feb 2017 06:43:29 GMT | Total Writes: 0
Thu, 09 Feb 2017 06:43:29 GMT | dump complete
Using elasticsearch-tools (es-export-bulk & es-import bulk) I again was able to backup the json successfully. But the import failed once again with an error:
"statusCode":400,"response":"{"error":{
I used the examples from es-bulk-export
Using the ElasticSearch built-in Snapshot & Restore.
curl -XPUT 'http://localhost:9200/_snapshot/my_backup' -d '{
"type": "fs",
"settings": {"location": "/home/shawn/backup", "compress": true}
}'
I believe I'm missing something as the execution gives me the following error. Do I need to create /_snapshot/my_backup? If so how?
{"error":{"root_cause":[
{"type":"repository_exception",
"reason":"[my_backup] location [/home/shawn/backup] doesn't match any of the locations specified by path.repo because this setting is empty"
}],
"type":"repository_exception","reason":"[my_backup] failed to create repository",
"caused_by":
{"type":"creation_exception","reason":"Guice creation errors:\n\n1) Error injecting constructor, RepositoryException[[my_backup] location [/home/shawn/backup] doesn't match any of the locations specified by path.repo because this setting is empty]\n at org.elasticsearch.repositories.fs.FsRepository.<init>(Unknown Source)\n while locating org.elasticsearch.repositories.fs.FsRepository\n while locating org.elasticsearch.repositories.Repository\n\n1 error","caused_by":{"type":"repository_exception","reason":"[my_backup] location [/home/shawn/backup] doesn't match any of the locations specified by path.repo because this setting is empty"}}},"status":500}

You are creating your _snapshot/my_backup just fine, You only need to add line to /etc/elasticsearch/elasticsearch.yml:
path.repo: ["/home/shawn/backup"]
Which is actual location of your snapshot. Then restart Elasticsearch.

Related

How can I find POEdit errors

I'm getting duplicate message definition errors in POEdit and I cannot update from that catalog.
POEdit doesn't show details and the line numbers in the errors don't match.
How can I find what is wrong?
Sat Mar 11 16:51:31 2017: /var/folders/qt/cg8qlhc159v9s15kmhy1h4440000gn/T/poeditgGtdFw/1input.po:2791: duplicate message definition...
Sat Mar 11 16:51:31 2017: /var/folders/qt/cg8qlhc159v9s15kmhy1h4440000gn/T/poeditgGtdFw/1input.po:65: ...this is the location of the first definition
Sat Mar 11 16:51:31 2017: /var/folders/qt/cg8qlhc159v9s15kmhy1h4440000gn/T/poeditgGtdFw/1input.po:2827: duplicate message definition...
Sat Mar 11 16:51:31 2017: /var/folders/qt/cg8qlhc159v9s15kmhy1h4440000gn/T/poeditgGtdFw/1input.po:1530: ...this is the location of the first definition
Sat Mar 11 16:51:31 2017: /Users/burakkilic/Applications/Poedit.app/Contents/PlugIns/GettextTools.bundle/Contents/MacOS/bin/msgmerge: found 2 fatal errors
Sat Mar 11 16:51:31 2017: Entries in the catalogue are probably incorrect.
Sat Mar 11 16:51:31 2017: Updating the catalogue failed. Click on 'Details >>' for details.
How can I find what is wrong?
msgfmt -v -c /path/to/1input.po

Hubot not responding to Ping

Installed hubot but it's not responding to Pinging on OSX. Sample error:
bin/hubot
botname> [Sun May 22 2016 00:24:06 GMT-0400 (EDT)] WARNING Loading scripts from hubot-scripts.json is deprecated and will be removed in 3.0 (https://github.com/github/hubot-scripts/issues/1113) in favor of packages for each script.
Your hubot-scripts.json is empty, so you just need to remove it.
[Sun May 22 2016 00:24:06 GMT-0400 (EDT)] ERROR hubot-heroku-alive included, but missing HUBOT_HEROKU_KEEPALIVE_URL. heroku config:set HUBOT_HEROKU_KEEPALIVE_URL=$(heroku apps:info -s | grep web-url | cut -d= -f2)
[Sun May 22 2016 00:24:06 GMT-0400 (EDT)] INFO hubot-redis-brain: Using default redis on localhost:6379

Teamcity Configuration Settings

I need to know the teamcity settings which prevents the re-trigger/trigger of outdated builds/jobs if the new builds are successful.
I am facing a issue where teamcity jobs can be re triggered even if the next builds are successful.And If the trigger event is fired before, then it must stop teamcity to run that job if the latest build is successful.
So I have to 2 jobs in TC for 1 branch -- Build-Precheck and the other is Build-compile
So I could see that Build-compile is just picking the latest available successful build from Build-Precheck and then queing up the next which may be the outdated build.
Build-Precheck is just taking 2 min to finish the builds , it quickly triggers the latest builds , I guess following the principal First In First Out
Build-Precheck
06 Oct 14 14:33 - 14:35 (2m:01s) –7.1.4345
06 Oct 14 14:41 - 14:43 (2m:16s)- 7.1.4346
06 Oct 14 14:45 - 14:47 (2m:10s)- 7.1.4347
Build-compile
06 Oct 14 14:35 - 15:00 -7.1.0.4345
06 Oct 14 14:52 - 15:20 (28m:02s)- 7.1.4347
06 Oct 14 16:08 - 16:33 (24m:52s)- 7.1.4346
Is there any fix for this that TC runs incremental builds rather than outdated ones
Sounds like you are looking for Configuring Build Trigger.
AFAIK, there isn't a way to cancel queued builds if a given build passes. However, you can adjust the Build Triggers that queue those builds. Most likely, you'll need to set the Quiet Period on your VCS Build Trigger to longer than it takes for your build.
For example, if your full build takes 5 minutes, you should set the Quiet Period to 7. This way additional builds wont queue while a build is running.

How to create a new timezone in Ruby?

I want to create a timezone definition for UTC+n without any DST changes for our testing purposes, but haven't figured out how to do it. I'm using ActiveSupport::TimeZone, but it seems that it has a hard-coded list of time zones which cannot be programmatically extended.
In particular, however I try to create a new timezone, it is always missing DST information. If I provide an existing timezone as a basis, it ignores the time offset I provided instead.
Below are some attempts I tried to create a timezone in UTC+1:
> ActiveSupport::TimeZone[1].now
=> Tue, 06 Aug 2013 12:39:35 CEST +02:00
> ActiveSupport::TimeZone.create("UTC", 3600).now
=> Tue, 06 Aug 2013 10:39:40 UTC +00:00
> ActiveSupport::TimeZone.create("foo", 3600).now
TZInfo::InvalidTimezoneIdentifier: cannot load such file -- tzinfo/definitions/foo
> ActiveSupport::TimeZone.create("foo", 3600, TZInfo::Timezone.get("UTC")).now
=> Tue, 06 Aug 2013 10:39:48 UTC +00:00

Single Column rpm command output

I have an application where I want to get the RPM info of several packages and format it in a pleasing and eye-catching way. *boom*.
My problem is that rpm wants to format the data into two columns, which is annoying.
Command
rpm -qi ruby
Result
Name : ruby Relocations: (not relocatable)
Version : 1.8.6.36 Vendor: (none)
Release : 2 Build Date: Thu 18 Sep 2008 10:50:30 AM EDT
Install Date: Thu 25 Sep 2008 12:12:53 PM EDT Build Host: kickbench
Group : Development/Languages Source RPM: ruby-1.8.6.36-2.src.rpm
Size : 1664035 License: Ruby License/GPL - see COPYING
Signature : (none)
URL : http://www.ruby-lang.org/
Summary : An interpreter of object-oriented scripting language.
Description : [truncated]
What I want
Name : ruby
Relocations : (not relocatable)
Version : 1.8.6.36
Vendor : (none)
Release : 2
Build Date : Thu 18 Sep 2008 10:50:30 AM EDT
Install Date: Thu 25 Sep 2008 12:12:53 PM EDT
Build Host : kickbench
Group : Development/Languages
Source RPM : ruby-1.8.6.36-2.src.rpm
Size : 1664035
License : Ruby License/GPL - see COPYING
Signature : (none)
URL : http://www.ruby-lang.org/
Summary : An interpreter of object-oriented scripting language.
Description : [truncated]
Is there any way to get this formatting without manually specifying a reaally long --queryformat?
Alternatively, any way to do this native to Ruby would be fine.
pairs = %x{rpm -qi ruby}
.split(/(?<!:)\s{2,}(?![\s:])|#$//)
.map{|line| line.split(/\s*:\s+/, 2)}
width = pairs.map{|pair| pair.first.length}.max
pairs.each{|k, v| puts "#{k.ljust(width)}: #{v}"}

Resources