spring cloud config server don't use profile - spring

I use:
<artifactId>spring-cloud-dependencies</artifactId>
<version>Brixton.SR6</version>
in my application.properties I have put
spring.profiles.active= prod
spring.application.name= FunctionalControl
I can see on the log :
2016-10-20 17:58:17.900 INFO c.o.l.f.Application - The following profiles are active: prod
But if I use tcpdump I can see than despite does a GET /FunctionalControl/prod
on my spring.cloud.config server It does GET /FunctionalControl/default
0x0030: 15f1 ddb3 4745 5420 2f46 756e 6374 696f ....GET./Functio
0x0040: 6e61 6c43 6f6e 7472 6f6c 2f64 6566 6175 nalControl/defau
0x0050: 6c74 2048 5454 502f 312e 310d 0a41 6363 lt.HTTP/1.1..Acc
0x0060: 6570 743a 2061 7070 6c69 6361 7469 6f6e ept:.application
0x0070: 2f6a 736f 6e2c 2061 7070 6c69 6361 7469 /json,.applicati
0x0080: 6f6e 2f2a 2b6a 736f 6e0d 0a55 7365 722d on/*+json..User-
So I never manage to use FunctionalControl-prod.properties from my git repo :-(

I found myself the solution and share it with everybody.
The active profile for config server is taken in consideration only if it is put into bootstrap.properties (or .yaml) not in application.properties
file bootstrap.properties:
spring.profiles.active= prod
spring.application.name= FunctionalControl
I add this file in my project and then it works fine as you can see on this tcpdump
0x0000: 4500 00d3 0799 4000 4006 348a 7f00 0001 E.....#.#.4.....
0x0010: 7f00 0001 9f40 22b8 8800 ede0 32d2 4aa8 .....#".....2.J.
0x0020: 8018 0156 fec7 0000 0101 080a 16da ea48 ...V...........H
0x0030: 16da ea48 4745 5420 2f46 756e 6374 696f ...HGET./Functio
0x0040: 6e61 6c43 6f6e 7472 6f6c 2f70 726f 6420 nalControl/prod.
0x0050: 4854 5450 2f31 2e31 0d0a 4163 6365 7074 HTTP/1.1..Accept
0x0060: 3a20 6170 706c 6963 6174 696f 6e2f 6a73 :.application/js
0x0070: 6f6e 2c20 6170 706c 6963 6174 696f 6e2f on,.application/
0x0080: 2a2b 6a73 6f6e 0d0a 5573 6572 2d41 6765 *+json..User-Age
0x0090: 6e74 3a20 4a61 7661 2f31 2e38 2e30 5f34 nt:.Java/1.8.0_4
0x00a0: 350d 0a48 6f73 743a 206c 6f63 616c 686f 5..Host:.localho
0x00b0: 7374 3a38 3838 380d 0a43 6f6e 6e65 6374 st:8888..Connect
0x00c0: 696f 6e3a 206b 6565 702d 616c 6976 650d ion:.keep-alive.

You need to define the active profile for your config client application (in your case "FunctionalControl"). There are multiple ways to do that.
E.g. you can add the following to your client's bootstrap.properties file
spring.profiles.active=prod
or e.g. start you config client application via
mvn spring-boot:run -Dspring.profiles.active=prod
Either way, your client server application doesn't need any active profile. It will server whatever file is available in your git repo.

Related

How to add path parameter in elasticsearch Java bulk Api

I'm using Java bulk Api,Elasticsearch 6.6.2, and expect to POST bulk request as
POST /<index>/_bulk,
but i got POST /_bulk instead (see packet captured below).
I know these 2 POST methods are the same, but there're a proxy between my app and ES, the path parameter is for the proxy for some reasons(I don't know why).
so, I wonder how to set the path parameter in Java ,so that request can be sent in POST /<index>/_bulk format.
Here's the code
RestHighLevelClient client;
/* some client init code ....*/
...
String INDEX = "api_track";
String DOC_TYPE = "_doc";
BulkProcessor.Builder builder = BulkProcessor.builder(
(request, bulkListener) ->
client.bulkAsync(request, RequestOptions.DEFAULT, bulkListener),
listener);
builder.setBulkActions(500);
builder.setBulkSize(new ByteSizeValue(1L, ByteSizeUnit.MB));
builder.setConcurrentRequests(0);
builder.setFlushInterval(TimeValue.timeValueSeconds(5L));
builder.setBackoffPolicy(BackoffPolicy
.constantBackoff(TimeValue.timeValueSeconds(1L), 3));
builder.setGlobalIndex(INDEX);
builder.setGlobalType(DOC_TYPE);
bulkProcessor = builder.build();
IndexRequest request = new IndexRequest().index(INDEX).type(DOC_TYPE).source(maps);
bulkProcessor.add(request);
and the packet captured:
0x0000: 0200 0000 4500 0277 0000 4000 4006 12c9 ....E..w..#.#...
0x0010: 1e27 dee3 0ba3 1d0b e142 23f0 85d1 ad67 .'.......B#....g
0x0020: 1540 31d3 8018 0801 3eb0 0000 0101 080a .#1.....>.......
0x0030: 1500 c5e6 b6ba 7ff0 504f 5354 202f 5f62 ........POST./_b <-- POST format here
0x0040: 756c 6b3f 7469 6d65 6f75 743d 316d 2048 ulk?timeout=1m.H
0x0050: 5454 502f 312e 310d 0a43 6f6e 7465 6e74 TTP/1.1..Content
0x0060: 2d4c 656e 6774 683a 2033 3335 0d0a 436f -Length:.335..Co
0x0070: 6e74 656e 742d 5479 7065 3a20 6170 706c ntent-Type:.appl
0x0080: 6963 6174 696f 6e2f 6a73 6f6e 0d0a 486f ication/json..Ho
0x0090: 7374 3a20 7365 6172 6368 2e61 6c69 7061 st:.search.alipa
0x00a0: 792e 6e65 743a 3932 3030 0d0a 436f 6e6e y.net:9200..Conn
0x00b0: 6563 7469 6f6e 3a20 4b65 6570 2d41 6c69 ection:.Keep-Ali
0x00c0: 7665 0d0a 5573 6572 2d41 6765 6e74 3a20 ve..User-Agent:.
0x00d0: 4170 6163 6865 2d48 7474 7041 7379 6e63 Apache-HttpAsync
0x00e0: 436c 6965 6e74 2f34 2e31 2e34 2028 4a61 Client/4.1.4.(Ja
0x00f0: 7661 2f31 2e38 2e30 5f31 3531 290d 0a41 va/1.8.0_151)..A
0x0100: 7574 686f 7269 7a61 7469 6f6e 3a20 4261 uthorization:.Ba
0x0110: 7369 6320 6247 6c75 6132 4d36 527a 4e61 sic.bGlua2M6RzNa
0x0120: 5954 4e4e 5755 593d 0d0a 0d0a 7b22 696e YTNNWUY=....{"in
0x0130: 6465 7822 3a7b 225f 696e 6465 7822 3a22 dex":{"_index":"
0x0140: 6170 695f 7472 6163 6b22 2c22 5f74 7970 api_track","_typ
0x0150: 6522 3a22 5f64 6f63 227d 7d0a 7b22 6372 e":"_doc"}}.{"cr
0x0160: 6561 7465 6441 7422 3a22 3230 3230 2d30 eatedAt":"2020-0
0x0170: 322d 3132 5430 383a 3438 3a34 332e 3333 2-12T08:48:43.33
0x0180: 3822 2c22 686f 7374 4164 6472 223a 2231 8","hostAddr":"1
0x0190: 3237 2e30 2e30 2e31 222c 2274 656e 616e 27.0.0.1","tenan
0x01a0: 7449 6422 3a6e 756c 6c2c 2273 6572 7665 tId":null,"serve
0x01b0: 724e 616d 6522 3a6e 756c 6c2c 2270 6172 rName":null,"par
0x01c0: 616d 7322 3a22 2670 7269 7661 7465 5f74 ams":"&private_t
0x01d0: 6f6b 656e 3d2a 2a2a 266f 7264 6572 5f62 oken=***&order_b
0x01e0: 793d 7369 7a65 222c 2275 7365 724e 616d y=size","userNam
0x01f0: 6522 3a22 6d79 7363 6d22 2c22 7573 6572 e":"myscm","user
0x0200: 4964 223a 3330 3034 2c22 7061 7468 6e61 Id":3004,"pathna
0x0210: 6d65 223a 222f 6170 692f 7634 2f70 726f me":"/api/v4/pro
0x0220: 6a65 6374 732f 3339 3235 352f 6c61 6e67 jects/39255/lang
0x0230: 7561 6765 7322 2c22 7265 6d6f 7465 4164 uages","remoteAd
0x0240: 6472 223a 2233 302e 3339 2e32 3232 2e32 dr":"30.39.222.2
0x0250: 3237 222c 2275 7064 6174 6564 4174 223a 27","updatedAt":
0x0260: 2232 3032 302d 3032 2d31 3254 3038 3a34 "2020-02-12T08:4
0x0270: 383a 3433 2e33 3338 227d 0a 8:43.338"}.
It can be done by client.getLowLevelClient().performRequest(new Request(method, endpoint)), fill INDEX into parameter of method.

How to format WHOIS response

I need to implement a "whois" service, that will be listening for whois queries (whois -h ) and will respond with information from phpipam API. I am listening on port 43 with xinetd that starts script.
My problem is that when I send a response from the server, the server after first new line, resets connection. For testing purposes I am simply trying to send two echo lines and result is the same.
Script:
#!/bin/bash
echo "One"
echo "Two"
lsof:
# lsof -i | grep whois
xinetd 17696 root 6u IPv4 91809 0t0 TCP *:whois (LISTEN)
whois query (from client):
# whois -h 192.168.4.129 44.44.44.44
One
fgets: Connection reset by peer
tcpdump:
# tcpdump -n -i any -X port 43
16:34:01.162440 IP 192.168.4.128.49374 > 192.168.4.129.nicname: Flags [S], seq 1949234703, win 29200, options [mss 1460,sackOK,TS val 337480796 ecr 0,nop,wscale 7], length 0
0x0000: 4500 003c 7a53 4000 4006 3617 c0a8 0480 E..<zS#.#.6.....
0x0010: c0a8 0481 c0de 002b 742e f60f 0000 0000 .......+t.......
0x0020: a002 7210 8a80 0000 0204 05b4 0402 080a ..r.............
0x0030: 141d 8c5c 0000 0000 0103 0307 0000 0000 ...\............
0x0040: 0000 0000 0000 0000 0000 0000 ............
16:34:01.162916 IP 192.168.4.129.nicname > 192.168.4.128.49374: Flags [S.], seq 3654394026, ack 1949234704, win 28960, options [mss 1460,sackOK,TS val 25631110 ecr 337480796,nop,wscale 7], length 0
0x0000: 4500 003c 0000 4000 4006 b06a c0a8 0481 E..<..#.#..j....
0x0010: c0a8 0480 002b c0de d9d1 a0aa 742e f610 .....+......t...
0x0020: a012 7120 eb31 0000 0204 05b4 0402 080a ..q..1..........
0x0030: 0187 1986 141d 8c5c 0103 0307 0000 0000 .......\........
0x0040: 0000 0000 0000 0000 0000 0000 ............
16:34:01.162946 IP 192.168.4.128.49374 > 192.168.4.129.nicname: Flags [.], ack 1, win 229, options [nop,nop,TS val 337480797 ecr 25631110], length 0
0x0000: 4500 0034 7a54 4000 4006 361e c0a8 0480 E..4zT#.#.6.....
0x0010: c0a8 0481 c0de 002b 742e f610 d9d1 a0ab .......+t.......
0x0020: 8010 00e5 8a78 0000 0101 080a 141d 8c5d .....x.........]
0x0030: 0187 1986 0000 0000 0000 0000 0000 0000 ................
0x0040: 0000 0000 ....
16:34:01.163348 IP 192.168.4.128.49374 > 192.168.4.129.nicname: Flags [P.], seq 1:14, ack 1, win 229, options [nop,nop,TS val 337480798 ecr 25631110], length 13
0x0000: 4500 0041 7a55 4000 4006 3610 c0a8 0480 E..AzU#.#.6.....
0x0010: c0a8 0481 c0de 002b 742e f610 d9d1 a0ab .......+t.......
0x0020: 8018 00e5 8a85 0000 0101 080a 141d 8c5e ...............^
0x0030: 0187 1986 3434 2e34 342e 3434 2e34 340d ....44.44.44.44.
0x0040: 0a00 0000 0000 0000 0000 0000 0000 0000 ................
0x0050: 00 .
16:34:01.164122 IP 192.168.4.129.nicname > 192.168.4.128.49374: Flags [.], ack 14, win 227, options [nop,nop,TS val 25631110 ecr 337480798], length 0
0x0000: 4500 0034 b92d 4000 4006 f744 c0a8 0481 E..4.-#.#..D....
0x0010: c0a8 0480 002b c0de d9d1 a0ab 742e f61d .....+......t...
0x0020: 8010 00e3 8a2c 0000 0101 080a 0187 1986 .....,..........
0x0030: 141d 8c5e 0000 0000 0000 0000 0000 0000 ...^............
0x0040: 0000 0000 ....
16:34:01.165523 IP 192.168.4.129.nicname > 192.168.4.128.49374: Flags [P.], seq 1:5, ack 14, win 227, options [nop,nop,TS val 25631110 ecr 337480798], length 4
0x0000: 4500 0038 b92e 4000 4006 f73f c0a8 0481 E..8..#.#..?....
0x0010: c0a8 0480 002b c0de d9d1 a0ab 742e f61d .....+......t...
0x0020: 8018 00e3 d5a7 0000 0101 080a 0187 1986 ................
0x0030: 141d 8c5e 4f6e 650a 0000 0000 0000 0000 ...^One.........
0x0040: 0000 0000 0000 0000 ........
16:34:01.165844 IP 192.168.4.128.49374 > 192.168.4.129.nicname: Flags [.], ack 5, win 229, options [nop,nop,TS val 337480800 ecr 25631110], length 0
0x0000: 4500 0034 7a56 4000 4006 361c c0a8 0480 E..4zV#.#.6.....
0x0010: c0a8 0481 c0de 002b 742e f61d d9d1 a0af .......+t.......
0x0020: 8010 00e5 8a78 0000 0101 080a 141d 8c60 .....x.........`
0x0030: 0187 1986 0000 0000 0000 0000 0000 0000 ................
0x0040: 0000 0000 ....
16:34:01.165914 IP 192.168.4.129.nicname > 192.168.4.128.49374: Flags [R.], seq 5, ack 14, win 227, options [nop,nop,TS val 25631111 ecr 337480798], length 0
0x0000: 4500 0034 b92f 4000 4006 f742 c0a8 0481 E..4./#.#..B....
0x0010: c0a8 0480 002b c0de d9d1 a0af 742e f61d .....+......t...
0x0020: 8014 00e3 8a23 0000 0101 080a 0187 1987 .....#..........
0x0030: 141d 8c5e 0000 0000 0000 0000 0000 0000 ...^............
0x0040: 0000 0000 ....
16:34:01.166350 IP 192.168.4.129.nicname > 192.168.4.128.49374: Flags [R], seq 3654394031, win 0, length 0
0x0000: 4500 0028 826a 4000 4006 2e14 c0a8 0481 E..(.j#.#.......
0x0010: c0a8 0480 002b c0de d9d1 a0af 0000 0000 .....+..........
0x0020: 5004 0000 ea03 0000 0000 0000 0000 0000 P...............
0x0030: 0000 0000 0000 0000 0000 0000 0000 ..............
If I try to connect to 43 with telnet, everything work fine:
# telnet 192.168.4.129 43
Trying 192.168.4.129...
Connected to 192.168.4.129.
Escape character is '^]'.
One
Two
Connection closed by foreign host.
UPDATE:
Loop like this works fine
while read line; do
echo "One"
echo "Two"
break
done
But still I can't understand why it is not working by just passing two echo lines, or even if I try to run a while loop with counter:
#!/bin/bash
COUNT=0
while [ $COUNT -lt 5 ]; do
echo The counter is $COUNT
let COUNT=COUNT+1
done
It stops after the first cycle:
# whois -h 192.168.4.129 44.44.44.44
The counter is 0
fgets: Connection reset by peer
Maybe you have any ideas, what is the reason/difference between these two loops?
My /etc/xinetd.d/whoistest:
service whois
{
port = 43
flags = REUSE
socket_type = stream
protocol = tcp
user = root
wait = no
type = UNLISTED
server = /path/to/shell/script
disable = no
log_type = SYSLOG authpriv
log_on_failure += USERID
}
Any support will be appreciated.
You need to make sure your xinetd is configured correctly. Your script also needs to handle input from whois client. This works:
#!/bin/bash
while read line; do
echo $line
echo "foo"
break
done
This reads your line, echoes it back and then something. Feel free to replace echo $line with echo whatever you want.
I had to configure xinetd like this for it to work:
service whois
{
port = 43
flags = REUSE
socket_type = stream
protocol = tcp
user = root
wait = no
type = UNLISTED
server = /tmp/w.sh
disable = no
}
Hope this helps. If your xinetd service is configured wrong, it just seems to drop the connection. If your server doesn't actually listen to input from client, you'll get connection reset by peer errors.
Hannu

Parsing of Windows BSOD minidump file

I am trying to write an utility (Python script) which parses the Windows BSOD minidump files. I can not use MS utility WinDbg for this purposes. Everything I find online considers mostly process (user-space) dumps. A couple of links which talk about BSOD dumps are apparently outdated (?).
For example, my dump file header looks like
1 0000000: 5041 4745 4455 3634 0f00 0000 b01d 0000 PAGEDU64........
2 0000010: 00d0 6d1e 0000 0000 20b2 9002 00f8 ffff ..m..... .......
3 0000020: 500e 8a02 00f8 ffff 302b 8802 00f8 ffff P.......0+......
4 0000030: 6486 0000 0100 0000 7f00 0000 5041 4745 d...........PAGE
5 0000040: 0800 0000 0000 0000 3100 0580 0000 0000 ........1.......
6 0000050: f806 0400 0000 0000 0b7c 1201 80f8 ffff .........|......
7 0000060: 5041 4745 5041 4745 5041 4745 5041 4745 PAGEPAGEPAGEPAGE
8 0000070: 5041 4745 5041 4745 5041 4745 5041 4745 PAGEPAGEPAGEPAGE
I would expect to find at the offset 0x1db0 (see https://crashpad.chromium.org/doxygen/structMINIDUMP__HEADER.html) an array of directories. Instead I see:
476 0001db0: 5041 4745 5041 4745 5041 4745 5041 4745 PAGEPAGEPAGEPAGE
477 0001dc0: 5041 4745 5041 4745 5041 4745 5041 4745 PAGEPAGEPAGEPAGE
478 0001dd0: 5041 4745 5041 4745 5041 4745 5041 4745 PAGEPAGEPAGEPAGE
479 0001de0: 5041 4745 5041 4745 5041 4745 5041 4745 PAGEPAGEPAGEPAGE
480 0001df0: 5041 4745 5041 4745 5041 4745 5041 4745 PAGEPAGEPAGEPAGE
481 0001e00: 5041 4745 5041 4745 5041 4745 5041 4745 PAGEPAGEPAGEPAGE
482 0001e10: 5041 4745 5041 4745 5041 4745 5041 4745 PAGEPAGEPAGEPAGE
There is another link with completely different structure http://computer.forensikblog.de/en/2008/02/64bit-crash-dumps.html . The structure is Ok for a large (full?) dump file, but for not a minidump one I see.
What am I missing?
Update1. Solution with pykd works, but I would really prefer something smaller and platform independent. It appears that BlueScreenView from nirsoft.net does the trick without dependencies on DbgEng.dll. At least I did not see LoadLibrary calls for DbgEng. It means that at least one person has cracked the "not full" BSOD dump. I have made some progress with the dump, but I do not have full picture yet. I can get only list of the loaded modules. Tip - one of the 16 bits offsets in the 0x2000 section points (in my case 0x2034 has value 0xd480) to the table of unicode strings.
Update2.
Something like this probably does the job https://github.com/larytet/parse_minidump/blob/master/parse_minidump.py
I am still missing lot of things. Specifically details how the stack is defined. This is probably good enough for my purposes.
You can use pykd
Simple script:
import pykd
pykd.loadDump("dump.dmp")
print pykd.getStack()

Why does this simple YAML fail to parse or validate appropriately?

This does not validate in several different YAML validators. It also fails to parse if read via PyYAML. Is this simply invalid YAML?
---
databases:
- name: mydb
    migrations:
      - type: liquibase
file: 1.xml
      - type: liquibase
file: 2.xml
For example, YAMLlint produces the following error:
(<unknown>): mapping values are not allowed in this context at line 6 column 13
This is terribly annoying, and had nothing to do with the list of maps structure. Somehow there are whitespace characters in there that aren't ASCII, and neither PyYAML, nor the validators I tried, cope with those well.
$ xxd foobar.txt
00000000: 2d2d 2d0a 6461 7461 6261 7365 733a 0a20 ---.databases:.
00000010: 202d 206e 616d 653a 206d 7964 620a e280 - name: mydb...
00000020: 8220 e280 8220 6d69 6772 6174 696f 6e73 . ... migrations
00000030: 3a0a e280 8220 e280 8220 e280 8220 2d20 :.... ... ... -
00000040: 7479 7065 3a20 6c69 7175 6962 6173 650a type: liquibase.
00000050: 2020 2020 2020 2020 6669 6c65 3a20 312e file: 1.
00000060: 786d 6c0a e280 8220 e280 8220 e280 8220 xml.... ... ...
00000070: 2d20 7479 7065 3a20 6c69 7175 6962 6173 - type: liquibas
00000080: 650a 2020 2020 2020 2020 6669 6c65 3a20 e. file:
00000090: 322e 786d 6c0a 2.xml.
See those e28082's? They're En Space characters.

CasperJS "unable to load script"

I have the strangest issue while writing a new script for casperJS, which I use all the time.
When I try to launch the script, using "casperjs myScript.js, I get the following :
Unable to load script /var/www/scrapers/myScript.js; check file syntax
I checked my syntax and found nothing wrong, chmod +x the file, sudoed everything, still the same.
Even stranger, I copypasted the example code from the quickstart (http://docs.casperjs.org/en/latest/quickstart.html), saved , and same message...
All my other scripts in the same directory are working, what do I try now?
Re: what do I try now?
You can use xxd to look inside the files. Looking at the comments, you now have a working version and a non-working version, so can compare them side-by-side to understand what the problem was. (And if they are identical then you know you are looking for something at the file system level, not inside the files.)
(xxd comes with the vi package, and apparently is also available for Windows if you install vim.)
Here is sample output for a script with and without the UTF-8 BOM (which, incidentally, made no difference: both these scripts worked, at least with CasperJS 1.1 on Ubuntu):
0000000: efbb bf76 6172 2063 6173 7065 7220 3d20 ...var casper =
0000010: 7265 7175 6972 6528 2763 6173 7065 7227 require('casper'
0000020: 292e 6372 6561 7465 2829 3b0a 0a63 6173 ).create();..cas
0000030: 7065 722e 7374 6172 7428 2768 7474 703a per.start('http:
0000040: 2f2f 676f 6f67 6c65 2e63 6f6d 2f27 2c20 //google.com/',
0000050: 6675 6e63 7469 6f6e 2829 207b 0a20 2020 function() {.
0000060: 2074 6869 732e 6563 686f 2874 6869 732e this.echo(this.
0000070: 6765 7454 6974 6c65 2829 293b 0a7d 293b getTitle());.});
0000080: 0a0a 6361 7370 6572 2e72 756e 2829 3b0a ..casper.run();.
0000000: 7661 7220 6361 7370 6572 203d 2072 6571 var casper = req
0000010: 7569 7265 2827 6361 7370 6572 2729 2e63 uire('casper').c
0000020: 7265 6174 6528 293b 0a0a 6361 7370 6572 reate();..casper
0000030: 2e73 7461 7274 2827 6874 7470 3a2f 2f67 .start('http://g
0000040: 6f6f 676c 652e 636f 6d2f 272c 2066 756e oogle.com/', fun
0000050: 6374 696f 6e28 2920 7b0a 2020 2020 7468 ction() {. th
0000060: 6973 2e65 6368 6f28 7468 6973 2e67 6574 is.echo(this.get
0000070: 5469 746c 6528 2929 3b0a 7d29 3b0a 0a63 Title());.});..c
0000080: 6173 7065 722e 7275 6e28 293b 0a asper.run();.

Resources