I am trying to load data from Elasticsearch but I only need a few fields. I tried this:
docker run \
-v $(pwd)/data:/data\
--rm -ti \
elasticdump/elasticsearch-dump \
--input=<INPUT> \
--type=data \
--size 5 \
--searchBody='{"_source": ["#time", "data.metadata.tmp", "data.metadata.super_tmp"]}' \
--output=/data/file.json
But instead this code loads every field. How to fix it?
P.S. I saw this question but it didn't help me.
Related
I am attempting to have the New Relic Infrastructure Agent monitor my heroku applications.
The documentation says to run the following:
docker run \
-d \
--name newrelic-infra \
--network=host \
--cap-add=SYS_PTRACE \
--privileged \
--pid=host \
-v "/:/host:ro" \
-v "/var/run/docker.sock:/var/run/docker.sock" \
-e NRIA_LICENSE_KEY=[Key]\
newrelic/infrastructure:latest
But where do I actually run or put this so it runs it on my Heroku apps?
I am trying to setup a simple stripprefix middleware using Docker Desktop with unix containers and i get a very wiered behavior.
The static traefik-config comes from environment-variables:
docker run -p 8080:8080 -p 80:80 -p 443:443 --rm \
-a STDOUT \
--name traefik \
--network elastic \
-e TRAEFIK_ACCESSLOG=false \
-e TRAEFIK_API_INSECURE=true \
-e TRAEFIK_PROVIDERS_DOCKER_ENDPOINT="tcp://docker.for.win.localhost:2375" \
-e TRAEFIK_PROVIDERS_DOCKER_NETWORK="elastic" \
-e TRAEFIK_PROVIDERS_DOCKER_SWARMMODE=false \
-e TRAEFIK_LOG_LEVEL=DEBUG \
-v c:/dev/repos/docker/dockerfiles/traefik/ssl/localhost.crt:/ssl/traefik-server.crt \
-v c:/dev/repos/docker/dockerfiles/traefik/ssl/localhost.key:/ssl/traefik-server.key \
${custom_image}
I start a service using a middleware defined with labels like this:
-l traefik.http.routers.test.middlewares=test \
-l traefik.http.middlewares.test.stripprefix.prefixes=/test/my-service \
-l traefik.http.middlewares.test.stripprefix.forceslash=false
As a result i would expect a stripprefix-middleware with "/test/my-service" appear in traefik dashboard.
Instead
A stripprefix-middleware with "C:/dev/tools/git/" appears in the treafik dashboard. Appearently traefik somehow resolves the first "/" into the directory-path.
I start the whole thing using Git-Bash.
If anyone encountered something like this, i would really appreciate some pointers...
P.S.: i also tried all kinds of escaping and quoting i could think of
Maybe to prevent others wasting as much time....
... Turns out that git-bash for windows does some crazy stuff before handing the commands to docker.
Executing the exact same config using IntelliJ (docker-integration) or powershell does not replace the leading "/" with a windows path.
How Do You Pass a Variable to Curl in Bash Script
I'm not able to get this curl to work in a bash script using an environment variable. The API key gets misinterpreted somehow when passed in via a variable and I'm getting authentication errors on submit. If I plug in the API key with no surrounding quotes as plaintext, this works just fine. I've tried various forms of escaping quotes and other combinations. Any help would be great.
Code
#!/bin/bash
source .env
curl -X POST https://api.easypost.com/v2/shipments \
-u "$EASYPOST_TEST_API_KEY": \
-d 'shipment[to_address][name]=Dr. Steve Brule' \
-d 'shipment[to_address][street1]=179 N Harbor Dr' \
-d 'shipment[to_address][city]=Redondo Beach' \
-d 'shipment[to_address][state]=CA' \
-d 'shipment[to_address][zip]=90277' \
-d 'shipment[to_address][country]=US' \
-d 'shipment[to_address][phone]=8573875756' \
-d 'shipment[to_address][email]=dr_steve_brule#gmail.com' \
-d 'shipment[from_address][name]=EasyPost' \
-d 'shipment[from_address][street1]=417 Montgomery Street' \
-d 'shipment[from_address][street2]=5th Floor' \
-d 'shipment[from_address][city]=San Francisco' \
-d 'shipment[from_address][state]=CA' \
-d 'shipment[from_address][zip]=94104' \
-d 'shipment[from_address][country]=US' \
-d 'shipment[from_address][phone]=4153334445' \
-d 'shipment[from_address][email]=support#easypost.com' \
-d 'shipment[parcel][length]=20.2' \
-d 'shipment[parcel][width]=10.9' \
-d 'shipment[parcel][height]=5' \
-d 'shipment[parcel][weight]=65.9' \
Per Gordon Davisson's suggestion to break it down into a minimal request, it started sending right. Then I rebuilt it just as in my question above and for whatever reason it worked like a charm. The -x flag was also incredibly helpful in troubleshooting this but come to find out there wasn't anything inherently wrong with it in the first place. Thanks for everyone's responses!
I've been trying to invoke conduit api using curl to add member to specific project and wrote following command:
curl https://test-caxzj6a226zp.phacility.com/api/project.edit -d api.token=api-[token] -d transactions[0][type]=parent -d transactions[0][value]=[project-phid] -d transactions[0][type]=members.add -d transactions[0][value][]=[user-phid]
I've followed the instructions on https://secure.phabricator.com/conduit/method/project.edit/ and was able to query project.search and user.search. But project.edit is giving me trouble. Any help will be appreciated.
PS: I get the following error: {"result":null,"error_code":"ERR-CONDUIT-CORE","error_info":"Validation errors:\n - Projects must have a name."}
You have a few errors in your call. First one is that you're specifying two transactions to apply within the same index. Formatting your call a bit, we get:
curl https://test-caxzj6a226zp.phacility.com/api/project.edit \
-d api.token=api-[token] \
-d transactions[0][type]=parent \
-d transactions[0][value]=[project-phid] \
-d transactions[0][type]=members.add \
-d transactions[0][value][]=[user-phid]
You have two definitions for transactions[0], if you want to apply multiple transformations, you need to increment your index:
curl https://test-caxzj6a226zp.phacility.com/api/project.edit \
-d api.token=api-[token] \
-d transactions[0][type]=parent \
-d transactions[0][value]=[project-phid] \
-d transactions[1][type]=members.add \
-d transactions[1][value][]=[user-phid]
Now, you didn't actually mention wanting to change the parent project, so I'm going to remove that transaction.
curl https://test-caxzj6a226zp.phacility.com/api/project.edit \
-d api.token=api-[token] \
-d transactions[0][type]=members.add \
-d transactions[0][value][]=[user-phid]
The users to add is also an array, so you need to specify an index for that argument:
curl https://test-caxzj6a226zp.phacility.com/api/project.edit \
-d api.token=api-[token] \
-d transactions[0][type]=members.add \
-d transactions[0][value][0]=[user-phid]
You could add additional users using -d transactions[0][value][1]=[user-phid], -d transactions[0][value][2]=[user-phid] etc.
Finally, you also need to specify which project you want to add the member to, since it's a required field. To do this, specify the project's numeric id (you can find this in its URL, /project/view/1/), its PHID, or its name/slug.
curl https://test-caxzj6a226zp.phacility.com/api/project.edit \
-d api.token=api-[token] \
-d transactions[0][type]=members.add \
-d transactions[0][value][0]=[user-phid] \
-d objectIdentifier=[project-phid]
You can actually submit a test call to conduit from within the conduit application, and after submitting it, one of the tabs in the example section will display the curl call required to perform that action via the API:
https://secure.phabricator.com/conduit/method/project.edit/
I'm running into an odd behavior on the latest version of vagrant in a Windows7/msys/Virtualbox environment setup, where after executing a vagrant up command I get an error with rsync; 'file has vanished: "/c/Users/spencerd/workspace/watcher/.LISTEN' doing the provisioning stage.
Since google, irc, and issue trackers have little to no documentation on this issue I wonder if anyone else ran into this and what would the fix be?
And for the record I have successfully build a box using the same vagrant file and provisioning script. For those that want to look, the project code is up at https://gist.github.com/denzuko/a6b7cce2eae636b0512d, with the debug log at gist.github.com/
After digging further into the directory structure and running into issues with git pushing code up I was able to find a non-existant file that needed to be removed after a reboot.
Thus, doing a reboot and a rm -rf -- "./.LISTEN\ \ \ \ \ 0\ \ \ \ \ \ 100\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ " did the trick.