I have got around 1000 contacts to import to Mailchimp. This is my company's old database, which we have exported from the CSM system, and we want every contact to confirm their subscription if they want to be on our subscription list.
When I try to import it through Mailchimp, I can't give the contact status pending.
So, I have managed how to do it with single contact through bash, but I will want to import the whole contact list.
I am not familiar with this scripting language that much, so can anybody advise me, is there a way to import the data from the CSV file and how can I do it?
Or maybe there is some other way to do it?
This is the code that is working for a single contact:
#!/bin/bash
set -euo pipefail
list_id="Add_LIST_ID"
user_email="Add_E_MAIL"
user_fname="Add_F_NAME"
user_lname="Add_L_NAME"
curl -sS --request POST \
--url "https://$API_SERVER.api.mailchimp.com/3.0/lists/$list_id/members" \
--user "key:$API_KEY" \
--header 'content-type: application/json' \
--data #- \
<<EOF | jq '.id'
{
"email_address": "$user_email",
"status": "pending",
"merge_fields": {
"FNAME": "$user_fname",
"LNAME": "$user_lname"
}
}
EOF
EDIT1
Okay, I have managed to load the data from csv file. The code is below.
while IFS=, read -r col1
do
{
#!/bin/bash
set -euo pipefail
list_id="LIST_ID"
echo "$col1"
curl -sS --request POST \
--url "https://$API_SERVER.api.mailchimp.com/3.0/lists/$list_id/members" \
--user "key:$API_KEY" \
--header 'content-type: application/json' \
--data #- \
<<EOF | jq '.id'
{
"email_address": "$(echo $col1)",
"status": "pending",
"merge_fields": {
"FNAME": "",
"LNAME": ""
}
}
EOF
}
done < mails.csv
I have put echo line after list_id to see if the data is imported correctly.
The code is working (no errors in the buildup), but I have managed to add a contact to the list only once (subscriber hash is the response). In other tries, I have got a "null" value in response. Does anybody know why?
I have Elasticsearch 7.4.1 cluster for storing logs, and i have this SLM pattern applied in my cluster
curl -X PUT "afghanistan-01.elastic:9200/_slm/policy/nightly-snapshots?pretty" -H 'Content-Type: application/json' -d'
{
"schedule": "0 30 1 * * ?",
"name": "<nightly-snap-{now/d}>",
"repository": "backup_repository",
"config": {
"indices": ["*"]
}
}
so this SLM takes snapshot daily at 01:30 AM.
And i want to delete indices which is older than 7 days after the successful snapshot completion
so is there any way to do that using the policies?
i know how to do it using Bash, but i think it's better to use internal mechanism to do this
I recently upgraded my ElasticStack instance from 5.5 to 6.0, and it seems that some of the breaking changes of this version have harmed my pipeline. I had a script that, depending on the indices inside ElasticSearch, created index-patterns automatically for some groups of similar indices. The problem is that with the new mapping changes of the 6.0 version, I cannot add any new index-pattern from the console. This was the request I used and worked fine in 5.5:
curl -XPOST "http://localhost:9200/.kibana/index-pattern" -H 'Content- Type: application/json' -d'
{
"title" : "index_name",
"timeFieldName" : "execution_time"
}'
This is the response I get now, in 6.0, from ElasticSearch:
{
"error": {
"root_cause": [
{
"type": "illegal_argument_exception",
"reason": "Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, doc]"
}
],
"type": "illegal_argument_exception",
"reason": "Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, doc]"
},
"status": 400
}
How could I add index-patterns from the console avoiding this multiple mapping issue?
The URL has been changed in version 6.0.0, here is the new URL:
http://localhost:9200/.kibana/doc/doc:index-pattern:my-index-pattern-name
This CURL should work for you:
curl -XPOST "http://localhost:9200/.kibana/doc/index-pattern:my-index-pattern-name" -H 'Content-Type: application/json' -d'
{
"type" : "index-pattern",
"index-pattern" : {
"title": "my-index-pattern-name*",
"timeFieldName": "execution_time"
}
}'
If you are Kibana 7.0.1 / 7+ then you can refer saved_objects API ex:
Refer: https://www.elastic.co/guide/en/kibana/master/saved-objects-api.html (Look for Get, Create, Delete etc).
In this case, we'll use: https://www.elastic.co/guide/en/kibana/master/saved-objects-api-create.html
$ curl -X POST -u $user:$pass -H "Content-Type: application/json" -H "kbn-xsrf:true" "${KIBANA_URL}/api/saved_objects/index-pattern/dummy_index_pattern" -d '{ "attributes": { "title":"index_name*", "timeFieldName":"sprint_start_date"}}' -w "\n" | jq
and
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 327 100 250 100 77 543 167 --:--:-- --:--:-- --:--:-- 543
{
"type": "index-pattern",
"id": "dummy_index_pattern",
"attributes": {
"title": "index_name*",
"timeFieldName": "sprint_start_date"
},
"references": [],
"migrationVersion": {
"index-pattern": "6.5.0"
},
"updated_at": "2020-02-25T22:56:44.531Z",
"version": "Wzg5NCwxNV0="
}
Where $KIBANA_URL was set to: http://my-elk-stack.devops.local:5601
If you don't have jq installed, remove | jq from the command (as listed above).
PS: When KIBANA's GUI is used to create an index-pattern, Kibana stores its i.e. index ID as an alpha-numeric value (ex: laskl32ukdflsdjflskadf-sdf-sdfsaldkjfhsdf-dsfasdf) which is hard to use/find/type when doing GET operation to find info about an existing index-pattern using the following curl command.
If you passed index pattern name (like we did above), then in Kibana/Elasticsearch, it'll story the Index-Pattern's ID by the name you gave to the REST call (ex: .../api/saved_objects/index-pattern/dummy_index_pattern")
here: dummy_index_pattern will become the ID (only visible if you hover over your mouse on the index-pattern name in Kibana GUI) and
it'll have it's index name as: index_name* (i.e. what's listed in GUI when you click on Kibana Home > Gear icon > Index Patterns and see the index patterns listed on the right side.
NOTE: The timeFieldName is very important. This is the field, which is used for looking for time-series events (i.e. especially TSVB Time Series Visual Builder Visualization type). By default, it uses #timestamp field, but if you recreate your index (instead of sending delta information to your target Elasticsearch index from a data source (ex: JIRA)) every time and send all data in one shot from scratch from a data source, then #timestamp won't help with Visualization's Time-Spanning/Window feature (where you change time from last 1 week to last 1 hour or last 6 months); in that case, you can set a different field i.e. sprint_start_date like I used (and now in Kibana Discover data page, if you select this index-pattern, it'll use sprint_start_date (type: date) field, for events.
To GET index pattern info about the newly created index-pattern, you can refer: https://www.elastic.co/guide/en/kibana/master/saved-objects-api-get.html --OR run the following where (the last value in the URL path is the ID value of the index pattern we created earlier:
curl -X GET "${KIBANA_URL}/api/saved_objects/index-pattern/dummy_index_pattern" | jq
or
otherwise (if you want to perform a GET on an index pattern which is created via Kibana's GUI/webpage under Page Index Pattern > Create Index Pattern, you'd have to enter something like this:
curl -X GET "${KIBANA_URL}/api/saved_objects/index-pattern/jqlaskl32ukdflsdjflskadf-sdf-sdfsaldkjfhsdf-dsfasdf" | jq
For Kibana 7.7.0 with Open Distro security plugin (amazon/opendistro-for-elasticsearch-kibana:1.8.0 Docker image to be precise), this worked for me:
curl -X POST \
-u USERNAME:PASSWORD \
KIBANA_HOST/api/saved_objects/index-pattern \
-H "kbn-version: 7.7.0" \
-H "kbn-xsrf: true" \
-H "content-type: application/json; charset=utf-8" \
-d '{"attributes":{"title":"INDEX-PATTERN*","timeFieldName":"#timestamp","fields":"[]"}}'
Please note, that kbn-xsrf header is required, but it seems like it's useless as from security point of view.
Output was like:
{"type":"index-pattern","id":"UUID","attributes":{"title":"INDEX-PATTERN*","timeFieldName":"#timestamp","fields":"[]"},"references":[],"migrationVersion":{"index-pattern":"7.6.0"},"updated_at":"TIMESTAMP","version":"VERSION"}
I can't tell why migrationVersion.index-pattern is "7.6.0".
For other Kibana versions you should be able to:
Open Kibana UI in browser
Open Developers console, navigate to Network tab
Create index pattern using UI
Open POST request in the Developers console, take a look on URL and headers, than rewrite it to cURL
Indices created in Elasticsearch 6.0.0 or later may only contain a single mapping type.
Indices created in 5.x with multiple mapping types will continue to function as before in Elasticsearch 6.x.
Mapping types will be completely removed in Elasticsearch 7.0.0.
Maybe you are creating a index with more than one doc_types in ES 6.0.0.
https://www.elastic.co/guide/en/elasticsearch/reference/current/removal-of-types.html
Create index-pattern in bulk with timestamp：
cat index_svc.txt
my-index1
my-index2
my-index3
my-index4
my-index5
my-index6
cat index_svc.txt | while read index; do
echo -ne "create index-pattern ${index} \t"
curl -XPOST "http://10.0.1.44:9200/.kibana/doc/index-pattern:${index}" -H 'Content-Type: application/json' -d "{\"type\":\"index-pattern\",\"index-pattern\":{\"title\":\"${index}2020*\",\"timeFieldName\":\"#timestamp\"}}"
echo
done
I am currently using:
$output = $adset->read(array("campaign_schedule"));
Which is similar to:
curl -G \
-d "fields=campaign_schedule" \
-d "access_token=___" \
-d "method=get" \
'https://graph.facebook.com/{adset_id}'
This returns the campaign_schedule array deep within the response object.
How can I get just the campaign schedule as an array/object?
Documentation here:
https://developers.facebook.com/docs/marketing-api/adset/pacing/v2.3
The campaign schedule will be a member variable of the object returned.
print_r($adset->campaign_schedule);
I"m using the following commands which i've copied over from the documentation. Assume that I'm passing in the application id and keys in each curl statement for brevity.
create player
curl -X POST \
-d '{"name":"abhinav", "rank" :"amazing"}' \
https://api.parse.com/1/classes/Player
Player objectId: HgMZF6H90L
Create game and add pointer relation
curl -X POST \
-d '{"Level" : "TWO"}' \
https://api.parse.com/1/classes/GameScore
GameScore objectId: cwYIEwFaq9
curl -X PUT \
-d '{"opponents":{"__op":"AddRelation","objects":[{"__type":"Pointer","className":"Player","objectId":"HgMZF6H90L"}]}}' \
https://api.parse.com/1/classes/GameScore/tDiFZSE0lQ
Now I check from the Data browser and the GameScore object has a field with Relation. clicking on the "View Relations" navigates me to the correct list underneath. Working fine except that it says "Relation" and not pointer. Not sure if that is relevant
HOWEVER, I'm not able to query this information from the REST API. A GET results in the relation data but no objectId for the player available.
curl -X GET \
https://api.parse.com/1/classes/GameScore/cwYIEwFaq9
result:
{"Level":"TWO","createdAt":"2014-06-11T08:49:43.325Z","objectId":"cwYIEwFaq9","opponents":{"__type":"Relation","className":"Player"},"updatedAt":"2014-06-11T08:51:01.093Z"}
tried with include option but that results in some error
curl -X GET \
--data-urlencode "include=opponents" \
https://api.parse.com/1/classes/GameScore/cwYIEwFaq9
result:
code 107, invalid JSON
What am I doing wrong?
why are you using "Remove" to add a relation? Look at the docs again.
go to Docs section "Creating Roles" again....
"__op": "AddRelation",
"objects": [
{
"__type": "Pointer",
"className": "_Role",
"objectId": "Ed1nuqPvc"
}
]
-- EDIT
to get using "include=opponents" you could create with this:
{"opponents":{"__op":"Add","objects":[{"__type":"Pointer","className":"_Role","objectId":"Ed1..."}]}}
Solved :
The problem is that i needed to create an array of pointers. So instead of the "AddRelation", i simply need to use the "AddUnique" operation to create an entry for the array. So
curl -X POST \
-d '{"Level" : "TWO", "opponents":{"__op":"AddUnique","objects":[{"__type":"Pointer","className":"Player","objectId":"5Q4QsKF8QR"}]}}' \
https://api.parse.com/1/classes/GameScore