Below is the code I am using:
"_source" : {
"name" : "hn name",
"user_id" : 553,
"email_id" : "ns#gmail.com",
"lres_id" : "",
"hres_id" : "hn image",
"followers" : 0,
"following" : 1,
"mentors" : 2,
"mentees" : 2,
"basic_info" : "hn developer",
"birth_date" : 1448451985397,
"charge_price" : 3000,
"org" : "mnc pvt ltd",
"located_in" : "Noidasec51 ",
"position" : "jjunior ava developer",
"requests" : 0,
"exp" : 5,
"video_bio_lres" : "hn test lres url",
"video_bio_hres" : "hn hres url",
"ratings" : [ {
"rating" : 1,
"ratedByUserId" : 777
}, {
"rating" : 1,
"ratedByUserId" : 555
} ],
"avg_rating" : 0.0,
"status" : 0,
"expertises" : [ 3345, 1234, 2345 ],
"blocked_users" : [ ]
}
In the Following Code, I want to delete rating ratedByUserId 555 only.But Some How I am unable for doing so.
How to do it?
its works for me:-
curl -XPOST 'localhost:9200/mentorz/users/555/_update' -d
'{" script":"ctx._source.ratings.remove(ratings)",
"params":{
"ratings":{
"rating":1,
"ratedByUserId":555
}
}
}'
I am trying to write a bash script that edits a json file to strip out entire objects, but I have hit a wall.
here is sample data
[
{
"MapObject" : {
"BoundingBox" : [ -1313.574, -1010.804, -1113.574, -810.804 ],
"Caption" : "Ref46",
"Comment" : "",
"Position" : {
"CPosition" : [ -1213.574, -910.804, 0, 0 ]
},
"Size" : {
"CPosition" : [ 100, 100, 0, 0 ]
},
"Type" : "moReferencePointID"
}
},
{
"MapObject" : {
"BoundingBox" : [ -1313.06, 495.39, -1113.06, 695.39 ],
"Caption" : "Ref47",
"Comment" : "",
"Position" : {
"CPosition" : [ -1213.06, 595.39, 0, 0 ]
},
"Size" : {
"CPosition" : [ 100, 100, 0, 0 ]
},
"Type" : "moReferencePointID"
}
},
{
"MapObject" : {
"BoundingBox" : [ -18070, 1180, -17870, 1380 ],
"Caption" : "Path way84",
"Comment" : "",
"ExecutableMapObject" : {
"BehaviourList" : [
{
"Speed" : 200,
"Type" : "bcSpeed"
},
{
"AvoidanceSpeed" : 150,
"FailureAction" : 0,
"FailureRetries" : 60,
"FailureWait" : 10,
"PathMode" : 2,
"PerSegment" : 15,
"ReturnToStart" : 0,
"ScanAngle" : 0.06981317007977778,
"Type" : "bcObstacleAvoidance",
"Wait" : 10,
"WaitUnits" : 2
},
{
"BinarySensorStates" : 3,
"SensorSensitivityList" : [
{
"IgnoreMapData" : false,
"Mask" : 65536,
"SpeedCapDistance" : 70,
"SpeedCapEnabled" : true,
"StopDistance" : 70
},
{
"IgnoreMapData" : false,
"Mask" : 131072,
"SpeedCapDistance" : 70,
"SpeedCapEnabled" : true,
"StopDistance" : 70
},
{
"IgnoreMapData" : false,
"Mask" : 262144,
"SpeedCapDistance" : 240,
"SpeedCapEnabled" : true,
"StopDistance" : 240
},
{
"IgnoreMapData" : false,
"Mask" : 524288,
"SpeedCapDistance" : 240,
"SpeedCapEnabled" : true,
"StopDistance" : 240
},
{
"IgnoreMapData" : false,
"Mask" : 1048576,
"SpeedCapDistance" : 240,
"SpeedCapEnabled" : true,
"StopDistance" : 240
},
{
"IgnoreMapData" : false,
"Mask" : 2097152,
"SpeedCapDistance" : 280,
"SpeedCapEnabled" : true,
"StopDistance" : 280
},
{
"IgnoreMapData" : false,
"Mask" : 4194304,
"SpeedCapDistance" : 320,
"SpeedCapEnabled" : true,
"StopDistance" : 320
},
{
"IgnoreMapData" : false,
"Mask" : 8388608,
"SpeedCapDistance" : 0,
"SpeedCapEnabled" : false,
"StopDistance" : 0
},
{
"IgnoreMapData" : false,
"Mask" : 16777216,
"SpeedCapDistance" : 0,
"SpeedCapEnabled" : false,
"StopDistance" : 0
}
],
"SpeedCap" : 30,
"Type" : "bcObstacleSensitivity2"
}
],
"EntryPoint" : {
"CPosition" : [ -13450, 1290, 0, 0 ]
},
"ExitPoint" : {
"CPosition" : [ -17970, 1280, 0, 0 ]
},
"RelocateOnExecute" : false,
"Required" : false,
"TwoWay" : false
},
"GoalSet" : [
{
"Path" : {
"PathType" : "pitLinear",
"Segments" : [
{
"Segment" : [ -13450, 1290, -17970, 1280 ]
}
]
}
}
],
"Position" : {
"CPosition" : [ -17970, 1280, 0, 0 ]
},
"Size" : {
"CPosition" : [ 0, 0, 0, 0 ]
},
"Type" : "moLinePathID"
}
},
{
"MapObject" : {
"AttributeList" : [
[ "Lock", "{} LabInt {}" ]
],
"BoundingBox" : [ -12350, -500, -12150, -300 ],
"Caption" : "QUEUE/LabInt",
"Comment" : "",
"ExecutableMapObject" : {
"EntryPoint" : {
"CPosition" : [ -12250, -400, 0, 0 ]
},
"ExitPoint" : {
"CPosition" : [ -12250, -400, 0, 0 ]
},
"RelocateOnExecute" : false,
"Required" : false,
"TwoWay" : false
},
"Position" : {
"CPosition" : [ -12250, -400, 0, 0 ]
},
"Size" : {
"CPosition" : [ 100, 100, 0, 0 ]
},
"Type" : "moHotPointID"
}
},
]
Each object begins with
{
and ends with
},
I want to remove all objects and their data that contain "Type" : "moReferencePointID" and "moReferenceLineID".
There are multiple instances of Type in some objects.
I've been monkeying with sed and awk using what I can find online and can't quite get it to work. Any help would be appreciated.
I'd post what I've done so far, but it definitely won't work...
Regex is the wrong answer for parsing structured text; using a parser is the right answer. Reading the whole file into memory is the wrong answer, too, but that's what parsers do, so meh.
That said, the wrong answer is sometimes okay for:
throwaway scripts on trivial, non-production datasets where you have absolute control over the input. For example, as a one-shot command to migrate a small amount of data between two systems, where you don't want to go writing a script for it.
where all you have is a text editor.
where you are on a live server and don't want to install or write additional utilities for a trivial one-off task.
where you are working on a readonly server and can't write scripts
various disaster recovery/mitigation situations.
It's important to know when not to use regexes, and how to use the more powerful and reliable alternatives: but when you really need them, knowing how to kludge a task with a regex can be a lifesaver.
At the very least, this is an interesting pedagogical exercise for exploring regular expressions and multiline-text-block handling with various unix command line utilities.
PHP:
php -r '$a=file_get_contents("data.json");file_put_contents("cleaned.json", preg_replace("/\{\s*\"MapObject(?:(?!MapObject)[\s\S])+\"Type\"\s*:\s*\"moReference(?:Point|Line)ID\"(?:(?!MapObject)[\s\S])+},?\s*/", "", $a));'
Perl:
The $_=do{local $/;<>}; just makes it read the file in as one line.
perl -e '$_=do{local $/;<>};$_=~s/\{\s*\"MapObject(?:(?!MapObject)[\s\S])+\"Type\"\s*:\s*\"moReference(?:Point|Line)ID\"(?:(?!MapObject)[\s\S])+},?\s*//g;open($fh,">","cleaned.json");print $fh $_;' data.json
Awk:
The RS="<{EOF}>" just makes it read the file in as one line. Replace the string with any string that definitely won't appear in your dataset.
awk -v RS="<{EOF}>" '{gsub(/\{[\r\n\t ]*\"MapObject/, "\x1E&");print $0}' < data.json > data1.json
awk -v RS="<{EOF}>" '{gsub(/\x1E[^\x1E]*\"Type\"[\t ]*:[\t ]*\"moReference(Point|Line)ID\"[^\x1E]*},[\r\n\t ]*|\x1E/, "");print $0}' < data1.json > cleaned.json
Sed:
The -e 1h;2,$H;$!d;g just makes it read the file in as one line.
sed -e '1h;2,$H;$!d;g' -E -e 's/\{\s*\"MapObject/\x1E\0/g' < data.json > data2.json
sed -e '1h;2,$H;$!d;g' -E -e 's/\x1E|\{[^\x1E]*\"Type\"\s*:\s*\"moReference(Point|Line)ID\"[^\x1E]*},\s*//g' < data2.json > cleaned.json
OSX Sed
echo 'Just use vi' | sed -e '' && vi data.json
In all cases above, I've done whatever it took to read the whole file in as a single string, then globally replaced what's matched by the regex with an empty string.
These solutions rely on the fact that each object begins with an identifiable opening pattern \{\s*"MapObject"\s*:\s*\{, which appears exactly once, at the beginning of each object.
PHP and Perl could just use this directly, using negative lookahead assertions.
However, Sed and Awk don't support look-around assertions, nor non-greedy matching. So there, I needed a single-width marker, so I had to do it in multiple passes: one to append some marker characters to the opening pattern (I picked 0x1E, ascii control code for "Record Separator"); and one to do the actual replacement.
If your code might contain arbitrary byte values, rather than readable text, then you will need another two passes, to first replace all 0x1E with a string that doesn't appear in your text, then replace that string back at the end.
Awk also don't seem to support (on my OSX machine) the \s escape for whitespace, so I've used character groups instead.
The regex can be described as:
\{\s*"MapObject" The opening pattern, starting an object.
(?:(?!MapObject)[\s\S])+ Some characters, not the opening pattern.
\"Type\" Literal quoted string "Type".
\s*:\s* Whitespace wrapped literal colon ':'.
\"moReference Literal doublequote and start of string.
(?: Non-capturing group of...
Point ...literal string 'Point'...
| ...or...
Line ...literal string 'Line'.
) Finish that logical grouping.
ID\" End of literal string and doublequote.
(?:(?!MapObject)[\s\S])+ More characters, not the opening pattern.
} Closing brace.
,? Optional comma.
\s* Optional trailing whitespace.
You can see this terrible beast in action here, where it can be seen working with an object containing multiple Type lines: https://regex101.com/r/XGULr5/4
jq is your friend in this situation. The cookbook for it has an example that should be adaptable to this.
{
if ($0 ~ /{/) {
strt=1;cnt++
}
if (strt=1) {
print cnt
bits[cnt]=bits[cnt]"\n"$0
if ( $0 ~ /Type/ ) {
typ[cnt]=$0
}
}
if ( $0 ~ /}/ ) {
strt=0
}
}
END{
for ( i=1;i<=cnt;i++ ) {
if ( typ[i] ~ /NotGood/ || typ[i] ~ /Bad/ ) {
}
else {
printf bits[i]
}
}
}
The above awk code should work (run with awk -f codefile datafile). Essentially we are checking each line (pattern matching) for a "{" and then begin writing each line to an array (bits) until "}" is pattern matched. In addition to this an additional array (typ) is written to track lines that pattern match "Type". At the end of the code, the bits array is looped through, referencing typ. If typ contains, "NotGood", the entry in bits is ignored, otherwise it is printed.
As a one liner, the solution would be:
awk '{ if ($0 ~ /{/) { strt=1;cnt++ } if (strt=1) { pnt;bits[cnt]=bits[cnt]"\n"$0;if ( $0 ~ /Type/ ) { typ[cnt]=$0 } } if ( $0 ~ /}/ ) { strt=0 } } END { for ( i=1;i<=cnt;i++ ) { if ( typ[i] ~ /NotGood/ || typ[i] ~ /Bad/ ) { } else { printf bits[i] } } }' filename
I have wrote simple search extension. It works fine on Windows but does not work very well on Mac. Extension sets newTab page, home page and default search provider. But on Mac search provider is not changed (I mean when I type in url field, omnibox), it is still google.
Here is the manifest:
{
"name" : "Example",
"short_name" : "example",
"manifest_version" : 2,
"version" : "2.6.1.0",
"description" : "Example Extension Description",
"chrome_settings_overrides" : {
"homepage" : "http://example.com/",
"search_provider" : {
"name": "Example Search Name",
"is_default" : true,
"encoding" : "UTF-8",
"favicon_url": "http://example.com/favicon.png",
"keyword" : "keyword.example.com",
"search_url" : "http://example.com/?q={searchTerms}&gid=SDD316",
"suggest_url" : "http://example.com/suggest.php?q={searchTerms}&gid=SDD316"
},
"startup_pages" : ["http://example.com"]
},
"chrome_url_overrides" : {
"newtab" : "html/newTab.html"
},
"icons" : {
"128" : "icon128.png",
"16" : "icon16.png",
"48" : "icon48.png"
},
"browser_action": {
"default_icon": "w.png",
"default_title": "Example Search",
"default_popup": "html/popup.html"
},
"background" : {
"scripts" : ["js/jquery-3.0.0.min.js", "js/background.js"]
},
"permissions" : ["storage", "http://*/*", "https://*/*", "unlimitedStorage", "chrome://favicon/*", "webRequest", "webRequestBlocking" ,"cookies", "*://*.test.com/*"],
"content_scripts": [
{
"js": [
"js/start.js"
],
"matches": [
"http://example.com/*"
],
"all_frames": false,
"run_at": "document_end"
}]
}
Wouldn't chrome_settings_overrides.search_provider change search in omnibox?
Settings Overrides is currently available in Windows and Mac, Mac in Chrome 56 and later. No Linux as of yet.
Chrome might be the only browser currently allowing Settings Overrides at some level.
Trivia: However general settings overriding (customization (for the user itself own client)) requires more serious security tuning and pruning by core Chromium.
General about browsers: Currently browsers are on fast dev cycle - Mozilla/Firefox coming along with Chrome, Opera, Vivaldi and Safari has really spiced up browser dev.
Settings Overrides soon then? I'm afraid that's a no. There's so many things happening atm.
Quoting the docs:
The API is available only on Windows.
I have an index with expireAfterSeconds set. How do I confirm this using the CLI when the output of db.my_collection.getIndexes() only shows this:
{
"v": 1,
"key": {
"last_used_at": 1
},
"name": "last_used_at_1",
"ns": "cms.user_sessions"
}
It looks just like a normal index without expireAfterSeconds set.
How can I confirm that an index was created with expireAfterSeconds?
Here's the Ruby code that creates the index. I have tried dropping the index and recreating it using this code several times.
#user_sessions.indexes.create_many([
{key: {token: 1}, unique: true},
{key: {last_used_at: 1}, expireAfterSeconds: ttl}
])
Version information:
$ mongod --version
db version v2.6.5
2016-07-13T16:11:45.844+0200 git version: e99d4fcb4279c0279796f237aa92fe3b64560bf6
$ gem list | grep mongo
mongo (2.2.5)
there is an issue with your index, try drop it and recreate.
When TTL is set you will get that from db.name.getIndexes() command
[
{
"v" : 1,
"key" : {
"_id" : 1
},
"name" : "_id_",
"ns" : "test.eventlog"
},
{
"v" : 1,
"key" : {
"lastModifiedDate" : 1.0
},
"name" : "lastModifiedDate_1",
"ns" : "test.eventlog",
"expireAfterSeconds" : 3600.0
}
]