Trying out filters for replication, I stumbled upon a problem.
While my filter is working as an entry in the _replicator database, I doesn't when using cURL.
The filter in the design document is:
{
"_id": "_design/partial",
"filters": {
"mobile": "function(doc, req) {
if (doc._attachments) {
var result = new Boolean(true);
for (attachment in doc._attachments) {
if (attachment.content_type == 'image/jpeg') {
return true;
}
if (doc._attachments.length > 1024) {
result = false;
}
}
return result;
} else {
return true;
}
}"
}
}
The cURL line:
curl -X POST http://admin:pass#192.168.178.13:5985/_replicate -d '{\"source\":\"http://admin:pass#192.168.2:5984/docs2\",\"target\":\"docs2_partial\",\"filter\":\"partial/mobile\",\"create_target\":true}' -H "Content-Type: application/json"
I created _design/partial document on both target and source, but all documents are being replicated. Even the one with an attached binary bigger than 1 MB.
Any help is appreciated!
The cURL reply is:
{"ok":true,"session_id":"833ff96d21278a24532d116f57c45f31","source_last_seq":32,"replication_id_version":2,"history":[{"session_id":"833ff96d21278a24532d116f57c45f31","start_time":"Wed, 17 Aug 2011 21:43:46 GMT","end_time":"Wed, 17 Aug 2011 21:44:22 GMT","start_last_seq":0,"end_last_seq":32,"recorded_seq":32,"missing_checked":0,"missing_found":28,"docs_read":28,"docs_written":28,"doc_write_failures":0}]}
Using either " instead of \" or " instead of ' the result is:
{"error":"bad_request","reason":"invalid UTF-8 JSON: [...]}
Now I think perhaps the logic of your filter function simply has a bug. Here is how I read your filter policy:
All docs that have no attachments pass
All docs that have an image/jpeg attachment pass
Docs with more than 1,024 attachments fail
In any other case, the docs pass
That sounds like perhaps an incorrect policy. Another way to restate this policy is "Docs with more than 1024 attachments fail, everything else passes." However since you wrote so much code, I suspect my summary is not the true policy.
Another quick note, on what looks like a bug. Given:
for (attachment in doc._attachments) { /* ... */ }
The attachment variable will be things like "index.html" or "me.jpeg", i.e. filenames. To get the attachment content-type, you need:
var type;
// This is WRONG
type = attachment.content_type; // type set to undefined
// This is RIGHT
type = doc._attachments[attachment].content_type; // type set to "text/html" etc.
To avoid this bug, you could change your code to make things more clear:
for (attachment_filename in doc._attachments) { /* ... */ }
Next, doc._attachments.length will tell you the number of attachments in the document, not for example the length of the current attachment. It is odd that you test for that inside the loop, because the expression will never change. Are you trying to test for attachment size instead?
What is the output from curl (i.e. from CouchDB)?
From your example, my first guess is that you have a quoting error. Inside single-quotes, you do not need to escape the double-quotes. Try removing all those backslashes. What happens?
If you are on Windows, the single quote is not valid in the shell. In that case, keep the backslashes and just change the single-quote to a double-quote.
Related
I'm sending the following POST request to my gRPC application:
curl \
--request POST \
--header 'Content-Type: application/json' \
--data-raw '{
"mandatory-key1": "value1",
"mandatory-key2": {
"arbitrary-optional-key1": [
"b",
"c"
],
"arbitrary-optional-key2": [
"e"
]
}
}' \
'http://localhost:11000/MyEndpoint'
The value associated with mandatory-key-1 must be a non-empty string.
The value associated with mandatory-key-2 must be a map where all keys are strings and all values are lists of strings.
Now I have to model this request's data structure in the gRPC proto file.
I am thinking of doing something like this:
message MyRequestData {
// pairs represents that map that the user will send in to the MyEndpoint.
map<string, string> pairs = 1;
}
But this specification is not general enough. I need to know how to write this specification correctly.
Question 1: How can I write this specification so it accepts strings in the values and also lists of strings?
Question 2: How can I do validation such that I ensure pairs has keys mandatory-key1 and mandatory-key2 and nothing else?
Question 3: How can I do validation such that I ensure:
pairs has keys mandatory-key1 and mandatory-key2 and nothing else?
pairs[mandatory-key1"] has value which is a non-empty string?
pairs["mandatory-key2"] has value which is a map of <strings, list of non-empty strings>?
Protobufs don't provide (the) validation (that you need).
You'd need to code your validation when you use the protoc-generated sources.
Protobuf doesn't support repeated map values directly, but you can:
message Request {
string mandatory_key1 = 1;
map<string, Value> mandatory_key2 = 2;
}
message Value {
repeated string value = 1;
}
I have the same problem as the people below, but the solutions offered for them does not work for me.
CodeIgniter - disallowed key characters
CodeIgniter Disallowed Key Characters
Disallowed key characters error message in Codeigniter (v2)
I get "Disallowed Key Characters" when I submit a form.
I have CSRF protection enabled, and I am using arrays in my form field names (i.e., search[] as the name as there are multiple selection dropdown options). I have a feeling it is the "[]" in the form name that bothers this form.
I have followed all advice I could see in the posts above.
I disabled CSRF temporarily,
I disabled XSS temporarily,
I edited $config['permitted_uri_chars'] and
I edited Input.php where this message is generated.
Anybody has any additional ideas of what could cause this problem on form submission?
Thanks!
Like my answer here — you just need to update the regex in MY_Input->_clean_input_keys() to allow more characters (eg escaped JSON, or escaped HTML/XML)
Allow just 'English': !preg_match("/^[a-z0-9\:\;\.\,\?\!\#\#\$%\^\*\"\~\'+=\\\ &_\/\.\[\]-\}\{]+$/iu", $str)
Allow Chinese Characters: !preg_match("/^[a-z0-9\x{4e00}-\x{9fa5}\:\;\.\,\?\!\#\#\$%\^\*\"\~\'+=\\\ &_\/\.\[\]-\}\{]+$/iu", $str)
My full working function looks like this:
public function _clean_input_keys($str) {
// NOTE: \x{4e00}-\x{9fa5} = allow chinese characters
// NOTE: 'i' — case insensitive
// NOTE: 'u' — UTF-8 mode
if (!preg_match("/^[a-z0-9\x{4e00}-\x{9fa5}\:\;\.\,\?\!\#\#\$%\^\*\"\~\'+=\\\ &_\/\.\[\]-\}\{]+$/iu", $str)) {
/**
* Check for Development enviroment - Non-descriptive
* error so show me the string that caused the problem
*/
if (is_env_dev()) {
var_dump($str);
}
exit('Disallowed Key Characters.');
}
// Clean UTF-8 if supported
if (UTF8_ENABLED === TRUE) {
return $this->uni->clean_string($str);
}
return $str;
}
my_helper.php
if (!function_exists('is_env_dev')) {
function is_env_dev() {
return (
defined('ENVIRONMENT') && strtolower(ENVIRONMENT) == 'development' ||
defined('ENVIRONMENT') && strtolower(ENVIRONMENT) == 'testing'
);
}
}
Thanks, but I found a comment hidden way below (right at the bottom at the time of this writing) on another post here: CodeIgniter Disallowed Key Characters
The comment suggested that I add $str to the exit() comment to test. This indicated that I had a missing double quote in my form fields. It is a very complex form built up dynamically, with 300 lines of code, so easy to miss.
Hope this answer (and the comment that inspired it) helps someone else.
Validating the source of the output could prevent problems such as this one :-)
Regards
I'm having some trouble with reading out the IPTC data of some images, the reason why I want to do this, is because my client has all the keywords already in the IPTC data and doesn't want to re-enter them on the site.
So I created this simple script to read them out:
$size = getimagesize($image, $info);
if(isset($info['APP13'])) {
$iptc = iptcparse($info['APP13']);
print '<pre>';
var_dump($iptc['2#025']);
print '</pre>';
}
This works perfectly in most cases, but it's having trouble with some images.
Notice: Undefined index: 2#025
While I can clearly see the keywords in photoshop.
Are there any decent small libraries that could read the keywords in every image? Or am I doing something wrong here?
I've seen a lot of weird IPTC problems. Could be that you have 2 APP13 segments. I noticed that, for some reasons, some JPEGs have multiple IPTC blocks. It's possibly the problem with using several photo-editing programs or some manual file manipulation.
Could be that PHP is trying to read the empty APP13 or even embedded "thumbnail metadata".
Could be also problem with segments lenght - APP13 or 8BIM have lenght marker bytes that might have wrong values.
Try HEX editor and check the file "manually".
I have found that IPTC is almost always embedded as xml using the XMP format, and is often not in the APP13 slot. You can sometimes get the IPTC info by using iptcparse($info['APP1']), but the most reliable way to get it without a third party library is to simply search through the image file from the relevant xml string (I got this from another answer, but I haven't been able to find it, otherwise I would link!):
The xml for the keywords always has the form "<dc:subject>...<rdf:Seq><rdf:li>Keyword 1</rdf:li><rdf:li>Keyword 2</rdf:li>...<rdf:li>Keyword N</rdf:li></rdf:Seq>...</dc:subject>"
So you can just get the file as a string using file_get_contents(get_attached_file($attachment_id)), use strpos() to find each opening (<rdf:li>) and closing (</rdf:li>) XML tag, and grab the keyword between them using substr().
The following snippet works for all jpegs I have tested it on. It will fill the array $keys with IPTC tags taken from an image on wordpress with id $attachment_id:
$content = file_get_contents(get_attached_file($attachment_id));
// Look for xmp data: xml tag "dc:subject" is where keywords are stored
$xmp_data_start = strpos($content, '<dc:subject>') + 12;
// Only proceed if able to find dc:subject tag
if ($xmp_data_start != FALSE) {
$xmp_data_end = strpos($content, '</dc:subject>');
$xmp_data_length = $xmp_data_end - $xmp_data_start;
$xmp_data = substr($content, $xmp_data_start, $xmp_data_length);
// Look for tag "rdf:Seq" where individual keywords are listed
$key_data_start = strpos($xmp_data, '<rdf:Seq>') + 9;
// Only proceed if able to find rdf:Seq tag
if ($key_data_start != FALSE) {
$key_data_end = strpos($xmp_data, '</rdf:Seq>');
$key_data_length = $key_data_end - $key_data_start;
$key_data = substr($xmp_data, $key_data_start, $key_data_length);
// $ctr will track position of each <rdf:li> tag, starting with first
$ctr = strpos($key_data, '<rdf:li>');
// Initialize empty array to store keywords
$keys = Array();
// While loop stores each keyword and searches for next xml keyword tag
while($ctr != FALSE && $ctr < $key_data_length) {
// Skip past the tag to get the keyword itself
$key_begin = $ctr + 8;
// Keyword ends where closing tag begins
$key_end = strpos($key_data, '</rdf:li>', $key_begin);
// Make sure keyword has a closing tag
if ($key_end == FALSE) break;
// Make sure keyword is not too long (not sure what WP can handle)
$key_length = $key_end - $key_begin;
$key_length = (100 < $key_length ? 100 : $key_length);
// Add keyword to keyword array
array_push($keys, substr($key_data, $key_begin, $key_length));
// Find next keyword open tag
$ctr = strpos($key_data, '<rdf:li>', $key_end);
}
}
}
I have this implemented in a plugin to put IPTC keywords into WP's "Description" field, which you can find here.
ExifTool is very robust if you can shell out to that (from PHP it looks like?)
I am new to groovy.I am reading values for 2 variables from console with below lines of code.
System.in.withReader {
println "Version: "
version = it.readLine()
println "Doc Type:"
Doc=it.readLine()
call getBillID(version,Doc)
}
getBillid method is as below,
def getBillID(int version,int doc)
{
allNodes.BillID.each {
theregularExpression=/\d+_\d+_\d+_\d_\d+_\d+_\d_${version}_${Doc}_\d+_\d+/
if(it != "" && it =~ theregularExpression) {
println "******" + it
}
}
}
now i want to use those variable values in my getBILLID method but i am getting error as
No signature of method: ReadXML.getBillID() is applicable for argument types: (java.lang.String, java.lang.String) values: [9, ]
where i went wrong.can any one tell me plz..
In addition to #Kalarani's answer, you could also do this:
System.in.withReader {
print "Version: "
int version = it.readLine() as int
print "Doc Type: "
int doc = it.readLine() as int
getBillID( version, doc )
}
As an aside; I would be careful with your capitalisation and variable names, ie: you have a variable called Doc with a capital letter. This is not the standard naming scheme, and you are best using all lowercase for variable names. You can see where it has got confused in the getBillID method. The parameter is called doc (all lowercase), but in the regular expression you reference ${Doc} (uppercase again).
This sort of thing is going to end up causing you a world of pain and bugs that might take you longer to find
Where is the getBillId() method defined? and what is the signature of the method? It would help understanding your problem if you could post that.
How do I create an if statement saying something like this?
Basically, how do you use the URI class to determine if there is a value in any segment?
$segment = value_of_any_segment;
if($segment == 1{
do stuff
}
I know this is pretty elementary, but I don't totally understand the URI class...
Your question is a little unclear to me, but I'll try to help. Are you wondering how to determine if a particular segment exists or if it contains a specific value?
As you are probably aware, you can use the URI class to access the specific URI segments. Using yoursite.com/blog/article/123 as an example, blog is the 1st segment, article is the 2nd segment, and 123 is the 3rd segment. You access each using $this->uri->segment(n)
You then can construct if statements like this:
// if segment 2 exists ("articles" in the above example), do stuff
if ($this->uri->segment(2)) {
// do stuff
}
// if segment 3 ("123" in the above example) is equal to some value, do stuff
if ($this->uri->segment(3) == $myValue) {
// do stuff
}
Hope that helps! Let me know if not and I can elaborate or provide additional information.
Edit:
If you need to determine if a particular string appears in any segment of the URI, you can do something like this:
// get the entire URI (using our example above, this is "/blog/article/123")
$myURI = $this->uri->uri_string()
// the string we want to check the URI for
$myString = "article";
// use strpos() to search the entire URI for $myString
// also, notice we're using the "!==" operator here; see note below
if (strpos($myURI, $myString) !== FALSE) {
// "article" exists in the URI
} else {
// "article" does not exist in the URI
}
A note regarding strpos() (from the PHP documentation):
This function may return Boolean
FALSE, but may also return a
non-Boolean value which evaluates to
FALSE, such as 0 or "". Please read
the section on Booleans for more
information. Use the === operator for
testing the return value of this
function.
I hope my edit helps. Let me know if I can elaborate.