I am trying to pass nginx variables to the passenger_app_env or passenger_user directives. It seems as though Passenger is not interpolating these as I'm seeing a literal $remote_user getting passed through.
location / {
set $something "something";
...
passenger_app_env $something;
...
}
Is this possible?
Related
All that I found about regsuball - it's description from docs: https://book.varnish-software.com/3.0/VCL_Basics.html#vcl-functions . But it still not clear for me what happens inside regsuball function. How does passed params (str, regex, sub) used? Is there a sandbox to test regsuball function online? Thanks!
Syntax
The regsuball() function will perform a regular expression match on a string and replace all occurences with another pattern.
regsuball(string, regex, sub)
The string argument is your input.
The regex argument is the regular expression you're using to match what you're looking for in the input string.
The sub argument is what the input string will be substituted with.
Example
The following example will look for cookies named original-cookie-123, where the numeric suffix could be any number. It replaces that with modified-cookie-123:
regsuball(req.http.Cookie, "original-cookie-([0-9+])", "modified-cookie-\1");
Imagine using passing the following cookie to Varnish:
Cookie: a=1; b=2; original-cookie-1=bla; c=3; original-cookie-2=test
The result after the find/replace would be:
Cookie: a=1; b=2; modified-cookie-1=bla; c=3; modified-cookie-2=test
Whereas regsub() matches and replaces the first occurrence of the pattern, regsuball() replaces all occurrences.
Sandbox
If you want to test regsuball() in a sandbox, the easiest way is by running varnishtest on a testcase. Here's an example:
varnishtest "Regsuball sandbox"
varnish v1 -vcl+backend {
vcl 4.1;
backend default none;
sub vcl_recv {
if (req.http.Cookie) {
set req.http.Cookie = regsuball(req.http.Cookie, "original-cookie-([0-9+])", "modified-cookie-\1");
return (synth(200, req.http.Cookie));
}
return (synth(400, "No cookie found"));
}
sub vcl_synth {
set resp.http.Content-Type = "text/plain; charset=utf-8";
set resp.body = req.http.Cookie;
return (deliver);
}
} -start
client c1 {
txreq -hdr "Cookie: a=1; b=2; original-cookie-1=bla; c=3; original-cookie-2=test"
rxresp
expect resp.body == "a=1; b=2; modified-cookie-1=bla; c=3; modified-cookie-2=test"
} -run
This test case uses the exact example mentioned earlier. After the find and replace, the VCL code in this test case will synthetically return the resulting value as output. There's no need for a backend server, all output is generated by the VCL code.
Running the test case, assuming it is stored in test.vtc can be done using the following command:
varnishtest test.vtc
You can also run this test case isolated within a Docker container. Just run the following command to bootstrap a Varnish Docker container and run the test:
docker run --rm --name varnishtest -v $(pwd)/test.vtc:/etc/varnish/test.vtc varnish:stable varnishtest /etc/varnish/test.vtc
I got this line in nginx conf, other line omitted
location /{
uwsgi_cache_key $request_uri;
}
and pure section
location ~ /_purge(/.*$){
uwsgi_cache_purge mycache $1;
}
It works well for url like http://host/test and http://host/_purge/test
but http://host/测试 with cache can't be purged by http://host/_purge/测试 or http://host/_purge/%E6%B5%8B%E8%AF%95
How to purge http://host/测试 cache ?
The var $request_uri in nginx is url encoded actually.
so the key cached is /%E6%B5%8B%E8%AF%95
but whether request to http://host/_purge/测试 or http://host/_purge/%E6%B5%8B%E8%AF%95 is the same. they are urlunescaped after the match /_purge(/.*$). The $1 actually is /测试.
So we can't purge the cache.
Solving:
extra module needed: https://github.com/openresty/set-misc-nginx-module#installation
location /{
set_unescape_uri $key $request_uri;
uwsgi_cache_key $key;
}
I have the following two settings in my elasticsearch.yml file. They are the only ones that pull from environment variables.
cloud.aws.access_key: ${AWS_ACCESS_KEY_ID}
cloud.aws.secret_key: ${AWS_SECRET_KEY}
When I restart elasticsearch to load these from the environment, I get an error that it can't resolve them. I've tested it and it will not resolve either, so this error applies to both (it just fails on the bottom one first)
- IllegalArgumentException[Could not resolve placeholder 'AWS_SECRET_KEY']
java.lang.IllegalArgumentException: Could not resolve placeholder 'AWS_SECRET_KEY'
at org.elasticsearch.common.property.PropertyPlaceholder.parseStringValue(PropertyPlaceholder.java:124)
at org.elasticsearch.common.property.PropertyPlaceholder.replacePlaceholders(PropertyPlaceholder.java:81)
at org.elasticsearch.common.settings.ImmutableSettings$Builder.replacePropertyPlaceholders(ImmutableSettings.java:1060)
at org.elasticsearch.node.internal.InternalSettingsPreparer.prepareSettings(InternalSettingsPreparer.java:101)
at org.elasticsearch.bootstrap.Bootstrap.initialSettings(Bootstrap.java:106)
at org.elasticsearch.bootstrap.Bootstrap.main(Bootstrap.java:177)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:32)
I did some investigating through elasticsearch's code repository on github and discovered this bit of code that pulls from the environment variables.
ImmutableSettings.java#resolvePlaceholder from elasticsearch#github
Namely. the lines inside that function that should be pulling from the environment variables are these one:
Code from resolvePlaceholder that pulls out environment variables
However, after resolvePlaceholder is run from inside function PropertyPlaceholder#parseStringValue, the System.getenv call must be returning null as that is the only way for that error to be thrown.
I wrote a simple test program that is essentially a copy of ImmutableSettings.java#resolvePlaceholder to test that System.getenv was pulling out the environment variables correctly on my system. This in fact returns the values I expect.
public class Cool {
public static void main(String[] args) {
System.out.println(resolvePlaceholder(args[0]));
}
public static String resolvePlaceholder(String placeholderName) {
if (placeholderName.startsWith("env.")) {
// explicit env var prefix
System.out.println("1: placeholderName.startsWith(\"env.\")");
return System.getenv(placeholderName.substring("env.".length()));
}
String value = System.getProperty(placeholderName);
if (value != null) {
System.out.println("2: System.getProperty");
return value;
}
value = System.getenv(placeholderName);
if (value != null) {
System.out.println("3: System.getenv");
return value;
}
return "Map should've had it";
}
}
When run, this is the output, showing we are getting the set environment variables (keys hidden for obvious reasons):
[ec2-user#ip-172-31-34-195 ~]$ java Cool AWS_SECRET_KEY
3: System.getenv
XXXXXXXXXXXXXXXXXX
[ec2-user#ip-172-31-34-195 ~]$ java Cool AWS_ACCESS_KEY_ID
3: System.getenv
XXXXXXXXXXXXXXXXXX
What is it about elasticsearch that isn't able to parse my environment variables from elasticsearch.yml? I've done quite a bit of digging at this point but I'm sure there is a simple solution around the corner. Any help would be very much appreciated.
I figured out the issue.
As I am running elasticsearch as a linux service, rather than a shell application, it has access to no environment variables except for a very select few.
I added the following line to the end of /etc/sysconfig/elasticsearch to load the environment variables I wanted available to the program:
. /path/to/environment/variables
I'm trying to implement nginx rewrite rules for the following situation
Request:
http://192.168.64.76/Shep.ElicenseWeb/Public/OutputDocuments.ashx?uinz=12009718&iinbin=860610350635
Should be redirected to:
http://localhost:82/Public/OutputDocuments.ashx?uinz=12009718&iinbin=860610350635
I tried this with no luck:
location /Shep.ElicenseWeb/ {
rewrite ^/Shep.ElicenseWeb/ /$1 last;
proxy_pass http://localhost:82;
}
What is the correct way to perform such a rewrite for nginx ?
Your rewrite statement is wrong.
The $1 on the right refers to a group (indicated by paratheses) in the matching section.
Try:
rewrite ^/Shep.ElicenseWeb/(.*) /$1 break;
You're missing a trailing slash:
location /Shep.ElicenseWeb/ {
proxy_pass http://localhost:82/;
}
This will work without a rewrite.
I want people type in http://www.myweb.com/like/1234456 will redirect to
http://www.myweb.com/item.php?itemid=1234456
I wrote something like this in the config but it doesn't work.
location = ^~/like/ {
rewrite ^1234456 ../likeitem.php?item=1234456break;
return 403;
}
this is just a test. I haven't used the $ matching yet.
I also restart my ngnix server but still.. it doesn't do the redirect.
The code above will not work because of a missing $ and poor use of the return command.
The code below works with Nginx, including version 0.8.54.
Format below is :
DesiredURL
Actual URL
Nginx_Rule
They must be inside location / {}
http://example.com/notes/343
http://example.com/notes.php?id=343
rewrite ^/notes/(.*)$ /notes.php?id=$1 last;
http://example.com/users/BlackBenzKid
http://example.com/user.php?username=BlackBenzKid
rewrite ^/users/(.*)$ /user.php?username=$1 last;
http://example.com/top
http://example.com/top.php
rewrite ^/top?$ /top.php last;
Complex and further
http://example.com/users/BlackBenzKid/gallery
http://example.com/user.php?username=BlackBenzKid&page=gallery
rewrite ^/users/(.*)/gallery$ /user.php?username=$1&page=gallery last;
Try this,
server {
server_name www.myweb.com;
rewrite ^/like/(.*) http://www.myweb.com/item.php?itemid=$1 permanent;
}