Istio EnvoyFilter Lua HttpCall doesn't work with HTTPS? - https

I need to decrypt the body of a request in an external API.
But, when I try to do it with an EnvoyFilter using lua it doesn't work.
If I try the same code that I'm posting here, but without HTTPS, works. But with HTTPS returns 503.
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: eva-decrypt-filter
namespace: istio-system
spec:
configPatches:
- applyTo: HTTP_FILTER
match:
context: ANY
listener:
filterChain:
filter:
name: "envoy.filters.network.http_connection_manager"
patch:
operation: INSERT_BEFORE
value:
name: envoy.lua
typed_config:
"#type": "type.googleapis.com/envoy.extensions.filters.http.lua.v3.Lua"
inlineCode: |
function envoy_on_request(request_handle)
local buffered = request_handle:body()
local bodyString = tostring(buffered:getBytes(0, buffered:length()))
print("bodyString ->")
print(bodyString)
if string.match(bodyString, "valcirtest") then
print("iniciando http_Call")
local responseHeaders, responseBody = request_handle:httpCall(
"thirdparty",
{
[":method"] = "POST",
[":path"] = "/decrypt",
[":authority"] = "keycloack-dev-admin.eva.bot",
[":scheme"] = "https",
["content-type"] = "application/json",
["content-length"] = bodyString:len(),
},
bodyString,
3000)
print("acabou a requisicao")
print("responseHeaders -> ")
print(responseHeaders)
print(responseHeaders[":status"])
print("responseBody -> ")
print(responseBody)
local content_length = request_handle:body():setBytes(responseBody)
request_handle:headers():replace("content-length", content_length)
else
print("nao entrou")
end
end
- applyTo: CLUSTER
match:
context: SIDECAR_OUTBOUND
patch:
operation: ADD
value: # cluster specification
name: thirdparty
connect_timeout: 1.0s
type: STRICT_DNS
dns_lookup_family: V4_ONLY
lb_policy: ROUND_ROBIN
load_assignment:
cluster_name: thirdparty
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
protocol: TCP
address: keycloack-dev-admin.eva.bot
port_value: 443
The response error is:
503
responseBody ->
upstream connect error or disconnect/reset before headers. reset reason: connection termination
I'm using Istio v.1.11.4.

It should be configured on your "thirdparty" cluster adding the following on your cluster config:
transport_socket:
name: envoy.transport_sockets.tls

To add to #koffi-kodjo's answer, you also need to specify the typed_config property. The transport_socket node should be placed at the same level of the name: thirdparty node.
transport_socket:
name: envoy.transport_sockets.tls
typed_config:
"#type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
ref:
https://github.com/envoyproxy/envoy/issues/11582#issuecomment-646427632
https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/transport_sockets/tls/v3/tls.proto.html#extensions-transport-sockets-tls-v3-upstreamtlscontext

Related

POST https://apm.<acme>.com/intake/v2/rum/events net::ERR_BLOCKED_BY_CLIENT

I have an elastic stack on kubernetes (k8s) using ECK.
Kibana version:
7.13.2
Elasticsearch version:
7.13.2
APM Server version:
7.13.2
APM Agent language and version:
https://www.npmjs.com/package/#elastic/apm-rum - 5.9.1
Browser version:
Chrome latest
Description of the problem
Frontend apm-run agent fails to send messages to apm server. if i disable cors on the browser it works - google-chrome --disable-web-security --user-data-dir=temp then navigate to my frontend http://localhost:4201/
[Elastic APM] Failed sending events! Error: https://apm.<redacted>.com/intake/v2/rum/events HTTP status: 0
at ApmServer._constructError (apm-server.js:120)
at eval (apm-server.js:48)
POST https://apm.<acme>.com/intake/v2/rum/events net::ERR_BLOCKED_BY_CLIENT
Code:
apm.yml
apiVersion: apm.k8s.elastic.co/v1
kind: ApmServer
metadata:
name: apm-server-prod
namespace: elastic-system
spec:
version: 7.13.2
count: 1
elasticsearchRef:
name: "elasticsearch-prod"
kibanaRef:
name: "kibana-prod"
http:
service:
spec:
type: NodePort
config:
apm-server:
rum.enabled: true
ilm.enabled: true
elastic.ingress.yml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: elastic-ingress
namespace: elastic-system
annotations:
kubernetes.io/ingress.class: alb
alb.ingress.kubernetes.io/group.name: "<redacted>"
alb.ingress.kubernetes.io/scheme: internet-facing
alb.ingress.kubernetes.io/listen-ports: '[{"HTTP":80,"HTTPS": 443}]'
alb.ingress.kubernetes.io/backend-protocol: 'HTTPS'
alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-east-2:<rd>:certificate/0250a551-8971-468d-a483-cad28f890463
alb.ingress.kubernetes.io/tags: Environment=prod,Team=dev
alb.ingress.kubernetes.io/healthcheck-path: /health
alb.ingress.kubernetes.io/healthcheck-interval-seconds: '300'
alb.ingress.kubernetes.io/load-balancer-attributes: access_logs.s3.enabled=true,access_logs.s3.bucket=<redacted>-aws-ingress-logs,access_logs.s3.prefix=dev-ingress
spec:
rules:
- host: elasticsearch.<redacted>.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: elasticsearch-prod-es-http
port:
number: 9200
- host: kibana.<redacted>.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: kibana-prod-kb-http
port:
number: 5601
- host: apm.<redacted>.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: apm-server-prod-apm-http
port:
number: 8200
frontend.js
import { init as initApm } from "#elastic/apm-rum";
import config from "<redacted>-web/config/environment";
export const apm = initApm({
serviceName: "frontend",
serverUrl: "https://apm.<redacted>.com",
environment: config.environment,
logLevel: "debug",
});
Errors in browser console:
apm server:
apm-server pod does not apear to display any errors in this case, i assume the client never reaches the server .
I was running into the same problem. Check your ad blocker. I found that UBlock was blocking requests to */rum/events.
I'm guessing that they consider this as a type of user "tracker" and that's why they're blocked, really no way around it though unless you change the endpoint path I guess.

Can reverse proxy for both grpc and grpc-web with envoy?

I have grpc server, web application and mobile application. With web application, I used envoy proxy for reverse from grpc web to grpc server by my domain. But this only grpc web can connect to my server over envoy, my application run with grpc cannot connect it. I want to connect to my gprc server over my domain with both grpc-web and grpc. Any one can help me explain and solve this issue. Thanks every one
This is my envoy setup
admin:
access_log_path: /tmp/admin_access.log
address:
socket_address: { address: 0.0.0.0, port_value: 9901 }
static_resources:
listeners:
- name: listener_0
address:
socket_address: { address: 0.0.0.0, port_value: 9090 }
filter_chains:
- filters:
- name: envoy.filters.network.http_connection_manager
typed_config:
"#type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
codec_type: auto
stat_prefix: ingress_http
route_config:
name: local_route
virtual_hosts:
- name: local_service
domains: ["*"]
routes:
- match: { prefix: "/" }
route:
cluster: echo_service
timeout: 0s
max_stream_duration:
grpc_timeout_header_max: 0s
cors:
allow_origin_string_match:
- prefix: "*"
allow_methods: GET, PUT, DELETE, POST, OPTIONS
allow_headers: keep-alive,user-agent,cache-control,content-type,content-transfer-encoding,custom-header-1,x-accept-content-transfer-encoding,x-accept-response-streaming,x-user-agent,x-grpc-web,grpc-timeout
max_age: "1728000"
expose_headers: custom-header-1,grpc-status,grpc-message
http_filters:
- name: envoy.filters.http.grpc_web
- name: envoy.filters.http.cors
- name: envoy.filters.http.router
clusters:
- name: echo_service
connect_timeout: 0.25s
type: logical_dns
http2_protocol_options: {}
lb_policy: round_robin
load_assignment:
cluster_name: cluster_0
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: server
port_value: 8080
Not sure what is the exact question? Currently you need to use Envoy to translate grpc-web to grpc.

Envoy lua filter - http call cluster invalid. Must be configured

I am trying to create an envoy filter for all of my microservices deployed on a GKE cluster which is running istio.
In the filter I want to read 2 header values and send a request to an external service on web to get list of all groups that the user has authority of.
When I send the request I get "http call cluster invalid. Must be configured" error.
As per my understanding we need to register the host as cluster so that envoy understand where to send the request and I have added a value in "clusters" section of my filter.yaml file.
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: authorization-filter
namespace: default
spec:
filters:
- listenerMatch:
listenerType: SIDECAR_INBOUND
listenerProtocol: HTTP
filterName: envoy.lua
filterType: HTTP
filterConfig:
inlineCode: |
function envoy_on_request(request_handle)
request_handle:logWarn("Inside filter")
local token = request_handle:headers():get("Authorization")
local partition = request_handle:headers():get("partition-id")
if token == nil or partition == nil then
request_handle:logErr("Token or Partition is empty")
request_handle:respond({[":status"] = "400",["Content-Type"] = "application/json"}, "{\"ErrorMessage\":\"Bad Request: Authorization or Partition-Id missing in headers\"}")
end
request_handle:logWarn("Sending request")
local headers, body = request_handle:httpCall(
"lua_cluster",
{
[":method"] = "GET",
[":authority"] = "example.com",
[":path"] = "/api/authorization/groups",
["Authorization"] = token,
["partition-id"] = partition
},
nil,
5000)
request_handle:headers():add("authorization-response-headers", headers)
end
clusters:
- name: lua_cluster
connect_timeout: 0.5s
type: strict_dns
lb_policy: round_robin
hosts:
- socket_address:
address: "example.com"
port_value: 8888
What am i missing here?

EnvoyProxy upstream connect error or disconnect/reset before headers

I'm currently trying to set up envoy to take in a https request, and pass the request on as an http request to a separately running(via Docker) local endpoint(Envoy is also being run locally). Unfortunately, when I make the request via Postman I get the following error "upstream connect error or disconnect/reset before headers. reset reason: connection failure". Any ideas on how this issue could be fixed?
The envoy.yaml file I'm using is the following:
# envoy
static_resources:
listeners:
- name: "listener"
address: { socket_address: { address: 0.0.0.0, port_value: 8443 } }
filter_chains:
- filters:
- name: envoy.filters.network.http_connection_manager
typed_config:
"#type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
codec_type: auto
stat_prefix: ingress_https
http_filters: { name: envoy.filters.http.router }
route_config:
name: route
virtual_hosts:
- name: host
domains: ["*"]
routes:
- match: { prefix: "/" }
route:
cluster: cluster
timeout: 10s
transport_socket:
name: envoy.transport_sockets.tls
typed_config:
"#type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext
common_tls_context:
tls_certificates:
- certificate_chain: { filename: "/etc/envoy/cert.crt" }
private_key: { filename: "/etc/envoy/cert.key" }
alpn_protocols: "h2,http/1.1"
clusters:
- name: cluster
connect_timeout: 20s
type: strict_dns
lb_policy: least_request
health_checks:
timeout: 5s
interval: 2s
unhealthy_threshold: 3
healthy_threshold: 1
http_health_check: { path: "/healthcheck" }
circuit_breakers:
thresholds:
max_retries: 300
load_assignment:
cluster_name: cluster
endpoints:
- lb_endpoints:
- endpoint: { address: { socket_address: { address: "0.0.0.0", port_value: 8080 } } }
If anything else is needed please let me know. Thanks in advance for the help/tips!
Increase your cluster connect_timeout and try again. Maybe that will fix the problem.

Can gRPC be integrated into flutter-web?

I tried to integrate gPRC into flutter-web, but it always failed. I don't know if there is a problem with my code or GRPC can't be integrated into flutter-web.
dependencies:
flutter:
sdk: flutter
grpc: ^2.1.3
protobuf: ^1.0.1
Here's my server-side code:
I have two questions.
The first one is whether the GRPC can be integrated into fluter-web.?
The second one is what libraries I need and whether there are any examples?
thank you.
The short answer, yes you can.
For now, grpc-web need a web proxy in front of the gRPC server to translate the requests and responses to something the browser can use. See https://grpc.io/blog/state-of-grpc-web/ for details.
You can use Envoy as the web proxy.
Here the steps to use envoy:
set your web client channel in flutter
GrpcWebClientChannel.xhr(Uri.parse('http://localhost:8080'));
set the server to listen on the following sample:
path := "127.0.0.1:3001"
Install envoy from https://www.envoyproxy.io/
create configuration for envoy like the following example. Save it as envoy.yaml:
admin:
access_log_path: /tmp/admin_access.log
address:
socket_address: { address: 0.0.0.0, port_value: 9901 }
static_resources:
listeners:
- name: listener_0
address:
socket_address: { address: 0.0.0.0, port_value: 8080 }
filter_chains:
- filters:
- name: envoy.filters.network.http_connection_manager
typed_config:
"#type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
codec_type: auto
stat_prefix: ingress_http
route_config:
name: local_route
virtual_hosts:
- name: local_service
domains: ["*"]
routes:
- match: { prefix: "/" }
route:
cluster: greeter_service
max_stream_duration:
grpc_timeout_header_max: 0s
cors:
allow_origin_string_match:
- prefix: "*"
allow_methods: GET, PUT, DELETE, POST, OPTIONS
allow_headers: keep-alive,user-agent,cache-control,content-type,content-transfer-encoding,x-accept-content-transfer-encoding,x-accept-response-streaming,x-user-agent,x-grpc-web,grpc-timeout
max_age: "1728000"
expose_headers: id,token,grpc-status,grpc-message
http_filters:
- name: envoy.filters.http.grpc_web
- name: envoy.filters.http.cors
- name: envoy.filters.http.router
clusters:
- name: greeter_service
connect_timeout: 0.25s
type: logical_dns
http2_protocol_options: {}
lb_policy: round_robin
# win/mac hosts: Use address: host.docker.internal instead of address: localhost in the line below
load_assignment:
cluster_name: cluster_0
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: 0.0.0.0
port_value: 3001
Run envoy with the configuration (sample in Linux box):
$ envoy -c envoy.yaml
Now, try to run the flutter web client and server.
See https://github.com/sigurdm/grpc_web_flutter_example or https://github.com/grpc/grpc-web/tree/master/net/grpc/gateway/examples/helloworld

Resources