Some favicon images come corrupt from Kubernetes cluster - image

Question

I'm trying to figure out why this is happening. In production, my site https://gordo.fitness returns the favicons apple-touch-icon.png, favicon-16x16.png, favicon-16x16.png and favicon.ico are not properly loaded/served to the browser (Firefox, Chrome, Firefox for Android, Chrome for Android). The browsers for desktop report that the resource in itself is corrupt and this favicon checker reports those same resources are "ill-formed"
But locally, the Docker containers (docker container run --rm my-container -p 80:80) that serve the images show them properly:
172.17.0.1 - - [10/Jul/2022:00:55:53 +0000] "GET /favicon.ico HTTP/1.1" 200 15086 "http://localhost/favicon-16x16.png" "Mozilla/5.0 (X11; Linux x86_64; rv:103.0) Gecko/20100101 Firefox/103.0" "-"
And, no complains or 404s coming from Ingress NGINX either:
10.124.0.3 - - [10/Jul/2022:00:37:47 +0000] "GET /favicon-16x16.png HTTP/1.1" 200 1189 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.106 Safari/537.36" 244 0.004 [gordo-fitness-gordo-frontend-staging-service-80] [] 10.244.1.4:3000 1189 0.004 200 7d0bb41030dea1e7152a6bfd3ce64e76
137.184.180.250 - - [10/Jul/2022:00:37:47 +0000] "GET /favicon-16x16.png HTTP/1.1" 200 1189 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.106 Safari/537.36" 244 0.003 [gordo-fitness-gordo-frontend-staging-service-80] [] 10.244.1.4:3000 1189 0.000 200 0024ae0458b1c5b7f80c12b224c2ef01
So I'm out of clues and don't know what's happening, maybe I did something wrong with my Ingress configuration?:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
cert-manager.io/issuer: letsencrypt-nginx
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"networking.k8s.io/v1","kind":"Ingress","metadata":{"annotations":{"cert-manager.io/issuer":"letsencrypt-nginx"},"name":"gordo-fitness-ingress","namespace":"gordo-fitness"},"spec":{"ingressClassName":"nginx","rules":[{"host":"gordo.fitness","http":{"paths":[{"backend":{"service":{"name":"gordo-frontend-staging-service","port":{"number":80}}},"path":"/","pathType":"Prefix"}]}}],"tls":[{"hosts":["gordo.fitness"],"secretName":"letsencrypt-nginx"}]}}
creationTimestamp: "2022-07-06T06:31:02Z"
generation: 5
name: gordo-fitness-ingress
namespace: gordo-fitness
resourceVersion: "1381386"
uid: b0377c4d-737f-4722-8ea9-d052df7970eb
spec:
ingressClassName: nginx
rules:
- host: gordo.fitness
http:
paths:
- backend:
service:
name: gordo-frontend-staging-static-service
port:
number: 8081
path: /[[:alnum:]]+\.(ico|png|svg|txt|webmanifest|xml)
pathType: Exact
- backend:
service:
name: gordo-frontend-staging-service
port:
number: 80
path: /
pathType: Prefix
tls:
- hosts:
- gordo.fitness
secretName: letsencrypt-nginx
status:
loadBalancer:
ingress:
- ip: 146.190.0.179
EDIT:
checking with dhex I see there are 2 bits changed in the file after being served vs the original from disk:

Answer 1

I believe the issue is not with the serving of the content, rather the content itself is corrupt. I mean there is nothing wrong with your ingress configuration, or the site traffic setup in general.
If you try and open the icon you downloaded with GET ==> 200 OK, you might see that the icon is indeed corrupted. The HTTP protocol does not verify the "sanity" of the data, and so, as far as HTTP is concerned, you can have a mixture of random bytes encoded into a PNG file, and it will transport them successfully as well. It will only verify the content encoding correctness, not the sanity of the payload.
Perhaps you can try and regenerate the icons.

Answer 2

Files were corrupted on the repository, probably something is wrong with my Git configuration (newline replacement, maybe?) since the generated files come from a third-party service, and thus something may be changed in the contents by Git.
I work around this activating Git LFS in my repository.