Static Media Failure, episode #2

[sleterrier]

Hi and thanks for maintaining this helm!

Running release 3.0.0, we are seeing the following when browsing to http://HOST.NET/netbox:

despite basePath and ingress.hosts[].paths[] being set in values.yaml:

basePath: 'netbox/'

ingress:
  enabled: true
  annotations: {}
  hosts:
    - host: HOST.NET
      paths:
        - /netbox
        - /ipam

The unit access logs seems to indicate that there might be an extra /netbox prefix leading to the 404s:

10.10.10.0 - - [25/Feb/2021:02:50:04 +0000] "GET /netbox/static/bootstrap-3.4.1-dist/css/bootstrap.min.css HTTP/1.1" 404 21545 "http://HOST.NET/netbox/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 11_2_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.182 Safari/537.36"                                                                         
10.10.10.0 - - [25/Feb/2021:02:50:04 +0000] "GET /netbox/static/select2-4.0.13/dist/css/select2.min.css HTTP/1.1" 404 21542 "http://HOST.NET/netbox/" "Mozilla/5.0 (Macin
tosh; Intel Mac OS X 11_2_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.182 Safari/537.36"
10.10.10.0 - - [25/Feb/2021:02:50:04 +0000] "GET /netbox/static/jquery-ui-1.12.1/jquery-ui.css HTTP/1.1" 404 21533 "http://HOST.NET/netbox/" "Mozilla/5.0 (Macintosh; Int
el Mac OS X 11_2_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.182 Safari/537.36"
10.10.10.0 - - [25/Feb/2021:02:50:04 +0000] "GET /netbox/static/materialdesignicons-5.4.55/css/materialdesignicons.min.css HTTP/1.1" 404 21561 "http://HOST.NET/netbox/"
"Mozilla/5.0 (Macintosh; Intel Mac OS X 11_2_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.182 Safari/537.36"
10.10.10.0 - - [25/Feb/2021:02:50:04 +0000] "GET /netbox/static/select2-bootstrap-0.1.0-beta.10/select2-bootstrap.min.css HTTP/1.1" 404 21560 "http://HOST.NET/netbox/" "
Mozilla/5.0 (Macintosh; Intel Mac OS X 11_2_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.182 Safari/537.36"
10.10.10.0 - - [25/Feb/2021:02:50:04 +0000] "GET /netbox/media-failure/?filename=bootstrap-3.4.1-dist/css/bootstrap.min.css HTTP/1.1" 200 1782 "http://HOST.NET/netbox/"
"Mozilla/5.0 (Macintosh; Intel Mac OS X 11_2_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.182 Safari/537.36"

Given the unit conf.json:

{"listeners":{"*:8080":{"pass":"routes"}},"routes":[{"match":{"uri":"/static/*"},"action":{"share":"/opt/netbox/netbox"}},{"action":{"pass":"applications/netbox"}}],"applications":{"netbox":{"type":"python 3","path":"/opt/netbox/netbox/","module":"netbox.wsgi","home":"/opt/netbox/venv","processes":{"max":4,"spare":1,"idle_timeout":120}}},"access_log":"/dev/stdout"}

#18 described a very similar issue back in May 2020, and commit e87f196 supposedly fixed it then. I suspect this might be related to the upstream's migration from Gunicorn to NGINX Unit, but I have not been able to figure out a fix yet.

[maximumG]

We had a similar issue when upgrading from Netbox 2.9 to 2.10 (still with nginx and gunicorn containers). We had to chmod the static folder (holding static files).

chmod g+w netbox-static-folder-path/

[bootc]

I think with the change to NGINX Unit this requires changing/overriding the nginx-unit.json file. You would need to change the path on line 11 to incorporate your desired static path. You should be able to do this using the extraVolumeMounts and extraVolumes functionality in the chart.

Alternatively you might be able to do something similar with NGINX Ingress rewrites, or potentially your chosen Ingress controller: just strip your chosen base path off the URL if the request is for $basePath/static/(.*).

[besteban1989]

Hi and thanks for maintaining this helm!

Running release 3.0.0, we are seeing the following when browsing to http://HOST.NET/netbox:

despite basePath and ingress.hosts[].paths[] being set in values.yaml:

basePath: 'netbox/'

ingress:
  enabled: true
  annotations: {}
  hosts:
    - host: HOST.NET
      paths:
        - /netbox
        - /ipam

The unit access logs seems to indicate that there might be an extra /netbox prefix leading to the 404s:

10.10.10.0 - - [25/Feb/2021:02:50:04 +0000] "GET /netbox/static/bootstrap-3.4.1-dist/css/bootstrap.min.css HTTP/1.1" 404 21545 "http://HOST.NET/netbox/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 11_2_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.182 Safari/537.36"                                                                         
10.10.10.0 - - [25/Feb/2021:02:50:04 +0000] "GET /netbox/static/select2-4.0.13/dist/css/select2.min.css HTTP/1.1" 404 21542 "http://HOST.NET/netbox/" "Mozilla/5.0 (Macin
tosh; Intel Mac OS X 11_2_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.182 Safari/537.36"
10.10.10.0 - - [25/Feb/2021:02:50:04 +0000] "GET /netbox/static/jquery-ui-1.12.1/jquery-ui.css HTTP/1.1" 404 21533 "http://HOST.NET/netbox/" "Mozilla/5.0 (Macintosh; Int
el Mac OS X 11_2_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.182 Safari/537.36"
10.10.10.0 - - [25/Feb/2021:02:50:04 +0000] "GET /netbox/static/materialdesignicons-5.4.55/css/materialdesignicons.min.css HTTP/1.1" 404 21561 "http://HOST.NET/netbox/"
"Mozilla/5.0 (Macintosh; Intel Mac OS X 11_2_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.182 Safari/537.36"
10.10.10.0 - - [25/Feb/2021:02:50:04 +0000] "GET /netbox/static/select2-bootstrap-0.1.0-beta.10/select2-bootstrap.min.css HTTP/1.1" 404 21560 "http://HOST.NET/netbox/" "
Mozilla/5.0 (Macintosh; Intel Mac OS X 11_2_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.182 Safari/537.36"
10.10.10.0 - - [25/Feb/2021:02:50:04 +0000] "GET /netbox/media-failure/?filename=bootstrap-3.4.1-dist/css/bootstrap.min.css HTTP/1.1" 200 1782 "http://HOST.NET/netbox/"
"Mozilla/5.0 (Macintosh; Intel Mac OS X 11_2_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.182 Safari/537.36"

Given the unit conf.json:

{"listeners":{"*:8080":{"pass":"routes"}},"routes":[{"match":{"uri":"/static/*"},"action":{"share":"/opt/netbox/netbox"}},{"action":{"pass":"applications/netbox"}}],"applications":{"netbox":{"type":"python 3","path":"/opt/netbox/netbox/","module":"netbox.wsgi","home":"/opt/netbox/venv","processes":{"max":4,"spare":1,"idle_timeout":120}}},"access_log":"/dev/stdout"}

#18 described a very similar issue back in May 2020, and commit e87f196 supposedly fixed it then. I suspect this might be related to the upstream's migration from Gunicorn to NGINX Unit, but I have not been able to figure out a fix yet.

Hey were you able to make it work?

[besteban1989]

I was able to make this work! Totally nightmare but here is the solution to make it work using /netbox basePath:

1. --set basePath='netbox/' in the helm parameters
2. Create an ingress for the netbox site

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: netbox-ingress
  namespace: netbox
  annotations:
    kubernetes.io/ingress.class: nginx
spec:
  ingressClassName: "nginx"  

  tls:
  - hosts:
    - 'yourhost.com'
    secretName: some-ssl-secret

  rules:
  - host: 'yourhost.com'
    http:
      paths:          
      - pathType: Prefix
        path: /netbox
        backend:
          service:
            name: netbox
            port: 
              number: 80

3. Create an ingress for the static content with a rewrite target that removes the basePath from the url:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: netbox-static-ingress
  namespace: netbox
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/rewrite-target: /static/$1
spec:
  ingressClassName: "nginx"  

  tls:
  - hosts:
    - 'yourhost.com'
    secretName: some-ssl-secret

  rules:
  - host: 'yourhost.com'
    http:
      paths:          
      - pathType: Prefix
        path: /netbox/static/(.*)
        backend:
          service:
            name: netbox
            port: 
              number: 80

This second ingress will rewrite any /netbox/static to /static.

Hope this helps.

[bootc]

Nice workaround @besteban1989, thanks for documenting it here!

[holmesb]

Be great to somehow template this, based on eg a new ingress.staticContent boolean. But obvs nginx.ingress.kubernetes.io/rewrite-target annotation is nginx-specific and won't work with other ingress controllers. So would need to allow customising using an ingress.staticContent.annotation value:

ingress:
  staticContent:
    enabled: true
    annotation: "nginx.ingress.kubernetes.io/rewrite-target: /static/$1"

[bootc]

I'd rather keep workarounds like this out of the chart to be honest. I'd be happy to have a note in the README saying how to deal with this particular issue with the NGINX Ingress, for example, but I don't think it belongs behind a flag in the chart.

[holmesb]

I was nearly finished creating a PR. Everyone who uses nginx and a path will experience this issue, possibly everyone who uses a path. Pretty common, no? These people will either have hard-coded hostname, path, namespace, etc or will have a messy task doing helm's job and templating them. Let me know if you change your mind.