redirects change https requests to http locations

[dennisobrien]

I'm running Caravel in AWS with this configuration:

• ELB terminates SSL (and accepts only https requests)
• Docker container runs gunicorn + caravel

Many requests hang in the browser because the https request is redirected to a http location.

$ curl -Ik https://caravel.example.com/
HTTP/1.1 302 FOUND
Content-Length: 239
Content-Type: text/html; charset=utf-8
Date: Thu, 18 Aug 2016 16:30:20 GMT
Location: http://caravel.example.com/caravel/welcome
Server: gunicorn/19.6.0
Connection: keep-alive

I'm not sure if this is an issue with Caravel or upstream in Flask or Flask-AppBuilder.

I tried setting PREFERRED_URL_SCHEME = 'https' in caravel_config.py hoping that would propagate to flask, but either it did not propagate, or it had no effect. (That config instructs flask what scheme to use when it cannot be determined.)

I think the right way to deal with this is to determine the protocol from the 'X-Forwarded-Proto' header. But I'm not sure if this is a bug in Caravel or Flask.

thanks,
Dennis

[mistercrunch]

We run behind a reverse proxy (nginx) that is https while the site is served by gunicorn on http and redirects are fine on our side.

Seems like a configuration issue in your reverse proxy.

[dennisobrien]

Thanks @mistercrunch
I'll try adding nginx to the mix and report back.

Just for kicks, I tried testing this to get an idea of where this might be breaking down. Here's a very simple app that just does a re-direct from '/a' to '/b'.

# app.py
from flask import Flask, redirect, url_for
app = Flask(__name__)

@app.route('/a')
def a():
    return redirect(url_for('b'))

@app.route('/b')
def b():
    return 'you made it!'

And testing this in straight-up Flask, the 'X-Forwarded-Proto' header is not recognized. (Notice the Location is https even though the header requested https.)

$ FLASK_APP=app.py flask run
...
$ curl -Ik -H "X-Forwarded-Proto: https" localhost:5000/a
HTTP/1.0 302 FOUND
Content-Type: text/html; charset=utf-8
Content-Length: 211
Location: http://localhost:5000/b
Server: Werkzeug/0.11.10 Python/3.5.2
Date: Sun, 21 Aug 2016 22:52:23 GMT

And running with gunicorn it does handle the 'X-Forwarded-Proto' header. (Notice the Location is https now.)

$ gunicorn -b 127.0.0.1:5000 app:app
...
$ curl -Ik -H "X-Forwarded-Proto: https" localhost:5000/a
HTTP/1.1 302 FOUND
Server: gunicorn/19.1.0
Date: Sun, 21 Aug 2016 22:53:13 GMT
Connection: close
Content-Type: text/html; charset=utf-8
Content-Length: 211
Location: https://localhost:5000/b

So gunicorn looks like it should be doing the right thing.

I think this can be closed since I believe the problem is not in Caravel. And I'll update this issue with whatever I learn.

thanks,
Dennis

[xrmx]

Thanks for the heads up, closing then.

[dennisobrien]

For the record, I found the cause of the problem and the fix. When gunicorn is run on a different machine from the load balancer (nginx or ELB), it needs to be told explicitly to trust the X-Forwarded-* headers sent. gunicorn takes an option --forwarded-allow-ips which can either be a comma separated list of ip addresses, or "*" to trust all.

I'm starting caravel with this command (with gunicorn running behind an ELB):

gunicorn \
  --error-logfile - \
  --access-logfile - \
  -w 8 \
  -k gevent \
  -b 0.0.0.0:8080 \
  --timeout 120 \
  --limit-request-line 0 \
  --limit-request-field_size 0 \
  --forwarded-allow-ips="*" \
  caravel:app

More details are in the gunicorn docs:
http://docs.gunicorn.org/en/stable/deploy.html

cheers,
Dennis

[rlei]

@dennisobrien In http://airbnb.io/superset/installation.html#configuration-behind-a-load-balancer

If the load balancer is inserting X-Forwarded-For/X-Forwarded-Proto headers, you should set ENABLE_PROXY_FIX = True in the superset config file to extract and use the headers.

Have you tried this setting?

[ecliptik]

I recently ran into this as well when trying to use a AWS load-balancer to a container and fixed it the following way.

1. superset v0.17.1 with docker image: python:3.6
2. Set ENABLE_PROXY_FIX = True in superset_config.py
3. Used a Application Load Balancer (ALB) instead of a classic ELB going to the container port over http and https

@ecliptik I'm using the below configuration, running superset in python virtualenv

1. Superset v0.17.0 : python:2.7.12,
2. Set ENABLE_PROXY_FIX = True in superset_config.py
3. Using the ALB.

I'm able to access the application using the ALB endpoint, while logging in, it says "Invalid login", eventhough giving the valid credentials

could help me on resolving this.

[Jie-Yang]

I am having similar problem: ALB works find with HTTP, but when I binding HTTPS to the ALB, I get error: 502 Bad Gateway. I have tried to figure out more details about the error, but not much information logged in S3. Is there anywhere I can see more information about this error? There is no any error from the superset console. I suspect the problem happens in ALB, some headers not matched maybe?

and I have tried ENABLE_PROXY_FIX = True in superset_config.py and also environment variable FORWARDED_ALLOW_IPS=*, but not luck.

my Gunicon version is 19.7.1, superset is 0.18.5.

Any suggestion or tip is appreciated! Thanks

Jay

[Jie-Yang]

For people may come across the same problem, I have found the problem. By mistake, I set protocol of target group to HTTPS. When I change it to HTTP, everything works!