Jinja filters not working with Global Async Queries (GAQ)

[himanshugarg574]

I have a working installation of superset 0.38. In this I am able to use jinja's filter_values function successfully. While upgrading to version 1.1.0 on my local machine, I used docker-compose to start superset and enabled GAQ.

Expected results

GAQ should work with templates

Actual results

Not able to apply templates along with GAQ. When I remove filter_values function from my query then the chart works fine. Other variables like to_dttm are working fine.

Screenshots

This is error log from celery worker

Working outside of request context.

This typically means that you attempted to use functionality that needed
an active HTTP request.  Consult the documentation on testing for
information about how to avoid this problem.
Traceback (most recent call last):
  File "/app/superset/viz.py", line 534, in get_df_payload
    df = self.get_df(query_obj)
  File "/app/superset/viz.py", line 275, in get_df
    self.results = self.datasource.query(query_obj)
  File "/app/superset/connectors/sqla/models.py", line 1313, in query
    query_str_ext = self.get_query_str_extended(query_obj)
  File "/app/superset/connectors/sqla/models.py", line 772, in get_query_str_extended
    sqlaq = self.get_sqla_query(**query_obj)
  File "/app/superset/connectors/sqla/models.py", line 1040, in get_sqla_query
    tbl = self.get_from_clause(template_processor)
  File "/app/superset/connectors/sqla/models.py", line 800, in get_from_clause
    from_sql = template_processor.process_template(from_sql)
  File "/app/superset/jinja_context.py", line 318, in process_template
    return template.render(context)
  File "/usr/local/lib/python3.7/site-packages/jinja2/environment.py", line 1090, in render
    self.environment.handle_exception()
  File "/usr/local/lib/python3.7/site-packages/jinja2/environment.py", line 832, in handle_exception
    reraise(*rewrite_traceback_stack(source=source))
  File "/usr/local/lib/python3.7/site-packages/jinja2/_compat.py", line 28, in reraise
    raise value.with_traceback(tb)
  File "<template>", line 9, in top-level template code
  File "/usr/local/lib/python3.7/site-packages/jinja2/sandbox.py", line 462, in call
    return __context.call(__obj, *args, **kwargs)
  File "/app/superset/jinja_context.py", line 218, in safe_proxy
    return_value = func(*args, **kwargs)
  File "/app/superset/jinja_context.py", line 86, in filter_values
    form_data, _ = get_form_data()
  File "/app/superset/views/utils.py", line 139, in get_form_data
    if request.is_json and "queries" in request.json
  File "/usr/local/lib/python3.7/site-packages/werkzeug/local.py", line 347, in __getattr__
    return getattr(self._get_current_object(), name)
  File "/usr/local/lib/python3.7/site-packages/werkzeug/local.py", line 306, in _get_current_object
    return self.__local()
  File "/usr/local/lib/python3.7/site-packages/flask/globals.py", line 38, in _lookup_req_object
    raise RuntimeError(_request_ctx_err_msg)
RuntimeError: Working outside of request context.

This typically means that you attempted to use functionality that needed
an active HTTP request.  Consult the documentation on testing for
information about how to avoid this problem.
[2021-05-24 13:13:48,194: ERROR/MainProcess] Task load_explore_json_into_cache[a5324433-6abd-4d5b-a638-f5d8020ed324] raised unexpected: SupersetVizException("[{'message': 'Working outside of request context.\\n\\nThis typically means that you attempted to use functionality that needed\\nan active HTTP request.  Consult the documentation on testing for\\ninformation about how to avoid this problem.', 'error_type': <SupersetErrorType.VIZ_GET_DF_ERROR: 'VIZ_GET_DF_ERROR'>, 'level': <ErrorLevel.ERROR: 'error'>, 'extra': None}]")
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/celery/app/trace.py", line 412, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/app/superset/app.py", line 116, in __call__
    return task_base.__call__(self, *args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/celery/app/trace.py", line 704, in __protected_call__
    return self.run(*args, **kwargs)
  File "/app/superset/tasks/async_queries.py", line 108, in load_explore_json_into_cache
    raise exc
  File "/app/superset/tasks/async_queries.py", line 86, in load_explore_json_into_cache
    raise SupersetVizException(errors=payload["errors"])
superset.exceptions.SupersetVizException: [{'message': 'Working outside of request context.\n\nThis typically means that you attempted to use functionality that needed\nan active HTTP request.  Consult the documentation on testing for\ninformation about how to avoid this problem.', 'error_type': <SupersetErrorType.VIZ_GET_DF_ERROR: 'VIZ_GET_DF_ERROR'>, 'level': <ErrorLevel.ERROR: 'error'>, 'extra': None}]

How to reproduce the bug

1. Enable GAQ.
2. Use filter_values function in sql template
3. It will result in error in celery worker container.

Environment

(please complete the following information):

• superset version: 1.1.0
• python version: 3.7.9
• node.js version: 14

Checklist

Make sure to follow these steps before submitting your issue - thank you!

• I have checked the superset logs for python stacktraces and included it here as text if there are any.
• I have reproduced the issue with at least the latest released version of superset.
• I have checked the issue tracker for the same issue and I haven't found one similar.

Additional context

Add any other context about the problem here.

[junlincc]

@robdiciuccio 🙏

[ramyarajasekaran]

@robdiciuccio Any updates on this issue? I'm trying to use filter_values with GAQ enabled and seeing the same error.

[robdiciuccio]

I'll be looking into this in the next few days. PRs welcome in the meantime!

[robdiciuccio]

@himanshugarg574 @ramyarajasekaran can you please test this PR to see if the issue is resolved? #16412