From e89be8f591ecf28712581d5f94f0d42572a21df8 Mon Sep 17 00:00:00 2001 From: romsharon98 Date: Mon, 19 Feb 2024 12:03:55 +0200 Subject: [PATCH] add presto connection documentation --- .../connections.rst | 56 +++++++++++++++++++ .../apache-airflow-providers-presto/index.rst | 1 + .../connections.rst | 2 +- 3 files changed, 58 insertions(+), 1 deletion(-) create mode 100644 docs/apache-airflow-providers-presto/connections.rst diff --git a/docs/apache-airflow-providers-presto/connections.rst b/docs/apache-airflow-providers-presto/connections.rst new file mode 100644 index 0000000000000..f0e3d8787347d --- /dev/null +++ b/docs/apache-airflow-providers-presto/connections.rst @@ -0,0 +1,56 @@ + .. Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + .. http://www.apache.org/licenses/LICENSE-2.0 + + .. Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. + +Presto Connection +================= + +The Presto connection type enables connection to Presto which is an open-source distributed SQL query engine designed for fast analytics on large-scale data sources, enabling interactive querying across multiple data platforms. + + +Default Connection IDs +---------------------- + +Presto Hook uses the parameter ``presto_conn_id`` for Connection IDs and the value of the parameter as ``presto_default`` by default. +Presto Hook supports multiple authentication types to ensure all users of the system are authenticated, the parameter ``auth`` can be set to enable authentication. The value of the parameter is ``None`` by default. + +Configuring the Connection +-------------------------- +Host + The host to connect to. + +Port + The port to connect to the host. Presto will use 8080 as default. + +Login + Effective user for connection. + +Password + This can be to pass to enable Basic Authentication. This is an optional parameter and is not required if a different authentication mechanism is used. + +Extra (optional, connection parameters) + Specify the extra parameters (as json dictionary) that can be used in Presto connection. The following parameters out of the standard python parameters are supported: + + * ``auth`` - Specifies which type of authentication needs to be enabled. The value can be ``kerberos``. + * ``source`` - Specifies source to connect to. Default value is ``airflow``. + * ``protocol`` - Specifies port to connect with. Default value is ``http``. + * ``catalog`` - Specifies which catalog to use. Default value is ``hive``. + * ``verify`` - Client certificate path to connect with SSL/TLS. + + + The following extra parameters can be used to configure authentication: + + * ``kerberos__service_name``, ``kerberos__config``, ``kerberos__mutual_authentication``, ``kerberos__force_preemptive``, ``kerberos__hostname_override``, ``kerberos__sanitize_mutual_error_response``, ``kerberos__principal``, ``kerberos__delegate``, ``kerberos__ca_bundle`` - These parameters can be set when enabling ``kerberos`` authentication. diff --git a/docs/apache-airflow-providers-presto/index.rst b/docs/apache-airflow-providers-presto/index.rst index fc15473d0717e..153eecc0b6fe3 100644 --- a/docs/apache-airflow-providers-presto/index.rst +++ b/docs/apache-airflow-providers-presto/index.rst @@ -35,6 +35,7 @@ :caption: Guides PrestoTransferOperator types + Connection types .. toctree:: :hidden: diff --git a/docs/apache-airflow-providers-trino/connections.rst b/docs/apache-airflow-providers-trino/connections.rst index 5b2a9b36478f9..f6bd0b46c1e5f 100644 --- a/docs/apache-airflow-providers-trino/connections.rst +++ b/docs/apache-airflow-providers-trino/connections.rst @@ -51,7 +51,7 @@ Extra (optional, connection parameters) * ``jwt__token`` - If jwt authentication should be used, the value of token is given via this parameter. * ``jwt__file`` - If jwt authentication should be used, the location on disk for the file containing the jwt token. * ``certs__client_cert_path``, ``certs__client_key_path``- If certificate authentication should be used, the path to the client certificate and key is given via these parameters. - * ``kerberos__service_name``, ``kerberos__config``, ``kerberos__mutual_authentication``, ``kerberos__force_preemptive``, ``kerberos__hostname_override``, ``kerberos__sanitize_mutual_error_response``, ``kerberos__principal``,``kerberos__delegate``, ``kerberos__ca_bundle`` - These parameters can be set when enabling ``kerberos`` authentication. + * ``kerberos__service_name``, ``kerberos__config``, ``kerberos__mutual_authentication``, ``kerberos__force_preemptive``, ``kerberos__hostname_override``, ``kerberos__sanitize_mutual_error_response``, ``kerberos__principal``, ``kerberos__delegate``, ``kerberos__ca_bundle`` - These parameters can be set when enabling ``kerberos`` authentication. * ``session_properties`` - JSON dictionary which allows to set session_properties. Example: ``{'session_properties':{'scale_writers':true,'task_writer_count:1'}}`` * ``client_tags`` - List of comma separated tags. Example ``{'client_tags':['sales','cluster1']}``` * ``timezone`` - The time zone for the session can be explicitly set using the IANA time zone name. Example: ``{'timezone':'Asia/Jerusalem'}``.