Skip to content

Commit

Permalink
Migrate/Update API documentation (#158)
Browse files Browse the repository at this point in the history
---------
Co-authored-by: Thomas Ubensee <[email protected]>
  • Loading branch information
Nicoretti authored Dec 5, 2024
1 parent 95414ac commit ad228bd
Show file tree
Hide file tree
Showing 10 changed files with 1,247 additions and 171 deletions.
41 changes: 41 additions & 0 deletions doc/api.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,44 @@
:octicon:`cpu` API Reference
=============================

.. autofunction:: pyexasol.connect

.. autofunction:: pyexasol.connect_local_config

.. autofunction:: pyexasol.http_transport

.. autoclass:: pyexasol.ExaConnection
:members:
:special-members: __init__
:undoc-members:
:show-inheritance:

.. autoclass:: pyexasol.ExaStatement
:members:
:special-members: __init__, __iter__
:undoc-members:
:show-inheritance:

.. autoclass:: pyexasol.ExaFormatter
:class-doc-from: init
:members:
:undoc-members:
:show-inheritance:

.. autoclass:: pyexasol.ExaMetaData
:class-doc-from: init
:members:
:undoc-members:
:show-inheritance:

.. autoclass:: pyexasol.ExaExtension
:class-doc-from: init
:members:
:undoc-members:
:show-inheritance:

.. autoclass:: pyexasol.ExaHTTPTransportWrapper
:class-doc-from: init
:members:
:undoc-members:
:show-inheritance:
4 changes: 4 additions & 0 deletions doc/changes/unreleased.md
Original file line number Diff line number Diff line change
@@ -1,2 +1,6 @@
# Unreleased

## 📚 Documentation

* Add sphinx based documention

2 changes: 2 additions & 0 deletions doc/user_guide/protocol_version.rst
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
.. _protocol_version:

WebSocket protocol versions
===========================

Expand Down
80 changes: 49 additions & 31 deletions pyexasol/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -62,22 +62,28 @@

def connect(**kwargs) -> ExaConnection:
"""
Constructor of connection objects
Please check ExaConnection object for list of arguments
Create a new connection object. For details regarding kwargs,
refer to the :class:`pyexasol.ExaConnection` class.
"""
return ExaConnection(**kwargs)


def connect_local_config(config_section, config_path=None, **kwargs) -> ExaConnection:
"""
Constructor of connection objects based on local config file
Default config path is ~/.pyexasol.ini
Constructor for connection objects based on a local config file.
Extra arguments override values from config
Info:
- The default config path is ~/.pyexasol.ini
- Extra arguments override values from config
:param config_section: Name of config section (required!)
:param config_path: Custom path to local config file
:param kwargs: Arguments for "connect()" function
Args:
config_section:
Name of config section (required!)
config_path:
Custom path to local config file
kwargs:
Arguments for "connect()" function
"""

conf = ExaLocalConfig(config_path)
Expand All @@ -88,28 +94,40 @@ def connect_local_config(config_section, config_path=None, **kwargs) -> ExaConne

def http_transport(ipaddr, port, compression=False, encryption=True) -> ExaHTTPTransportWrapper:
"""
Constructor of HTTP Transport wrapper for parallel HTTP Transport (EXPORT or IMPORT)
Compression and encryption arguments should match pyexasol.connect()
How to use:
1) Parent process opens main connection to Exasol with pyexasol.connect()
2)
2) Parent process creates any number of child processes (possibly on remote host or another container)
3) Every child process starts HTTP transport sub-connection with pyexasol.http_transport()
and gets "ipaddr:port" string using ExaHTTPTransportWrapper.address
4) Every child process sends address string to parent process using any communication method (Pipe, Queue, Redis, etc.)
5) Parent process runs .export_parallel() or .import_parallel(), which initiates EXPORT or IMPORT query in Exasol
6) Every child process receives or sends a chunk of data using ExaHTTPTransportWrapper.export_*() or .import_*()
7) Parent process waits for Exasol query and for child processes to finish
All child processes should run in parallel.
It is NOT possible to process some data first, and process some more data later.
If an exception is raised in child process, it will close the pipe used for HTTP transport.
Closing the pipe prematurely will cause SQL query to fail and will raise an exception in parent process.
Parent process is responsible for closing other child processes and cleaning up.
PyEXASOL does not provide a complete solution to manage child processes, only examples.
The final solution depends on your hardware, network configuration, cloud provider and container orchestration software.
Constructor for HTTP Transport wrapper for parallel HTTP Transport (EXPORT or IMPORT)
Args:
ipaddr:
IP address of one of Exasol nodes received from :meth:`pyexasol.ExaConnection.get_nodes`
port:
Port of one of Exasol nodes received from :meth:`pyexasol.ExaConnection.get_nodes`
compression:
Use zlib compression for HTTP transport, must be the same as `compression` of main connection
encryption:
Use SSL/TLS encryption for HTTP transport, must be the same as `encryption` of main connection
Info:
Compression and encryption arguments should match :func:`pyexasol.connect`
How to use:
#. Parent process opens main connection to Exasol with pyexasol.connect()
#. Parent process creates any number of child processes (possibly on remote host or another container)
#. Every child process starts HTTP transport sub-connection with pyexasol.http_transport()
#. and gets "ipaddr:port" string using ExaHTTPTransportWrapper.address
#. Every child process sends address string to parent process using any communication method (Pipe, Queue, Redis, etc.)
#. Parent process runs .export_parallel() or .import_parallel(), which initiates EXPORT or IMPORT query in Exasol
#. Every child process receives or sends a chunk of data using ExaHTTPTransportWrapper.export_*() or .import_*()
#. Parent process waits for Exasol query and for child processes to finish
All child processes should run in parallel.
It is NOT possible to process some data first, and process some more data later.
If an exception is raised in child process, it will close the pipe used for HTTP transport.
Closing the pipe prematurely will cause SQL query to fail and will raise an exception in parent process.
Parent process is responsible for closing other child processes and cleaning up.
PyEXASOL does not provide a complete solution to manage child processes, only examples.
The final solution depends on your hardware, network configuration, cloud provider and container orchestration software.
"""
return ExaHTTPTransportWrapper(ipaddr, port, compression, encryption)
Loading

0 comments on commit ad228bd

Please sign in to comment.