Asabru Proxy

Objective

Asabru is a high-performance SQL Proxy developed by ChistaDATA, designed to improve the scalability and availability of database servers like ClickHouse, PostgreSQL, and MySQL. Asabru server enables users to easily configure and manage their database connections while enhancing their database performance.

At present, Asabru supports TCP/IP, HTTP/HTTPS, and TLS/SSL protocols for ClickHouse. We are also working on adding TLS/SSL support for MySQL and PostgreSQL, while wire-level protocol is already supported.

ChistaDATA Asabru Proxy is available in two forms

• as part of ChistaDATA DBAAS Portal
• as an independently deployable proxy

Staring Proxy along with DBAAS

At present, the ChistaDATA Asabru Proxy is seamlessly connected to the ChistaDATA DBAAS portal.

The deployment of the proxy can be initiated during the cluster creation process by choosing the “Deploy Proxy” option on the cluster creation screen.In this case proxy will be using the default port number. Example proxy will be available at the port # 9000 for ClickHouse Wire-level Protocol Proxying

Independently deployable proxy

You can follow the below guidelines to build a server from the source code.

Installation

To install and run the Chista Asabru server, first clone the code on the server from the GitHub repository, install essential packages and dependencies then configure and build the Asabru server from the source code.

Prerequisite

Make https://www.gnu.org/software/make

CMake https://cmake.org/install/

Install docker and docker-compose https://docs.docker.com/engine/install/ https://docs.docker.com/compose/install/

CURL https://everything.curl.dev/get

Run the following command to create a symlink

ln -s /usr/bin/curl /opt/bin/curl

Python Asabru requires Python version >=3.11

Download Page - https://www.python.org/downloads/

Submodules

Asabru project consists of the following sub-modules :

1. asabru-engine: This is the core engine of the project, which provides an abstraction over platform dependent networking APIs.
2. asabru-commons: This repo contains the common utility classes for the chista database proxy.
3. asabru-parsers: This repo contains the code for the ChistaDATA database proxy parsers.
4. asabru-handlers: This repo contains the various handlers for the chista database proxy.

Development setup

Clone the project using the following command. This will clone the submodules of the project and the external dependencies into the lib folder.

git clone --recurse-submodules

To initialize ClickHouse cluster

docker compose up -d

Build

Build the dependency libraries

sudo chmod u+x ./build_libs.sh
./build_libs.sh

Set environment variables

Asabru proxy’s configuration is based on an xml file. This file can be fetched from a remote server or from local PC

Remote configuration

export CONFIG_FILE_URL=<url-to-config>;
export CONFIG_FILE_PATH=<path-to-config>;

Eg.
export CONFIG_FILE_URL=https://pastebin.com/raw/RcyrYLMc;
export CONFIG_FILE_PATH=/tmp/config.xml;

Local configuration (skip setting the CONFIG_FILE_URL environment variable)

export CONFIG_FILE_PATH=<path-to-config>;

Eg.
export CONFIG_FILE_PATH=/Users/username/workplace/chistadata/chista-asabru/build/config.xml

The plugins folder path is the directory where plugins have been built and the public folder path is the folder where the HTTP protocol handler serves static files.

export PLUGINS_FOLDER_PATH=<path-to-plugins-folder>;
export PUBLIC_FOLDER_PATH=<path-to-public-folder>;

Eg.
export PLUGINS_FOLDER_PATH=/Users/username/workplace/chistadata/chista-asabru/lib/asabru-handlers/build;
export PUBLIC_FOLDER_PATH=/Users/username/workplace/chistadata/chista-asabru/build/public;

Running Proxy using SSL clients requires setting the following SSL configuration. Please see the documentation on how to create a self-signed certificates for testing

export SSL_CERT_FILE_PATH = <path-to-SSL-certificate-directory>
export SSL_KEY_FILE_PATH = <path-to -SSL-key-directory>

If a passphrase is set , give the passphrase also

export SSL_CERT_PASSPHRASE = <passphrase>

Eg.
export SSL_CERT_FILE_PATH = /Users/josephabraham/ChistaV2/ssl_test1/cert.pem
export SSL_KEY_FILE_PATH = /Users/josephabraham/ChistaV2/ssl_test1/key.pem
export SSL_CERT_PASSPHRASE = mypassword

If you are connecting to a target server with a self-signed certificate, add this environment variable

export SSL_VERIFY_CERT=false

Logging

The proxy currently supports logging to these outputs

• text file (FILE)
• sqlite database (DB)
• network tcp channel (NET)

For configuration, set the env variable LOGGER_TYPE to any of the above.

export LOGGER_TYPE=FILE

The text file will be generated in the build folder with the current date appended for the logs.

Build the asabru app from the root of the repository

mkdir build
cd build
cmake ..
make

Build in Debug mode

mkdir debug
cd debug
cmake -DCMAKE_BUILD_TYPE=Debug ..
cmake --build .

Docker Build

If you want to build the proxy in a docker container. Follow the below steps.

docker build --no-cache --progress=plain -t asabru-proxy .

Run Docker Image

docker run -it -d -p 9000-9999:9000-9999 -e CONFIG_FILE_URL=<config-file-url> -e CONFIG_FILE_PATH=<config-file-path> --name asabru-proxy asabru-proxy

Eg. docker run -it -d -p 9000-9999:9000-9999 -e CONFIG_FILE_URL=https://pastebin.com/raw/qAPt4KNQ -e CONFIG_FILE_PATH=/tmp/config.xml --name asabru-proxy asabru-proxy

Run with Docker compose

Update the values for the environment variables in the docker-compose.yaml file and run the following command.

docker compose up -d --build

Usage

The proxy configurations are given in the config.xml file, we need to set the config file path location in ENV variables. Proxy will fetch the configurations from that location and start the server.

 ./build/Chista_Asabru

Run the ClickHouse client in SSL mode

./clickhouse client --config <your-config-path>/clickhouse-client-ssl.xml --host 127.0.0.1 --port 9120

Eg. ./clickhouse client --config /Users/username/software/openssl/clickhouse-client-ssl.xml --host 127.0.0.1 --port 9120

The above-mentioned steps will start the Asabru server and enable it to proxy connections for ClickHouse using TCP/TLS with the WireLevel protocol. The proxy will now be ready to accept incoming connections on the specified port number.

Asabru servers support various configurations to suit different needs and use cases. The proxy server operates on specific port numbers that correspond to the protocol used by the underlying database system. By leveraging the Asabru server, users can benefit from its efficient and reliable proxying capabilities, making it an excellent tool for database management and optimization.

Asabru server provides users with an efficient and flexible way to manage their database connections through a range of proxy port numbers for various config types, allowing users to configure and manage their database connections with ease. Here are the available configurations:

To run the proxy for other configurations, refer to the Table above and replace the port number in the command above with the corresponding port number for the desired configuration. For example, if you want to run the proxy for PostgreSQL using the WireLevel protocol with port number 9140, use the following command:

./Chista_Asabru 9140

By following these simple steps, you can easily run the Asabru server proxy from the command line for various database systems and configurations. This provides users with a powerful tool for managing their database connections and optimizing their database performance.

Our current version of Asabru serves as a highly efficient forward proxy for client connections to ClickHouse, PostgreSQL, and MySQL databases. However, our long-term vision is to expand its capabilities to support multiple popular databases with a more robust reverse proxy that can intelligently interpret SQL and take appropriate actions for load balancing and high availability. With this enhanced functionality, Asabru will not only continue to serve as a forward proxy for clients but will also act as a reverse proxy for backend database servers.

Visual aid included

Glossary

Proxy Mode - Passthrough mode

Passthrough mode refers to a mode of operation where the proxy simply forwards the incoming network traffic to the destination database server without performing any modification or interpretation of the data. In Passthrough mode, the Asabru server acts as a transparent intermediary between the client and the database server, relaying network packets without any modification to the underlying protocol or data. This mode is useful in situations where the client application and the database server have a direct network connection and the Asabru server is simply acting as a “pass-through” proxy, providing an extra layer of security or performance optimization without interfering with the communication between the client and server.

Wirelevel Protocol

The Wirelevel Protocol is a low-level communication protocol used by MySQL/ClickHouse/PostgreSQL to exchange information between the client and server. It is based on TCP/IP and supports a binary format making it efficient for transmitting large amounts of data. It includes features such as authentication, encryption, and compression to ensure secure and efficient communication between the client and the server. Examples of client applications that use the Wirelevel Protocol include the MySQL command-line client, MySQL Workbench, and various programming languages connectors such as JDBC for Java or the MySQL-python library for Python.