What is Apache Airflow stack
Is a web stack of applications built with Docker Compose containing preconfigured Apache Airflow, RabbitMQ, Redis, and SMTP server. Apache Airflow is difficult to set up manually, so the main goal of TryDirect here is to streamline the deployment process to the most famous clouds.
What does Apache Airflow stack include
- Airflow docker container
- Nginx docker container
- PostgreSQL docker container
- Redis docker container
- RabbitMQ docker container
Currently, following versions are supported:
Who is using Apache Airflow
- Data Engineers
- Data Scientists
- Software developers
- DevOps Engineers
What is Apache Airflow
Apache Airflow is an open-source platform for creating Python workflows, scheduling and monitoring them. The platform supports REST API, WebHDFS, allows progress monitoring, release management, and production optimization. Apache Airflow supports Jinja templating engine and Apache Pig integration. It was built by an Airbnb developer and has grown from an internal product to a widely-spread workflow management solution.
What are the benefits of using Apache Airflow stack
- Automatic deployment of the Airflow suite to the prefered cloud
- Straightforward upgrade to a new business apps suite version
- The accurate stack configuration
- Smooth app migration to another cloud provider
- Fast startup and better performance of the app
- Lower system overhead
Number of servers required: 1
Companion apps
knockd is a port-knock server. It listens to all traffic on an ethernet (or PPP) interface, looking for special knock sequences of port-hits. A client makes these port-hits by sending a TCP (or UDP) packet to a port on the server. This port need not be open -- since knockd listens at the link-layer level, it sees all traffic even if it's destined for a closed port. When the server detects a specific sequence of port-hits, it runs a command defined in its configuration file. This can be used to open up holes in a firewall for quick access.
Fail2ban scans log files and bans IPs that show the malicious signs. Highly recommended for simple VPS server setup
Minimal docker container management panel. Allows you to check the status and logs of all services of this stack. It also simplifies SSL setup and configuration.
Portainer is a lightweight management UI which allows you to easily manage your different Docker environments (Docker hosts or Swarm clusters)
Real-time performance monitoring, done right!
Netdata positions itself as an alloy of the best features that the paid infrastructure monitoring solutions and open-source tools have. Its multifunctionality, velocity, and ease of use make it secure, reliable, and scalable. This tool enables you effortlessly perform your infrastructure slowdowns and anomalies, troubleshooting, and overall system monitoring with little to no configuration.