GitLab CI with Python and Docker!

ALT TEXT

Show me the Code!

This is a simple Flask application that counts the number of POST and GET requests it receives.

Setup

  1. Clone the repository:
    git clone https://gitlab.com/devozs/counter-service
cd counter-service
  2. Create a virtual environment and activate it:
    python -m venv venv
source venv/bin/activate  # On Windows, use `venv\Scripts\activate`
  3. Install the requirements:
    pip install -r requirements.txt

Project Structure

counter_service/
├── app/
│   ├── __init__.py
│   ├── routes.py
│   └── counter.py
├── certs/
│   ├── generate-self-signed-cert.sh
│   ├── cert.pem (generated)
│   └── key.pem (generated)
├── tests/
│   ├── __init__.py
│   └── test_routes.py
├── config.py
├── counter_service.py
├── run_server.sh
├── run_pytest.sh
├── requirements.txt
└── README.md
  • app/: Contains the main application code
    • __init__.py: Initializes the Flask application
    • routes.py: Defines the application routes
    • counter.py: Implements the Counter class for tracking requests
  • certs/: Contains SSL certificate-related files
    • generate-self-signed-cert.sh: Script to generate self-signed SSL certificates
    • cert.pem: SSL certificate (generated by the script)
    • key.pem: SSL private key (generated by the script)
  • tests/: Contains test files
    • test_routes.py: Tests for the application routes
  • config.py: Configuration settings for the application
  • counter_service.py: Main script to run the Flask application
  • run_server.sh: Bash script to run the application with or without SSL
  • run_pytest.sh: Bash script to run the application tests
  • requirements.txt: List of Python dependencies
  • README.md: Project documentation

SSL Configuration

SSL configuration is separate from the environment configuration. You can run any environment (development, production, testing) with or without SSL by adding the 'ssl' argument to the script.

If running with SSL, the application expects the following files in the certs directory:

  • cert.pem: SSL certificate
  • key.pem: SSL private key

To generate self-signed certificates for testing purposes, use the provided script:

  1. Navigate to the project root directory.
  2. Run the following command: 
    (cd certs && ./generate-self-signed-cert.sh)

This script will generate a self-signed certificate and key with the following details:

  • Country: IL
  • State: Haifa
  • Locality: Haifa
  • Organization: DevOzs
  • Organizational Unit: DevOps
  • Common Name: localhost
  • Email: admin@devozs.com

The certificate will be valid for 365 days.

Note: These self-signed certificates should only be used for development and testing purposes. For production environments, always use certificates from a trusted Certificate Authority.

Running the Application

You can run the application using the run_server.sh script with various configurations:

  1. Development mode without SSL (default):
    ./run_server.sh

    or
    ./run_server.sh development
  2. Production mode without SSL:
    ./run_server.sh production
  3. Testing mode without SSL:
    ./run_server.sh testing
  4. Development mode with SSL:
    ./run_server.sh development ssl
  5. Production mode with SSL:
    ./run_server.sh production ssl

Note: Running with SSL requires root privileges and will prompt for your password.

Using the API

You can interact with the API using curl commands:

  1. To make a GET request and view the current count:
    curl http://localhost:5000  # If running without SSL
curl -k https://localhost   # If running with SSL
  2. To make a POST request and increment the counter:
    curl -X POST http://localhost:5000  # If running without SSL
curl -k -X POST https://localhost   # If running with SSL

The -k flag is used to allow insecure connections when using self-signed certificates.

You can alternate between these commands to see the counter increment for both GET and POST requests.

Running the tests

Run the tests using pytest:

pytest tests/

Configuration

The application uses different configuration classes for different environments:

  • DevelopmentConfig: Used for development.
  • TestingConfig: Used for testing.
  • ProductionConfig: Used for production.

You can modify these configurations in the config.py file.

API Endpoints

  • GET /: Returns the current count of POST and GET requests
  • POST /: Increments the POST counter and returns the updated count

Running Tests

To run the tests for the Counter Service application:

  1. Ensure you're in the root directory of the project.
  2. Run the test script:
    ./run_pytest.sh

This script will:

  • Activate the virtual environment
  • Set the FLASK_ENV to 'testing'
  • Run pytest on the tests in the tests/ directory
  • Deactivate the virtual environment after tests are complete

The tests use the TestingConfig configuration, which is designed for testing purposes. This ensures that your tests are run in an environment that mimics your production setup but is safe for testing.

Test Configuration

The TestingConfig in config.py is used for running tests. You can modify this configuration to set up any specific settings needed for your test environment. For example:

class TestingConfig(Config):
    TESTING = True
    # Add any other test-specific configurations here

Writing Tests

When writing new tests, ensure they are placed in the tests/ directory. The test_routes.py file contains examples of how to write tests for your routes.

Docker Setup

This project includes a Dockerfile for containerizing the Counter Service application. The Docker setup is optimized for running the application on port 5000 without SSL.

Building the Docker Image

To build the Docker image:

docker build -t counter-service .

Running the Application in Docker

You can run the application in different modes using Docker: Please note that the current Docker configuration does not support running the application with SSL on port 443. The application is set up to run on port 5000 without SSL when using Docker.

  1. Production mode (default):
    docker run -p 5000:5000 counter-service

    or
    docker run -p 5000:5000 counter-service production
  2. Development mode:
    docker run -p 5000:5000 counter-service development

The application will be accessible at http://localhost:5000

Future Enhancements and Notes

As we continue to develop and improve this project, we've identified several areas for future enhancements and some important notes:

CI/CD Improvements

  • Docker Run in CI: The run_docker step at the end of the CI pipeline is currently included for demonstration purposes only. In a production environment, this step would be unnecessary as the image is already pushed to the registry.
  • Image Scanning: Implement image scanning using tools like Trivy or similar to enhance security and identify vulnerabilities in our Docker images.
  • Merge Request Approvals: Consider implementing additional merge request approvals and other GitLab approval processes to enhance code quality and security.

Dockerfile Optimizations

  • Single Layer Build: The current Dockerfile uses a two-layer build for demonstration purposes. In the future, we may optimize this to a single layer for efficiency.
  • Rootless Docker Build: Explore using Podman for rootless Docker builds to enhance security.
  • Requirement Separation: Investigate the option to split requirements.txt into separate files for application and testing dependencies.

These enhancements and notes serve as a roadmap for future development. They reflect our commitment to continuous improvement in terms of efficiency, security, and best practices. We welcome contributions and suggestions in any of these areas.

CI Test Build Docker Python ECR