POC replace travis by Github Actions (#2066)

* POC replace travis by Github Actions * add libldap2-dev * add libsasl2-dev libssl-dev * Delete .travis.yml * Add specific configuration for PostgreSQL * Retrieve PostgreSQL or MySQL port from env var * Print affected port for the docker service * Add Database type and dburl * Fix dbtype * Use environment variable to define the port in use for tests. * Configured password for postgres connection. * Added redis service. * Enabled OpenLDAP service. * Use a container for OpenLDAP too. * Removed wrong option. * Fixed volume syntax. * Try using the option directly. * Try using an absolute path. * Fixed wrong syntax. * Try different format for env variable. * Try a different way to load data. * Added missing method call. * Fix envar * Try to wait a bit before starting tests. * Display openldap service logs. * Fixed ldif scripts path. * Try to use a custom docker image. * Try to provide credentials. * Try with github token. * use main reference * Changed LDAP Admin password. * Added debug. * Added more debug * Removed debug. * dd test for MySQL * Fix name * Test is independant * Link build * add envar DB for MySQL * remove depends * fix fast failed * fix mysql user * Use 127.0.0.1 to force for IPv4 * For parameters for MySQL * Database name test_modoboa * change mysql user to launch test * Move to the last mysql version lose connection on 5.7 * TEst some timout parameters * remove bad parameters * Add CONN_MAX_AGE parameters * Add docker logs * Try to reset db connections before we spawn processes. * POC replace travis by Github Actions * add libldap2-dev * add libsasl2-dev libssl-dev * Delete .travis.yml * Add specific configuration for PostgreSQL * Retrieve PostgreSQL or MySQL port from env var * Print affected port for the docker service * Add Database type and dburl * Fix dbtype * Use environment variable to define the port in use for tests. * Configured password for postgres connection. * Added redis service. * Enabled OpenLDAP service. * Use a container for OpenLDAP too. * Removed wrong option. * Fixed volume syntax. * Try using the option directly. * Try using an absolute path. * Fixed wrong syntax. * Try different format for env variable. * Try a different way to load data. * Added missing method call. * Fix envar * Try to wait a bit before starting tests. * Display openldap service logs. * Fixed ldif scripts path. * Try to use a custom docker image. * Try to provide credentials. * Try with github token. * use main reference * Changed LDAP Admin password. * Added debug. * Added more debug * Removed debug. * dd test for MySQL * Fix name * Test is independant * Link build * add envar DB for MySQL * remove depends * fix fast failed * fix mysql user * Use 127.0.0.1 to force for IPv4 * For parameters for MySQL * Database name test_modoboa * change mysql user to launch test * Move to the last mysql version lose connection on 5.7 * TEst some timout parameters * remove bad parameters * Add CONN_MAX_AGE parameters * Add docker logs * Try to reset db connections before we spawn processes. * Add relase on pypi (test server first) * Fix setuptools scm to fix upload on Pypi test * Retrieve all repos when publish on pypi * if file exists, skip it * Deploy only on master * Added python 3.5 in matrix. * Removed comments. * Add translation and codecov * Missing argument for upload in production * fix bad indentation * move codecov * Codecov only in one build * fix command * Try to run codecov after PG tests. * Fixed expressions syntax. * Fixed path issue. * Fixed path issue. * Try workflow_run statement. * Removed useless triggers. * Test trigger on one workflow * Another test * Exclude master and main branch * Regroup all steps in one workflow. * Fixed typo * Fixed bad syntax * Added missing requirement. * Push on Pypi Test only on master branch Co-authored-by: Antoine Nguyen <tonio@ngyn.org>
2026-10-18 05:51:26 +02:00 · 2020-11-09 20:44:48 +01:00
parent b6b07f2d15
commit 02efd7f0e3
9 changed files with 226 additions and 97 deletions
@@ -0,0 +1,180 @@
+# This workflow will install Python dependencies, run tests and lint with a variety of Python versions
+# For more information see: https://help.github.com/actions/language-and-framework-guides/using-python-with-github-actions
+
+name: Modoboa App
+
+on:
+  push:
+    branches: [ master ]
+  pull_request:
+    branches: [ master ]
+
+env:
+  POSTGRES_HOST: localhost
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    services:
+      postgres:
+        image: postgres:11
+        env:
+          POSTGRES_USER: postgres
+          POSTGRES_PASSWORD: postgres
+          POSTGRES_DB: postgres
+        ports:
+        # will assign a random free host port
+        - 5432/tcp
+        # needed because the postgres container does not provide a healthcheck
+        options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5
+      mysql:
+        image: mysql:8.0
+        env:
+          MYSQL_ROOT_PASSWORD: root
+          MYSQL_USER: modoboa
+          MYSQL_PASSWORD: modoboa
+          MYSQL_DATABASE: modoboa
+        ports:
+          - 3306/tcp
+        options: --health-cmd="mysqladmin ping" --health-interval=10s --health-timeout=5s --health-retries=3
+      redis:
+        image: redis
+        ports:
+          - 6379/tcp
+      openldap:
+        image: docker.pkg.github.com/modoboa/docker-openldap/docker-openldap
+        ports:
+          - 389/tcp
+        env:
+          LDAP_DOMAIN: example.com
+          LDAP_ADMIN_PASSWORD: test
+        credentials:
+          username: tonioo
+          password: ${{ secrets.GITHUB_TOKEN }}
+
+    strategy:
+      matrix:
+        database: ['postgres', 'mysql']
+        python-version: [3.6, 3.7, 3.8]
+      fail-fast: false
+
+    steps:
+      - uses: actions/checkout@v2
+      - name: Set up Python ${{ matrix.python-version }}
+        uses: actions/setup-python@v2
+        with:
+          python-version: ${{ matrix.python-version }}
+      - name: Install dependencies
+        run: |
+          sudo apt-get install postfix librrd-dev rrdtool libldap2-dev libsasl2-dev libssl-dev
+          python -m pip install --upgrade pip
+          pip install -r requirements.txt
+          pip install -r ldap-requirements.txt
+          pip install -r test-requirements.txt
+          python setup.py develop
+      - name: Install postgres requirements
+        if: ${{ matrix.database == 'postgres' }}
+        run: |
+          pip install -r postgresql-requirements.txt
+          pip install coverage
+          echo "DB=postgres" >> $GITHUB_ENV
+      - name: Install mysql requirements
+        if: ${{ matrix.database == 'mysql' }}
+        run: |
+          pip install -r mysql-requirements.txt
+          echo "DB=mysql" >> $GITHUB_ENV
+      - name: Test with pytest
+        if: ${{ matrix.python-version != '3.8' || matrix.database != 'postgres' }}
+        run: |
+          python ./tests.py
+          cd test_project
+          python3 manage.py test modoboa.core modoboa.lib modoboa.admin modoboa.limits modoboa.transport modoboa.relaydomains modoboa.dnstools modoboa.ldapsync modoboa.policyd modoboa.maillog
+        env:
+          # use localhost for the host here because we are running the job on the VM.
+          # If we were running the job on in a container this would be postgres
+          POSTGRES_PORT: ${{ job.services.postgres.ports[5432] }} # get randomly assigned published port
+          MYSQL_HOST: 127.0.0.1
+          MYSQL_PORT: ${{ job.services.mysql.ports[3306] }} # get randomly assigned published port
+          MYSQL_USER: root
+          REDIS_PORT: ${{ job.services.redis.ports[6379] }}
+          LDAP_SERVER_PORT: ${{ job.services.openldap.ports[389] }}
+
+      - name: Test with pytest and coverage
+        if: ${{ matrix.python-version == '3.8' && matrix.database == 'postgres' }}
+        run: |
+          python ./tests.py
+          cd test_project
+          coverage run manage.py test modoboa.core modoboa.lib modoboa.admin modoboa.limits modoboa.transport modoboa.relaydomains modoboa.dnstools modoboa.ldapsync modoboa.policyd modoboa.maillog
+          coverage combine
+        env:
+          # use localhost for the host here because we are running the job on the VM.
+          # If we were running the job on in a container this would be postgres
+          POSTGRES_PORT: ${{ job.services.postgres.ports[5432] }} # get randomly assigned published port
+          MYSQL_HOST: 127.0.0.1
+          MYSQL_PORT: ${{ job.services.mysql.ports[3306] }} # get randomly assigned published port
+          MYSQL_USER: root
+          REDIS_PORT: ${{ job.services.redis.ports[6379] }}
+          LDAP_SERVER_PORT: ${{ job.services.openldap.ports[389] }}
+      - name: Upload coverage result
+        if: ${{ matrix.python-version == '3.8' }}
+        uses: actions/upload-artifact@v2
+        with:
+          name: coverage-results
+          path: test_project/.coverage
+
+  coverage:
+    needs: test
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v2
+      - name: Set up Python 3.8
+        uses: actions/setup-python@v2
+        with:
+          python-version: '3.8'
+      - name: Install dependencies
+        run: |
+          pip install codecov
+      - name: Download coverage results
+        uses: actions/download-artifact@v2
+        with:
+          name: coverage-results
+      - name: Report coverage
+        run: |
+          coverage report
+          codecov
+
+  release:
+    needs: coverage
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v2
+        with:
+          fetch-depth: 0
+      - name: Set up Python 3.8
+        uses: actions/setup-python@v2
+        with:
+          python-version: '3.8'
+      - name: Install dependencies
+        run: |
+          sudo apt-get install librrd-dev rrdtool libssl-dev gettext
+          python -m pip install --upgrade pip setuptools wheel
+          pip install -r requirements.txt
+          cd modoboa
+          django-admin compilemessages
+          cd ..
+          python setup.py sdist bdist_wheel
+      - name: Publish to Test PyPI
+        if: endsWith(github.event.ref, '/master')
+        uses: pypa/gh-action-pypi-publish@master
+        with:
+          user: __token__
+          password: ${{ secrets.test_pypi_password }}
+          repository_url: https://test.pypi.org/legacy/
+          skip_existing: true
+      - name: Publish distribution to PyPI
+        if: startsWith(github.event.ref, 'refs/tags') || github.event_name == 'release'
+        uses: pypa/gh-action-pypi-publish@master
+        with:
+          user: __token__
+          password: ${{ secrets.pypi_password }}
+          skip_existing: true