Merge pull request #3012 from modoboa/specific-mailbox

Send only mailboxes

doc/upgrade.rst

@@ -6,7 +6,7 @@ Modoboa
 *******
 
 .. note::
-   In this doc, ``by default`` mean that you used `modoboa installer
+   In this doc, ``by default`` means that you used `modoboa installer
    <https://github.com/modoboa/modoboa-installer>`_ to install modoboa
    and that your didn't change your configuration.
 
@@ -119,9 +119,23 @@ Specific instructions
 .. note::
    This version has not been released yet.
 
-* Django has been updated to 4.2LTS. Please upgrade all your extensions alongside modoboa.
-* Support for Python 3.7 has been dropped, minimum Python version is now 3.8.
-* Support for Postgres 11 has been dropped, minimum Postgres version is now 12.
+.. note::
+   If you use the installer to perform the upgrade, you only need to edit ``urls.py`` and ``settings.py``
+
+* Django has been updated to 4.2LTS. Please upgrade all your extensions alongside modoboa
+* Support for Python 3.7 has been dropped, minimum Python version is now 3.8
+* Support for Postgres 11 has been dropped, minimum Postgres version is now 12
+
+You need to change the first line in ``urls.py`` in
+(``/srv/modoboa/instance/instance/urls.py`` by default):
+
+.. sourcecode:: python
+
+   # from django.conf.urls import include, url
+   from django.urls import include, path
+
+And replace `url` with `path` in the file.
+
 
 If you use Postgresql, you need to install pyscopg3+:
 
@@ -131,7 +145,7 @@ If you use Postgresql, you need to install pyscopg3+:
 
 RQ has been added. This should replace the use of cron jobs in the future.
 For now, only ``manage_dkim_keys`` has been migrated. This will make the dkim
-key generation asynchronious but the task will be started as soon as the generation
+key generation asynchronous but the task will be started as soon as the generation
 is required.
 
 Follow these instructions to perform the update in case you used supervisord for the
@@ -140,44 +154,37 @@ installation (this apply if you used ``modoboa-installer``):
 1. Edit settings.py (``/srv/modoboa/instance/instance/settings.py`` by default) and add:
 
 .. sourcecode:: python
+
    INSTALLED_APPS = (
-   ...,
-   'django_rq',
+       ...,
+       'django_rq',
    )
 
 and add the RQ section bellow the ``#REDIS`` section:
 
 .. sourcecode:: python
-   # RQ
 
+   # RQ
    RQ_QUEUES = {
-      'default': {
+      'dkim': {
          'HOST': REDIS_HOST,
          'PORT': REDIS_PORT,
          'DB': 0,
       },
-      'high': {
-         'HOST': REDIS_HOST,
-         'PORT': REDIS_PORT,
-         'DB': 0,
-      },
-      'low': {
-         'HOST': REDIS_HOST,
-         'PORT': REDIS_PORT,
-         'DB': 0,
-      }
    }
 
-Then by default you will need to restart ``uwsgi`` service. ``systemctl restart uwsgi`` on Debian.
+Then by default you will need to restart ``uwsgi`` service (``systemctl restart uwsgi`` on Debian).
 
-2. Create a new supervisord config (``/etc/supervisor/conf.d/modoboaworkerd.conf`` by default) :
+2. Create a new supervisord config (``/etc/supervisor/conf.d/modoboa-worker.conf`` by default) :
 
 .. sourcecode:: ini
+
    [program:modoboa-worker]
    autostart=true
    autorestart=true
-   command={%python env path%} {% manage.py instance path%} worker high default low
+   command={%python env path%} {% manage.py instance path%} rqworker dkim
    directory={%modoboa home dir%}
+   user=%{dkim manager user}
    numprocs=1
    stopsignal=TERM
 
@@ -187,11 +194,44 @@ You will find it here  ``/srv/modoboa/venv/bin/python`` by default.
 You will find it here : ``/srv/modoboa/instance/manage.py``by default.
 ``Modoboa home dir``: Home dir of the user running modooba.
 You will find it here ``/srv/modoboa/`` by default.
+``dkim manager user`` : User managing dkim signing (opendkim by default).
 
 You can help you with ``/etc/supervisor/conf.d/policyd.conf`` (by default).
 
 Then restart supervisor. ``#> supervisorctl reread && supervisorctl update`` on Debian.
 
+Admins now have the option to setup a send-only mailbox.
+
+Send-only mailboxes do not have access to IMAP.
+You need to change your dovecot configuration to enable it.
+
+.. note::
+   modoboa-webmail may not work with send-only user for now.
+
+If you use ``Postgres``:
+Change ``user_query`` and ``password_query`` in ``/etc/dovecot/dovecot-sql.conf.ext``:
+
+.. sourcecode::
+
+   user_query = SELECT '%{home_dir}/%%d/%%n' AS home, %mailboxes_owner_uid as uid, %mailboxes_owner_gid as gid, '*:bytes=' || mb.quota || 'M' AS quota_rule FROM admin_mailbox mb INNER JOIN admin_domain dom ON mb.domain_id=dom.id INNER JOIN core_user u ON u.id=mb.user_id WHERE (mb.is_send_only IS NOT TRUE OR '%s' NOT IN ('imap', 'pop3', 'lmtp')) AND mb.address='%%n' AND dom.name='%%d'
+
+   password_query = SELECT email AS user, password, '%{home_dir}/%%d/%%n' AS userdb_home, %mailboxes_owner_uid AS userdb_uid, %mailboxes_owner_gid AS userdb_gid, CONCAT('*:bytes=', mb.quota, 'M') AS userdb_quota_rule FROM core_user u INNER JOIN admin_mailbox mb ON u.id=mb.user_id INNER JOIN admin_domain dom ON mb.domain_id=dom.id WHERE (mb.is_send_only IS NOT TRUE OR '%s' NOT IN ('imap', 'pop3')) AND email='%%u' AND is_active AND dom.enabled
+
+
+You basically simply need to add ``(mb.is_send_only IS NOT TRUE OR '%s' NOT IN ('imap', 'pop3', 'lmtp')) AND`` for ``user_query`` after ``WHERE`` and ``(mb.is_send_only IS NOT TRUE OR '%s' NOT IN ('imap', 'pop3')) AND`` for ``password_query`` after ``WHERE``.
+
+
+If you use ``MySQL``:
+Change ``user_query`` and ``password_query`` in ``/etc/dovecot/dovecot-sql-master.ext``:
+
+.. sourcecode::
+
+   user_query = SELECT '%{home_dir}/%%d/%%n' AS home, %mailboxes_owner_uid as uid, %mailboxes_owner_gid as gid, CONCAT('*:bytes=', mb.quota, 'M') AS quota_rule FROM admin_mailbox mb INNER JOIN admin_domain dom ON mb.domain_id=dom.id INNER JOIN core_user u ON u.id=mb.user_id WHERE (mb.is_send_only=0 OR '%s' NOT IN ('imap', 'pop3', 'lmtp')) AND mb.address='%%n' AND dom.name='%%d'
+
+   password_query = SELECT email AS user, password, '%{home_dir}/%%d/%%n' AS userdb_home, %mailboxes_owner_uid AS userdb_uid, %mailboxes_owner_gid AS userdb_gid, CONCAT('*:bytes=', mb.quota, 'M') AS userdb_quota_rule FROM core_user u INNER JOIN admin_mailbox mb ON u.id=mb.user_id INNER JOIN admin_domain dom ON mb.domain_id=dom.id WHERE (mb.is_send_only=0 OR '%s' NOT IN ('imap', 'pop3')) AND u.email='%%u' AND u.is_active=1 AND dom.enabled=1
+
+You basically simply need to add ``(mb.is_send_only=0 OR '%s' NOT IN ('imap', 'pop3', 'lmtp')) AND`` for ``user_query`` after ``WHERE`` and ``(mb.is_send_only=0 OR '%s' NOT IN ('imap', 'pop3'))`` for ``password_query`` after ``WHERE``.
+
 
 2.1.0
 =====

docker-compose.yml

@@ -16,7 +16,7 @@ services:
       DEBUG: "True"
       REDIS_HOST: redis
     command: >
-      sh -c "cd test_project; python3 manage.py rqworker high default low"
+      sh -c "cd test_project; python3 manage.py rqworker dkim"
     volumes:
       - .:/code
 

frontend/src/components/identities/AccountCreationForm.vue

@@ -107,6 +107,11 @@ export default {
             {
               key: this.$gettext('Message sending limit'),
               value: this.account.mailbox.message_limit
+            },
+            {
+              key: this.$gettext('Send only account'),
+              value: this.account.mailbox.is_send_only,
+              type: 'yesno'
             }
           ]
         },

frontend/src/components/identities/AccountForm.vue

@@ -88,6 +88,11 @@
                   <v-col cols="6" v-else>
                     <translate class="mr-2">Quota: </translate> {{ account.mailbox.quota }}
                   </v-col>
+                  <v-col cols="6">
+                    <translate class="mr-2">Send only</translate>
+                    <v-icon color="success" v-if="account.mailbox.is_send_only">mdi-check-circle-outline</v-icon>
+                    <v-icon v-else>mdi-close-circle-outline</v-icon>
+                  </v-col>
                 </template>
               </v-row>
             </v-fade-transition>

frontend/src/components/identities/AccountGeneralForm.vue

@@ -81,7 +81,6 @@ export default {
       return await this.$refs.observer.validate() && await this.$refs.passwordForm.validate()
     },
     update () {
-      console.log(this.form)
       this.$emit('input', this.form)
     }
   },

frontend/src/components/identities/AccountMailboxForm.vue

@@ -39,6 +39,18 @@
       @input="update"
       />
   </validation-provider>
+  <validation-provider
+    v-slot="{ errors }"
+    rules=""
+    >
+    <v-switch
+      v-model="form.mailbox.is_send_only"
+      :label="'Send only account' | translate"
+      dense
+      :error-messages="errors"
+      @input="update"
+      />
+  </validation-provider>
 </validation-observer>
 </template>
 

modoboa/admin/api/v1/serializers.py

@@ -156,7 +156,7 @@ class MailboxSerializer(serializers.ModelSerializer):
         model = models.Mailbox
         fields = (
             "pk", "full_address", "use_domain_quota", "quota", "message_limit",
-            "quota_usage"
+            "quota_usage", "is_send_only"
         )
 
     def get_quota_usage(self, mb) -> int:

modoboa/admin/api/v2/serializers.py

@@ -393,6 +393,7 @@ class MailboxSerializer(serializers.ModelSerializer):
         model = models.Mailbox
         fields = (
             "pk", "use_domain_quota", "quota", "message_limit",
+            "is_send_only",
         )
 
     def validate_quota(self, value):

modoboa/admin/api/v2/tests.py

@@ -258,7 +258,7 @@ class AccountViewSetTestCase(ModoAPITestCase):
         resp = self.client.patch(url, data, format="json")
         self.assertEqual(resp.status_code, 200)
 
-    def test_message_limit_update(self):
+    def test_mailbox_options_update(self):
         account = core_models.User.objects.get(username="user@test.com")
         url = reverse("v2:account-detail", args=[account.pk])
         data = {
@@ -266,13 +266,15 @@ class AccountViewSetTestCase(ModoAPITestCase):
             "role": "SimpleUsers",
             "password": "Toto12345",
             "mailbox": {
-                "message_limit": 10
+                "message_limit": 10,
+                "is_send_only": True,
             }
         }
         resp = self.client.patch(url, data, format="json")
         self.assertEqual(resp.status_code, 200)
         account.refresh_from_db()
         self.assertEqual(account.mailbox.message_limit, 10)
+        self.assertEqual(account.mailbox.is_send_only, True)
 
         data = {
             "username": "user@test.com",
@@ -292,13 +294,15 @@ class AccountViewSetTestCase(ModoAPITestCase):
             "role": "SimpleUsers",
             "password": "Toto12345",
             "mailbox": {
-                "message_limit": None
+                "message_limit": None,
+                "is_send_only": False,
             }
         }
         resp = self.client.patch(url, data, format="json")
         self.assertEqual(resp.status_code, 200)
         account.refresh_from_db()
         self.assertEqual(account.mailbox.message_limit, None)
+        self.assertEqual(account.mailbox.is_send_only, False)
 
     def test_update_admin(self):
         account = core_models.User.objects.get(username="admin")

modoboa/admin/forms/account.py

@@ -249,6 +249,11 @@ class AccountFormMail(forms.Form, DynamicForm):
         help_text=gettext_lazy(
             "Number of messages this mailbox can send per day")
     )
+    is_send_only = forms.BooleanField(
+        required=False,
+        help_text=gettext_lazy(
+            "Block IMAP connection and incoming mails for this mailbox")
+    )
     aliases = lib_fields.UTF8AndEmptyUserEmailField(
         label=gettext_lazy("Alias(es)"),
         required=False,
@@ -296,6 +301,7 @@ class AccountFormMail(forms.Form, DynamicForm):
                 self.fields["quota"].initial = self.mb.quota
             if self.mb.message_limit:
                 self.fields["message_limit"].initial = self.mb.message_limit
+            self.fields["is_send_only"].initial = self.mb.is_send_only
             self.fields["create_alias_with_old_address"].initial = (
                 params["create_alias_on_mbox_rename"]
             )
@@ -384,7 +390,8 @@ class AccountFormMail(forms.Form, DynamicForm):
         self.mb = models.Mailbox(
             address=self.locpart, domain=self.domain, user=account,
             use_domain_quota=self.cleaned_data["quota_act"],
-            message_limit=self.cleaned_data.get("message_limit")
+            message_limit=self.cleaned_data.get("message_limit"),
+            is_send_only=self.cleaned_data.get("is_send_only", False)
         )
         override_rules = (
             user.is_superuser or

modoboa/admin/handlers.py

@@ -51,7 +51,8 @@ def update_domain_mxs_and_mailboxes(sender, instance, **kwargs):
 def create_dkim_key(sender, instance, **kwargs):
     if not instance.enable_dkim:
         return
-    django_rq.enqueue(call_command,
+    queue = django_rq.get_queue('dkim')
+    queue.enqueue(call_command,
                       "modo",
                       "manage_dkim_keys",
                       f"--domain={instance.name}")

modoboa/admin/migrations/0021_mailbox_is_send_only.py

@@ -0,0 +1,17 @@
+# Generated by Django 4.2.1 on 2023-06-13 16:43
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+    dependencies = [
+        ("admin", "0020_auto_20210130_2226"),
+    ]
+
+    operations = [
+        migrations.AddField(
+            model_name="mailbox",
+            name="is_send_only",
+            field=models.BooleanField(default=False),
+        ),
+    ]

modoboa/admin/models/mailbox.py

@@ -103,6 +103,8 @@ class Mailbox(mixins.MessageLimitMixin, AdminObject):
     domain = models.ForeignKey(Domain, on_delete=models.CASCADE)
     user = models.OneToOneField(User, on_delete=models.CASCADE)
 
+    is_send_only = models.BooleanField(default=False)
+
     objects = MailboxManager()
 
     class Meta:
@@ -324,6 +326,8 @@ class Mailbox(mixins.MessageLimitMixin, AdminObject):
             self.set_quota(values["quota"], override_rules)
         if "message_limit" in values:
             self.message_limit = values["message_limit"]
+        if "is_send_only" in values:
+            self.is_send_only = values["is_send_only"]
         if newaddress:
             self.rename(local_part, domain)
         self.save()

modoboa/admin/templates/admin/mailform.html

@@ -6,5 +6,6 @@
 {% endif %}
 {% include "common/generic_field.html" with field=form.quota help_display_mode="tooltip" label_width="col-sm-4" activator=True activator_value=form.quota_act.value deactivate_if_empty=False activator_label=_("Use domain default value") activator_size="col-sm-5" %}
 {% render_field form.message_limit %}
+{% render_field form.is_send_only %}
 {% render_fields_group form "aliases" %}
 {% render_fields_group form "senderaddress" %}

modoboa/core/commands/deploy.py

@@ -3,7 +3,9 @@
 import getpass
 import os
 from os.path import isfile
+import secrets
 import shutil
+import string
 import subprocess
 import sys
 
@@ -81,6 +83,10 @@ class DeployCommand(Command):
             "--admin-username", default="admin",
             help="Username of the initial super administrator"
         )
+        self._parser.add_argument(
+            "--secret-key-length", default=60, type=int,
+            help="Specify the length of the secret key"
+        )
 
     def _exec_django_command(self, name, cwd, *args):
         """Run a django command for the freshly created project
@@ -156,6 +162,14 @@ class DeployCommand(Command):
         official_exts = ModoAPIClient(url).list_extensions()
         return [extension["name"] for extension in official_exts]
 
+    def _generate_secret_key(self, secret_key_length):
+        """Generate a secret key since the one generated by django
+        is marked as insecure.
+        """
+        c = string.ascii_letters + string.digits + string.punctuation
+        secret_key = ''.join(secrets.choice(c) for i in range(secret_key_length))
+        return secret_key
+
     def find_extra_settings(self, extensions):
         """Install one or more extensions.
 
@@ -233,7 +247,7 @@ class DeployCommand(Command):
         tpl = self._render_template(
             "%s/settings.py.tpl" % self._templates_dir, {
                 "db_connections": connections,
-                "secret_key": mod.settings.SECRET_KEY,
+                "secret_key": self._generate_secret_key(parsed_args.secret_key_length),
                 "name": parsed_args.name,
                 "allowed_host": allowed_host,
                 "lang": parsed_args.lang,

modoboa/core/commands/templates/settings.py.tpl

@@ -257,21 +257,11 @@ REDIS_URL = 'redis://{}:{}/{}'.format(REDIS_HOST, REDIS_PORT, REDIS_QUOTA_DB)
 # RQ
 
 RQ_QUEUES = {
-    'default': {
+    'dkim': {
         'HOST': REDIS_HOST,
         'PORT': REDIS_PORT,
         'DB': 0,
     },
-    'high': {
-        'HOST': REDIS_HOST,
-        'PORT': REDIS_PORT,
-        'DB': 0,
-    },
-    'low': {
-        'HOST': REDIS_HOST,
-        'PORT': REDIS_PORT,
-        'DB': 0,
-    }
 }
 
 # Password validation

test_project/test_project/settings.py

@@ -240,15 +240,9 @@ REDIS_URL = 'redis://{}:{}/{}'.format(REDIS_HOST, REDIS_PORT, REDIS_QUOTA_DB)
 # RQ
 
 RQ_QUEUES = {
-    'default': {
+    'dkim': {
         'URL': REDIS_URL,
     },
-    'high': {
-         'URL': REDIS_URL,
-    },
-    'low': {
-         'URL': REDIS_URL,
-    }
 }
 
 # Password validation