Add support for multiple servers #12

Janfred · 2016-03-14T09:15:45Z

Add support for multiple servers.

Useful for hot standby or backup systems with replicated databases.

The main problem is that, if you want to have a backup mx, you need to copy all mailbox setting changes you make on the first system to the second one.

With this new feature you can just replicate your database and use it for both, the primary and the secondary mailserver.

You can also do load balancing by sending one domain to the first and an other domain to the second mailserver as primary.

DavidGoodwin · 2016-03-14T09:54:16Z

Thank you for taking the time to submit the pull request. I've had a quick eye-ball of it, and nothing immediate is jumping out.
(The relay_recipient_maps is a useful addition too - as I've found myself needing this too)

Janfred · 2016-03-14T10:14:15Z

There is one small todo, i don't have any idea how to solve it:
At the server list is a select field for the admin. I could not remove that like in the admin list.
Since only super-admins can see the smtp-address and all other users can see all servers, this is not necessary.
Maybe you can give me a hint how to remove that.

cboltz · 2016-03-14T12:04:41Z

DOCUMENTS/POSTFIX_CONF.txt

@@ -13,12 +13,19 @@ postconf -m)  Three main.cf variables are involved:

 virtual_mailbox_domains = proxy:mysql:/etc/postfix/sql/mysql_virtual_domains_maps.cf
 virtual_alias_maps =
-   proxy:mysql:/etc/postfix/sql/mysql_virtual_alias_maps.cf,


There is a reason why mysql_virtual_alias_maps.cf is listed first - an alias domain can "override" some aliases. To keep this working, mysql_virtual_alias_maps.cf needs to be the first in the list.

Why did you move it around?

The reason i remember was that recursive lookups failed.
e.g. some_mail@host aliases to webmaster@host
webmaster@host aliases to my_name@host

In the default settings this lookup failed.
Could you name a setting where this order of handling is needed?

Hmm, I'm surprised that this fails because IIRC I have more than one "chained" alias ;-) - and I'm even more surprised that changing the order fixed it for you.

The setting that will probably break is:

alias a@host -> foo@example.com

alias a@otherhost -> bar@example.com

alias b@otherhost -> b@example.com (and some other aliases that don't exist @host)

alias domain host -> otherhost

The goal is: all mails that are sent to @host should be forwarded to the same address @Otherhost (if this address exists) - except a@ because the a@host alias overrides this.

Janfred · 2016-03-14T17:36:33Z

Ok, i've done several changes:

The language keys are changed.
The list_servers method 'moved' to the DomainHandler. I don't know if this is the best coding style, but it is better placed then in the functions.inc.php
The 2nd Migration is merged into one. ( Just one small question: does this already has effect with the sqlite-Implementation from @phyrog ? )
The primary key is set to the correct one (copy-paste-error)
The primary key is now checked that it is not empty on creation

The postcreation and postdeletion hook is only there because i followed the implementation of domains. I don't know if there is a need for server-post(creation|deletion)-scripts.

I still dont get what problem the changed order in the virtual_alias_maps would cause.
If you could tell me some usecases, I could check. if there is a solution to allow recursive lookups as well.

Janfred · 2016-03-14T20:54:40Z

For the postfix-issue: I checked it with my local setup. You were right. I don't know what went wrong in my normal setup...
I changed this in the POSTFIX_CONF.txt

Janfred · 2016-03-20T15:50:13Z

I did one final change: The server tab is now only visible for super-admins.

I thought of making only servers visible to non-global admins which are primary servers for their domain(s).
But I don't think this is very important to look at for non-global admins.

I left the primary_mx column in the domain overview, maybe this is useful. If it is not, i'll remove it too.

cboltz · 2016-05-22T15:16:50Z

model/PFAHandler.php

@@ -59,6 +59,9 @@
    # set in __construct()
    protected $allowed_domains = false;

+    # array with the server list set in __construct()
+    protected $allowed_servers = false;
+


(sorry for the long delay!)

This is only used in DomainHandler, so it should be defined there. (If it's used by more *Handler classes in the future, PFAHandler would be ok.)

tumd · 2016-09-25T11:15:08Z

DOCUMENTS/POSTFIX_CONF.txt

+password = password
+hosts = localhost
+dbname = postfix
+query = SELECT address FROM server JOIN domain ON domain.primarymx = server.server WHERE (domain.domain = '%s' OR CONCAT('.', domain.domain) = '%s') AND domain.primarymx!='@@myhostname@@'


Shouldn't the 'transport_maps'-query also include AND active = '1'?

FWIW, I love this pull request, and are working on implementing it in my own mail-server setup.

Janfred · 2017-01-29T19:08:46Z

I've rebased all changes to comply with the new master status.

You might want to recheck my implementations if they comply with your code style.

I haven't done the adjustment @cboltz suggested to use the already available 'transport' field.
The transport is not the same for the primary and secondary server. If the secondary server receives a mail for the primary it will be a relay transport. For the primary it will be (depending on the overall setup) a virtual or local transport.

jdsn · 2017-04-21T08:12:55Z

config.inc.php

@@ -523,6 +528,22 @@ function x_struct_admin_modify($struct) {
 // $CONF['domain_postdeletion_script']='sudo -u courier /usr/local/bin/postfixadmin-domain-postdeletion.sh';
 $CONF['domain_postdeletion_script'] = '';

+// Optional
+// Script to run after creation of servers.
+// Not that this may fail if PHP is nun in "safe mode", or if


Not or Note ?

jdsn · 2017-04-21T08:13:16Z

config.inc.php

+
+// Optional
+// Script to run after deletion of servers.
+// Not that this may fail if PHP is nun in "safe mode", or if


Not or Note ?

jdsn · 2017-04-21T08:14:13Z

languages/bg.lang

+$PALANG['server_button'] = 'Add server'; # XXX
+$PALANG['server_create_error'] = 'Creating server %s failed'; # XXX
+$PALANG['server_edit_error'] = 'Modifying server %s failed'; # XXX
+$PALANG['server_store_success'] = 'The Server %s has successfully been added.'; # XXX


just nitpicking: why is Server uppercase here?

jdsn · 2017-04-21T08:14:36Z

languages/bg.lang

+$PALANG['server_does_not_exist'] = 'The server %s does not exist.'; # XXX
+$PALANG['confirm_delete_server'] = 'Do you really want to delete the server %s?'; # XXX
+$PALANG['delete_server_domain_target'] = 'There are still domains, where this server is the primary one. Change them first'; # XXX
+$PALANG['server_updated'] = 'The Server %s was updated'; # XXX


just nitpicking again :)
... and a few more places below

Janfred · 2020-07-25T15:43:32Z

I have updated this pull request to Postfixadmin v3.
There is still work to do, but I'd like to hear if this goes in the right direction @cboltz

I haven't updated the language files yet, nor the postfix configuration script.

DavidGoodwin · 2020-07-25T17:19:51Z

model/ServerHandler.php

+        $firstline='';
+        $firstline=exec($command,$output,$retval);
+        if (0!=$retval) {
+            error_log("Running $command yielded return value=$retval, first line of output=$firstline");


Why not (for example) just json_encode($output) at the end of this message, rather than restricting it to the first line ?

To be honest, I've just copied this piece of code from model/DomainHandler.php#L292

I could change that in my pull request, but I guess it should then be changed in all model handlers.

DavidGoodwin · 2020-07-25T17:23:04Z

....

....nor the postfix configuration script.

By that - do you mean whatever config is needed for Postfix to use this functionality ?

Janfred · 2020-07-26T21:37:37Z

By that - do you mean whatever config is needed for Postfix to use this functionality ?

In the previous version of my pull request I changed the POSTFIX_CONF.txt to support the multiple server feature. But I think my changes were not compatible with the standard-setup.
I'd be happy about any idea how to address this issue.

DavidGoodwin · 2022-01-01T20:56:33Z

Any chance you could rebase your branch onto master and add some basic documentation - e..g what changes are needed to the database queries postfix uses for this to be in a usable state to merge in?

Neustradamus · 2023-10-15T19:16:12Z

@Janfred: It is possible to rebase your PR?

Janfred · 2023-10-15T23:09:24Z

I'll add it to my todo-List for when I have a bit time to spare.

ETE-Design · 2024-01-04T23:19:53Z

@Janfred Did you manage to rebase your PR? Would be a really nice feature to have :-)

cboltz reviewed Mar 14, 2016
View reviewed changes

cboltz reviewed May 22, 2016
View reviewed changes

tumd reviewed Sep 25, 2016

View reviewed changes

Janfred force-pushed the MultipleServerSetup branch from 29a9877 to 1371bd4 Compare January 29, 2017 19:02

jdsn reviewed Apr 21, 2017

View reviewed changes

cboltz mentioned this pull request May 10, 2020

FREQ: Add support for multi-relay mail architectures #354

Closed

DavidGoodwin assigned cboltz May 10, 2020

Add support for multiple servers

271bb09

Janfred force-pushed the MultipleServerSetup branch from 1371bd4 to 271bb09 Compare July 25, 2020 15:38

DavidGoodwin reviewed Jul 25, 2020

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for multiple servers #12

Add support for multiple servers #12

Janfred commented Mar 14, 2016

DavidGoodwin commented Mar 14, 2016

Janfred commented Mar 14, 2016

cboltz Mar 14, 2016

Janfred Mar 14, 2016

cboltz Mar 14, 2016

Janfred commented Mar 14, 2016

Janfred commented Mar 14, 2016

Janfred commented Mar 20, 2016

cboltz May 22, 2016

tumd Sep 25, 2016

Janfred commented Jan 29, 2017

jdsn Apr 21, 2017

jdsn Apr 21, 2017

jdsn Apr 21, 2017

jdsn Apr 21, 2017 •

edited

Janfred commented Jul 25, 2020

DavidGoodwin Jul 25, 2020

Janfred Jul 26, 2020

DavidGoodwin commented Jul 25, 2020

Janfred commented Jul 26, 2020

DavidGoodwin commented Jan 1, 2022

Neustradamus commented Oct 15, 2023

Janfred commented Oct 15, 2023

ETE-Design commented Jan 4, 2024

Add support for multiple servers #12

Are you sure you want to change the base?

Add support for multiple servers #12

Conversation

Janfred commented Mar 14, 2016

DavidGoodwin commented Mar 14, 2016

Janfred commented Mar 14, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Janfred commented Mar 14, 2016

Janfred commented Mar 14, 2016

Janfred commented Mar 20, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Janfred commented Jan 29, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jdsn Apr 21, 2017 • edited

Choose a reason for hiding this comment

Janfred commented Jul 25, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

DavidGoodwin commented Jul 25, 2020

Janfred commented Jul 26, 2020

DavidGoodwin commented Jan 1, 2022

Neustradamus commented Oct 15, 2023

Janfred commented Oct 15, 2023

ETE-Design commented Jan 4, 2024

jdsn Apr 21, 2017 •

edited