View Issue Details

IDProjectCategoryView StatusLast Update
0005342SOGoDocumentationpublic2021-07-05 15:34
Reporterjean38 Assigned Tofrancis  
PrioritynormalSeveritymajorReproducibilityalways
Status resolvedResolutionno change required 
Product Version5.1.1 
Summary0005342: Upgrading broke Sogo
Description

Today after upgrading via the official, supported repositories, Sogo became unavailable. I could not identify anything relevant in the logs, even after activating debug, and did not find anything relevant in the changelog. Eventually I had to restore the system to a previous version.

Steps To Reproduce

Upgrade to Sogo v5.1.1 with LDAP auth enabled.

Additional Information

So, after the drama I reproduced the issue on a test server, and it turned out that the issue is with the new default for SOGoXSRFValidationEnabled

"Parameter used to enable or not XSRF (also known as CSRF) protection in SOGo. Default value is YES, or enabled."

From the changelog :

5.1.0

The XSRF protection is now enabled by default in SOGo. If you use a single sign-on mechanisim such as C.A.S. or SAML2, you need to disable XSRF by adding SOGoXSRFValidationEnabled = NO to your configuration file.

→ LDAP in itself is not single sign-on, so I didn't care much about that note. Yet it is very much impacted!

Other issues mentioning this partly:
https://www.sogo.nu/bugs/bug_relationship_graph.php?bug_id=3668&graph=relation

TagsNo tags attached.

Activities

francis

francis

2021-06-21 14:30

administrator   ~0015325

Please share an anonymized version of your sogo.conf file. SOGoXSRFValidationEnabled is supported for internal authentication (LDAP and SQL).

jean38

jean38

2021-06-21 14:31

reporter   ~0015326

sogo_error_XSRF.png (169,547 bytes)   
sogo_error_XSRF.png (169,547 bytes)   
jean38

jean38

2021-06-21 14:36

reporter   ~0015327

Anonymised version of sogo.conf

{
/* ***** Main SOGo configuration file **

  • *
  • Since the content of this file is a dictionary in OpenStep plist format, *
  • the curly braces enclosing the body of the configuration are mandatory. *
  • See the Installation Guide for details on the format. *
  • *
  • C and C++ style comments are supported. *
  • *
  • This example configuration contains only a subset of all available *
  • configuration parameters. Please see the installation guide more details. *
  • *
  • ~sogo/GNUstep/Defaults/.GNUstepDefaults has precedence over this file, *
  • make sure to move it away to avoid unwanted parameter overrides. *
  • *
  • **/

    / Database configuration (mysql://, postgresql:// or oracle://) /
    SOGoProfileURL = "mysql://adminsql:XXXXXX@sogodb:3306/sogo/sogo_user_profile";
    OCSFolderInfoURL = "mysql://adminsql:XXXXXX@sogodb:3306/sogo/sogo_folder_info";
    OCSSessionsFolderURL = "mysql://adminsql:XXXXXX@sogodb:3306/sogo/sogo_sessions_folder";
    MySQL4Encoding = "utf8mb4";

    / Mail /
    SOGoDraftsFolderName = Drafts;
    SOGoSentFolderName = Sent;
    SOGoTrashFolderName = Trash;
    SOGoJunkFolderName = Spam;

    SOGoIMAPServer = "imaps://imap.example.com:993";
    SOGoSMTPServer = "smtp.example.com";
    SOGoMailDomain = example.com;
    SOGoMailingMechanism = smtp;

    / Notifications /
    SOGoAppointmentSendEMailNotifications = YES;
    SOGoACLsSendEMailNotifications = NO;
    SOGoFoldersSendEMailNotifications = NO;

    / Public sharing of calendar and address book /
    SOGoEnablePublicAccess = NO;

    / Authentication /
    SOGoPasswordChangeEnabled = NO;

    / LDAP authentication /
    SOGoUserSources = (
    {
    type = ldap;
    CNFieldName = cn;
    IDFieldName = uid; // first field of the DN for direct binds
    UIDFieldName = uid;

    bindFields = (uid, mail); // array of fields to use for indirect binds
    baseDN = "ou=People,dc=example,dc=com";
    bindDN = "cn=sogo,dc=example,dc=com";
    bindPassword = "XXXXXXXXXXX";
    bindAsCurrentUser = NO;

    canAuthenticate = YES;
    filter = "(mail='*')";

    displayName = "Example Addresses";
    hostname = "ldaps://ldap1.example.com";
    id = directory;
    isAddressBook = YES;
    }
    );

    / Web Interface /
    SOGoPageTitle = "Example Webmail";
    SOGoVacationEnabled = NO;
    SOGoForwardEnabled = NO;
    SOGoSieveScriptsEnabled = NO;

    //SOGoTrustProxyAuthentication = NO;
    //SOGoXSRFValidationEnabled = YES;

    / General - SOGoTimeZone MUST be defined /
    SOGoLanguage = English;
    SOGoTimeZone = Europe/Paris;
    SOGoCalendarDefaultRoles = (
    PublicDAndTViewer,
    ConfidentialDAndTViewer
    );
    //SxVMemLimit = 384;
    WOPidFile = "/var/run/sogo/sogo.pid";
    SOGoMemcachedHost = "/var/run/memcached/memcached.sock";

    SOGoMaximumFailedLoginCount = 5;
    SOGoMaximumFailedLoginInterval = 10; // seconds
    SOGoFailedLoginBlockInterval = 300; // seconds
    SOGoCacheCleanupInterval = 300;

    / Debug /
    //SOGoDebugRequests = YES;
    SoDebugBaseURL = YES;
    //ImapDebugEnabled = YES;
    //LDAPDebugEnabled = YES;
    //PGDebugEnabled = YES;
    //MySQL4DebugEnabled = YES;
    SOGoUIxDebugEnabled = YES;
    //WODontZipResponse = YES;
    WOLogFile = /var/log/sogo/sogo.log;
    }

jean38

jean38

2021-06-21 14:40

reporter   ~0015328

{
  /* *********************  Main SOGo configuration file  **********************
   *                                                                           *
   * Since the content of this file is a dictionary in OpenStep plist format,  *
   * the curly braces enclosing the body of the configuration are mandatory.   *
   * See the Installation Guide for details on the format.                     *
   *                                                                           *
   * C and C++ style comments are supported.                                   *
   *                                                                           *
   * This example configuration contains only a subset of all available        *
   * configuration parameters. Please see the installation guide more details. *
   *                                                                           *
   * ~sogo/GNUstep/Defaults/.GNUstepDefaults has precedence over this file,    *
   * make sure to move it away to avoid unwanted parameter overrides.          *
   *                                                                           *
   * **************************************************************************/

  /* Database configuration (mysql://, postgresql:// or oracle://) */
  SOGoProfileURL = "mysql://adminsql:XXXXXX@sogodb:3306/sogo/sogo_user_profile";
  OCSFolderInfoURL = "mysql://adminsql:XXXXXX@sogodb:3306/sogo/sogo_folder_info";
  OCSSessionsFolderURL = "mysql://adminsql:XXXXXX@sogodb:3306/sogo/sogo_sessions_folder";
  MySQL4Encoding = "utf8mb4";

  /* Mail */
  SOGoDraftsFolderName = Drafts;
  SOGoSentFolderName = Sent;
  SOGoTrashFolderName = Trash;
  SOGoJunkFolderName = Spam;

  SOGoIMAPServer = "imaps://imap.example.com:993";
  SOGoSMTPServer = "smtp.example.com";
  SOGoMailDomain = example.com;
  SOGoMailingMechanism = smtp;

  /* Notifications */
  SOGoAppointmentSendEMailNotifications = YES;
  SOGoACLsSendEMailNotifications = NO;
  SOGoFoldersSendEMailNotifications = NO;

  /* Public sharing of calendar and address book */
  SOGoEnablePublicAccess = NO;

  /* Authentication */
  SOGoPasswordChangeEnabled = NO;

  /* LDAP authentication */
  SOGoUserSources = (
    {
     type = ldap;
     CNFieldName = cn;
     IDFieldName = uid; // first field of the DN for direct binds
     UIDFieldName = uid;

     bindFields = (uid, mail); // array of fields to use for indirect binds
     baseDN = "ou=People,dc=example,dc=com";
     bindDN = "cn=sogo_ldap,dc=example,dc=com";
     bindPassword = "XXXXXXXXXXX";
     bindAsCurrentUser = NO;

     canAuthenticate = YES;
     filter = "(mail='*')";

     displayName = "Example Addresses";
     hostname = "ldaps://ldap1.example.com";
     id = directory;
     isAddressBook = YES;
    }
  );

  /* Web Interface */
  SOGoPageTitle = "Example Webmail";
  SOGoVacationEnabled = NO;
  SOGoForwardEnabled = NO;
  SOGoSieveScriptsEnabled = NO;

  //SOGoTrustProxyAuthentication = NO;
  //SOGoXSRFValidationEnabled = YES;

  /* General - SOGoTimeZone *MUST* be defined */
  SOGoLanguage = English;
  SOGoTimeZone = Europe/Paris;
  SOGoCalendarDefaultRoles = (
    PublicDAndTViewer,
    ConfidentialDAndTViewer
  );
  WOPidFile = "/var/run/sogo/sogo.pid";
  SOGoMemcachedHost = "/var/run/memcached/memcached.sock";

  SOGoMaximumFailedLoginCount = 5;
  SOGoMaximumFailedLoginInterval = 10; // seconds
  SOGoFailedLoginBlockInterval = 300; // seconds
  SOGoCacheCleanupInterval = 300;

  /* Debug */
  //SOGoDebugRequests = YES;
  SoDebugBaseURL = YES;
  //ImapDebugEnabled = YES;
  //LDAPDebugEnabled = YES;
  //PGDebugEnabled = YES;
  //MySQL4DebugEnabled = YES;
  SOGoUIxDebugEnabled = YES;
  //WODontZipResponse = YES;
  WOLogFile = /var/log/sogo/sogo.log;
}
francis

francis

2021-06-21 15:03

administrator   ~0015329

Last edited: 2021-06-21 15:04

It should work with this configuration. Try to empty your browser's cache, restart your Web server and memcached instance.

jean38

jean38

2021-06-21 15:10

reporter   ~0015330

I did it all before, including connecting from private browsing. I even rebooted the server before restoring.

Also, aside from memcached there are no caching servers associated with sogo.

Now that I have a test server handy, I will triple check though. However the 500 error is delivered from the local Apache which sends all to sogo daemon, so it looks like there is a issue bigger than cache at hand.

jean38

jean38

2021-06-22 07:37

reporter   ~0015332

I've re-run tests this morning, still reproducible 100% as soon as I enable XSRF and reload.

[code]
2021-06-22 09:32:31.472 sogod[3762:3762] <MySQL4Channel[0x0x560c94fd93b0] connection=0x0x560c94e201b0> query has results, entering fetch-mode.
Jun 22 09:32:31 sogod [3762]: |SOGo| request took 0.044940 seconds to execute
Jun 22 09:32:31 sogod [3762]: 192.168.100.54, 193.48.252.35 "GET /SOGo/so/test/Calendar/alarmslist?browserTime=1624347151 HTTP/1.1" 500 36/0 0.047 - - 0 - 13
Jun 22 09:32:31 sogod [3764]: |SOGo| request took 0.104119 seconds to execute
Jun 22 09:32:31 sogod [3764]: 192.168.100.54, 193.48.252.35 "GET /SOGo/so/test/Mail/0/view HTTP/1.1" 500 36/0 0.113 - - 3M - 13
Jun 22 09:32:31 sogod [3763]: |SOGo| request took 0.145917 seconds to execute
Jun 22 09:32:31 sogod [3763]: 192.168.100.54, 193.48.252.35 "POST /SOGo/so/test/Mail/0/folderINBOX/view HTTP/1.1" 500 36/126 0.150 - - 3M - 13
[/code]

With SOGoXSRFValidationEnabled set to 'NO', everything is back, no more HTTP error code 500.

francis

francis

2021-06-29 14:02

administrator   ~0015341

Is the XSRF-TOKEN cookie properly sent by the server?
Is the x-xsrf-token header properly sent by the browser? Can you sniff the HTTP traffic on port 20000 (sogod) to make sure the header reaches SOGo?

xsrf-token cookie.jpg (121,018 bytes)   
xsrf-token cookie.jpg (121,018 bytes)   
x-xsrf-token header.jpg (215,199 bytes)   
x-xsrf-token header.jpg (215,199 bytes)   
jean38

jean38

2021-06-30 07:08

reporter   ~0015342

sogo_error_XSRF-2.png (169,547 bytes)   
sogo_error_XSRF-2.png (169,547 bytes)   
jean38

jean38

2021-06-30 07:10

reporter   ~0015343

2021-06-30_09-08_XSRF_500.png (100,513 bytes)   
2021-06-30_09-08_XSRF_500.png (100,513 bytes)   
jean38

jean38

2021-06-30 07:29

reporter   ~0015344

Below is the packet capture from the Sogo server, as it receives the packets from the reverse proxy.

jean38

jean38

2021-06-30 07:29

reporter   ~0015345

francis

francis

2021-06-30 12:37

administrator   ~0015346

How about the x-xsrf-token header sent by the client? I don't see it in your packet capture.

jean38

jean38

2021-07-05 14:04

reporter   ~0015348

Putting the solution here as well: if you have enabled httpOnly for your cookies on Apache, you need to disable it for XSRF to work and not break Sogo.

Typically, I had to comment out the following line in my Apache configuration :

#Header edit Set-Cookie ^(.*)$ $1;HttpOnly;Secure

Reference: https://github.com/angular/angular/issues/20511#issuecomment-473740783

jean38

jean38

2021-07-05 14:06

reporter   ~0015349

Then I do get the X-XSRF-TOKEN header. I hope this helps others facing the same issue.

francis

francis

2021-07-05 14:48

administrator   ~0015350

Indeed, the CSRF token cookie must not have httpOnly flag, as it is intended to be read by the JavaScript by design.

Related Changesets

sogo: master f8f4de60

2021-07-05 11:33

francis


Details Diff
docs: add notice regarding the HttpOnly flag and XSRF

Fixes 0005342
Affected Issues
0005342
mod - Documentation/SOGoInstallationGuide.asciidoc Diff File

Issue History

Date Modified Username Field Change
2021-06-21 14:24 jean38 New Issue
2021-06-21 14:30 francis Note Added: 0015325
2021-06-21 14:31 jean38 Note Added: 0015326
2021-06-21 14:31 jean38 File Added: sogo_error_XSRF.png
2021-06-21 14:36 jean38 Note Added: 0015327
2021-06-21 14:40 jean38 Note Added: 0015328
2021-06-21 15:03 francis Note Added: 0015329
2021-06-21 15:04 francis Note Edited: 0015329
2021-06-21 15:10 jean38 Note Added: 0015330
2021-06-22 07:37 jean38 Note Added: 0015332
2021-06-29 14:02 francis Note Added: 0015341
2021-06-29 14:02 francis File Added: xsrf-token cookie.jpg
2021-06-29 14:02 francis File Added: x-xsrf-token header.jpg
2021-06-30 07:08 jean38 Note Added: 0015342
2021-06-30 07:08 jean38 File Added: sogo_error_XSRF-2.png
2021-06-30 07:10 jean38 Note Added: 0015343
2021-06-30 07:10 jean38 File Added: 2021-06-30_09-08_XSRF_500.png
2021-06-30 07:29 jean38 Note Added: 0015344
2021-06-30 07:29 jean38 Note Added: 0015345
2021-06-30 07:29 jean38 File Added: 2021-06-30_09-08_XSRF_500_capture.png
2021-06-30 12:37 francis Note Added: 0015346
2021-07-05 14:04 jean38 Note Added: 0015348
2021-07-05 14:04 jean38 File Added: 2021-07-05_15-43_XSRF_Http_only_off.png
2021-07-05 14:06 jean38 Note Added: 0015349
2021-07-05 14:06 jean38 File Added: 2021-07-05_15-43_XSRF_Http_only_off_header_OK.png
2021-07-05 14:48 francis Note Added: 0015350
2021-07-05 15:34 francis Changeset attached => sogo master f8f4de60
2021-07-05 15:34 francis Assigned To => francis
2021-07-05 15:34 francis Resolution open => fixed
2021-07-05 15:34 francis Status new => resolved
2021-07-05 15:34 francis Resolution fixed => no change required