View Issue Details

IDProjectCategoryView StatusLast Update
0005330SOGoActiveSyncpublic2022-11-22 11:20
Reporterrbicelli Assigned Tosmizrahi  
PrioritynormalSeveritymajorReproducibilityalways
Status resolvedResolutionfixed 
Platform[Client] GoogleOSAndroidOS VersionJelly Bean
Product Version5.0.1 
Fixed in Version5.8.0 
Summary0005330: Replying from gmail app sometimes result in message being truncated
Description

Sometimes replying from gmail app result in message being truncated.

The relevant part of attached payload is:
<Data><div dir='auto'>Ciao Max .<div dir="auto">Ignora la mCIAO RICKY, MI HA CHIAMATO REDACTED PER SAPERE SE CI SONO SCONTI...<br /><br />NELLO STORICO CDU NON NE VEDO...<br /><br />DAMMI TUE INDICAZIONI, OPPURE CONTATTALO DIRETTAMENTE.<br /><br />MASSI<br /><br /><br /><br /><br /><br /><br /><br /></Data>

Between "Ignora la m" and "CIAO" should be present another part of message.

Tagsactive sync, android

Activities

rbicelli

rbicelli

2021-05-27 14:39

reporter  

sogo-aes-debug.txt (4,792 bytes)   
<!DOCTYPE ActiveSync PUBLIC "-//MICROSOFT//DTD ActiveSync//EN" "http://www.microsoft.com/">
<Sync xmlns="AirSync:">
 <Collections>
  <Collection>
   <Class>Email</Class>
   <SyncKey>4120-20704</SyncKey>
   <CollectionId>mail%2F81a33f177a9bf859bb6300004ca6fcd2</CollectionId>
   <Status>1</Status>
   <Commands>
    <SoftDelete>
     <ServerId>3966</ServerId>
    </SoftDelete>
    <Add>
     <ServerId>4120</ServerId>
     <ApplicationData>
      <To xmlns="Email:">&quot;REDACTED.1@example.com&quot; &lt;REDACTED.1@example.com&gt;</To>
      <DisplayTo xmlns="Email:">REDACTED.1@example.com</DisplayTo>
      <From xmlns="Email:">&quot;REDACTED.2&quot; &lt;REDACTED.2@example.com&gt;</From>
      <Subject xmlns="Email:">Fwd: offerta x fustini</Subject>
      <ThreadTopic xmlns="Email:">offerta x fustini</ThreadTopic>
      <DateReceived xmlns="Email:">2021-05-27T10:44:20.000Z</DateReceived>
      <Importance xmlns="Email:">1</Importance>
      <Read xmlns="Email:">1</Read>
      <MessageClass xmlns="Email:">IPM.Note</MessageClass>
      <ContentClass xmlns="Email:">urn:content-classes:message</ContentClass>
      <InternetCPID xmlns="Email:">65001</InternetCPID>
      <Body xmlns="AirSyncBase:">
       <Type>2</Type>
       <Truncated>0</Truncated>
       <Preview/>
       <Data>&lt;div dir=&apos;auto&apos;&gt;&lt;div&gt;Ciao Mauro .&lt;/div&gt;&lt;div dir=&quot;auto&quot;&gt;Ecco l offerta aggiornata.&amp;nbsp;&lt;/div&gt;&lt;div dir=&quot;auto&quot;&gt;&lt;br&gt;&lt;div class=&quot;gmail_quote&quot; dir=&quot;auto&quot;&gt;---------- Messaggio inoltrato ----------&lt;br&gt;Da: &quot;REDACTED&quot; &amp;lt;REDACTED.4@example.com&amp;gt;&lt;br&gt;Data: 27 Mag 2021 11:46&lt;br&gt;Oggetto: offerta x Caffaro&lt;br&gt;A: REDACTED.3@example.com&lt;br&gt;Cc: &lt;br&gt;&lt;br type=&quot;attribution&quot;&gt;&lt;blockquote class=&quot;quote&quot; style=&quot;$
&lt;br&gt;

&lt;br&gt;
in allegato offerta da inviare al cliente attenzione nuova quotazione
&lt;br&gt;

&lt;br&gt;
Consegna meta&apos; Giugno.
&lt;br&gt;

&lt;br&gt;
Grazie
&lt;br&gt;

&lt;br&gt;

&lt;br&gt;

&lt;br&gt;

&lt;br&gt;

&lt;br&gt;
&lt;/p&gt;
&lt;/blockquote&gt;&lt;/div&gt;&lt;br&gt;&lt;/div&gt;&lt;/div&gt;</Data>
       <EstimatedDataSize>1168</EstimatedDataSize>
      </Body>
      <Attachments xmlns="AirSyncBase:">
       <Attachment>
        <DisplayName>15736_OFF_CLI_2021-OFCBX-0006221.pdf</DisplayName>
        <FileReference>mail/Sent/4120/2</FileReference>
        <Method>1</Method>
        <EstimatedDataSize>357764</EstimatedDataSize>
       </Attachment>
      </Attachments>
      <Flag xmlns="Email:">
       <FlagStatus>0</FlagStatus>
      </Flag>
      <Categories xmlns="Email:"/>
      <ConversationId xmlns="Email2:">PDk3N2E3MzFmLWI3ODQtNGQwMi05ZjNhLTllZWJjMGJjMjE4ZUBlbWFpbC5hbmRyb2lkLmNvbT4=</ConversationId>
      <LastVerbExecuted xmlns="Email2:">0</LastVerbExecuted>
      <NativeBodyType xmlns="AirSyncBase:">2</NativeBodyType>
     </ApplicationData>
    </Add>
    <Add>
     <ServerId>4121</ServerId>
     <ApplicationData>
      <To xmlns="Email:">&quot;REDACTED&quot; &lt;REDACTED.3@example.com&gt;</To>
      <DisplayTo xmlns="Email:">REDACTED</DisplayTo>
      <From xmlns="Email:">&quot;REDACTED.2&quot; &lt;REDACTED.2@example.com&gt;</From>
      <Subject xmlns="Email:">Re: REDACTED SUBJECT</Subject>
      <ThreadTopic xmlns="Email:">REDACTED SUBJECT</ThreadTopic>
      <DateReceived xmlns="Email:">2021-05-27T12:59:35.000Z</DateReceived>
      <Importance xmlns="Email:">1</Importance>
      <Read xmlns="Email:">1</Read>
      <MessageClass xmlns="Email:">IPM.Note</MessageClass>
      <ContentClass xmlns="Email:">urn:content-classes:message</ContentClass>
      <InternetCPID xmlns="Email:">65001</InternetCPID>
      <Body xmlns="AirSyncBase:">
       <Type>2</Type>
       <Truncated>0</Truncated>
       <Preview/>
       <Data>&lt;div dir=&apos;auto&apos;&gt;Ciao Max .&lt;div dir=&quot;auto&quot;&gt;Ignora la mCIAO RICKY, MI HA CHIAMATO REDACTED PER SAPERE SE CI SONO SCONTI...&lt;br /&gt;&lt;br /&gt;NELLO STORICO CDU NON NE VEDO...&lt;br /&gt;&lt;br /&gt;DAMMI TUE INDICAZIONI, OPPURE CONTATTALO DIRETTAMENTE.&lt;br /&gt;&lt;br /&gt;MASSI&lt;br /&gt;&lt;br /&gt;&lt;br /&gt;&lt;br /&gt;&lt;br /&gt;&lt;br /&gt;&lt;br /&gt;&lt;br /&gt;</Data>
       <EstimatedDataSize>406</EstimatedDataSize>
      </Body>
      <Flag xmlns="Email:">
       <FlagStatus>0</FlagStatus>
      </Flag>
      <Categories xmlns="Email:"/>
      <ConversationId xmlns="Email2:">PDIxMDUxMzEwMTkxMDAwMDcxNzU2OThAWERTMDMuc2Vydml6aTAxLmFkZHM+</ConversationId>
      <LastVerbExecuted xmlns="Email2:">0</LastVerbExecuted>
      <NativeBodyType xmlns="AirSyncBase:">2</NativeBodyType>
     </ApplicationData>
    </Add>
   </Commands>
  </Collection>
 </Collections>
</Sync>

sogo-aes-debug.txt (4,792 bytes)   
rbicelli

rbicelli

2021-05-27 15:47

reporter   ~0015287

Attached the Smartreply log, looks like the reply text is base64 encoded in base64 encoded body.

smartreply-eas-log.txt (1,135 bytes)   
<!DOCTYPE ActiveSync PUBLIC "-//MICROSOFT//DTD ActiveSync//EN" "http://www.microsoft.com/">
<SmartReply xmlns="ComposeMail:">
 <ClientId>SendMail-105848786755999</ClientId>
 <SaveInSentItems/>
 <Source>
  <ItemId>15051</ItemId>
  <FolderId>mail%2F41c96c0cd6446f60c85f0000c343691f</FolderId>
 </Source>
 <MIME>RGF0ZTogVGh1LCAyNyBNYXkgMjAyMSAxNDo1OTozNSArMDIwMA0KU3ViamVjdDogUmU6IEludmlvIHByZXZlbnRpdm8gbnIuIDIwMjEtT0ZDQlgtMDAwNTQ2OQ0KTWVzc2FnZS1JRDogPDk1N2FiYTc5LTE1MjMtNGJlNi1hZGYyLWFiZDNhZjlmMmEyZkBlbWFpbC5hbmRyb2lkLmNvbT4NClgtQW5kcm9pZC1NZXNzYWdlLUlEOiA8OTU3YWJhNzktMTUyMy00YmU2LWFkZjItYWJkM2FmOWYyYTJmQGVtYWlsLmFuZHJvaWQuY29tPg0KRnJvbTogci5jb2xsaXZhQGRlZ2FlZ3JhemlvbGkuaXQNClRvOiAiTWFzc2ltaWxpYW5vIEN1Z2xpYW5kb2xvIC0gREVHQSBlIEdSQVpJT0xJIFMuci5sLiINCiA8bS5jdWdsaWFuZG9sb0BkZWdhZWdyYXppb2xpLml0Pg0KSW1wb3J0YW5jZTogTm9ybWFsDQpYLVByaW9yaXR5OiAzDQpYLU1TTWFpbC1Qcmlvcml0eTogTm9ybWFsDQpNSU1FLVZlcnNpb246IDEuMA0KQ29udGVudC1UeXBlOiB0ZXh0L2h0bWw7IGNoYXJzZXQ9dXRmLTgNCkNvbnRlbnQtVHJhbnNmZXItRW5jb2Rpbmc6IGJhc2U2NA0KDQpQR1JwZGlCa2FYSTlKMkYxZEc4blBrTnBZVzhnVFdGNElDNDhaR2wySUdScGNqMGlZWFYwYnlJK1NXZHViM0poSUd4aElHMD0NCg==</MIME>
</SmartReply>
smartreply-eas-log.txt (1,135 bytes)   
tfu

tfu

2021-10-03 11:37

reporter   ~0015507

I cannot reproduce the problem with the lastest version of the gmail app.
Do you still see the problem?

plantroon

plantroon

2022-10-29 16:07

reporter   ~0016322

I am having a similar problem where Gmail app on Android breaks an attachment.

When I send an e-mail with an attachment (a jpeg picture) it sends it just fine.

However when I reply with this same attachment to another e-mail, it will do something to the attachment that makes it unreadable and the reported size in sogo webui goes from 4.5 MiB (working) down to 3.3 MiB (broken) - after download this size is 3398 KB for the broken picture and 3397 KB for the working one (yes the bigger/smaller is flipped compared to webui)

I could provide the jpeg picture for testing - it's in the attachment (Woz.jpg and the broken version Woz - broken.jpg)

SOGO version: 5.7.1

Woz.jpg (3,478,453 bytes)
Woz - broken.jpg (3,478,952 bytes)
Message with attachment.eml (4,700,883 bytes)
plantroon

plantroon

2022-11-18 16:48

reporter   ~0016399

I have info to add, by analyzing attachments in the reply, I found that the broken one is missing CRLF across the whole file (only has LF). Apparently during some processing Sogo does this which renders binary file formats unreadable (pdf, jpeg tested).

tfu

tfu

2022-11-18 18:37

reporter   ~0016401

Can you please set SOGoEASDebugEnabled = YES and check sogo.log for <SmartReply ...> and <SendMail ...> requests?

plantroon

plantroon

2022-11-19 19:12

reporter   ~0016403

Sending mail with the attachment:

<SendMail xmlns="ComposeMail:">
<ClientId>SendMail-8476503515667</ClientId>
<SaveInSentItems/>
<MIME>RGF0ZTogU2F0LCAxOSBOb3YgMjAyMiAxOTo1NTo0NyArMDEwMA0KU3ViamVjdDogamRqZA0KTW...
</SendMail>

Replying to any mail with the same attachment:

<SmartReply xmlns="ComposeMail:">
<ClientId>SendMail-8516482523716</ClientId>
<SaveInSentItems/>
<Source>
<ItemId>33149</ItemId>
<FolderId>mail%2Fa4a8082531e91361d11d00006d1336ae</FolderId>
</Source>
<MIME>RGF0ZTogU2F0LCAxOSBOb3YgMjAyMiAxOTo1NjoyOCArMDEwMA0KU3ViamVjdDogUmU6IG...
</SmartReply>

In Sent mails the attachment will be unreadable.

The MIME parts compared here (was too big to upload here):
http://upload.plantroon.com/goA.log

tfu

tfu

2022-11-20 20:01

reporter   ~0016406

This is a side effect of bug : fix(mime): encode text parts individually - Fixes 0005376 - https://github.com/inverse-inc/sope/commit/0eb64db6e1e3ba8fc5e03831ac31873f88a75859.
Similar bug was fixed in 0005427.

plantroon

plantroon

2022-11-20 20:36

reporter   ~0016408

Any way I can help further? Samsung Mail on Android has the same bug. This is universally reproducible on Android it seems. I don't know of any other good EAS apps and Gmail app by itself must be very popular.

Why this is a serious problem imo: Right now, Sogo EAS always breaks attached files to replies from Android - meaning I have to write a new mail or use a webUI or use SMTP, which is not always possible on some "corporate" wifi networks (don't ask me why somebody blocked sending through 465 and 587, but EAS being entirely https is a huge advantage in this case and blocking 465 and 587 seems to be a common malpractice). I wouldn't even know about this if the recipient didn't tell me, which is the saddest part. Most people use Gmail or Outlook and their mail sending runs over https, so again, they won't hit this problem. But even people with their company IMAP setups run into this and it's a big reason to use EAS to avoid such bs.

Btw, don't get me wrong, I appreciate this great software, but how did nobody run into this yet or do people even know it happens? Without the recipient notifying me, I wouldn't have known to this day.

tfu

tfu

2022-11-21 10:17

reporter   ~0016409

https://github.com/Alinto/sogo/pull/334

smizrahi

smizrahi

2022-11-22 11:20

administrator   ~0016412

Merged : https://github.com/Alinto/sogo/pull/334/commits/dff907a158f9b8d7d2ca69e2ad33c7a4dd562abf

Thanks @tfu

Issue History

Date Modified Username Field Change
2021-05-27 14:39 rbicelli New Issue
2021-05-27 14:39 rbicelli Tag Attached: active sync
2021-05-27 14:39 rbicelli Tag Attached: android
2021-05-27 14:39 rbicelli File Added: sogo-aes-debug.txt
2021-05-27 15:47 rbicelli Note Added: 0015287
2021-05-27 15:47 rbicelli File Added: smartreply-eas-log.txt
2021-10-03 11:37 tfu Note Added: 0015507
2022-10-29 16:07 plantroon Note Added: 0016322
2022-10-29 16:07 plantroon File Added: Woz.jpg
2022-10-29 16:07 plantroon File Added: Woz - broken.jpg
2022-10-29 16:07 plantroon File Added: Empty essage sent from Gmail.eml
2022-10-29 16:07 plantroon File Added: Message with attachment.eml
2022-10-29 16:07 plantroon File Added: Reply with attachment to_ Empty essage sent from Gmail.eml
2022-11-18 16:48 plantroon Note Added: 0016399
2022-11-18 18:37 tfu Note Added: 0016401
2022-11-19 19:12 plantroon Note Added: 0016403
2022-11-20 20:01 tfu Note Added: 0016406
2022-11-20 20:36 plantroon Note Added: 0016408
2022-11-21 10:17 tfu Note Added: 0016409
2022-11-22 11:20 smizrahi Note Added: 0016412
2022-11-22 11:20 smizrahi Assigned To => smizrahi
2022-11-22 11:20 smizrahi Status new => resolved
2022-11-22 11:20 smizrahi Resolution open => fixed
2022-11-22 11:20 smizrahi Fixed in Version => 5.8.0