View Issue Details
ID | Project | Category | View Status | Date Submitted | Last Update |
---|---|---|---|---|---|
0002667 | SOGo | ActiveSync | public | 2014-03-18 22:21 | 2014-03-19 15:45 |
Reporter | vinyard | Assigned To | ludovic | ||
Priority | normal | Severity | minor | Reproducibility | always |
Status | closed | Resolution | fixed | ||
Platform | [Client] Google | OS | Android | OS Version | Kit Kat |
Product Version | nightly v2 | ||||
Target Version | 2.2.2 | Fixed in Version | 2.2.2 | ||
Summary | 0002667: Decode base64 and quoted-printable encoded multipart emails | ||||
Description | Currently multipart emails do not get decoded. Android 4.4.2 activesync client does not decode base64 or quoted-printable emails. First patch adds base64/quoted-printable decoding to encoded multipart emails. Second patch, I'm not sure if it's needed, adds quoted-printable decoding for html and plain-text emails. | ||||
Steps To Reproduce | View base64 or quoted-printable emails in Android 4.4.2 ActiveSync client. | ||||
Tags | No tags attached. | ||||
0001-Decode-base64-and-quoted-printable-encoded-multipart.patch (1,912 bytes)
From 19d4aca11f5215d2fba4f9aafc746c45e86f378d Mon Sep 17 00:00:00 2001 From: David Rivera <vinyard@vnetgaming.net> Date: Tue, 18 Mar 2014 13:31:27 -0700 Subject: [PATCH] Decode base64 and quoted-printable encoded multipart emails --- ActiveSync/SOGoMailObject+ActiveSync.m | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/ActiveSync/SOGoMailObject+ActiveSync.m b/ActiveSync/SOGoMailObject+ActiveSync.m index 9f0ce50..4285c58 100644 --- a/ActiveSync/SOGoMailObject+ActiveSync.m +++ b/ActiveSync/SOGoMailObject+ActiveSync.m @@ -42,6 +42,7 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. #import <NGCards/iCalTimeZone.h> #import <NGExtensions/NGBase64Coding.h> +#import <NGExtensions/NGQuotedPrintableCoding.h> #import <NGExtensions/NSString+misc.h> #import <NGExtensions/NSString+Encoding.h> #import <NGImap4/NGImap4Envelope.h> @@ -196,7 +197,7 @@ struct GlobalObjectId { // - (NSData *) _preferredBodyDataInMultipartUsingType: (int) theType { - NSString *key, *plainKey, *htmlKey, *type, *subtype; + NSString *key, *plainKey, *htmlKey, *type, *subtype, *encoding; NSDictionary *textParts, *part; NSEnumerator *e; NSData *d; @@ -222,12 +223,19 @@ struct GlobalObjectId { if (theType == 2) { d = [[self fetchPlainTextParts] objectForKey: htmlKey]; + encoding = [[self lookupInfoForBodyPart: htmlKey] objectForKey: @"encoding"]; } else if (theType == 1) { d = [[self fetchPlainTextParts] objectForKey: plainKey]; + encoding = [[self lookupInfoForBodyPart: plainKey] objectForKey: @"encoding"]; } + if ([encoding caseInsensitiveCompare: @"base64"] == NSOrderedSame) + d = [d dataByDecodingBase64]; + else if ([encoding caseInsensitiveCompare: @"quoted-printable"] == NSOrderedSame) + d = [d dataByDecodingQuotedPrintableTransferEncoding]; + return d; } -- 1.7.12.4 |
|
0002-Decode-quoted-printable-encoded-html-and-plain-text.patch (990 bytes)
From 8bf6e24f9332d0e533a7f770afea554a2f8ce928 Mon Sep 17 00:00:00 2001 From: David Rivera <vinyard@vnetgaming.net> Date: Tue, 18 Mar 2014 14:14:24 -0700 Subject: [PATCH] Decode quoted-printable encoded html and plain text emails --- ActiveSync/SOGoMailObject+ActiveSync.m | 2 ++ 1 file changed, 2 insertions(+) diff --git a/ActiveSync/SOGoMailObject+ActiveSync.m b/ActiveSync/SOGoMailObject+ActiveSync.m index 4285c58..51425eb 100644 --- a/ActiveSync/SOGoMailObject+ActiveSync.m +++ b/ActiveSync/SOGoMailObject+ActiveSync.m @@ -274,6 +274,8 @@ struct GlobalObjectId { if ([encoding caseInsensitiveCompare: @"base64"] == NSOrderedSame) d = [d dataByDecodingBase64]; + else if ([encoding caseInsensitiveCompare: @"quoted-printable"] == NSOrderedSame) + d = [d dataByDecodingQuotedPrintableTransferEncoding]; // Check if we must convert html->plain if (theType == 1 && [subtype isEqualToString: @"html"]) -- 1.7.12.4 |
|
Thanks - I've already produced a more extensive patch. Can you test it? I've attached it to this ticket. |
|
outlook-encoding.diff (7,392 bytes)
diff --git a/ActiveSync/SOGoMailObject+ActiveSync.m b/ActiveSync/SOGoMailObject+ActiveSync.m index 9f0ce50..933bee3 100644 --- a/ActiveSync/SOGoMailObject+ActiveSync.m +++ b/ActiveSync/SOGoMailObject+ActiveSync.m @@ -42,12 +42,21 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. #import <NGCards/iCalTimeZone.h> #import <NGExtensions/NGBase64Coding.h> +#import <NGExtensions/NGQuotedPrintableCoding.h> #import <NGExtensions/NSString+misc.h> #import <NGExtensions/NSString+Encoding.h> #import <NGImap4/NGImap4Envelope.h> #import <NGImap4/NGImap4EnvelopeAddress.h> #import <NGObjWeb/WOContext+SoObjects.h> +#import <NGMime/NGMimeBodyPart.h> +#import <NGMime/NGMimeFileData.h> +#import <NGMime/NGMimeMultipartBody.h> +#import <NGMime/NGMimeType.h> +#import <NGMail/NGMimeMessageParser.h> +#import <NGMail/NGMimeMessage.h> +#import <NGMail/NGMimeMessageGenerator.h> + #include "iCalTimeZone+ActiveSync.h" #include "NSData+ActiveSync.h" #include "NSDate+ActiveSync.h" @@ -196,7 +205,7 @@ struct GlobalObjectId { // - (NSData *) _preferredBodyDataInMultipartUsingType: (int) theType { - NSString *key, *plainKey, *htmlKey, *type, *subtype; + NSString *encoding, *key, *plainKey, *htmlKey, *type, *subtype; NSDictionary *textParts, *part; NSEnumerator *e; NSData *d; @@ -219,15 +228,132 @@ struct GlobalObjectId { plainKey = key; } + key = nil; + if (theType == 2) + key = htmlKey; + else if (theType == 1) + key = plainKey; + + if (key) { d = [[self fetchPlainTextParts] objectForKey: htmlKey]; + + encoding = [[self lookupInfoForBodyPart: key] objectForKey: @"encoding"]; + + if ([encoding caseInsensitiveCompare: @"base64"] == NSOrderedSame) + d = [d dataByDecodingBase64]; + else if ([encoding caseInsensitiveCompare: @"quoted-printable"] == NSOrderedSame) + d = [d dataByDecodingQuotedPrintable]; } - else if (theType == 1) + + return d; +} + +// +// +// +- (void) _sanitizedMIMEPart: (id) thePart + performed: (BOOL *) b +{ + if ([thePart isKindOfClass: [NGMimeMultipartBody class]]) + { + NGMimeBodyPart *part; + NSArray *parts; + int i; + + parts = [thePart parts]; + + for (i = 0; i < [parts count]; i++) + { + part = [parts objectAtIndex: i]; + + [self _sanitizedMIMEPart: part + performed: b]; + } + } + else if ([thePart isKindOfClass: [NGMimeBodyPart class]]) { - d = [[self fetchPlainTextParts] objectForKey: plainKey]; + NGMimeFileData *fdata; + id body; + + body = [thePart body]; + + if ([body isKindOfClass: [NGMimeMultipartBody class]]) + { + [self _sanitizedMIMEPart: body + performed: b]; + } + else if ([body isKindOfClass: [NSData class]] && + [[[thePart contentType] type] isEqualToString: @"text"]) + { + // We make sure everything is encoded in UTF-8 + NSString *charset, *s; + int encoding; + + charset = [[thePart contentType] valueOfParameter: @"charset"]; + encoding = [NGMimeType stringEncodingForCharset: charset]; + + s = [[NSString alloc] initWithData: body encoding: encoding]; + AUTORELEASE(s); + + if (s) + { + body = [s dataUsingEncoding: NSUTF8StringEncoding]; + } + + NGMimeType *mimeType; + + mimeType = [NGMimeType mimeType: [[thePart contentType] type] + subType: [[thePart contentType] subType] + parameters: [NSDictionary dictionaryWithObject: @"utf-8" forKey: @"charset"]]; + [thePart setHeader: mimeType forKey: @"content-type"]; + + fdata = [[NGMimeFileData alloc] initWithBytes: [body bytes] + length: [body length]]; + + [thePart setBody: fdata]; + RELEASE(fdata); + *b = YES; + } } +} + +// +// +// +- (NSData *) _sanitizedMIMEMessage +{ + NGMimeMessageParser *parser; + NGMimeMessage *message; + NSData *d; + + BOOL b; + + d = [self content]; + + parser = [[NGMimeMessageParser alloc] init]; + AUTORELEASE(parser); + + message = [parser parsePartFromData: d]; + b = NO; + + if (message) + { + [self _sanitizedMIMEPart: [message body] + performed: &b]; + if (b) + { + NGMimeMessageGenerator *generator; + + generator = [[NGMimeMessageGenerator alloc] init]; + AUTORELEASE(generator); + + d = [generator generateMimeFromPart: message]; + } + } + return d; } @@ -266,6 +392,8 @@ struct GlobalObjectId { if ([encoding caseInsensitiveCompare: @"base64"] == NSOrderedSame) d = [d dataByDecodingBase64]; + else if ([encoding caseInsensitiveCompare: @"quoted-printable"] == NSOrderedSame) + d = [d dataByDecodingQuotedPrintable]; // Check if we must convert html->plain if (theType == 1 && [subtype isEqualToString: @"html"]) @@ -286,7 +414,8 @@ struct GlobalObjectId { } else if (theType == 4) { - d = [self content]; + d = [self _sanitizedMIMEMessage]; + //d = [self content]; } return d; @@ -531,6 +660,7 @@ struct GlobalObjectId { // [s appendFormat: @"<Reply-To xmlns=\"Email:\">%@</Reply-To>", [addressFormatter stringForArray: replyTo]]; // InternetCPID - 65001 == UTF-8, we use this all the time for now. + // - 20127 == US-ASCII [s appendFormat: @"<InternetCPID xmlns=\"Email:\">%@</InternetCPID>", @"65001"]; // Body - namespace 17 @@ -544,11 +674,16 @@ struct GlobalObjectId { NSString *content; int len, truncated; + //d = [d dataByEncodingBase64]; content = [[NSString alloc] initWithData: d encoding: NSUTF8StringEncoding]; // FIXME: This is a hack. We should normally avoid doing this as we might get // broken encodings. We should rather tell that the data was truncated and expect // a ItemOperations call to download the whole base64 encoding multipart. + // + // See http://social.msdn.microsoft.com/Forums/en-US/b9944e49-9bc9-4ab8-ba33-a9fc08557c5b/mime-raw-data-in-eas-sync-response?forum=os_exchangeprotocols + // for an "interesting" discussion around this. + // if (!content) content = [[NSString alloc] initWithData: d encoding: NSISOLatin1StringEncoding]; @@ -561,10 +696,14 @@ struct GlobalObjectId { [s appendString: @"<Body xmlns=\"AirSyncBase:\">"]; [s appendFormat: @"<Type>%d</Type>", preferredBodyType]; - [s appendFormat: @"<EstimatedDataSize>%d</EstimatedDataSize>", len]; - [s appendFormat: @"<Truncated>%d</Truncated>", 0]; + [s appendFormat: @"<Truncated>%d</Truncated>", truncated]; + [s appendFormat: @"<Preview></Preview>"]; + if (!truncated) - [s appendFormat: @"<Data>%@</Data>", content]; + { + [s appendFormat: @"<Data>%@</Data>", content]; + [s appendFormat: @"<EstimatedDataSize>%d</EstimatedDataSize>", len]; + } [s appendString: @"</Body>"]; } |
|
Of course, I'll post back as soon as I get the chance. |
|
Found one error in your patch. line 240 in SOGoMailObject+ActiveSync.m: |
|
Also, I used dataByDecodingQuotedPrintableTransferEncoding instead of dataByDecodingQuotedPrintable as it was replacing underscore wiich was breaking linked images in emails adn some URLs. (URLs had undescores) |
|
Good catches. |
|
With the above changes to your patch, the emails are displayed properly on android. |
|
Can you disable sanitization at line 417 and retest? Normally, it should only be used for Outlook clients. |
|
Sure, but I wouldn't think it would hit this code path as the preferred body type would be 2, line 417 would only be hit if the preferred body type would be 4, correct? I'll disable it and let you know. |
|
You would only hit it if your device asks for the complete MIME content. |
|
Oh I see, thanks. I've disabled sanitization, still displaying correctly on android. |
|
Perfect, thanks for your testing time, much appreciated. |
|
Glad I could help out. |
|
https://github.com/inverse-inc/sogo/commit/fc56493b782009e19fbc5d017bdbdb3b71dba6f6 |
|
Date Modified | Username | Field | Change |
---|---|---|---|
2014-03-18 22:21 | vinyard | New Issue | |
2014-03-18 22:21 | vinyard | File Added: 0001-Decode-base64-and-quoted-printable-encoded-multipart.patch | |
2014-03-18 22:22 | vinyard | File Added: 0002-Decode-quoted-printable-encoded-html-and-plain-text.patch | |
2014-03-18 22:24 | ludovic | Note Added: 0006720 | |
2014-03-18 22:27 | ludovic | File Added: outlook-encoding.diff | |
2014-03-18 22:40 | vinyard | Note Added: 0006721 | |
2014-03-18 22:57 | vinyard | Note Added: 0006722 | |
2014-03-18 23:09 | vinyard | Note Added: 0006724 | |
2014-03-18 23:10 | ludovic | Note Added: 0006725 | |
2014-03-18 23:20 | vinyard | Note Added: 0006726 | |
2014-03-18 23:22 | ludovic | Note Added: 0006727 | |
2014-03-18 23:30 | vinyard | Note Added: 0006728 | |
2014-03-18 23:38 | ludovic | Note Added: 0006729 | |
2014-03-18 23:47 | vinyard | Note Added: 0006730 | |
2014-03-18 23:57 | ludovic | Note Added: 0006731 | |
2014-03-18 23:57 | ludovic | Target Version | => 2.2.2 |
2014-03-19 00:22 | vinyard | Note Added: 0006732 | |
2014-03-19 15:45 | ludovic | Note Added: 0006737 | |
2014-03-19 15:45 | ludovic | Status | new => closed |
2014-03-19 15:45 | ludovic | Assigned To | => ludovic |
2014-03-19 15:45 | ludovic | Resolution | open => fixed |
2014-03-19 15:45 | ludovic | Fixed in Version | => 2.2.2 |