Spontaneous thought; Caudium truncates the data it sends to MIME (timeout?). Wasn't this a also problem in WebServer that has been fixed?
/ Martin Nilsson (har bott i google)
Previous text:
2003-04-13 22:59: Subject: MIME multipart message improperly terminated bug [3414]
Product: Pike Version: 7.4 Component: Modules Reporter: funk@softhome.net URL: https://community.roxen.com/crunch/show_bug.cgi?id=3414
I'm using pike 7.4.21 with the Caudium web server. I've noticed a bug related to uploading files using MIME attachments. IT seems that small files work fine, however larger files do not. The web browser keeps saying it's loading, but the caudium log gives error ouput as follows:
multipart message improperly terminated ("\r\nContent-Disposition: form-data; name="file"; filename="drummer.png"\r\nContent-Type: image/png***Here it shows the text ouput of the file i cut it out to save lots of space in the bug report*** ) /usr/local/pike/7.4.21/lib/modules/MIME.pmod/module.pmod:1078: MIME.Message()->create("-----------------------------47449013511489176341833204461\r\nContent-Disposition: form-data; name="file"; filename="drummer.png"\r\nContent-Type: image/png\r\n\r\n\211PNG\r\n\32\n\0\0\0\rIHDR\0\0\1g\0\0\1j\b\6\0"+[16854],mapping[11],0,0) object_program(MIME): MIME->Message() protocols/http.pike:200: protocols/http()->handle_body_encoding(21665) protocols/http.pike:455: protocols/http()->parse_got() protocols/http.pike:1657: protocols/http()->got_data(0,"7\366$\336\31t\34\223\245\314)\262r\16\302\361D_\25\36PjZuk\27\7!\301""6\345D9s\224\277\7\341\317N|\24\215\255\234\233\220ES\355\273\274\36\361\310\310\373\354\350u\340-\334\23\212q\210\37\177=\322\216[9\367 \35\345<\31\231\350\254\t\274\215< \343\234\261\307J\243(\347\t1\367\347\227V`\245\30\373\253\364\203nD\345\274#\372\33\322?\220\207\n\177lU<\217\373\\276A}+g\27q\377\336""5\345\274\21b\352*\323\216\230\275\216#\207o+Y\257\352\247\365e\335\25s\177YQiG\323L2\365ls\326r\253|I\261\24""3\300\177\225\355CS\225"]4_\347\264"+[1254]) /usr/local/pike/7.4.21/lib/modules/Stdio.pmod/module.pmod:695: Stdio.File("socket", "192.168.0.2 58269", 777 /* fd=19 */)->__stdio_read_callback() module.c:58()->Builtin()->Backend: module.c:58()->Builtin()->__backend->`()(3600.000000) /usr/local/pike/7.4.21/lib/master.pike:1951: master()->_main(({"bin/caudium","-DCAUDIUM","-DCAUDIUM_CACHE","-DROXEN","-DMODULE_DEBUG","-w","-Mlib/7.4.21/",,,6}),({"SSH_CLIENT=192.168.0.2 63575 22","USER=root","MAIL=/var/mail/root","SHLVL=1",,,15}))
The log keeps repeating this until i press stop in my browser...
Here i'll include the code i use to upload the images.
HTML PAGE:
<form name="imhouploadform" method="POST" enctype="multipart/form-data" action="/(dumpid)/upload"> <br /> <input name="file" type="file"> <br /> <input type="submit" name="1" value="Upload to attachments" /> </form>
excerpt from Caudium Pike Module:
mixed find_file( string f, object id ) { string sucpage = id->variables->success; string fpage = id->variables->failed; string contents = id->variables->file; string filename = id->variables["file.filename"]; string gallery; if (id->variables->gallery) gallery = id->variables->gallery; else gallery = QUERY(defaultgallery); string galpath = Stdio.append_path(QUERY(albumpath), gallery); string path = Stdio.append_path(QUERY(albumpath), gallery, filename); if ( contents && filename && Stdio.is_dir(galpath) && !Stdio.is_file(path) && isgood(filename) && sizeof(contents) < QUERY(maxsize) ) { Stdio.write_file ( path, contents ); return http_string_answer("Upload Succeeded","text/html"); }
I hope this helps someone locate and troubleshoot this bug, it's seriously stumping my productivity... Well thanks for any help.
/ Brevbäraren
I also suspect the problem is with Caudium rather than with Pike. Maybe it does something inappropriate if the request comes in more than one TCP packet?
/ Marcus Comstedt (ACROSS) (Hail Ilpalazzo!)
Previous text:
2003-04-13 23:44: Subject: MIME multipart message improperly terminated bug [3414]
Spontaneous thought; Caudium truncates the data it sends to MIME (timeout?). Wasn't this a also problem in WebServer that has been fixed?
/ Martin Nilsson (har bott i google)
Maybe, but then - maybe not.
For a while I have been passing all my incoming mail through pike's MIME.Message (a total of about 16k messages by now), and I do observe a similar error on a small percentage of messages. No TCP is involved here (the messages are passed on through a pipe from procmail). In all cases I looked at, the rejected messages were spam from the Far East, so I assumed they were really malformed MIME and didn't worry too much about it. I guess I'll have a closer look now. /rjb
/ rjb
Previous text:
2003-04-14 02:16: Subject: MIME multipart message improperly terminated bug [3414]
I also suspect the problem is with Caudium rather than with Pike. Maybe it does something inappropriate if the request comes in more than one TCP packet?
/ Marcus Comstedt (ACROSS) (Hail Ilpalazzo!)
If you find something which is rejected but which isn't actually malformed, please do share it.
/ Marcus Comstedt (ACROSS) (Hail Ilpalazzo!)
Previous text:
2003-04-14 09:43: Subject: MIME multipart message improperly terminated bug [3414]
Maybe, but then - maybe not.
For a while I have been passing all my incoming mail through pike's MIME.Message (a total of about 16k messages by now), and I do observe a similar error on a small percentage of messages. No TCP is involved here (the messages are passed on through a pipe from procmail). In all cases I looked at, the rejected messages were spam from the Far East, so I assumed they were really malformed MIME and didn't worry too much about it. I guess I'll have a closer look now. /rjb
/ rjb
Well, I took a close look at one of my rejects, and it seems that the problem was related to having a multipart message nested in another, _and_ the boundary string of the outer one being a substring of that for the inner. To be specific:
crap;
(1) Result: "Content-type: multipart/mixed;boundary="XXX"\n\ncrap\n\n--XXX\nContent-type: multipart/mixed;boundary="XXXAA"\n\ndung\n\n--XXXAA\n\nabc\n\n--XXXAA--\n--XXX--\n"
MIME.Message(crap);
multipart message improperly terminated ("\ndung\n") /usr/local/pike/7.4.13/lib/modules/MIME.pmod/module.pmod:1078: MIME.Message()->create("Content-type: multipart/mixed;boundary="XXXAA"\n\ndung\n",0,0,0) object_program(MIME): MIME->Message() /usr/local/pike/7.4.13/lib/modules/MIME.pmod/module.pmod:1092: MIME.Message()->__lambda_65679_4_line_1083("Content-type: multipart/mixed;boundary="XXXAA"\n\ndung\n") /usr/local/pike/7.4.13/lib/modules/MIME.pmod/module.pmod:1082: MIME.Message()->create("Content-type: multipart/mixed;boundary="XXX"\n\ncrap\n\n--XXX\nContent-type: multipart/m
ixed;boundary="XXXAA"\n\ndung\n\n--XXXAA\n\nabc\n\n--XXXAA--\n--XXX--\n",0,0,0) object_program(MIME): MIME->Message() HilfeInput:1: HilfeInput()->___HilfeWrapper()
crap=replace(crap, "XXXAA", "YYY");
(2) Result: "Content-type: multipart/mixed;boundary="XXX"\n\ncrap\n\n--XXX\nContent-type: multipart/mixed;boundary="YYY"\n\ndung\n\n--YYY\n\nabc\n\n--YYY--\n--XXX--\n"
MIME.Message(crap);
(3) Result: MIME.Message()
(I have no idea whether this violates MIME...)
/ rjb
Previous text:
2003-04-14 12:28: Subject: MIME multipart message improperly terminated bug [3414]
If you find something which is rejected but which isn't actually malformed, please do share it.
/ Marcus Comstedt (ACROSS) (Hail Ilpalazzo!)
Umm, actually being a substring is not enough. The outer's boundary must be a prefix of the inner's to trigger the error. /rjb
/ rjb
Previous text:
2003-04-14 12:45: Subject: MIME multipart message improperly terminated bug [3414]
Well, I took a close look at one of my rejects, and it seems that the problem was related to having a multipart message nested in another, _and_ the boundary string of the outer one being a substring of that for the inner. To be specific:
crap;
(1) Result: "Content-type: multipart/mixed;boundary="XXX"\n\ncrap\n\n--XXX\nContent-type: multipart/mixed;boundary="XXXAA"\n\ndung\n\n--XXXAA\n\nabc\n\n--XXXAA--\n--XXX--\n"
MIME.Message(crap);
multipart message improperly terminated ("\ndung\n") /usr/local/pike/7.4.13/lib/modules/MIME.pmod/module.pmod:1078: MIME.Message()->create("Content-type: multipart/mixed;boundary="XXXAA"\n\ndung\n",0,0,0) object_program(MIME): MIME->Message() /usr/local/pike/7.4.13/lib/modules/MIME.pmod/module.pmod:1092: MIME.Message()->__lambda_65679_4_line_1083("Content-type: multipart/mixed;boundary="XXXAA"\n\ndung\n") /usr/local/pike/7.4.13/lib/modules/MIME.pmod/module.pmod:1082: MIME.Message()->create("Content-type: multipart/mixed;boundary="XXX"\n\ncrap\n\n--XXX\nContent-type: multipart/m
ixed;boundary="XXXAA"\n\ndung\n\n--XXXAA\n\nabc\n\n--XXXAA--\n--XXX--\n",0,0,0) object_program(MIME): MIME->Message() HilfeInput:1: HilfeInput()->___HilfeWrapper()
crap=replace(crap, "XXXAA", "YYY");
(2) Result: "Content-type: multipart/mixed;boundary="XXX"\n\ncrap\n\n--XXX\nContent-type: multipart/mixed;boundary="YYY"\n\ndung\n\n--YYY\n\nabc\n\n--YYY--\n--XXX--\n"
MIME.Message(crap);
(3) Result: MIME.Message()
(I have no idea whether this violates MIME...)
/ rjb
That does indeed violate MIME. RFC2046, section 5.1.1:
"Boundary delimiters must not appear within the encapsulated material, and must be no longer than 70 characters, not counting the two leading hyphens."
Here, the boundary delimiter ("--XXX") _does_ appear inside the encapsulated material (as part of the boundary delimiter "--XXXAA"), so the message is invalid and Pike is correct in rejecting it. Note that the line break following the delimiter is not part of the delimiter itself.
/ Marcus Comstedt (ACROSS) (Hail Ilpalazzo!)
Previous text:
2003-04-14 12:45: Subject: MIME multipart message improperly terminated bug [3414]
Well, I took a close look at one of my rejects, and it seems that the problem was related to having a multipart message nested in another, _and_ the boundary string of the outer one being a substring of that for the inner. To be specific:
crap;
(1) Result: "Content-type: multipart/mixed;boundary="XXX"\n\ncrap\n\n--XXX\nContent-type: multipart/mixed;boundary="XXXAA"\n\ndung\n\n--XXXAA\n\nabc\n\n--XXXAA--\n--XXX--\n"
MIME.Message(crap);
multipart message improperly terminated ("\ndung\n") /usr/local/pike/7.4.13/lib/modules/MIME.pmod/module.pmod:1078: MIME.Message()->create("Content-type: multipart/mixed;boundary="XXXAA"\n\ndung\n",0,0,0) object_program(MIME): MIME->Message() /usr/local/pike/7.4.13/lib/modules/MIME.pmod/module.pmod:1092: MIME.Message()->__lambda_65679_4_line_1083("Content-type: multipart/mixed;boundary="XXXAA"\n\ndung\n") /usr/local/pike/7.4.13/lib/modules/MIME.pmod/module.pmod:1082: MIME.Message()->create("Content-type: multipart/mixed;boundary="XXX"\n\ncrap\n\n--XXX\nContent-type: multipart/m
ixed;boundary="XXXAA"\n\ndung\n\n--XXXAA\n\nabc\n\n--XXXAA--\n--XXX--\n",0,0,0) object_program(MIME): MIME->Message() HilfeInput:1: HilfeInput()->___HilfeWrapper()
crap=replace(crap, "XXXAA", "YYY");
(2) Result: "Content-type: multipart/mixed;boundary="XXX"\n\ncrap\n\n--XXX\nContent-type: multipart/mixed;boundary="YYY"\n\ndung\n\n--YYY\n\nabc\n\n--YYY--\n--XXX--\n"
MIME.Message(crap);
(3) Result: MIME.Message()
(I have no idea whether this violates MIME...)
/ rjb
Right. Also note that it is not clear whether this has anything to do with the reason for the problem originally reported as #3414.
Fortunately, my experience is that all messages I have seen that violate this rule were spam, and seem to have been generated by buggy mass-mailing software (so I hardly see this as a problem, unless my aim were to collect a corpus of spam mails...) /rjb
/ rjb
Previous text:
2003-04-14 12:56: Subject: MIME multipart message improperly terminated bug [3414]
That does indeed violate MIME. RFC2046, section 5.1.1:
"Boundary delimiters must not appear within the encapsulated material, and must be no longer than 70 characters, not counting the two leading hyphens."
Here, the boundary delimiter ("--XXX") _does_ appear inside the encapsulated material (as part of the boundary delimiter "--XXXAA"), so the message is invalid and Pike is correct in rejecting it. Note that the line break following the delimiter is not part of the delimiter itself.
/ Marcus Comstedt (ACROSS) (Hail Ilpalazzo!)
pike-devel@lists.lysator.liu.se