For some days I'm running the mod_http_upload instead of the python script and noticed that sending a file causes a hugh memory usage on my Cent OS system with prosody trunk. Sending a 36 MiB file increases ram usage to more about 6,5 GiB so I switched back to the python script. Any ideas?

I have the same problem here. If I send a 70MB file, the lua process eats 100% of my RAM+Swap. Finally Linux kills it (and sometimes other processes as well) due to OOM. Its always reproducible. I am using the latest stable 0.9.10 on Ubuntu 14.04 from the repo.

Are there any soloutions for this problem?

Thanks for the report. What network backend is used (select or event)? Does the memory growth appear when uploading or downloading? (or both?) As for solutions, using something other than an experimental non-core plugin is one way. Having another http server handle the serving of files could also help. Or just don't send files that large. FWIW, I was unable to reproduce using a 700M file.

• owner Zash
• tags Status-NeedInfo HTTP

Hi, thanks for your answer. I'm using libevent if you meant that as backend. The growth appears while uploading files and sizes until 50 MB and is higher on bigger files. I didn't try another http server yet.

@kriztan: I am seeing something similar, do you by any chance run this on arm ? I also used event so it might be related to that.

@mt: I didn't spend more time with this prosody module yet, due to this problem. I'm still using the python script from https://github.com/siacs/HttpUploadComponent

I'm seeing this as well. My prosody server uses about 100 times more memory than the filesize (above 1GB ram usage for a simple 10MB file). Because of this I wrote a new external upload component in php. This component is capable of uploading and downloading files at arbitrary sizes only limited by the amount of disk space you have available and nothing more. You can get it at github: https://github.com/tmolitor-stud-tu/http_upload_component Feedback welcome :)

Fixed in https://hg.prosody.im/0.9/rev/3ae247af68f4 There are now also limits on the size of HTTP uploads and the size of buffered data. Since this issue was in the HTTP stream parser, it is not limited to mod_http_upload, but affects every use of HTTP. You may want to follow #726 for future improvements in this area.

• tags Status-Fixed Milestone-0.9
• title [mod_http_upload] memory usage on file upload Inefficient buffering of incoming HTTP payloads