01/27/2016 (Wed) 00:16:40 Id: bcdc6f
>>255>1. 8chan has an active working set of at least 100GB. Their servers only have 8GB of RAM. Sure, they could probably upgrade to 32GB, but with 8GB they're probably going to end up churning their working set a lot. This may be unhealthy for mongodb and lead to performance issues.\>they're probably>This may>opinions
So far so good over endchan.
Given how mongodb is meant to handle insanely large amounts of data, the chance it will handle this moderately large amount of data bad is negligible.
All benchmarks of mongo indicate the scenario at hand won't present any issues.>2. Assuming that doesn't fuck them, gridfs has contention issues. Basically gridfs is only safe, i.e. it won't throw errors and/or corrupt gridfs data files if>* Files are strictly read-only, they can never be deleted or changed>* The engine will never attempt to access a file when any kind of write or delete is in progress
Errors have ALREADY happened 2 or 3 versions ago and I already handled with them.
In software, shit always happen and you just deal with it.
Ask uguu when he managed to crash it by simply posting like mad and requesting the thread page at the same time.
The tl,dr; is:
If you read a file the exact same time you are writing, you might get an error. 95% of the time you get this error before reading anything at all. In this case I retry. The other 5% you get during reading. In that case I interrupt the reading and spit a broken page with the error at the end of the text, is up to the user to refresh and try again.
These errors will happen almost never on 1.5, though, because of how I made the page regeneration queue smarter so it will prioritize much better what should be written and when.