archive/pillar
0
0
Fork 0
Code Issues 42 Pull Requests Projects Releases Wiki Activity

refresh_links_for_backend: lowering batch size

Browse Source 
MongoDB will close an idle connection after 10 minutes. By lowering the
batch size, I hope we'll often request a small batch of data, therefore
avoiding the connection to be idle for too long and timing out.
This commit is contained in:
Sybren A. Stüvel
2016-07-27 11:54:49 +02:00
parent 2124e4335d
commit 37bf9733f6
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
pillar/application/modules/file_storage.py
View File

@@ -466,7 +466,7 @@ def refresh_links_for_backend(backend_name, chunk_size, expiry_seconds):
{'$or': [{'backend': backend_name, 'link_expires': None}, {'$or': [{'backend': backend_name, 'link_expires': None},
{'backend': backend_name, 'link_expires': {'$lt': expire_before}}, {'backend': backend_name, 'link_expires': {'$lt': expire_before}},
{'backend': backend_name, 'link': None}] {'backend': backend_name, 'link': None}]
}).sort([('link_expires', pymongo.ASCENDING)]).limit(chunk_size) }).sort([('link_expires', pymongo.ASCENDING)]).limit(chunk_size).batch_size(5)
if to_refresh.count() == 0: if to_refresh.count() == 0:
log.info('No links to refresh.') log.info('No links to refresh.')