Modified kafka max buffer size

Setting max_buffer_size to None in the kafka consumer object lets the
consumer to grow its internal buffer without limit allowing it to read messages
of any size from the kafka queue.

Change-Id: Ic47068cf1c12a9d40c6d9fe4424cf2d658955139
This commit is contained in:
Joe Keen 2015-10-05 10:42:00 -06:00
parent ca21c1feed
commit f9403b3a6e
1 changed files with 2 additions and 1 deletions

View File

@ -49,7 +49,8 @@ class KafkaConsumer(BaseProcessor):
group,
self._kafka_topic,
auto_commit=False,
iter_timeout=5)
iter_timeout=5,
max_buffer_size=None)
self._consumer.provide_partition_info() # Without this the partition is not provided in the response
self._consumer.fetch_last_known_offsets()