Catch the case when no new messages are available

Confluent Kafka client throws the KafkaError._PARTITION_EOF exception
when no new messages are available in a given partitiion. We should
catch this case and continue consuming after a short sleep.

Story: 2003705
Task: 30116
Change-Id: I44add24df764bbc4e718358a8af75903e035f3f0
This commit is contained in:
Witek Bedyk 2019-03-19 09:46:15 +01:00
parent 5cfcbda66c
commit 5bdd98662b
1 changed files with 4 additions and 0 deletions

View File

@ -70,6 +70,10 @@ class KafkaConsumer(object):
continue
elif not message.error():
yield message
elif message.error().code() == \
confluent_kafka.KafkaError._PARTITION_EOF:
time.sleep(0.1)
continue
else:
log.error("Kafka error: %s", message.error().str())
raise confluent_kafka.KafkaException(message.error())