Poll for messages always returns empty regardless of index committed

[lhmzhou]

Hi,

Trying to write a python script that consumes the entire Kafka history of messages from index 0 using the library kafka-python, but am having trouble receiving any messages for some reason. Regardless of what I commit my index to be a poll for messages always returns empty. Is there something wrong with the order of operations?

Here's the output I'm receiving thus far:

part=TopicPartition(topic='SharedKafka2_500000958_Aquila-Vulnerability-Event', partition=2), pos=0, max=61
part=TopicPartition(topic='SharedKafka2_500000958_Aquila-Vulnerability-Event', partition=1), pos=0, max=67
part=TopicPartition(topic='SharedKafka2_500000958_Aquila-Vulnerability-Event', partition=4), pos=0, max=66
part=TopicPartition(topic='SharedKafka2_500000958_Aquila-Vulnerability-Event', partition=3), pos=0, max=64
part=TopicPartition(topic='SharedKafka2_500000958_Aquila-Vulnerability-Event', partition=0), pos=0, max=65
{}

Notice the poll at the bottom is empty despite having roughly 60 messages in each partition (...what?!).

I'm assuming committed offsets are only used on rebalance, otherwise there's an in memory offset that just increases with each poll. If im using auto.offset.reset, do I have to use a random group-id? I've added the reset and commented out the commit and so far I still don't receive any empirical results.