Now receive messages from the message queue and then write them to the file. Write one every hour, which is a batch write. But there is a problem. Every time a new file is written, there is some data at the beginning that should belong to the previous hour, and the message body has time. How to ensure that all messages are strictly written to your own files. What I think of now is to divide the buffers written in batches by hour, Map<hour,buffer>, one buffer corresponds to one hour, and the file name is generated based on the hour field corresponding to the buffer, so as to ensure that the data in each buffer belongs to itself. document. Is there any other better way?
The file is generated on the hour, and each message has its own timestamp, matching writing
I feel like your problem may be caused by the following reasons:
producer sent the message repeatedly
mq repeatedly sent a message to consumer
The consumer did not inform mq that it had consumed the message one hour ago, so mq sent another message the next hour
Reference:
duplicated-messages-on-activemq
avoiding-duplicated-messages-on-jms-activemq
duplicate-detection