I am a newbie in operation and maintenance. After recently doing simple processing of company logs, I added a certain amount to a list, and then used the ES interface to add the data in batches to another ElasticSearch, and then displayed it in some ways. There is a problem here. If data is inserted, an exception occurs. For example, the es host to be inserted crashes, etc., but there is no problem with the data source. This should move the processed data to the cache and then stop the program. (The data source is kafka. Even if the program is stopped, it can still be consumed after restarting.)
So I would like to ask, for relatively lightweight python programs or scripts, how should these exceptions be stored? What about the data that needs to be stored first?