Data mining trades & ticker feed in Influxdb but ticker feed stops randomly

[NashTools]

Im mining trades and ticker feed across all platforms that have btc-usd pairs into influxdb using influxdb backend handler. trade feed runs normally, but ticker feeds stops at some point, sometimes after a few minutes, sometimes after a few hours. All ticker feeds end at the same time. Trade feed is not affected by the abrupt termination of ticker feed. I can't find any useful logs regarding this issue. Any help is highly appreciated!

code snippet of the handler

def create_feed_handler():
    config = {
        'uvloop': True,
        'log': {'filename': 'logs/system.log', 'level': 'DEBUG'},
    }

    handler = FeedHandler(config)

    feed_callback = {TRADES: TradeInflux(influx.INFLUX_ADDR, influx.ORG, influx.BUCKET_TRADES, influx.TOKEN),
                     TICKER: TickerInflux(influx.INFLUX_ADDR, influx.ORG, influx.BUCKET_TICKER, influx.TOKEN)}

    handler.add_feed(Binance(symbols=['BTC-USDT'], channels=[TRADES, TICKER], callbacks=feed_callback))
    handler.add_feed(Binance(symbols=['BTC-BUSD'], channels=[TRADES, TICKER], callbacks=feed_callback))
    handler.add_feed(Binance(symbols=['BTC-USDC'], channels=[TRADES, TICKER], callbacks=feed_callback))
    handler.add_feed(BinanceFutures(symbols=['BTC-USDT-PERP'], channels=[TRADES, TICKER], callbacks=feed_callback))
    handler.add_feed(BinanceDelivery(symbols=['BTC-USD-PERP'], channels=[TRADES, TICKER], callbacks=feed_callback))
    handler.add_feed(BinanceFutures(symbols=['BTC-BUSD-PERP'], channels=[TRADES, TICKER], callbacks=feed_callback))
    handler.add_feed(FTX(symbols=['BTC-USD'], channels=[TRADES, TICKER], callbacks=feed_callback))
    handler.add_feed(FTX(symbols=['BTC-USDT'], channels=[TRADES, TICKER], callbacks=feed_callback))
    handler.add_feed(FTX(symbols=['BTC-USD-PERP'], channels=[TRADES, TICKER], callbacks=feed_callback))
    handler.add_feed(FTXUS(symbols=['BTC-USD'], channels=[TRADES, TICKER], callbacks=feed_callback))
    handler.add_feed(FTXUS(symbols=['BTC-USDT'], channels=[TRADES, TICKER], callbacks=feed_callback))
    handler.add_feed(Huobi(symbols=['BTC-USDT'], channels=[TRADES], callbacks=feed_callback))
    handler.add_feed(HuobiLinearSwap(symbols=['BTC-USDT-PERP'], channels=[TRADES], callbacks=feed_callback))
    handler.add_feed(HuobiSwap(symbols=['BTC-USD-PERP'], channels=[TRADES], callbacks=feed_callback))
    handler.add_feed(Bitfinex(symbols=['BTC-USD'], channels=[TRADES, TICKER], callbacks=feed_callback))
    handler.add_feed(Bitstamp(symbols=['BTC-USD'], channels=[TRADES], callbacks=feed_callback))
    handler.add_feed(Coinbase(symbols=['BTC-USD'], channels=[TRADES, TICKER], callbacks=feed_callback))
    handler.add_feed(Coinbase(symbols=['BTC-USDC'], channels=[TRADES, TICKER], callbacks=feed_callback))
    handler.add_feed(Coinbase(symbols=['BTC-USDT'], channels=[TRADES, TICKER], callbacks=feed_callback))
    handler.add_feed(Gateio(symbols=['BTC-USDT'], channels=[TRADES, TICKER], callbacks=feed_callback))
    handler.add_feed(Kraken(symbols=['BTC-USD'], channels=[TRADES, TICKER], callbacks=feed_callback))
    handler.add_feed(Kraken(symbols=['BTC-USDT'], channels=[TRADES, TICKER], callbacks=feed_callback))
    handler.add_feed(KrakenFutures(symbols=['BTC-USD-PERP'], channels=[TRADES, TICKER], callbacks=feed_callback))
    handler.add_feed(KuCoin(symbols=['BTC-USDT'], channels=[TRADES, TICKER], callbacks=feed_callback))
    return handler

if __name__ == '__main__':
    fh = feed.create_feed_handler()
    fh.run()

[NashTools]

i figured out the problem. the ticker feed import was lagging behind thats why it appeared to me as it was not importing.

[bmoscon]

ok glad you figured it out - I meant to say that the issue was probably a throughput issue - thats a lot of data for a single python process to handle and also write to influx. you might try breaking that up into several processes, and/or offload the write to a separate process