How to implement GPU memory recycling in CUDA C++ for data streaming in TensorFlow? - Stack Overflow

I have to decide on the specification of a project for my HPC course, which involves optimizing GPU memory usage in a data streaming context. Specifically, I aim to implement a mechanism for recycling allocated memory on the GPU to improve efficiency while processing a stream of input data.

I've been considering TensorFlow as the framework for this task because of its built-in support for GPU operators and i was wondering if TensorFlow's API includes features to simulate or handle streaming. However, I'm unsure how to approach the problem of memory recycling in this context.

Here are my specific questions:

1. Memory Recycling in TensorFlow: Does TensorFlow have built-in tools or patterns for recycling GPU memory during continuous data processing, or would i need to implement custom solutions? The scope of my project is to implement CUDA C++ code, so i'm particularly interested in whether TensorFlow lacks a solution for handling GPU memory recycling in contexts where the input is a data stream(e.g., Sparse matrices or other data structures where dimensions significantly impact performance).
2. Custom GPU Operators: If I need to create custom GPU operators to manage memory more efficiently, how should I approach this in TensorFlow? Are there resources or examples for implementing such custom operators?
3. Profiling Memory Usage: What are the best practices for profiling and monitoring GPU memory usage in TensorFlow and CUDA, especially when working with data streams? The goal is to otpimize GPU memory usage and minimize the impact of the PCIe transfer bottleneck. I am considering to use nvprof and its graphical version for profiling CUDA execution.
4. Streaming in TensorFlow: Does TensorFlow provide APIs for handling streaming inputs of data? or perhaps tools to emulate this behavior?

If TensorFlow isn't the best choice for this type of project, I would also appreciate suggestions for alternative frameworks or tools that might be better suited for GPU memory management in a streaming context. To provide some context, this project has already been implemented in WindFlow library. My professor and i were discussing the possibility of implementing this feature in another streaming tool like Flink, but Flink doesn't support GPU "operators". As a result, the scope of the project following that path might become too large for an exam worth only 9 CFUs

I apologize in advance if my question seems somewhat vague; I am currently navigating a phase filled with ambiguity and multiple potential directions. Any guidance, references, or sample code to get started with this would be greatly appreciated!