First of all, it depends on the session state mode. Your thoughts will relate to the operating mode or other user mode, which will store objects in memory.
On the other hand, removing an object from the session state does not mean that you killed it. This is just a collection of objects, like any other (fe List).
Objects are destroyed and deleted from memory by the garbage collector.
You can learn more about the garbage collector in this MSDN article: