Java program crashes when replacing a base jar

I have a piece of software contained in one .jar that does its job, but sometimes I need to quickly click on a bug fix, requiring a .jar file to be replaced in a central location, unfortunately, if there is a jar currently executing from that file if I replace it, it crashes with a "class not found" error. I thought that as soon as the jar file is executed, the JVM will cache it in memory and will not do any reads from the disk, but apparently this is not how it can (if possible at all) fix it?

EDIT:

The application is not based on a web interface. This is normal Java SE.