Sounds like there is a process being left behind. You should make sure all child processes are closed when exiting an application. Since you say you have Java packages I suspect it is Java auto updater being the issue here. But before you know for sure it's hard to give you a workaround.
Try this:
In the package's package.ini add this in the build options section:
ProcessExternalNameBehavior=Original
..and rebuild.
Now you should be able to use Process Explorer (SysInternals) to see which process is being left behind (never use task manager, it doesn't show you everything). Once identified it should be easy to solve.
