That is interesting and definitely not normal behavior. It could be a glitch within the Task Manager, however, selecting them and clicking End task should close them out.
If the issue is still persisting, please click on the gear icon on the top right of the VyprVPN app.
Then select Help, and then Report a Bug.
Enter a description of the issue and ensure you check the box next to Include application logs.
Then click Report a Bug so that we may review and investigate further.
Thank you for contacting us today!