Clicking Stop will disable the app. But it will stay on your list in case you want to click Start which will refresh the app with the latest version (this happens automatically as well but not always immediately on updates). It will stay on your list until you start another app search, click on another app you are also running, or you log off.