There is no direct way to logout of Amazon android appstore . From within the settings, it displays "logged in as ___' but does not let you log out. So if you want to use apps that are associated with a different account of yours, you can not.
The work around is to goto Settings-apps, select the appstore app and then clear data. Since you are clearing the data associated with the appstore, you are clearing your login data too. If you open the app again now, you can login again with the same or a different user id.
Most apps that are installed using Amazon appstore require you to be logged in to the appstore to be able to use the app. So the moment you logout of the app, you can not use the apps you have already installed unless you login to the appstore.