ADFS can be very unforgiving if you need to uninstall or reconfigure. The following steps should steer you clear of any issues that you might run into. (ACL List issues are covered on my blog as well).

Step 1:
Uninstall ADFS using the control panel, remove program
NOTE:- ADFS will be shown as an installed update not a program. System will require a reboot.

Step 2:
Open IIS, find the ADFS Website, now delete the folders
Delete LS Folder
Delete ADFS Folder

Step 3:
View ADFS Application Pool
Note: You should see that there no applications now assigned to.
Delete ADFS Application Pool

Step 4:
Delete Program Files\ADFS installation path

Step 5:
IIS Restart

Now you’re ready to reinstall ADFS. Please make sure you download the latest ADFS 2.0 from the Microsoft Download Center, and don’t install/add the ADFS role from the Windows Server.