Bruce Pearl clears the air about his comments made regarding "loyalty"