First of all, it depends on whether or not it was an officiated match. If so, the official would've called a let AND a hindrance warning (if it happens again, person losing hat loses point.) In an unofficiated match, the person who lost the hat cannot call a let (rules says that YOU CANNOT CREATE YOUR OWN LET). The opponent could have called a let but since she/he didn’t, it now becomes part of the court. Thus a ball hitting the hat out of bounds is out. If the hat was in the middle of the court and a ball in play hits it and it bounces up, it is in play.
