"Does that bother you?" Fin asked. It bothered him sometimes, but he didn't like to complain.
Other options:
no subject
"Does that bother you?" Fin asked. It bothered him sometimes, but he didn't like to complain.