Yes, you can deny users access in the SSHD config. However, going by the principle of not giving a user more access then they need, a better solution in view of the FTP-only requirement would be changing that users shell to /sbin/nologin